Target detection method and device, electronic equipment and medium

Abstract

The embodiment of the invention discloses a target detection method and device, electronic equipment and a medium. The method comprises the following steps: determining a sound source area corresponding to each sound collection device according to the sound intensity collected by microphones in the sound collection devices for at least two sound collection devices which collect sound; determining candidate objects associated with each sound collection device which collects the sound; and determining a target object from the candidate objects according to the relative relationship between the positions of the candidate objects and the sound source area. According to the scheme, the target object which is sounding at present can be accurately positioned through the sound intensity acquired by the microphone in the sound acquisition equipment which can acquire the sound, and the specific position of the target object is accurately determined.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a target detection method, device, electronic equipment and medium. Background Art

[0002] At present, conference systems that integrate screen sharing and sound sharing are widely used. The use of conference tablets and conference whiteboards is becoming more and more popular, and they are widely used in schools, businesses and other fields. Conference tablets and conference whiteboards integrate display functions, writing functions, video conferencing, wireless connections and other functions, and are rich in functions. The video conferencing function involves two-way video interaction and voice interaction, including image acquisition, audio acquisition, speaker playback and other functions.

[0003] In the process of audio collection, if the conference whiteboard's own microphone is used for audio collection, it is difficult to effectively collect the sound of the sound source far away due to the limited sound collection distance. If a wireless cascade microphone is used for audio collection, it is impossible to determine the speaker due to the large number of microphones. Summary of the invention

[0004] The present application provides a target detection method, device, electronic device and medium to accurately determine the target object currently giving a lecture.

[0005] According to one aspect of the present application, a target detection method is provided, the method comprising:

[0006] For at least two sound collection devices that collect sound, determine the sound source area corresponding to each sound collection device according to the sound intensity collected by the microphone in the sound collection device;

[0007] Determining candidate objects associated with each sound collection device that collects the sound;

[0008] According to the relative relationship between the positions of the candidate objects and the sound source area, a target object is determined from the candidate objects.

[0009] According to another aspect of the present application, a target detection device is provided, the device comprising:

[0010] A sound source region determination module, for determining the sound source region corresponding to each sound collection device according to the sound intensity collected by the microphone in the sound collection device for at least two sound collection devices that collect sound;

[0011] A candidate object determination module, used to determine candidate objects associated with each sound collection device that collects sound;

[0012] The target object determination module is used to determine the target object from the candidate objects according to the relative relationship between the positions of the candidate objects and the sound source area.

[0013] According to another aspect of the present application, an electronic device is provided, the device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the target detection method of any embodiment of the present application.

[0017] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the target detection method of any embodiment of the present application when executed.

[0018] The technical solution of the embodiment of the present application is to determine the sound source area corresponding to each sound collection device according to the sound intensity collected by the microphone in the sound collection device for at least two sound collection devices that collect sound; determine the candidate objects associated with each sound collection device that collects sound; and determine the target object from the candidate objects according to the relative relationship between the position of the candidate object and the sound source area. The above solution can accurately determine the sound source area according to the sound intensity collected by the microphone of the sound collection device for the sound collection device that collects sound, and then accurately locate the target object that is currently making sound, and accurately determine the specific position of the target object.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of a target detection method provided according to Embodiment 1 of the present application;

[0022] Figure 2 It is a schematic diagram of the internal components of the sound collection device provided according to the first embodiment of the present application;

[0023] Figure 3is a first schematic diagram of the conference system layout provided according to the first embodiment of the present application;

[0024] Figure 4 is a second schematic diagram of the conference system layout provided according to the first embodiment of the present application;

[0025] Figure 5 is a third schematic diagram of the conference system layout provided according to the first embodiment of the present application;

[0026] Figure 6 is a fourth schematic diagram of the conference system layout provided according to the first embodiment of the present application;

[0027] Figure 7 is a fifth schematic diagram of the conference system layout provided according to the first embodiment of the present application;

[0028] Figure 8 is a flow chart of a target detection method provided according to Embodiment 2 of the present application;

[0029] Fig. 9 A first schematic diagram is provided for determining a sound source area according to the second embodiment of the present application;

[0030] Fig.10 The second schematic diagram is determined according to the sound source area provided in the second embodiment of the present application;

[0031] Fig.11 A third schematic diagram for determining the sound source area according to the second embodiment of the present application;

[0032] Fig.12 is a flow chart of a target detection method provided according to Embodiment 3 of the present application;

[0033] Fig.13 This is a schematic diagram showing a preset area provided in Example 3 of the present application;

[0034] Fig.14 is a structural schematic diagram of a target detection device provided according to Embodiment 4 of the present application;

[0035] Fig.15 It is a structural schematic diagram of an electronic device provided according to Embodiment 5 of the present application. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0037] It should be noted that the terms "first", "second", "third", "fourth", "actual", "preset", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] Embodiment 1

[0039] Figure 1 This is a flow chart of a target detection method provided in the first embodiment of the present application. The embodiment of the present application can be applied to the case where a target object that is making a sound is detected. Typically, it can be applied to the case where multiple omnidirectional microphones are used to detect the target object that is speaking. The method can be executed by a target detection device, which can be implemented in the form of hardware and / or software, and the target detection device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0040] S110. For at least two sound collection devices that have collected sound, determine a sound source area corresponding to each sound collection device according to the sound intensity collected by the microphones in the sound collection devices.

[0041] Among them, at least two sound collection devices can be devices capable of sound collection, such as omnidirectional microphones, bidirectional microphones, directional microphones, etc. The sound collection device may include at least one microphone. Taking an omnidirectional microphone as an example, an omnidirectional microphone may include multiple microphones, such as Figure 2 As shown, it may include 6 microphones, microphone 1 to microphone 6. The largest circular boundary is the farthest range that the omnidirectional microphone can detect sound. Within the largest circular boundary, the fan-shaped area with the microphone as the center and α as the central angle is the sound collection sub-area of ​​the microphone. The sound collection sub-areas of each microphone together constitute the sound collection area of ​​the omnidirectional microphone.

[0042] Taking the omnidirectional microphone as an example, the layout of the conference system is as follows: Figure 3As shown, it includes a conference whiteboard and at least two omnidirectional microphones. The conference whiteboard and the omnidirectional microphones include wireless communication modules, and wireless communication can be performed between the conference whiteboard and the omnidirectional microphones, as well as between the omnidirectional microphones. The relative position relationship between the omnidirectional microphone and the conference whiteboard can be determined by the wireless communication module, and the position relationship between the conference whiteboard and the omnidirectional microphone can also be determined by the positioning module integrated in the omnidirectional microphone and the conference whiteboard. The position point of the conference whiteboard can be used as the origin to establish a coordinate system to determine the coordinates of each sound collection device in the coordinate system.

[0043] In the application scenario, there may be at least one object located at the sound collection device to make a sound. The object may be located in the sound collection area of ​​some sound collection devices but not in the sound collection area of ​​other sound collection devices. Therefore, only some sound collection devices may be able to collect the sound. The intensity of the sound collected by the sound collection devices may be different or the same. For the sound collection devices that collect the sound, the sound intensity collected by each microphone in the sound collection device is determined respectively, and the sound source area corresponding to the sound collection device is determined according to the sound intensity.

[0044] Specifically, Figure 2 As shown, due to the different orientations of the microphones in the sound collection device and the different sound collection sub-areas, when collecting the sound of a sound source, the sound intensity collected by each microphone may be different. The greater the sound intensity of the microphone, the greater the possibility that the microphone is facing the sound source, and the greater the possibility that the sound source is located in the sound collection sub-area of ​​the microphone. The sound source area can be located based on the sound intensity collected by the microphone and the corresponding sound collection sub-area. For example, the sound collection sub-area of ​​the microphone with the strongest sound intensity can be used as the sound source area to accurately determine the target object currently giving the main speech.

[0045] S120: Determine candidate objects associated with each sound collecting device that collects sound.

[0046] Exemplarily, after determining the sound collection device that collects the sound, the candidate object associated with the sound collection device can be determined. Specifically, if the sound collection device can collect the sound of the object, that is, the object is located in the sound collection area of ​​the sound collection device, then the object is determined to be a candidate object associated with the sound collection device. The sound collection device may or may not have associated candidate objects. Similarly, an object may or may not have an associated sound collection device.

[0047] In the embodiment of the present application, determining candidate objects associated with each sound collection device that collects sound includes:

[0048] Based on the association relationship between the sound collection devices and the objects, determining the candidate objects associated with each sound collection device that collects the sound;

[0049] The process of determining the association relationship between the sound collection device and the object includes:

[0050] Determine that the sound collection area includes at least two sound collection devices at the location of the object, and establish an association relationship between the at least two sound collection devices and the object.

[0051] Exemplarily, the sound collection area of ​​each sound collection device can be determined first, and the sound collection range of the sound collection device can be determined. Objects existing in the current scene are detected, and the position of the object is determined. According to the position of the object and the sound collection area, it is determined whether the object is located in the sound collection area. If it is, an association relationship between the object and the sound collection area is established. When establishing the association relationship, for each object, an association relationship between the object and at least two sound collection devices is established, so that when positioning is performed according to the sound intensity later, the object can be accurately positioned by the sound collected by at least two sound collection devices, avoiding errors when positioning is performed only by the sound collected by one sound collection device. Specifically, if an object is only located in the sound collection area of ​​one sound collection device, the sound collection area of ​​the sound collection device within the preset range of the object can be expanded according to actual conditions, so that after the sound collection area is expanded, the object can fall into the sound collection area of ​​at least two sound collection devices, thereby establishing an association relationship between the object and at least two sound collection devices.

[0052] For example, Figure 4 As shown, objects S1, omnidirectional microphone 1, and omnidirectional microphone 2 establish an association relationship, objects S2, omnidirectional microphone 4, and omnidirectional microphone 7 establish an association relationship, objects S3, omnidirectional microphone 2, and omnidirectional microphone 3 establish an association relationship, and objects S4, omnidirectional microphone 4, and omnidirectional microphone 5 establish an association relationship. If the sound collection devices that collect sound are omnidirectional microphone 2, omnidirectional microphone 3, omnidirectional microphone 5, and omnidirectional microphone 6, it can be determined that the candidate object associated with omnidirectional microphone 2 and omnidirectional microphone 3 is object S3, and the candidate object associated with omnidirectional microphone 5 and omnidirectional microphone 8 is object S4.

[0053] S130: Determine a target object from the candidate objects according to a relative relationship between the positions of the candidate objects and the sound source area.

[0054] Exemplarily, the sound source area can reflect the direction and position information of the sound source. According to the relative relationship between the position of the candidate object and the sound source area, the target object for the speaker can be determined from the candidate objects, thereby accurately identifying the current speaker, facilitating highlighting the speaker's information in the conference scene, enabling other participants to know the current speaker, and facilitating interaction and communication.

[0055] In the embodiment of the present application, determining the target object from the candidate objects according to the relative relationship between the position of the candidate object and the sound source area includes:

[0056] If there is a candidate object located in all sound source areas, then the candidate object is used as the target object; or;

[0057] An overlapping area of ​​all sound source areas is determined, and a candidate object located in the overlapping area is used as a target object.

[0058] For example, for each candidate object, it can be determined whether the candidate object is located in all sound source areas. If the candidate object is located in all sound source areas, the candidate object is used as the target object. It is also possible to determine the overlapping area of ​​all sound source areas, and determine whether the candidate object is located in the overlapping area based on the position of the candidate object. If so, the candidate object is determined as the target object. Figure 5 As shown in FIG. 1 , if the sound is collected by omnidirectional microphone 5 and omnidirectional microphone 8, the associated candidate object is S4. The sound source area corresponding to omnidirectional microphone 5 is the area corresponding to α1 emitted from omnidirectional microphone 5, and the sound source area corresponding to omnidirectional microphone 8 is the area corresponding to α2 emitted from omnidirectional microphone 8. Candidate object S4 is located in the sound source areas of omnidirectional microphone 5 and omnidirectional microphone 8, and candidate object S4 is taken as the target object. Figure 6 As shown in the figure, if the sound collected by omnidirectional microphone 5, omnidirectional microphone 8 and omnidirectional microphone 6 is omnidirectional microphone 5, the associated candidate object is S4. The sound source area corresponding to omnidirectional microphone 5 is the area corresponding to α1 emitted from omnidirectional microphone 5, the sound source area corresponding to omnidirectional microphone 8 is the area corresponding to α2 emitted from omnidirectional microphone 8, and the sound source area corresponding to omnidirectional microphone 6 is the area corresponding to β emitted from omnidirectional microphone 8. Candidate object S4 is located in the sound source areas of omnidirectional microphone 5, omnidirectional microphone 8 and omnidirectional microphone 6, and candidate object S4 is taken as the target object. Figure 7As shown, if the sound collected is from omnidirectional microphone 2, omnidirectional microphone 3, omnidirectional microphone 5 and omnidirectional microphone 8, the associated candidate objects are S3 and S4. The sound source area corresponding to omnidirectional microphone 5 is the area corresponding to α1 emitted from omnidirectional microphone 5, the sound source area corresponding to omnidirectional microphone 8 is the area corresponding to α2 emitted from omnidirectional microphone 8, the sound source area corresponding to omnidirectional microphone 2 is the area corresponding to α3 emitted from omnidirectional microphone 2, and the sound source area corresponding to omnidirectional microphone 3 is the area corresponding to α4 emitted from omnidirectional microphone 3. Candidate object S4 is in the sound source areas of omnidirectional microphone 2, omnidirectional microphone 3, omnidirectional microphone 5 and omnidirectional microphone 8, and candidate object S4 is determined to be the target object. Candidate object S3 is in the sound source areas of omnidirectional microphone 3 and omnidirectional microphone 8, but not in the sound source areas of omnidirectional microphone 2 and omnidirectional microphone 5, and candidate object S3 is determined not to be the target object. Alternatively, the overlapping area of ​​the sound source areas of the omnidirectional microphones 2, 3, 5, and 8 may be determined first, and only the candidate object S4 is located in the overlapping area, so the candidate object S4 is the target object. If there are at least two candidate objects that meet the requirements, for example, at least two candidate objects are located in all the sound source areas, or at least two candidate objects are located in the overlapping area of ​​the sound source areas, the range of the sound source areas may be adaptively narrowed so that there is only one candidate object that meets the requirements, thereby accurately determining the target object.

[0059] The technical solution of the embodiment of the present application is to determine the sound source area corresponding to each sound collection device according to the sound intensity collected by the microphone in the sound collection device for at least two sound collection devices that collect sound; determine the candidate objects associated with each sound collection device that collects sound; and determine the target object from the candidate objects according to the relative relationship between the position of the candidate object and the sound source area. The above solution can accurately determine the sound source area according to the sound intensity collected by the microphone in the sound collection device that collects sound, and then accurately locate the target object that is currently making sound, and accurately determine the specific position of the target object.

[0060] Embodiment 2

[0061] Figure 8 This is a flow chart of a target detection method provided in Example 2 of the present application. The present application example is optimized based on the above example. For solutions not described in detail in the present application example, see the above example. Figure 8 As shown, the method of the embodiment of the present application specifically includes the following steps:

[0062] S210: For at least two sound collection devices that collect sound, determine a sound collection sub-area corresponding to a microphone in the sound collection device.

[0063] Exemplarily, there may be at least two microphones in the sound collection device for collecting sound signals from different directions. At least two microphones have their sound collection directions and collection ranges, and the corresponding sound collection sub-areas can be determined for each microphone in the sound collection device to locate the sound source.

[0064] S220. Determine the sound source area corresponding to each sound collection device according to the sound intensity collected by the microphone and the sound collection sub-area; wherein the range of the sound source area is negatively correlated with the sound intensity collected by the sound collection device.

[0065] For example, if the microphone is facing the sound source, the intensity of the sound collected is the strongest, and if the microphone is facing away from the sound source, the intensity of the sound collected is the weakest. Similarly, if the sound collection direction of the microphone is closer to the direction facing the sound source, the intensity of the sound collected is stronger, and if the sound collection direction of the microphone is farther away from the direction facing the sound source, the intensity of the sound collected is weaker. The sound source area corresponding to the sound collection device can be located according to the sound intensity collected by the microphone and the sound collection sub-area to reflect the position of the sound source.

[0066] In the embodiment of the present application, the stronger the sound intensity, the closer the sound collection direction of the sound collection device is to the direction facing the sound source, and the more accurately the sound source area located by the sound collection device is. The range of the sound source area can be set to be negatively correlated with the sound intensity collected by the sound collection device, that is, the greater the sound intensity collected by the sound collection device, the smaller the range of the sound source area determined by the sound collection device. Thus, the approximate range of the sound source is determined by the larger sound source area corresponding to the sound collection device with weaker sound intensity, and the position of the sound source is accurately located by the smaller sound source area corresponding to the sound collection device with stronger sound intensity, thereby improving the accuracy of the target object determination. For example, Fig. 9 As shown, assuming that both the omnidirectional microphone 5 and the omnidirectional microphone 8 collect sound, and the sound intensity collected by the omnidirectional microphone 5 is greater than the sound intensity collected by the omnidirectional microphone 8, the central angle α1 of the sound source area corresponding to the omnidirectional microphone 5 can be set to be smaller than the central angle α2 of the sound source area corresponding to the omnidirectional microphone 8, so that the range of the sound source area corresponding to the omnidirectional microphone 5 is smaller than the range of the sound source area corresponding to the omnidirectional microphone 8, thereby accurately locating the sound source position by appropriately narrowing the range of the sound source area.

[0067] In the embodiment of the present application, the sound source area corresponding to each sound collection device is determined according to the sound intensity and the sound collection sub-area collected by the microphone, including:

[0068] Determine a first microphone that collects the strongest sound intensity and a second microphone that collects the second strongest sound intensity;

[0069] The sound source area is determined according to the relative positions of the first microphone and the second microphone, and a target sound collection sub-area; wherein the target sound collection sub-area is the sound collection sub-area corresponding to the first microphone.

[0070] Exemplarily, the stronger the sound intensity collected by each microphone in the sound collection device, the greater the possibility that the sound collection direction of the microphone is toward the sound source. Therefore, from the microphones of the sound collection device, the first microphone with the strongest sound intensity and the second microphone with the second strongest sound intensity can be selected. According to the relative positions of the first microphone and the second microphone, and the target sound collection sub-area corresponding to the first microphone, the sound source area corresponding to the sound collection device is determined to accurately determine the area where the sound source is located.

[0071] Specifically, a partial area close to the sound collection sub-area corresponding to the second microphone can be selected from the sound collection sub-area corresponding to the first microphone as the target sound collection sub-area. The sound collection sub-area corresponding to the first microphone and the sound collection sub-area corresponding to the second microphone can also be combined to obtain an area as the target sound collection sub-area, or a partial area can be selected from the combined area as the target sound collection sub-area. The specific determination can be made according to actual conditions.

[0072] In the embodiment of the present application, the target sound collection sub-area and the sound source area are fan-shaped areas;

[0073] Determining a sound source area according to a relative position of the first microphone and the second microphone and a target sound collection sub-area includes:

[0074] If the preset angle is less than or equal to the central angle of the target sound collection sub-area, a region in the target sound collection sub-area with a radius close to the second microphone as the radius of the sound source region and a central angle of the preset angle as the sound source region;

[0075] If the preset angle is greater than the central angle of the sound collection sub-area, the radius of the target sound collection sub-area away from the second microphone is used as the radius of the sound source area, and the area with the preset angle as the central angle and including the target sound collection sub-area is used as the sound source area.

[0076] For example, a preset angle may be determined for each sound collection device. If the preset angle is less than or equal to the center angle of the target sound collection sub-area, a sound source area is selected from the inside of the target sound collection sub-area. If the preset angle is greater than the center angle of the target sound collection sub-area, the target sound collection sub-area is expanded toward the second microphone to obtain a sound source area. Fig.10As shown, θ is a preset angle. If θ is smaller than the center angle α of the sound collection sub-area of ​​microphone 1, then from the target sound collection sub-area, that is, the fan-shaped area corresponding to the center angle α, a radius close to the second microphone (i.e. Fig.10 The radius of the right boundary of the sector area corresponding to the central angle α is the radius of the sound source area, and the area with the preset angle as the central angle is the sound source area, that is, Fig.10 The fan-shaped area corresponding to the central angle θ. Fig.11 As shown, if θ is greater than the central angle α of the sound collection sub-area of ​​microphone 1, the radius of the target sound collection sub-area away from the second microphone is used as the radius of the sound source area, and the area with θ as the central angle and including the target sound collection sub-area is used as the sound source area, that is, Fig.11 The fan-shaped area corresponding to the central angle θ of the circle. If θ is equal to the central angle α of the sound collection sub-area of ​​microphone 1, the sound collection sub-area of ​​microphone 1 is used as the target sound collection sub-area. The above scheme can accurately determine the sound source area based on the microphones with the strongest and second strongest sound intensities to determine the location of the sound source. In addition, the sound source area is determined by the relative position of the microphone with the second strongest sound intensity and the microphone with the strongest sound intensity. The direction of the sound source can be further accurately determined based on the sound collection sub-area corresponding to the microphone with the strongest sound intensity, so as to reduce the error of the single microphone intensity judgment.

[0077] In an embodiment of the present application, the preset angle can be determined according to actual conditions. For example, since it has been mentioned in the above embodiment that the range of the sound source area is negatively correlated with the sound intensity collected by the sound collection device, the preset angle can be negatively correlated with the sound intensity collected by the sound collection device. The specific value is not limited and can be 30 degrees, 45 degrees, 60 degrees, 90 degrees, 120 degrees, etc.

[0078] In the embodiment of the present application, the method for determining the preset angle includes:

[0079] Determine a difference between the intensity of the sound collected by the first microphone and the intensity of the sound collected by the second microphone;

[0080] Based on a positive correlation or an increasing step function relationship, a preset angle is determined according to the difference.

[0081] Exemplarily, for each sound collection device, a positive correlation or an increasing step function relationship between the difference and the preset angle can be established in advance. For example, a positive correlation relationship y=a*x+b can be established, where y is the preset angle, x is the difference, and a and b are constants. An increasing step function relationship can be established: Among them, c < d < e, and TH1 < TH2 < TH3. The actual values of c, d, e, TH1, TH2, and TH3 can be determined according to the actual situation. It should be noted that the above step function formula is only an example. The value range of x can also be divided into two intervals, including two stages, or the value range of x can be divided into more intervals, including more stages, which is not limited here.

[0082] Specifically, if there are three sound collection devices that collect sound, for the sound collection device with the strongest collected sound intensity, if the difference between the sound intensity collected by the first microphone and the sound intensity collected by the second microphone is greater than or equal to 0 and less than 5 decibels, the preset angle is determined to be 15 degrees; if the difference is greater than or equal to 5 decibels and less than 10 decibels, the preset angle is determined to be 30 degrees; if the difference is greater than or equal to 10 decibels and less than 15 decibels, the preset angle is determined to be 45 degrees. For the sound collection device with the second strongest collected sound intensity, if the difference between the sound intensity collected by the first microphone and the sound intensity collected by the second microphone is greater than or equal to 0 and less than 5 decibels, the preset angle is determined to be 60 degrees; if the difference is greater than or equal to 5 decibels and less than 10 decibels, the preset angle is determined to be 75 degrees; if the difference is greater than or equal to 10 decibels and less than 15 decibels, the preset angle is determined to be 90 degrees. For the sound collection device with the weakest collected sound intensity, if the difference between the sound intensity collected by the first microphone and the sound intensity collected by the second microphone is greater than or equal to 0 and less than 5 decibels, the preset angle is determined to be 100 degrees; if the difference is greater than or equal to 5 decibels and less than 10 decibels, the preset angle is determined to be 105 degrees; if the difference is greater than or equal to 10 decibels and less than 15 decibels, the preset angle is determined to be 120 degrees.

[0083] S230. Determine candidate objects associated with each sound collection device that collects sound.

[0084] S240. Determine a target object from the candidate objects according to the relative relationship between the positions of the candidate objects and the sound source area.

[0085] The embodiment of the present application provides a target detection method, which determines a sound collection sub-region corresponding to a microphone in a sound collection device; determines a sound source area corresponding to each sound collection device according to the sound intensity collected by the microphone and the sound collection sub-region, wherein the range of the sound source area is negatively correlated with the sound intensity collected by the sound collection device. The above solution can determine sound source areas with different ranges for sound collection devices that collect different sound intensities, so as to determine a large-range approximate sound source area by a sound collection device that collects weak sound intensity, and determine a small-range precise sound source area by a sound collection device that collects strong sound intensity. The combination of each sound source area accurately determines the position of the sound source, and then determines the current sounding target object in combination with the position of the candidate object.

[0086] Embodiment 3

[0087] Fig.12 This is a flow chart of a target detection method provided in Example 3 of the present application. The present application example is optimized based on the above example. For solutions not described in detail in the present application example, see the above example. Fig.12 As shown, the method of the embodiment of the present application specifically includes the following steps:

[0088] S310: For at least two sound collection devices that have collected sound, determine the sound source area corresponding to each sound collection device according to the sound intensity collected by the microphones in the sound collection devices.

[0089] S320: If candidate objects associated with each sound collecting device that collects sound are determined.

[0090] S330: Determine a target object from the candidate objects according to a relative relationship between the positions of the candidate objects and the sound source area.

[0091] S340. Determine, according to the position of the target object, an image collector whose field of view includes the target object.

[0092] Exemplarily, after the position of the target object is determined, the field of view range of each image collector can be determined for the image collectors in the current scene, and the image collector whose field of view includes the target object is selected.

[0093] S350: Acquire image information of the target object through the image collector, and display the image information in a preset area of ​​a display device.

[0094] Among them, the preset area can be determined according to actual conditions, and can be a fixed position, such as the upper left corner, lower right corner, etc. It can also be an adaptively movable position, that is, moving the display to an area where there is no content on the display device. If the current preset area needs to display other content, the preset area is adaptively moved to a blank area that does not display other content to avoid blocking the displayed content. Exemplarily, the image information of the target object can be obtained in a targeted manner through an image collector, or the image information of the target object can be intercepted from the overall image collected by the image collector, and the image information can be displayed in a preset area of ​​the display device, so as to highlight the target object of the speaker in the current scene and facilitate interactive communication. For example, Fig.13 As shown, assuming that the determined target object is S4, the image information of S4 can be obtained through the image collector and displayed in the upper left corner of the conference whiteboard. When the determined target object changes, the object displayed in the preset area changes adaptively.

[0095] The embodiment of the present application provides a target detection method, which determines an image collector whose field of view includes the target object according to the position of the target object; obtains image information of the target object through the image collector, and displays the image information in a preset area of ​​a display device. The above scheme can highlight the image information of the target object of the speaker in the preset area of ​​the display device, so that other objects in the scene can clearly know the target object of the current speaker, which is convenient for interactive communication and understanding of the sound content.

[0096] Embodiment 4

[0097] Fig.14 This is a schematic diagram of the structure of a target detection device provided in the fourth embodiment of the present application. The device can execute the target detection method provided in any embodiment of the present application and has the corresponding functional modules and beneficial effects of the execution method. Fig.14 As shown, the device comprises:

[0098] A sound source region determination module 410 is used to determine the sound source region corresponding to each sound collection device according to the sound intensity collected by the microphone in the sound collection device for at least two sound collection devices that collect sound;

[0099] A candidate object determination module 420, for determining candidate objects associated with each sound collection device that collects sound;

[0100] The target object determination module 430 is used to determine the target object from the candidate objects according to the relative relationship between the positions of the candidate objects and the sound source area.

[0101] In the embodiment of the present application, the sound source area determination module 410 includes:

[0102] A sound collection sub-area determination unit, used to determine the sound collection sub-area corresponding to the microphone in the sound collection device;

[0103] The area determination unit is used to determine the sound source area corresponding to each sound collection device according to the sound intensity collected by the microphone and the sound collection sub-area; wherein the range of the sound source area is negatively correlated with the sound intensity collected by the sound collection device.

[0104] In the embodiment of the present application, the region determination unit is specifically used to:

[0105] Determine a first microphone that collects the strongest sound intensity and a second microphone that collects the second strongest sound intensity;

[0106] The sound source area is determined according to the relative positions of the first microphone and the second microphone, and a target sound collection sub-area; wherein the target sound collection sub-area is the sound collection sub-area corresponding to the first microphone.

[0107] In the embodiment of the present application, the target sound collection sub-area and the sound source area are fan-shaped areas;

[0108] The area determination unit is specifically used for:

[0109] If the preset angle is less than or equal to the central angle of the target sound collection sub-area, a region in the target sound collection sub-area with a radius close to the second microphone as the radius of the sound source region and a central angle of the preset angle as the sound source region;

[0110] If the preset angle is greater than the central angle of the sound collection sub-area, the radius of the target sound collection sub-area away from the second microphone is used as the radius of the sound source area, and the area with the preset angle as the central angle and including the target sound collection sub-area is used as the sound source area.

[0111] In the embodiment of the present application, the preset angle is negatively correlated with the sound intensity collected by the sound collection device.

[0112] In the embodiment of the present application, the device further includes:

[0113] A difference determination module, used to determine the difference between the intensity of the sound collected by the first microphone and the intensity of the sound collected by the second microphone;

[0114] The preset angle determination module is used to determine the preset angle according to the difference based on a positive correlation relationship or an increasing step function relationship.

[0115] In the embodiment of the present application, the target object determination module 430 includes:

[0116] A first judgment unit is used to take a candidate object as a target object if there is a candidate object located in all sound source areas; or;

[0117] The second judgment unit is used to determine an overlapping area of ​​all sound source areas, and take the candidate object located in the overlapping area as the target object.

[0118] In the embodiment of the present application, the device further includes:

[0119] An image collector determining module, used for determining an image collector whose field of view includes the target object according to the position of the target object;

[0120] The display module is used to obtain the image information of the target object through the image collector and display the image information in a preset area of ​​the display device.

[0121] In the embodiment of the present application, the candidate object determination module 420 includes:

[0122] an association determination unit, configured to determine candidate objects associated with each sound collection device that has collected sound, based on the association relationship between the sound collection device and the object;

[0123] The device also includes:

[0124] The association relationship establishing module is used to determine that the sound collection area includes at least two sound collection devices at the position of the object, and establish an association relationship between the at least two sound collection devices and the object.

[0125] The technical solution of the embodiment of the present application includes a sound source area determination module, which is used to determine the sound source area corresponding to each sound collection device according to the sound intensity collected by the microphone in the sound collection device for at least two sound collection devices that collect sound; a candidate object determination module, which is used to determine the candidate objects associated with each sound collection device that collects sound; and a target object determination module, which is used to determine the target object from the candidate objects according to the relative relationship between the position of the candidate object and the sound source area. The above solution can accurately determine the sound source area according to the sound intensity collected by the microphone in the sound collection device that collects sound, thereby accurately locating the target object that is currently making sound and accurately determining the specific position of the target object.

[0126] Embodiment 5

[0127] Fig.15 A schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0128] like Fig.15As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0129] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0130] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a target detection method.

[0131] In some embodiments, the target detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the target detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the target detection method in any other appropriate manner (e.g., by means of firmware).

[0132] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0133] The computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable target detection device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0134] In the context of the present application, a computer readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device or equipment. A computer readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer readable storage medium may be a machine readable signal medium. A more specific example of a machine readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0135] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0136] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0137] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0138] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the information expected by the technical solution of this application can be achieved, and this document is not limited here.

[0139] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A target detection method, characterized in that: The method comprises: For at least two sound collection devices that collect sound, determine the sound source area corresponding to each sound collection device according to the sound intensity collected by the microphone in the sound collection device; Determining candidate objects associated with each sound collection device that collects the sound; According to the relative relationship between the positions of the candidate objects and the sound source area, a target object is determined from the candidate objects.

2. The method according to claim 1, characterized in that According to the sound intensity collected by the microphone in the sound collection device, the sound source area corresponding to each sound collection device is determined, including: Determine the sound collection sub-area corresponding to the microphone in the sound collection device; The sound source area corresponding to each sound collection device is determined according to the sound intensity collected by the microphone and the sound collection sub-area; wherein the range of the sound source area is negatively correlated with the sound intensity collected by the sound collection device.

3. The method according to claim 2, characterized in that According to the sound intensity collected by the microphone and the sound collection sub-area, the sound source area corresponding to each sound collection device is determined, including: Determine a first microphone that collects the strongest sound intensity and a second microphone that collects the second strongest sound intensity; The sound source area is determined according to the relative positions of the first microphone and the second microphone, and a target sound collection sub-area; wherein the target sound collection sub-area is the sound collection sub-area corresponding to the first microphone.

4. The method according to claim 3, characterized in that The target sound collection sub-area and the sound source area are fan-shaped areas; Determining a sound source area according to a relative position of the first microphone and the second microphone and a target sound collection sub-area includes: If the preset angle is less than or equal to the central angle of the target sound collection sub-area, a region in the target sound collection sub-area with a radius close to the second microphone as the radius of the sound source region and a central angle of the preset angle as the sound source region; If the preset angle is greater than the central angle of the sound collection sub-area, the radius of the target sound collection sub-area away from the second microphone is used as the radius of the sound source area, and the area with the preset angle as the central angle and including the target sound collection sub-area is used as the sound source area.

5. The method according to claim 4, characterized in that The preset angle is negatively correlated with the intensity of the sound collected by the sound collection device.

6. The method according to claim 4 or 5, characterized in that: The method for determining the preset angle includes: Determine a difference between the intensity of the sound collected by the first microphone and the intensity of the sound collected by the second microphone; Based on a positive correlation or an increasing step function relationship, a preset angle is determined according to the difference.

7. The method according to claim 1, characterized in that Determining a target object from the candidate objects according to a relative relationship between a position of the candidate object and the sound source area includes: If there is a candidate object located in all sound source areas, then the candidate object is used as the target object; or; An overlapping area of ​​all sound source areas is determined, and a candidate object located in the overlapping area is used as a target object.

8. The method according to claim 1, characterized in that: The method further comprises: Determine, according to the position of the target object, an image collector whose field of view includes the target object; The image information of the target object is acquired through the image collector, and the image information is displayed in a preset area of ​​a display device.

9. The method according to claim 1, characterized in that: Determine candidate objects associated with each sound collection device that collects sound, including: Based on the association relationship between the sound collection devices and the objects, determining the candidate objects associated with each sound collection device that collects the sound; The process of determining the association relationship between the sound collection device and the object includes: Determine that the sound collection area includes at least two sound collection devices at the location of the object, and establish an association relationship between the at least two sound collection devices and the object.

10. A target detection device, characterized in that: The device comprises: A sound source region determination module, for determining the sound source region corresponding to each sound collection device according to the sound intensity collected by the microphone in the sound collection device for at least two sound collection devices that collect sound; A candidate object determination module, used to determine candidate objects associated with each sound collection device that collects sound; The target object determination module is used to determine the target object from the candidate objects according to the relative relationship between the positions of the candidate objects and the sound source area.

11. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the target detection method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the target detection method according to any one of claims 1 to 9 when executed.

Images