Method and apparatus for determining microphone pickup

Abstract

This application discloses a method and apparatus for determining a microphone, relating to the field of sound signal processing. It helps improve the flexibility of stereo scene setup and, more importantly, improves the accuracy of microphone determination, resulting in better stereo effects. This method can be applied to conference systems. The method includes: first, determining the camera's shooting direction information, which indicates the camera's shooting direction; then, based on the camera's shooting direction, determining a first reference direction at a first preset angle to the shooting direction. The first reference direction is the reference direction of a target microphone unit in the array microphones, where the target microphone unit is either a left-channel microphone unit or a right-channel microphone unit; the left-channel microphone unit is used to pick up left-channel sound signals, and the right-channel microphone unit is used to pick up right-channel sound signals. Next, based on the first reference direction, the target microphone unit is determined from the array microphones.

Description

Technical Field

[0001] This application relates to the field of sound signal processing, and in particular to a method and apparatus for determining a microphone. Background Technology

[0002] Currently, in conference systems, when array microphones are used for stereo sound pickup, the generated stereo direction needs to correspond to the direction of the image captured by the cameras in the conference system. For example, when a speaker speaks on the left side of the room, the image of the speaker captured by the camera will be displayed on the left side of the screen, and the speaker's voice will also originate from the left side of the room. To ensure that the image and sound direction of the speaker correspond during the conference, the camera needs to be placed directly facing the array microphones during installation. Furthermore, one or more specific microphone units in the array microphones should be positioned on the left side of the room, and another one or more specific microphone units should be positioned on the right side. The microphone units on the left and right sides of the room will pick up the sound signals from the left and right sides respectively. Subsequently, the array microphones can send the picked-up sound signals from the left and right sides of the room to the main control conference terminal in the conference system, which will then generate a stereo signal based on the sound signals from the two directions.

[0003] However, the existing method of determining the individual microphone units requires precise placement of the camera and array microphones (i.e., the camera facing the array microphones). This makes the placement method inconvenient and prone to errors. Even if the array microphones are correctly positioned for stereo sound (i.e., the camera facing the array microphones), movement of the array microphones during use can lead to incorrect sound pickup orientation, resulting in an inability to reproduce the stereo effect corresponding to the camera's captured image. Summary of the Invention

[0004] This application provides a method and apparatus for determining a microphone, which helps to improve the flexibility of stereo scene setup and improves the accuracy of determining the microphone, thereby resulting in better stereo effects.

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] A first aspect provides a method for determining a microphone, comprising: determining camera shooting direction information, wherein the shooting direction information is used to indicate the camera's shooting direction; determining a first reference direction based on the camera's shooting direction, wherein the first reference direction is the reference direction of a target microphone unit in an array microphone, and the target microphone unit is a left channel microphone unit or a right channel microphone unit; the shooting direction forms a first preset angle with the first reference direction; the left channel microphone unit is used to pick up left channel sound signals, and the right channel microphone unit is used to pick up right channel sound signals; and determining the target microphone unit from the array microphone based on the first reference direction.

[0007] Compared to existing technologies, this solution provides a method for dynamically determining the microphone, which is not limited by the placement of the camera or the array microphones. Therefore, it offers greater flexibility in stereo scene setup. Furthermore, even if the array microphones move during use, this method allows for dynamic re-determination of the microphones, improving accuracy and resulting in better stereo sound when applied to stereo environments.

[0008] In one possible design, the camera's shooting direction is the same as when it was in its initial position. This could mean the camera's shooting direction remains constant in its initial position, or it returns to its initial shooting direction. The initial position refers to the position where the camera lens is perpendicular to the camera's base. This technical solution is proposed considering that "in conference scenarios, it is usually required that the camera's shooting direction be the same as when it was in its initial position." The detailed implementation section below is also based on this example. In practice, if the camera is not in its initial position, the camera can be brought back to that position by adjusting the camera lens direction. Specific adjustment methods can be found in existing technologies and will not be elaborated here.

[0009] In one possible design, the first preset angle is 90°.

[0010] For example, based on Figure 9 As shown on the left and right sides of the venue, when the target microphone is a left channel microphone, the first reference direction is the direction after rotating the camera's shooting direction 90° counterclockwise; when the target microphone is a right channel microphone, the first reference direction is the direction after rotating the camera's shooting direction 90° clockwise.

[0011] For example, based on and Figure 9 The definitions of the left and right sides of the meeting room shown are opposite, that is... Figure 9 The left side of the meeting room in the diagram is actually defined as the right side of the meeting room. Figure 9 In practice, the right side of the venue is defined as the left side of the venue. When the target microphone is a left channel microphone, the first reference direction is the direction after rotating the camera's shooting direction 90° clockwise; when the target microphone is a right channel microphone, the first reference direction is the direction after rotating the camera's shooting direction 90° counterclockwise.

[0012] In one possible design, determining a target microphone unit from the array microphones based on a first reference direction includes: selecting microphone units in the array microphones that satisfy a first preset condition as target microphone units, based on the first reference direction. The microphone units satisfying the first preset condition include: a first preset number of microphone units whose angle between their direction and the first reference direction is less than or equal to a first threshold; or, the first first preset number of microphone units corresponding to the angles between their direction and the first reference direction, sorted from smallest to largest; or, the last first preset number of microphone units corresponding to the angles between their direction and the first reference direction, sorted from largest to smallest.

[0013] Therefore, using the microphone unit with the smaller angle between its direction and the first reference direction from at least two microphone units in the array microphone as the target microphone unit helps to improve the sound pickup effect of the target microphone unit, thereby helping to improve the effect of the synthesized stereo signal.

[0014] In one possible design, the method further includes: receiving magnetic field orientation information of any microphone unit in the array microphone; determining the orientation information of the microphone unit based on the magnetic field orientation information of the microphone unit; and determining the angle of the orientation of the microphone unit relative to a first reference direction based on the orientation information of the microphone unit. Alternatively, receiving the orientation information of any microphone unit in the array microphone; and determining the angle of the orientation of the microphone unit relative to the first reference direction based on the orientation information of the microphone unit. Alternatively, receiving the angle of the orientation of any microphone unit in the array microphone, as transmitted by the array microphone, relative to the first reference direction.

[0015] This possible design provides a specific implementation of determining the orientation of any microphone unit in the array microphone relative to a first reference direction. The actual implementation is not limited to this, and can be found in the following detailed implementation section.

[0016] In one possible design, the array microphone includes a first magnetic sensor for measuring the magnetic field orientation information of any microphone unit. Since magnetic sensors offer high accuracy in measuring magnetic field direction, using a magnetic sensor to measure the magnetic field orientation information and then converting this information into the orientation information of the microphone unit helps improve the accuracy of determining the orientation of the microphone unit, thereby improving the accuracy of identifying the individual microphone unit.

[0017] In one possible design, the array microphone includes a first infrared sensor for measuring the orientation information of any microphone unit. Since the infrared sensor can directly measure the orientation information of the microphone unit, compared to a magnetic sensor, it eliminates the need to convert magnetic field orientation information into orientation information, thus saving processing complexity and shortening the time required to determine the angle of the microphone unit's orientation relative to a first reference direction, thereby reducing the time required to determine the microphone unit.

[0018] In addition, compared to setting the magnetic sensor (or infrared sensor) and the array microphone separately, setting the magnetic sensor (or infrared sensor) and the array microphone together can determine the angle of any microphone unit in the array microphone relative to the first reference direction without adding external devices, which is convenient and simple to implement.

[0019] In one possible design, determining the camera's shooting direction information includes: acquiring the camera's magnetic field orientation information, which indicates the camera's magnetic field orientation; and determining the camera's shooting direction information based on the camera's magnetic field orientation information. Alternatively, receiving the camera's shooting direction information. This possible design provides a specific implementation method for determining the camera's shooting direction information, but actual implementations are not limited to this.

[0020] In one possible design, the camera includes a second magnetic sensor for measuring the camera's magnetic field orientation. This helps improve the accuracy of determining the camera's shooting direction.

[0021] In one possible design, the camera includes a second infrared sensor for measuring the camera's shooting direction information. This eliminates the need for converting magnetic field orientation information into direction information compared to a magnetic sensor, thus reducing processing complexity and shortening the time required to determine the camera's shooting direction, which in turn reduces the time required to determine the individual microphone units.

[0022] In one possible design, determining the camera's shooting direction information includes: acquiring sound signals collected by at least two microphone units (e.g., at least three microphone units) in an array microphone, the sound signals originating from the same sound source whose sound source direction is consistent with (or substantially consistent with) the camera's shooting direction; and using a sound source localization algorithm based on the sound signals collected by the at least two microphone units to determine the camera's shooting direction information.

[0023] In one possible design, the camera includes a sound source. Based on this approach, the sound source is fixed within the camera during manufacturing, ensuring that the sound source's direction aligns (or is substantially aligned) with the camera's shooting direction. This eliminates the need for manual placement of the camera and sound source during installation, reducing setup requirements and improving usability. Furthermore, compared to errors caused by assuming the camera and sound source positions are correctly positioned, this technical solution improves the accuracy of the determined camera shooting direction information, thereby increasing the accuracy of microphone selection.

[0024] In one possible design, after determining the target microphone unit from the array microphones based on a first reference direction, the method further includes synthesizing a stereo signal based on the left and right channel audio signals. This possible design provides one application scenario for determining the microphone unit, but its implementation is not limited to this.

[0025] In one possible design, the method further includes: determining a second reference direction based on the camera's shooting direction; the second reference direction is the reference direction of a mute microphone in the array microphone; the shooting direction forms a second preset angle with the second reference direction; and, based on the second reference direction, designating microphone units in the array microphone that meet the second preset condition as mute microphone units. The mute microphone units do not pick up sound. Microphone units meeting the second preset condition include: a second preset number of microphone units whose angle between their direction and the second reference direction is less than or equal to a second threshold; or, the microphone units corresponding to the first second preset number of angles between their direction and the second reference direction, sorted from smallest to largest; or the microphone units corresponding to the last second preset number of angles between their direction and the second reference direction, sorted from largest to smallest.

[0026] In one possible design, the second preset angle is 180 degrees.

[0027] Secondly, a microphone identification device is provided.

[0028] In one possible design, the microphone determining device is used to execute the microphone determining method provided in the first aspect above. This application can divide the microphone determining device into functional modules based on the method provided in the first aspect above. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. For example, this application can divide the microphone determining device into a first determining unit, a second determining unit, and a third determining unit, etc., according to function. The descriptions of the possible technical solutions and beneficial effects performed by the above-described functional modules can be found in the technical solutions provided in the first aspect above or its corresponding possible designs, and will not be repeated here.

[0029] In another possible design, the microphone determining device includes a memory and one or more processors coupled together. The memory stores computer instructions, and the processor invokes these instructions to perform any of the methods provided by the first aspect and any of its possible design embodiments.

[0030] Thirdly, this application provides a computer-readable storage medium, such as a non-transient computer-readable storage medium. A computer program (or instructions) is stored thereon, which, when executed on a microphone-picking device, causes the microphone-picking device to perform any method provided by any possible implementation of the first aspect described above.

[0031] Fourthly, this application provides a computer program product that, when run on a computer, causes any method provided by any possible implementation of the first aspect to be executed.

[0032] Fifthly, this application provides a chip system comprising: a processor, the processor being configured to retrieve and execute a computer program stored in a memory, performing any of the methods provided in the implementation of the first aspect.

[0033] Sixthly, this application provides a conference system, including: a camera, an array microphone, and a main control conference terminal. The camera is used to acquire video signals. The array microphone is used to pick up audio signals and convert them into electrical signals. The main control conference terminal is used to receive the video signals and audio signals transmitted by the camera and the array microphone, respectively.

[0034] It is understood that any of the microphone pickup device, computer storage medium, computer program product or conference system provided above can be applied to the corresponding method provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding method, and will not be repeated here.

[0035] In this application, the name of the aforementioned microphone detection device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this application, it falls within the scope of the claims of this application and its equivalents.

[0036] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0037] Figure 1 One of the structural schematic diagrams of the conference system provided in the embodiments of this application;

[0038] Figure 2 A second schematic diagram of the structure of the conference system provided in this application embodiment;

[0039] Figure 3 The third schematic diagram of the conference system provided in the embodiments of this application;

[0040] Figure 4 This application provides a schematic diagram illustrating a scenario application of a conference system.

[0041] Figure 5 This application provides a schematic diagram of the structure of a master control conference terminal.

[0042] Figure 6 This application provides a flowchart illustrating a method for determining a microphone.

[0043] Figure 7 A schematic diagram of the shooting direction of a camera provided in an embodiment of this application;

[0044] Figure 8 A schematic diagram of magnetic field orientation provided for an embodiment of this application;

[0045] Figure 9 A schematic diagram showing the shooting direction of a camera and the reference directions of the left and right channel microphone units, provided for an embodiment of this application;

[0046] Figure 10 A schematic diagram showing the orientation of any microphone unit in the array microphone provided in this application embodiment;

[0047] Figure 11 This is a schematic diagram of the structure of a microphone detection device provided in an embodiment of this application;

[0048] Figure 12 This is a schematic diagram of the structure of a chip system provided in an embodiment of this application;

[0049] Figure 13 A conceptual partial view of a computer program product provided for an embodiment of this application. Detailed Implementation

[0050] The following describes some of the terms used in this application.

[0051] 1) Array microphone

[0052] An array microphone is a system device consisting of a certain number of acoustic sensors (usually individual microphone units) used to sample and process the spatial characteristics of a sound field.

[0053] 2) Microphone unit

[0054] A microphone unit is an energy conversion device that converts sound signals into electrical signals. Microphones can be classified into moving-coil, condenser, electret, and the recently emerging silicon micro-microphones, as well as liquid microphones and laser microphones.

[0055] 3) Magnetic field orientation

[0056] The direction of the magnetic field, or magnetic azimuth, is defined as the direction of the magnetic force experienced by the north pole of a small compass needle at a point in a magnetic field. Magnetic azimuth is the direction from the north pole to the south pole. Inside a magnet, it is from the south pole to the north pole; externally, it can be represented by the tangent direction of the magnetic field lines or the direction the north pole of a small compass needle points when at rest.

[0057] 4) Magnetic sensor

[0058] A magnetic sensor is a device that converts changes in the magnetic properties of a sensitive element caused by external factors such as magnetic field, current, stress, strain, temperature or light into electrical signals, thereby detecting the corresponding physical quantity.

[0059] 5) Sound source

[0060] A sound source is a device that can produce sound, such as a loudspeaker, a piezoelectric element, or a vibrating motor.

[0061] 6) Infrared sensor

[0062] Infrared sensors are sensors that use infrared light for data processing. They have advantages such as high sensitivity and can control the operation of drive devices.

[0063] 7) Other terms

[0064] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0065] In the embodiments of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0066] In the description of this application, unless otherwise stated, "a plurality of" means two or more. The term "at least one" in this application means one or more, and the term "a plurality of" in this application means two or more.

[0067] It should be understood that the terminology used in the description of the various examples herein is for the purpose of describing particular examples only and is not intended to be limiting. As used in the description of the various examples and the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0068] It should also be understood that the term "and / or" as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. The term "and / or" describes an association between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this application generally indicates that the preceding and following related objects are in an "or" relationship.

[0069] It should also be understood that, in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0070] It should be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.

[0071] It should also be understood that the term “comprising” (also referred to as “includes”, “including”, “comprises” and / or “comprising”) as used in this specification specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0072] It should also be understood that the term "if" can be interpreted as meaning "when" or "upon" or "in response to determination" or "in response to detection." Similarly, depending on the context, the phrases "if determination..." or "if detection [the stated condition or event]" can be interpreted as meaning "when determination..." or "in response to determination..." or "when detection [the stated condition or event]" or "in response to detection [the stated condition or event]."

[0073] It should be understood that the phrases "an embodiment," "an embodiment," and "a possible implementation" used throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment or implementation is included in at least one embodiment of this application. Therefore, the phrases "in an embodiment," "an embodiment," or "a possible implementation" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0074] Figure 1 This is a schematic diagram of the structure of a conference system 10 provided in an embodiment of this application. The conference system 10 can be a conference room system, a desktop conference system, a video conferencing system, a computer conferencing system, or a video conferencing system, etc. Of course, this embodiment of the application does not specifically limit the actual form of the conference system 10. Figure 1 As shown, the conference system 10 includes a camera 11, an array microphone 12, and a main control conference terminal 13. Optionally, the conference system 10 may also include a display screen 14 and speakers 15. The speakers 15 can be replaced with other sound playback devices / devices, and this embodiment does not limit their use.

[0075] Camera 11 is used to acquire video signals and encode them to obtain a bitstream. This bitstream is then sent to display screen 14 and / or main control conference terminal 13.

[0076] An array microphone 12 is used to pick up audio signals and convert them into electrical signals. The converted electrical signals are then transmitted to the speaker 15 and / or the main control conference terminal 13. The array microphone 12 includes at least two microphone units, which may include a left channel microphone unit 121 and a right channel microphone unit 122. The left channel microphone unit 121 is used to pick up left channel audio signals. The right channel microphone unit 122 is used to pick up right channel audio signals. Optionally, these microphone units may also include mute microphone units, which do not pick up audio.

[0077] This application embodiment does not limit the number or arrangement of the microphone units included in the array microphone 12. For example, the number can be 3, 4, or 5. The arrangement of the array microphone 12 can be circular, triangular, rectangular, etc. The specific examples below are all based on the example of the array microphone 12 including 3 microphone units, and these 3 microphone units being evenly distributed on a circular ring. Furthermore, this application embodiment does not limit the number of left channel microphone unit 121, right channel microphone unit 122, and mute microphone unit. In specific implementation, the number of different types of microphone units can be predefined, specifically one or more. For the sake of brevity, the specific examples below are all based on the example of one left channel microphone unit 121, one right channel microphone unit 122, and one mute microphone unit.

[0078] The main control conference terminal 13 is used to receive video signals sent by camera 11 and audio signals sent by array microphone 12. Then, the video signal sent by camera 11 is processed and sent to display screen 14 for display; the audio signal is processed and sent to speaker 15 for playback.

[0079] When applied to the embodiments of this application, the main control conference terminal 13 is further configured to acquire the shooting direction information of the camera 11, and determine at least one of the left channel microphone unit 121 and the right channel microphone unit 122 of the array microphone based on the shooting direction information. For details regarding the shooting direction information of the camera 11 and the specific implementation of determining the left channel microphone unit 121 and the right channel microphone unit 122, please refer to the following text, which will not be repeated here.

[0080] The display screen 14 is used to receive video signals sent by the camera 11 or the main control conference terminal 13 and display the video signals on the display screen 14.

[0081] Speaker 15 is used to receive sound signals sent by array microphone 12 or main control conference terminal 13 and play the received sound signals.

[0082] It should be noted that, Figure 1 The structure shown does not constitute a limitation on the conference system 10 and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0083] For example, one or more of the camera 11, array microphone 12, display screen 14, or speaker 15 may be installed in the main control conference terminal 13. However, the array microphone 12 and the camera are usually not installed in the main control conference terminal 13 at the same time.

[0084] To identify the left channel microphone unit 121 and / or the right channel microphone unit 122 from the array microphone 12, several alternative implementations are provided below:

[0085] Optional, such as Figure 2 As shown, the conference system 10 also includes a first magnetic sensor 123 or a first infrared sensor 124. Figure 2 Is Figure 1 It was drawn based on that.

[0086] The first magnetic sensor 123 is used to measure the magnetic field orientation information of any microphone unit in the array microphone 12. This magnetic field orientation information is used to determine the left channel microphone unit 121 and / or the right channel microphone unit 122 from the array microphone 12. The specific implementation method can be found below.

[0087] The first infrared sensor 124 is used to measure the directional information of any microphone unit in the array microphone. This directional information is used to determine the left channel microphone unit 121 and / or the right channel microphone unit 122 from the array microphone 12. The specific implementation can be found below.

[0088] The first magnetic sensor 123 or the first infrared sensor 124 can be integrated with the array microphone 12 or set separately from the array microphone 12. This application embodiment does not impose specific limitations on this.

[0089] To obtain the shooting direction information of camera 11, the following two optional implementation methods are provided:

[0090] In one alternative implementation, such as Figure 2 As shown, the conference system 10 also includes a second magnetic sensor 111 or a second infrared sensor 112.

[0091] The second magnetic sensor 111 is used to measure the magnetic field orientation information of the camera 11, which is used to determine the camera's shooting direction information. The second infrared sensor 112 is used to measure the shooting direction information of the camera 11.

[0092] The second magnetic sensor 111 or the second infrared sensor 112 can be integrated with the camera 11 or set separately from the camera 11. This application embodiment does not impose specific restrictions on this.

[0093] In another alternative implementation, such as Figure 3 As shown, the conference system 10 also includes a sound source 113. The sound source 113 emits sound in the same direction or roughly the same direction as the camera 11 captures the image.

[0094] The sound source 113 is used to output a sound signal so that the individual microphones in the array microphone 12 can collect the sound signal and thus determine the shooting direction of the camera 11. The specific implementation method can be found in the following text.

[0095] The sound source 113 and the camera 11 can be electrically connected by a wire. The sound source 113 can be placed in a position where the array microphone 12 can pick up the signal. The sound source 113 can be placed at a preset distance from the camera 11, but it is necessary to ensure that the sound source 113 emits sound in the same or roughly the same direction as the camera 11.

[0096] The sound signal emitted by the sound source 113 can be a signal of a specific frequency so that the main control conference terminal 13 can identify it. Specifically, the sound source 113 can be an in-band sound source (frequency range: 20Hz~22KHz) or an out-of-band sound source (frequency <20Hz or >20KHz). Out-of-band sound sources can avoid interfering with normal conference communication.

[0097] The following is a schematic diagram illustrating an application scenario of the conference system 10 provided in this application embodiment:

[0098] like Figure 4 In this system, the conference system 10 is applied in a scenario comprising a first conference room and a second conference room. The first conference room includes a first camera, a first array microphone, a first main control conference terminal, a first display screen, and a first speaker. The second conference room includes a second camera, a second array microphone, a second main control conference terminal, a second display screen, and a second speaker. The first and second main control conference terminals exchange data via a network switching device. For example, the video and audio signals from the first conference room are transmitted from the first main control conference terminal to the second main control conference terminal via the network switching device and output to the second display screen and the second speaker in the second conference room.

[0099] The first camera is used to capture video signals from the first meeting room and send the captured video signals to the first master control conference terminal. The first array microphone is used to pick up audio signals from the first meeting room, convert the audio signals from the first meeting room into electrical signals, and send the converted electrical signals to the first master control conference terminal.

[0100] The first master control conference terminal processes the video signal sent by the first camera. It then sends the processed video signal to the first display screen for local display and to the network switching equipment for transmission to the second conference terminal. The second conference terminal then sends the processed video signal to the second display screen for display, thus enabling the video signal from the first conference room to be displayed in the second conference room.

[0101] Furthermore, the first master control conference terminal processes the electrical signals sent by the first array microphones, and then sends the processed audio signal to the first speaker for local playback, and simultaneously transmits the processed audio signal to the second conference terminal. The second conference terminal receives the processed audio signal and sends it to the second speaker for playback, thereby enabling the audio signal from the first conference room to be played in the second conference room.

[0102] Correspondingly, the second camera is used to capture video signals from the second meeting room for local display on the second screen, and to transmit them to the first master control conference terminal via network switching equipment, so as to display the video signals from the second meeting room in the first meeting room. The second array microphone is used to capture audio signals from the second meeting room for local playback by the second speaker, and to transmit them to the first master control conference terminal via network switching equipment, so as to play the audio signals from the second meeting room in the first meeting room.

[0103] It should be noted that, Figure 4 This is merely an example of one application scenario of the conference system 10 provided in the embodiments of this application, and it does not constitute a limitation on the application scenario of the conference system 10 provided in the embodiments of this application.

[0104] refer to Figure 5 This application provides a schematic diagram of the structure of a master control conference terminal 50. The technical terms used in this application are described below.

[0105] like Figure 5 As shown, the master control conference terminal 50 may include a processor 51, a memory 52, a communication interface 53, and a bus 54. The processor 51, the memory 52, and the communication interface 53 can be connected to each other via the bus 54.

[0106] The processor 51 is the control center of the main control conference terminal 50. It can be a general-purpose central processing unit (CPU) or other general-purpose processors. The general-purpose processor can be a microprocessor or any conventional processor.

[0107] As an example, processor 51 may include one or more CPUs, for example Figure 5CPU 0 and CPU 1 are shown in the diagram.

[0108] The memory 52 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0109] In one possible implementation, the memory 52 can exist independently of the processor 51. The memory 52 can be connected to the processor 51 via a bus 54 and is used to store data, instructions, or program code. When the processor 51 calls and executes the instructions or program code stored in the memory 52, it can implement the audio rendering method provided in the embodiments of this application.

[0110] In another possible implementation, the memory 52 can also be integrated with the processor 51.

[0111] The communication interface 53 is used for the main control conference terminal 50 to connect with other devices (such as network switching equipment) via a communication network, which can be Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The communication interface 53 may include a receiving unit for receiving data and a transmitting unit for sending data.

[0112] Bus 54 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, a peripheral component interconnect express (PCIe) bus, or an extended industry standard architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0113] It should be pointed out that, Figure 5 The structure shown does not constitute a limitation on the master control conference terminal 50, except Figure 5 In addition to the components shown, the master conference terminal 50 may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0114] This application provides a method and apparatus for determining a microphone. This method can be applied to any of the master control conference terminal 13 or master control conference terminal 50 described above. When this method is applied... Figure 5 When the main control conference terminal 50 shown is executed, the microphone determination method provided in this application embodiment can be implemented by the processor 51 executing the program instructions in the memory 52. ​​By executing the microphone determination method provided in this application embodiment, the accuracy of selecting individual microphones can be improved, and it helps to avoid the problem of poor sound pickup when the array microphones are moved.

[0115] The method for determining a microphone provided in the embodiments of this application will now be described with reference to the accompanying drawings.

[0116] Please refer to Figure 6 , Figure 6 A flowchart illustrating a microphone detection method according to an embodiment of this application is shown. The method may include the following steps:

[0117] S101, The main control conference terminal determines the camera's shooting direction information.

[0118] Among them, the shooting direction information is used to indicate the shooting direction of the camera. For example... Figure 7 As shown, the camera's shooting direction is the shooting direction when the camera returns to its initial position, which refers to the position when the lens is perpendicular to the main body of the camera base.

[0119] Optionally, S101 can be implemented in the following ways:

[0120] Method 1: The main control conference terminal acquires the camera's magnetic field orientation information, which is used to indicate the camera's magnetic field orientation. Based on this information, the main control conference terminal determines the camera's shooting direction.

[0121] For example, combining Figure 2After the second magnetic sensor 111 determines the magnetic field orientation of the camera, it sends the camera's magnetic field orientation to the camera's processing module. The camera's processing module then sends the camera's magnetic field orientation information to the main control conference terminal. The main control conference terminal receives the camera's magnetic field orientation information and, based on this information, determines the camera's shooting orientation.

[0122] like Figure 8 As shown, the magnetic field orientation is determined by magnetic field calibration to be due east, due west, due south, and due north. In this application, due north is defined as 0°, and the range from 0° to 360° is defined clockwise from due north. Based on this, in one example, if the camera is located in the due north direction in the magnetic field orientation, the camera's shooting direction information is 0°. In another example, such as... Figure 8 As shown, if the camera is located in the northeast direction in the magnetic field, then the camera's shooting direction information is 45°.

[0123] Method 2: The main control conference terminal acquires sound signals from at least two individual microphones in the array microphones. The sound signals originate from the same sound source, and the direction of the sound source is consistent with or approximately consistent with the camera's shooting direction. Based on the sound signals acquired by these at least two individual microphones, the main control conference terminal uses a sound source localization algorithm to determine the camera's shooting direction information.

[0124] For example, combining Figure 3 The sound source 113 emits a sound signal, and at least two microphones in the array microphone 12 pick up the sound signal. The processing module in the array microphone 12 sends the sound signal picked up by the at least two microphones to the main control conference terminal 13. Based on the sound signal collected by the at least two microphones, the main control conference terminal 13 uses a sound source localization algorithm to determine the direction of sound emission from the sound source 13. Since the direction of sound emission from the sound source 13 is the same as or approximately the same as the shooting direction of the camera, the main control conference terminal 13 can use the direction of sound emission from the sound source 13 as the shooting direction of the camera.

[0125] Method 3: The main control conference terminal receives the camera's shooting direction information.

[0126] For example, combining Figure 2 The second magnetic sensor 111 determines the magnetic field orientation of the camera and sends it to the camera's processing module. The processing module in the camera can determine the camera's shooting direction based on this magnetic field orientation information, and then send the shooting direction information to the main control conference terminal.

[0127] For example, combining Figure 2The second infrared sensor 112 measures the camera's shooting direction information and sends the measured shooting direction information to the camera's processing module. The camera's processing module then sends the camera's shooting direction information to the main control module.

[0128] For example, combining Figure 3 The sound source 113 emits a sound signal, and at least two microphones in the array microphone 12 pick up the sound signal. The processing module in the array microphone 12 determines the sound direction of the sound source 113 according to the sound source localization algorithm, and sends the sound direction information as the shooting direction information of the camera to the main control conference terminal.

[0129] S102. The main control conference terminal determines the first reference direction of the target microphone unit based on the camera's shooting direction.

[0130] The first reference direction is the reference direction of the target microphone unit in the array microphone, which is either the left channel microphone unit or the right channel microphone unit. The camera's shooting direction forms a first preset angle with the first reference direction. The left channel microphone unit is used to pick up the left channel audio signal, and the right channel microphone unit is used to pick up the right channel audio signal.

[0131] Specifically, when the target microphone is a left-channel microphone, the first reference direction is: ideally, the reference direction where the left-channel microphone is located, specifically perpendicular to the camera's shooting direction and located on the left side of the venue. When the target microphone is a right-channel microphone, the first reference direction is: ideally, the reference direction where the right-channel microphone is located, specifically perpendicular to the camera's shooting direction and located on the right side of the venue.

[0132] like Figure 9 The image shown illustrates the shooting direction of a camera and the reference directions of the left and right channel microphone units provided in an embodiment of this application. Wherein, in Figure 9 In this diagram, the camera's shooting direction is marked as S1, the reference direction of the left channel microphone is marked as S2, and the reference direction of the right channel microphone is marked as S3. Additionally, Figure 9 The diagram illustrates the left and right sides of the meeting room. The left and right sides are relative and can be predefined in the specific implementation. This application does not limit how the left and right sides of the meeting room are determined; for example, any direction of the meeting room can be considered the left side, and the opposite direction can be considered the right side.

[0133] S103, the main control conference terminal determines the target microphone unit from the array microphones based on the first reference direction. The target microphone unit includes the left channel microphone unit and / or the right channel microphone unit.

[0134] When the target microphone unit is a left channel microphone unit, it can be achieved in the following way:

[0135] Based on the first reference direction, the main control conference terminal selects the microphone unit in the array microphone that meets the first preset condition as the left channel microphone unit.

[0136] The microphone units that meet the first preset condition include: a first preset number of microphone units whose angle between their current direction and the first reference direction is less than or equal to the first threshold; or, the microphone units corresponding to the first preset number of angles between their current direction and the first reference direction, sorted from smallest to largest; or, the microphone units corresponding to the last preset number of angles between their current direction and the first reference direction, sorted from largest to smallest.

[0137] When the target microphone unit is a left channel microphone unit, the first reference direction is, ideally, the reference direction where the left channel microphone unit is located. Therefore, using the microphone unit with the smaller angle between its direction and the first reference direction as the left channel microphone unit helps improve its pickup effect, thus contributing to a better synthesized stereo effect. Specifically, using a microphone unit with a smaller angle to the first reference direction as the left channel microphone unit further enhances its pickup effect, resulting in a better synthesized stereo effect.

[0138] Similarly, since the first reference direction is the ideal reference direction of the right channel microphone unit when the target microphone unit is the right channel microphone unit, using the microphone unit with the smaller angle between the direction of at least two microphone units in the array and the first reference direction as the right channel microphone unit helps to improve the sound pickup effect of the right channel microphone unit, thereby helping to make the synthesized stereo effect better.

[0139] The following explains how to determine the angle between the orientation of any microphone unit in the array microphone and the first reference direction:

[0140] Method 1: The master control conference terminal receives the magnetic field orientation information of any microphone unit in the array microphones. Then, based on this magnetic field orientation information, the master control conference terminal determines the orientation information of that microphone unit. Next, based on the orientation information of that microphone unit, the master control conference terminal determines the angle of that microphone unit's orientation relative to a first reference direction.

[0141] For example, combining Figure 2The first magnetic sensor 123 measures the magnetic field orientation information of any microphone unit in the array microphone and sends this information to the array microphone processing module. The processing module then sends this information to the main control conference terminal. Subsequently, the main control conference terminal determines the orientation information of the microphone unit based on its magnetic field orientation information. The specific implementation of this step can be found in the above-described implementation of determining the camera's shooting direction information based on the camera's magnetic field orientation information, and will not be repeated here.

[0142] In one example, the orientation of a single microphone unit is: the direction of the line connecting the geometric center of the array microphones to the geometric center of that single microphone unit. For example... Figure 10 The diagram shown illustrates the orientation of the microphone unit. Figure 10 The example described uses an array microphone consisting of three microphones, with each microphone evenly distributed on a circular ring.

[0143] It should be noted that the direction of the microphone unit of the array microphone in the embodiments of this application is not limited to passing through the geometric center point of the microphone unit.

[0144] Method 2: The main control conference terminal receives the direction information of any microphone unit in the array microphones. Then, based on the direction information of the array microphones, the main control conference terminal determines the angle of the direction of that microphone unit relative to the first reference direction.

[0145] For example, combining Figure 2 The first magnetic sensor 123 measures the magnetic field orientation information of any microphone unit in the array microphone and sends this information to the array microphone processing module. The processing module determines the orientation information of the microphone unit based on its magnetic field orientation information and then sends the determined orientation information to the main control conference terminal. The specific implementation methods for subsequent steps can refer to Method 1 described above, and will not be repeated here.

[0146] For example, combining Figure 2 The first infrared sensor 124 measures the directional information of any individual microphone in the array microphone and sends this directional information to the array microphone processing module; then, the array microphone processing module sends this directional information to the main control conference terminal. The specific implementation methods for subsequent steps can refer to Method 1 above, and will not be repeated here.

[0147] Method 3: The main control conference terminal receives the angle of the direction of any microphone unit in the array microphone relative to the first reference direction.

[0148] For example, combining Figure 2First, the first magnetic sensor 123 measures the magnetic field orientation information of any microphone unit in the array microphone and sends this information to the array microphone processing module. The processing module determines the orientation information of the microphone unit based on this magnetic field orientation information. Alternatively, the first infrared sensor 124 measures the orientation information of any microphone unit in the array microphone and sends this information to the array microphone processing module. Then, the array microphone processing module determines the angle of the microphone unit's orientation relative to a first reference direction based on the orientation information, and sends the determined angle to the main control conference terminal. The specific implementation methods for subsequent steps can refer to the first method described above, and will not be repeated here.

[0149] It should be noted that for any single microphone unit in the array microphone, the angle of its orientation relative to the first reference direction can be determined using any of the methods provided above, but not limited to those described above. Furthermore, since the angles between the orientations of different microphone units are fixed once the arrangement of the microphones in the array microphone is determined, in one optional implementation, the orientation of one or more microphone units in the array microphone can be determined using the methods provided above, and the orientations of other microphone units can be determined based on the angular relationships between their orientations. This helps to save on the large hardware resource overhead caused by using magnetic sensors or infrared sensors to measure the orientation of microphone units.

[0150] Optionally, the main control conference terminal can determine the individual microphones to be muted in the following way:

[0151] First, the main control conference terminal determines a second reference direction based on the camera's shooting direction. This second reference direction is the reference direction of the mute microphone in the array microphone. The shooting direction and the second reference direction form a second preset angle.

[0152] Optionally, the second preset angle is 180°, and the second reference direction A0 is as follows: Figure 9 As shown. However, in actual operation, the second preset angle may deviate from 180°. That is, the second preset angle can be 180° ± a preset angle range. The preset angle range can be 0°-5°, 0°-10°, or 0°-15°, etc. This application embodiment does not impose specific limitations on this.

[0153] Secondly, based on the second reference direction, the main control conference terminal selects the microphone units in the array microphones that meet the second preset conditions as the microphone units to be muted.

[0154] The microphone units that meet the second preset condition include: a second preset number of microphone units whose angle between their current direction and the second reference direction is less than or equal to the second threshold; or, the microphone units corresponding to the first second preset number of angles between their current direction and the second reference direction, sorted from smallest to largest; or, the microphone units corresponding to the last second preset number of angles between their current direction and the second reference direction, sorted from largest to smallest.

[0155] It should be noted that, in actual implementation, the main control conference terminal can determine the left and right channel microphone units based on S103, and then use the remaining microphone units in the array microphones as mute microphone units. Alternatively, the main control conference terminal can determine the left channel microphone unit based on S103, and determine the mute microphone unit based on the above optional implementation method, and then use the remaining microphone units in the array microphones as right channel microphone units. Alternatively, the main control conference terminal can determine the right channel microphone unit based on S103, and determine the mute microphone unit based on the above optional implementation method, and then use the remaining microphone units in the array microphones as left channel microphone units.

[0156] S104. The main control terminal device receives the left channel sound signal picked up by the left channel microphone and the right channel sound signal picked up by the right channel microphone, and synthesizes a stereo signal based on the left channel sound signal and the right channel sound signal.

[0157] Specifically, firstly, the left and right channel microphones in the array microphone pick up the sound signals and send them to the array microphone's processing module. Secondly, the array microphone's processing module sends the received sound signals to the main control terminal device. Then, the main control terminal device can synthesize the left and right channel sound signals into stereo sound using any method provided by existing technology.

[0158] For example, audio panning technology can be used to produce stereo signals.

[0159] Subsequently, the main control terminal device can send the synthesized stereo sound to the local speakers for playback. Optionally, the main control terminal device can transmit the synthesized stereo sound to a remote location for playback via a network switching device. This application embodiment does not limit this aspect.

[0160] It should be noted that S104 and the subsequent steps are all application scenarios of the microphone determination method provided in the embodiments of this application. In actual implementation, the embodiments of this application are not limited to this.

[0161] The microphone identification method provided in this application determines the reference direction (i.e., the ideal direction of the left and / or right channel microphone units) of the array microphone based on the camera's shooting direction. Since the camera's shooting direction and the reference directions of the left and right channel microphone units are at preset angles, the ideal reference directions of the left and / or right channel microphone units can be determined based on the camera's shooting direction. Subsequently, the left and / or right channel microphone units are identified from the array microphone based on the determined reference directions.

[0162] Furthermore, compared to existing technologies, this solution provides a method for dynamically determining the microphone, which is not limited by the placement of the camera or the array microphones. Therefore, it offers greater flexibility in stereo scene setup. Moreover, even if the array microphones move during use, this method allows for dynamic re-determination of the new microphones, thus contributing to better stereo sound.

[0163] The following specific example illustrates the method for determining the microphone provided in this application. In this embodiment, an array microphone is used, comprising microphone units M1, M2, and M3, which are evenly distributed in a circular pattern.

[0164] Step 1: The main control terminal device converts the camera's magnetic field direction S1 into the camera's shooting direction information S1°. Then, based on the relationship between the camera's shooting direction and the reference directions of the left and right channel microphones, the main control terminal device determines the direction information of the reference direction of the left channel microphone as: S2° = S1° - 90°, that is, the reference direction S2 of the left channel microphone is the direction after rotating the camera's shooting direction S1 counterclockwise by 90°; the direction information of the reference direction of the right channel microphone is: S3° = S1° + 90°, that is, the reference direction S3 of the right channel microphone is the direction after rotating the camera's shooting direction S1 clockwise by 90°.

[0165] Step 2: Based on the 180° angle between the camera's shooting direction and the reference direction A0 (i.e., the second reference direction mentioned above) where the microphone unit is located, the reference direction information of the microphone unit can be obtained as A0° = S1° + 180°. Here, A0° is the direction information of direction A0.

[0166] Step 3: The main control terminal device obtains the direction information A1° of the direction A1 where microphone unit M1 is located. For the specific implementation method, please refer to the above. Since the three microphone units are arranged at a 120-degree angle, the angle A2° of the direction A2 where microphone unit M2 is located is A2° = A1° + 120°, and the angle A3° of the direction A3 where microphone unit M3 is located is A3° = A1° + 240°.

[0167] Step 4: The main control terminal device determines the microphone unit N for muting according to Formula 1:

[0168] Formula 1: N=min{|(A1°-A0°)|,|(A2°-A0°)|,|A3°-A0°|}.

[0169] In other words, the main control terminal device will use the microphone unit facing the direction close to A0 as the microphone unit N with the microphone muted.

[0170] Step 5: The main control terminal device determines the left channel microphone unit L from the microphone units in the array microphone (excluding the mute microphone units) according to Formula 2:

[0171] Formula 2: L=min{|(A1°-S2°)|,|(A2°-S2°)|}.

[0172] Formula 2 is illustrated using the assumption that the microphone unit N determined in step 4 is microphone unit M3.

[0173] Step 6: The main control terminal device will designate all microphone units in the array microphone except for the mute microphone unit and the left channel microphone unit as the right channel microphone unit.

[0174] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0175] This application embodiment can divide the microphone pickup device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0176] like Figure 11 As shown, Figure 11 A schematic diagram of the structure of a microphone determination device 110 provided in an embodiment of this application is shown. This microphone determination device 110 is used to perform the aforementioned microphone determination method, for example, to perform... Figure 6 The method for determining a microphone is shown. For example, the microphone determining device 110 may include a first determining unit 1101, a second determining unit 1102, and a third determining unit 1103.

[0177] The first determining unit 1101 is used to determine the shooting direction information of the camera, which indicates the shooting direction of the camera. The second determining unit 1102 is used to determine a first reference direction based on the camera's shooting direction; wherein the first reference direction is the reference direction of the target microphone unit in the array microphone, and the target microphone unit is either a left channel microphone unit or a right channel microphone unit; the shooting direction forms a first preset angle with the first reference direction; the left channel microphone unit is used to pick up the left channel sound signal, and the right channel microphone unit is used to pick up the right channel sound signal. The third determining unit 1103 determines the target microphone unit from the array microphone based on the first reference direction.

[0178] As an example, the camera can be camera 11 as described above, the array microphone can be array microphone 12 as described above, and the first determining unit 1101, the second determining unit 1102, and the third determining unit 1103 are configured in the main control conference terminal 13 described above. Combined with... Figure 6 The first determining unit 1101 can execute S101, the second determining unit 1102 can execute S102, and the third determining unit 1103 can execute S103.

[0179] Optionally, the camera's shooting direction is the shooting direction of the camera when it is in (e.g., always or in recovery) its initial position, which refers to the position when the camera lens is perpendicular to the camera's base body.

[0180] Optionally, the third determining unit 1103 is specifically used to: based on the first reference direction, select microphone units in the array microphones that meet the first preset condition as target microphone units. The microphone units that meet the first preset condition include: a first preset number of microphone units whose angle between their current direction and the first reference direction is less than or equal to a first threshold; or, microphone units corresponding to the first preset number of angles between their current direction and the first reference direction, sorted from smallest to largest.

[0181] Optionally, the microphone identification device 110 further includes a receiving unit 1104.

[0182] Optionally, the receiving unit 1104 is used to receive the magnetic field orientation information of any microphone unit in the array microphone; the third determining unit 1103 is further used to determine the orientation information of the microphone unit based on the magnetic field orientation information of the microphone unit; and to determine the angle of the direction of the microphone unit relative to the first reference direction based on the orientation information of the microphone unit.

[0183] Optionally, the receiving unit 1104 is used to receive the direction information of any microphone unit in the array microphone; the third determining unit 1103 is further used to determine the angle of the direction of the microphone unit relative to the first reference direction based on the direction information of the microphone unit.

[0184] Optionally, the receiving unit 1104 is used to receive the angle of the direction of any microphone unit in the array microphone relative to the first reference direction.

[0185] Optionally, the array microphone includes a first magnetic sensor for measuring the magnetic field orientation information of any individual microphone unit.

[0186] Optionally, the array microphone includes a first infrared sensor for measuring the orientation information of any individual microphone unit.

[0187] Optionally, the first determining unit 1101 is specifically used to: acquire the magnetic field orientation information of the camera, the magnetic field orientation information being used to indicate the magnetic field orientation of the camera; and determine the shooting direction information of the camera based on the magnetic field orientation information of the camera.

[0188] Optionally, the receiving unit 1104 is used to receive the shooting direction information of the camera.

[0189] Optionally, the camera includes a second magnetic sensor for measuring the camera's magnetic field orientation information.

[0190] Optionally, the camera includes a second infrared sensor for measuring the camera's shooting direction information.

[0191] Optionally, the first determining unit 1101 is specifically used to: acquire sound signals collected by at least two individual microphones in the array microphone, the sound signals coming from the same sound source, the sound source’s direction of sound being consistent with the camera’s shooting direction; and based on the sound signals collected by at least two individual microphones, use a sound source localization algorithm to determine the camera’s shooting direction information.

[0192] Optionally, the camera includes a sound source.

[0193] Optionally, the first determining unit 1101 is specifically used to: receive the shooting direction information of the camera.

[0194] Optionally, the second determining unit 1102 is specifically used to: determine a second reference direction based on the camera's shooting direction; the second reference direction is the reference direction of the mute microphone units in the array microphone; the shooting direction and the second reference direction form a second preset angle; the third determining unit 1103 is specifically used to: based on the second reference direction, select microphone units in the array microphone that meet the second preset condition as mute microphone units; wherein, the mute microphone units do not pick up sound. The microphone units that meet the second preset condition include: a second preset number of microphone units whose angle between their direction and the second reference direction is less than or equal to a second threshold; or, the microphone units corresponding to the first second preset number of angles after sorting the angles between their direction and the second reference direction from smallest to largest.

[0195] Optionally, the first preset angle is 90 degrees.

[0196] Optionally, the microphone determining device further includes a synthesis unit 1105 for synthesizing a stereo signal based on the left channel sound signal and the right channel sound signal.

[0197] For a detailed description of the above-mentioned optional methods, please refer to the foregoing method embodiments, which will not be repeated here. Furthermore, the explanation of any of the microphone detection devices 110 provided above, as well as the description of their beneficial effects, can be found in the corresponding method embodiments described above, and will not be repeated here.

[0198] As an example, combined Figure 5 The functions implemented by some or all of the first determining unit 1101, the second determining unit 1102, the third determining unit 1103, and the synthesis unit 1105 in the microphone pickup determining device can be achieved through... Figure 5 The processor 51 in the middle executes Figure 5 The program code in memory 52 is used for implementation. The receiving unit 1104 can be accessed via... Figure 5 The receiving unit in the communication interface 53 is implemented.

[0199] This application also provides a chip system, such as... Figure 12 As shown, the chip system includes at least one processor 131 and at least one interface circuit 132. As an example, when the chip system 130 includes a processor and an interface circuit, the processor can be... Figure 12 The processor 131 shown in the solid box (or the processor 131 shown in the dashed box) can be an interface circuit. Figure 12 The interface circuit 132 is shown in the solid box (or the dashed box). When the chip system 130 includes two processors and two interface circuits, the two processors include... Figure 12 The processor 131 shown in the solid box and the processor 131 shown in the dashed box, these two interface circuits include Figure 12 Interface circuit 132 is shown in both solid and dashed boxes. No limitations are imposed on it.

[0200] Processor 131 and interface circuit 132 can be interconnected via lines. For example, interface circuit 132 can be used to receive signals (e.g., signals from a vehicle speed sensor or edge service unit). As another example, interface circuit 132 can be used to send signals to other devices (e.g., processor 131). Exemplarily, interface circuit 132 can read instructions stored in memory and send those instructions to processor 131. When the instructions are executed by processor 131, the microphone detection device can perform the steps in the above embodiments. Of course, this chip system may also include other discrete components, which are not specifically limited in this application embodiment.

[0201] Another embodiment of this application also provides a computer-readable storage medium storing instructions that, when executed on a microphone-detecting device, cause the microphone-detecting device to perform each step of the method flow shown in the above-described method embodiment.

[0202] In some embodiments, the disclosed method may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of art.

[0203] Figure 13 A conceptual partial view of a computer program product provided in an embodiment of this application is shown schematically. The computer program product includes a computer program for executing computer processes on a computing device.

[0204] In one embodiment, a computer program product is provided using a signal bearer medium 140. The signal bearer medium 140 may include one or more program instructions that, when executed by one or more processors, can provide the above-mentioned... Figure 6 The described function or part of the function. Therefore, for example, refer to... Figure 6 One or more features of S101 to S104 can be assumed by one or more instructions associated with the signal carrying medium 140. Furthermore, Figure 13 The program instructions in the document also describe example instructions.

[0205] In some examples, the signal carrying medium 140 may include a computer-readable medium 141, such as, but not limited to, a hard disk drive, a compact disc (CD), a digital video optical disc (DVD), a digital magnetic tape, a memory, a read-only memory (ROM), or a random access memory (RAM), etc.

[0206] In some implementations, the signal carrying medium 140 may include a computer recordable medium 142, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, and so on.

[0207] In some implementations, the signal carrying medium 140 may include a communication medium 143, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0208] The signal-bearing medium 140 can be transmitted by a wireless communication medium 143 (e.g., a wireless communication medium conforming to the IEEE 802.11 standard or other transmission protocols). One or more program instructions can be, for example, computer-executable instructions or logical implementation instructions.

[0209] In some examples, such as targeting Figure 13 The described microphone identification device can be configured to provide various operations, functions, or actions in response to one or more program instructions in a computer-readable medium 141, a computer-recordable medium 142, and / or a communication medium 143.

[0210] It should be understood that the arrangements described herein are for illustrative purposes only. Therefore, those skilled in the art will understand that other arrangements and other elements (e.g., machines, interfaces, functions, sequences, and functional groups, etc.) can be used instead, and some elements may be omitted depending on the desired outcome. Furthermore, many of the described elements are functional entities that can be implemented as discrete or distributed components, or in any suitable combination and location with other components.

[0211] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs).

[0212] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for determining a microphone, characterized in that, The method includes: Determine the camera's shooting direction information, which is used to indicate the camera's shooting direction; Based on the camera's shooting direction, a first reference direction is determined; wherein, the first reference direction is the reference direction of the target microphone unit in the array microphone, and the target microphone unit is either a left channel microphone unit or a right channel microphone unit; the shooting direction forms a first preset angle with the first reference direction; the left channel microphone unit is used to pick up the left channel sound signal, and the right channel microphone unit is used to pick up the right channel sound signal. Based on the first reference direction, the microphone unit in the array microphone that meets the first preset condition is taken as the target microphone unit; The microphone unit that meets the first preset condition includes: The angle between the current direction and the first reference direction is less than or equal to a first preset number of microphone units of a first threshold. Alternatively, the angle between the direction it is in and the first reference direction, and the microphone unit corresponding to the first preset number of angles after sorting from smallest to largest.

2. The method according to claim 1, characterized in that, The method further includes: Receive the magnetic field orientation information of any microphone unit in the array microphones; determine the orientation information of any microphone unit based on the magnetic field orientation information of the microphone unit; determine the angle of the orientation of any microphone unit relative to the first reference direction based on the orientation information of the microphone unit. Alternatively, receive the orientation information of any microphone unit in the array microphones; determine the angle of the orientation of any microphone unit relative to the first reference direction based on the orientation information of the microphone unit; Alternatively, receive the angle of the direction of any microphone unit in the array microphone relative to the first reference direction.

3. The method according to claim 2, characterized in that, The array microphone includes a first magnetic sensor, which is used to measure the magnetic field orientation information of any one of the microphone units.

4. The method according to claim 2, characterized in that, The array microphone includes a first infrared sensor, which is used to measure the directional information of any one of the microphone units.

5. The method according to any one of claims 1-4, characterized in that, The determination of the camera's shooting direction information includes: The magnetic field orientation information of the camera is obtained, and the magnetic field orientation information is used to indicate the magnetic field orientation of the camera; based on the magnetic field orientation information of the camera, the shooting direction information of the camera is determined. Alternatively, it can receive the camera's shooting direction information.

6. The method according to claim 5, characterized in that, The camera includes a second magnetic sensor, which is used to measure the magnetic field orientation information of the camera.

7. The method according to claim 5, characterized in that, The camera includes a second infrared sensor, which is used to measure the camera's shooting direction information.

8. The method according to any one of claims 1-4, characterized in that, Determining the camera's shooting direction information includes: Acquire sound signals from at least two individual microphones in the array microphone, wherein the sound signals originate from the same sound source, and the direction of the sound source is consistent with the shooting direction of the camera; Based on the sound signals collected by the at least two individual microphones, a sound source localization algorithm is used to determine the shooting direction information of the camera.

9. The method according to claim 8, characterized in that, The camera includes the sound source.

10. The method according to any one of claims 1-9, characterized in that, After determining the target microphone unit from the array microphones based on the first reference direction, the method further includes: A stereo signal is synthesized based on the left channel audio signal and the right channel audio signal.

11. The method according to any one of claims 1-10, characterized in that, The method further includes: Based on the camera's shooting direction, a second reference direction is determined; the second reference direction is the reference direction of the mute microphone in the array microphone; the shooting direction and the second reference direction form a second preset angle; Based on the second reference direction, the microphone units in the array microphones that meet the second preset conditions are designated as mute microphone units; wherein, the mute microphone units do not pick up sound. The microphone unit that satisfies the second preset condition includes: The angle between the current direction and the second reference direction is less than or equal to the second threshold, which is a second preset number of microphone units; Alternatively, the angle between the current direction and the second reference direction, and the microphone units corresponding to the second preset number of angles after sorting from smallest to largest.

12. The method according to claim 11, characterized in that, The second preset angle is 180 degrees.

13. The method according to any one of claims 1 to 12, characterized in that, The first preset angle is 90 degrees.

14. The method according to any one of claims 1 to 13, characterized in that, The camera's shooting direction is the shooting direction of the camera in its initial position, which refers to the position where the camera lens is perpendicular to the camera's base body.

15. A microphone pickup device, characterized in that, The device includes: The first determining unit is used to determine the shooting direction information of the camera, wherein the shooting direction information is used to indicate the shooting direction of the camera; The second determining unit is used to determine a first reference direction based on the shooting direction of the camera; wherein, the first reference direction is the reference direction of the target microphone unit in the array microphone, and the target microphone unit is a left channel microphone unit or a right channel microphone unit; the shooting direction forms a first preset angle with the first reference direction; the left channel microphone unit is used to pick up the left channel sound signal, and the right channel microphone unit is used to pick up the right channel sound signal. The third determining unit is specifically used to determine the target microphone unit based on the first reference direction, which is the microphone unit in the array microphone that meets the first preset condition. The microphone unit that meets the first preset condition includes: The angle between the current direction and the first reference direction is less than or equal to a first preset number of microphone units of a first threshold. Alternatively, the angle between the direction it is in and the first reference direction, and the microphone unit corresponding to the first preset number of angles after sorting from smallest to largest.

16. The apparatus according to claim 15, characterized in that, The device further includes: a receiving unit; The receiving unit is configured to receive the magnetic field orientation information of any microphone unit in the array microphone; the third determining unit is further configured to determine the direction information of any microphone unit based on the magnetic field orientation information of any microphone unit; and determine the angle of the direction of any microphone unit relative to the first reference direction based on the direction information of any microphone unit. Alternatively, the receiving unit is configured to receive the direction information of any microphone unit in the array microphones; the third determining unit is further configured to determine the angle of the direction of any microphone unit relative to the first reference direction based on the direction information of the microphone unit. Alternatively, the receiving unit is configured to receive the angle of the direction of any microphone unit in the array microphone relative to the first reference direction.

17. The apparatus according to claim 16, characterized in that, The array microphone includes a first magnetic sensor, which is used to measure the magnetic field orientation information of any one of the microphone units.

18. The apparatus according to claim 16, characterized in that, The array microphone includes a first infrared sensor, which is used to measure the directional information of any one of the microphone units.

19. The apparatus according to any one of claims 15-18, characterized in that, The first determining unit is specifically used to: acquire the magnetic field orientation information of the camera, the magnetic field orientation information being used to indicate the magnetic field orientation of the camera; and determine the shooting direction information of the camera based on the magnetic field orientation information of the camera. Alternatively, the device may further include a receiving unit for receiving the shooting direction information of the camera.

20. The apparatus according to claim 19, characterized in that, The camera includes a second magnetic sensor, which is used to measure the magnetic field orientation information of the camera.

21. The apparatus according to claim 19, characterized in that, The camera includes a second infrared sensor, which is used to measure the camera's shooting direction information.

22. The apparatus according to any one of claims 15-18, characterized in that, The first determining unit is specifically used for: Acquire sound signals from at least two individual microphones in the array microphone, wherein the sound signals originate from the same sound source, and the direction of the sound source is consistent with the shooting direction of the camera; Based on the sound signals collected by the at least two individual microphones, a sound source localization algorithm is used to determine the shooting direction information of the camera.

23. The apparatus according to claim 22, characterized in that, The camera includes the sound source.

24. The apparatus according to any one of claims 15-23, characterized in that, The device further includes: A synthesis unit is used to synthesize a stereo signal based on the left channel audio signal and the right channel audio signal.

25. The apparatus according to any one of claims 15-24, characterized in that, The second determining unit is further configured to determine a second reference direction based on the shooting direction of the camera; the second reference direction is the reference direction of the mute microphone in the array microphone; the shooting direction and the second reference direction form a second preset angle; The third determining unit is further configured to, based on the second reference direction, designate the microphone units in the array microphone array that meet the second preset conditions as mute microphone units; wherein, the mute microphone units do not pick up sound; The microphone unit that satisfies the second preset condition includes: The angle between the current direction and the second reference direction is less than or equal to the second threshold, which is a second preset number of microphone units; Alternatively, the angle between the current direction and the second reference direction, and the microphone units corresponding to the second preset number of angles after sorting from smallest to largest.

26. The apparatus according to claim 25, characterized in that, The second preset angle is 180 degrees.

27. The apparatus according to any one of claims 15 to 26, characterized in that, The first preset angle is 90 degrees.

28. The apparatus according to any one of claims 15 to 27, characterized in that, The camera's shooting direction is the shooting direction of the camera in its initial position, which refers to the position where the camera lens is perpendicular to the camera's base body.

29. A microphone pickup device, characterized in that, include: A memory and a processor, the memory for storing a computer program, the processor for invoking the computer program to perform the method according to any one of claims 1-14.

30. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the method described in any one of claims 1-14.

Images