Microphone array networking method and system, electronic equipment, storage medium and program product

Abstract

The invention discloses a microphone array networking method and system, electronic equipment, a storage medium and a program product, and the method comprises the steps: enabling a microphone array in a conference space to return feedback information to a configuration terminal in response to the active broadcast of the configuration terminal; determining a plurality of microphone arrays for networking from the microphone arrays in the conference space according to networking information sent by the configuration terminal; the master device initializes and sends master device ready state information to the slave device; and after the slave device is initialized and receives the master device ready state information, slave device ready state information is sent to the master device so as to complete networking of the plurality of microphone arrays. According to the method and the device, the plurality of microphone arrays are selected for networking through the networking information issued by the configuration terminal, and the master-slave relationship among the plurality of microphone arrays is configured through the master-slave device relationship information set in the configuration terminal, so that decentralization of the plurality of microphone arrays is realized, and the problem of dependence on the external central device is solved.

Description

Technical Field

[0001] The present application relates to the technical field of array microphones, and in particular to a microphone array networking method, system, electronic device, storage medium and program product. Background Art

[0002] With the continuous development of voice technology, more and more microphone arrays are used in conferences, teaching, exhibition halls and other scenes that require microphone pickup and amplification. However, since a single microphone array cannot cover the entire pickup space (generally, the room is too large and the distance between the speaker and the microphone array is far, resulting in poor pickup effect), in order to solve the problem of blind spots in sound pickup, a feasible solution is to network multiple microphone arrays to form a microphone array cluster, which leads to a networking linkage control solution between microphone arrays.

[0003] The networking method adopted by the networking linkage control scheme in the related technology is the network communication scheme of the CS (Client / Server) architecture, that is, a fixed definition of an independent control center (server), and other devices actively connect and communicate with the control center as clients. However, in this way, the instructions need to be sent to a server first, and the server processes them one by one and forwards them to the client in turn. It is impossible to achieve efficient one-to-many simultaneous management and has a strong dependence on the control center. Summary of the invention

[0004] The embodiments of the present application provide a microphone array networking method, system, electronic device, storage medium and program product, which are used to solve at least one of the above-mentioned technical problems.

[0005] In a first aspect, an embodiment of the present application provides a microphone array networking method for multiple microphone arrays, the method comprising: In response to the active broadcast of the configuration terminal, the microphone array in the conference space sends feedback information back to the configuration terminal; Determine, according to the networking information sent by the configuration terminal, a plurality of microphone arrays for networking from the microphone arrays in the conference space; wherein the networking information includes master-slave device relationship information, which is used to determine the master device and the slave device in the plurality of microphone arrays; The master device is initialized and sends master device ready status information to the slave device; The slave device is initialized and, after receiving the ready state information of the master device, sends the slave device ready state information to the master device to complete the networking of the multiple microphone arrays.

[0006] In some embodiments, the networking information further includes identification information of multiple microphone arrays; and the method further includes: The slave device is initialized and sends slave device identification information to the master device after receiving the ready state information of the master device; The master device performs slave device legitimacy verification on the slave device according to the identification information of the plurality of microphone arrays and the slave device identification information, and marks the slave device that passes the legitimacy verification as ready; When the slave devices are all marked as ready, the master device notifies the slave devices that networking is complete.

[0007] In some embodiments, the plurality of microphone arrays respectively include a communication bus unit, and the master device initializing includes: Initialize a first data structure required by the master device; A first preset message topic is published, so that the slave device can subscribe to the first preset message topic through the communication bus unit of the master device.

[0008] In some embodiments, initializing from the device includes: Initialize a second data structure required by the slave device; A second preset message topic is published, so that the master device can subscribe to the second preset message topic through the communication bus unit of the master device.

[0009] In some embodiments, the master device ready status information is sent using a reserved message mechanism.

[0010] In some embodiments, it also includes: after the master device notifies the slave device that networking is complete, starting a disconnection monitoring mechanism between the master device and the slave device.

[0011] In a second aspect, an embodiment of the present application provides a microphone array system, including multiple microphone arrays, configured in a conference space, characterized in that: The microphone array in the conference space is configured to: in response to the active broadcast of the configuration terminal, send feedback information back to the configuration terminal; determine multiple microphone arrays for networking from the microphone arrays in the conference space according to the networking information sent by the configuration terminal; wherein the networking information includes master-slave device relationship information, which is used to determine the master device and the slave device in the multiple microphone arrays; The master device is configured to: initialize and send master device ready status information to the slave device; The slave device is configured to: perform initialization and, after receiving the ready state information of the master device, send the slave device ready state information to the master device, so as to complete the networking of the multiple microphone arrays.

[0012] In some embodiments, the networking information further includes identification information of multiple microphone arrays; wherein the slave device is further configured to: initialize and send slave device identification information to the master device after receiving the ready state information of the master device; The master device is further configured to: perform slave device legitimacy verification on the slave device according to the identification information of the plurality of microphone arrays and the slave device identification information, and mark the slave device that has passed the legitimacy verification as ready; The master device is further configured to: when all the slave devices are marked as ready, the master device notifies the slave devices that networking is completed.

[0013] In some embodiments, the plurality of microphone arrays respectively include a communication bus unit, and the master device initializing includes: Initialize a first data structure required by the master device; A first preset message topic is published, so that the slave device can subscribe to the first preset message topic through the communication bus unit of the master device.

[0014] In some embodiments, the initializing from the device includes: Initialize a second data structure required by the slave device; A second preset message topic is published, so that the master device can subscribe to the second preset message topic through the communication bus unit of the master device.

[0015] In some embodiments, the master device ready status information is sent using a reserved message mechanism.

[0016] In some embodiments, the master device is further configured to: after the master device notifies the slave device that networking is complete, start a disconnection monitoring mechanism between the master device and the slave device.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the microphone array networking method described in any embodiment of the present application.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program / instruction is stored, characterized in that when the computer program / instruction is executed by a processor, the steps of the microphone array networking method described in any embodiment of the present application are implemented.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the steps of the microphone array networking method described in any embodiment of the present application are implemented.

[0020] The embodiment of the present application discovers the microphone arrays already installed in the conference space by actively broadcasting the configuration terminal, further selects multiple microphone arrays for networking to meet the conference needs by configuring the networking information of the terminal, and configures the master-slave relationship between the multiple microphone arrays by configuring the master-slave device relationship information set in the configuration terminal, thereby realizing the decentralization of multiple microphone arrays and solving the problem of dependence on central devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A schematic diagram of a flow chart of an embodiment of a microphone array networking method of the present application; Figure 2 A schematic diagram of another embodiment of the microphone array networking method of the present application; Figure 3 A schematic diagram of another embodiment of the microphone array networking method of the present application; Figure 4 A schematic diagram of another embodiment of the microphone array networking method of the present application; Figure 5 A schematic diagram of the connection between the configuration software in this application and multiple microphone arrays; Figure 6 A schematic diagram of the connection between the microphone arrays and the configuration software in this application; Figure 7 A schematic diagram of a one-to-many publish-subscribe mode between a master device and a slave device in this application; FIG8 is a schematic diagram of a many-to-one publish-subscribe mode between a master device and a slave device in the present application; FIG9 is a schematic diagram of the connection relationship between multiple microphone arrays in this application; Fig.10 A schematic diagram of a receiving arbitration unit in the present application sampling instructions through a practice window; Fig.11 It is a schematic structural diagram of an embodiment of an electronic device of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0024] It should also be noted that, in this article, the terms "include" and "comprise" include not only those elements, but also other elements not explicitly listed, or elements inherent to such processes, methods, articles or equipment. In the absence of further restrictions, the elements defined by the sentence "include..." do not exclude the existence of other identical elements in the process, method, article or equipment that includes the elements.

[0025] The networking linkage control scheme in the related technology adopts the network communication scheme of CS (Client / Server) architecture, and the networked devices must support the function of synchronous control. For example, a single microphone array generally supports infrared remote control functions, such as controlling the volume of the device, muting the device, and switching the sound amplification. However, when there are multiple microphone arrays networked in a room, when the user presses the mute button of the infrared remote control at a certain location, not all devices in the group can receive the infrared operation (due to the limitation of infrared transmission power or receiving radius). Even if they can receive it, they may not be received at the same time (the signal may have reflection problems), resulting in inconsistent device status in the group, thereby affecting the overall sound pickup effect of the microphone array cluster.

[0026] In the process of implementing the present application, the applicant discovered that due to the network communication scheme of the CS (Client-Server model) architecture adopted in the prior art, the microphone array as an independent device cannot perceive the existence of other devices, and therefore cannot synchronously process external instructions. In other words, the execution of instructions is in sequence, resulting in differences in the final results.

[0027] In order to solve the technical problems existing in the prior art, the present application proposes networking among multiple microphone arrays, including how devices are networked (i.e., networking mode), networking process (devices complete self-networking according to networking information), and linkage control of networked devices. For example, the microphone array can be implemented as a ceiling microphone product for conferences, which has one or more of the following functions: sound pickup function, sound amplification function, audio signal processing function, voice enhancement function, voice signal processing function, etc.

[0028] In some embodiments, networking occurs when: the configuration terminal sends networking information, when multiple microphone arrays in the conference room are powered on (automatically according to the saved networking information), and when a microphone array is disconnected and automatically re-networked. The following takes the configuration terminal sending networking information as an example for explanation.

[0029] like Figure 1 As shown, an embodiment of the present application provides a microphone array networking method for multiple microphone arrays, the method comprising: S10. In response to the active broadcast of the configuration terminal, the microphone array in the conference space sends feedback information back to the configuration terminal.

[0030] Exemplarily, the configuration terminal may be, for example, a computer device (e.g., a laptop, a tablet computer, a smart phone, or other dedicated configuration device, which is not limited in this application) with configuration software installed. The user sends an active broadcast to search for the microphone array in the conference space through operation on the operation interface of the configuration software. After receiving the active broadcast from the configuration terminal, the corresponding microphone array in the conference space sends feedback information to the configuration terminal to inform the configuration terminal of the existence of the microphone array.

[0031] S20. Determine multiple microphone arrays for networking from the microphone arrays in the conference space according to the networking information sent by the configuration terminal; wherein the networking information includes master-slave device relationship information, which is used to determine the master device and the slave device in the multiple microphone arrays.

[0032] Exemplarily, after the configuration terminal finds the microphone arrays in the conference space through active broadcasting, they are connected to these microphone arrays respectively, and can be displayed on the operation interface of the configuration software. The user can select multiple microphone arrays from them as needed, and set one of them as the main microphone array (i.e., the master device), and the remaining microphone arrays are set as slave microphone arrays (i.e., the slave devices), and the master-slave device relationship information is generated accordingly and sent to the microphone arrays in the conference space.

[0033] It should be noted that in actual applications, it is not necessarily necessary to use all microphone arrays in the conference space for networking at the same time, so this embodiment facilitates the user to select appropriate multiple microphone arrays from the microphone arrays arranged in the conference space as needed.

[0034] Exemplarily, the master-slave device relationship information includes master device identification information and slave device identification information, wherein the master device identification information corresponds to the ID information of the master microphone array, and the slave device identification information corresponds to the ID information of the slave microphone array.

[0035] Each microphone array connected to the configuration terminal in the conference room can determine whether it is one of the multiple microphone arrays selected for networking and whether it is a master device or a slave device according to the received networking information.

[0036] S30, the master device is initialized and sends master device ready status information to the slave device.

[0037] S40, the slave device is initialized and after receiving the ready state information of the master device, sends the slave device ready state information to the master device to complete the networking of the multiple microphone arrays.

[0038] This embodiment discovers the microphone arrays already installed in the conference space by actively broadcasting the configuration terminal, further selects multiple microphone arrays for networking to meet the conference needs by configuring the networking information of the terminal, and configures the master-slave relationship between the multiple microphone arrays by configuring the master-slave device relationship information set in the configuration terminal, thereby realizing the decentralization of multiple microphone arrays and solving the problem of dependence on external central devices.

[0039] like Figure 2 FIG. 2 is a flow chart of another embodiment of the microphone array networking method of the present application. In this embodiment, the networking information also includes identification information of multiple microphone arrays; the microphone array networking method also includes: S50, the slave device is initialized and sends slave device identification information to the master device after receiving the ready state information of the master device; S60, the master device performs slave device legitimacy verification on the slave device according to the identification information of the plurality of microphone arrays and the slave device identification information, and marks the slave device that passes the legitimacy verification as ready; S70: When all the slave devices are marked as ready, the master device notifies the slave devices that networking is complete.

[0040] This embodiment sets the identification information of multiple microphone arrays in the networking information, so that after receiving the identification information of the slave device uploaded by the slave device during the networking process, it can check whether the slave device identification information exists in the identification information of the multiple microphone arrays. If so, it indicates that the slave device is a legal device. If not, it indicates that the slave device is an illegal device, thereby realizing the legitimacy verification of the slave devices participating in the networking.

[0041] like Figure 3 FIG. 2 is a flow chart of another embodiment of the microphone array networking method of the present application. In this embodiment, the plurality of microphone arrays respectively include a communication bus unit, and the master device initialization includes: S31, initializing a first data structure required by the master device; S32. Publish a first preset message topic, for the slave device to subscribe to the first preset message topic through the communication bus unit of the master device.

[0042] This embodiment implements subscription of messages from the microphone array to the main microphone array, so that synchronous control of multiple networked microphone arrays can be achieved through the main microphone array, that is, a one-to-many synchronous control effect is achieved.

[0043] like Figure 4 FIG. 2 is a flow chart of another embodiment of the microphone array networking method of the present application. In this embodiment, initializing the slave device includes: S41, initializing a second data structure required by the slave device; S42: Publish a second preset message topic, for the master device to subscribe to the second preset message topic through the communication bus unit of the master device.

[0044] This embodiment implements the subscription of the main microphone array to the messages of the slave microphone array, so that the control instructions can be synchronized to the main microphone array through any slave microphone array, thereby realizing synchronous control of multiple networked microphone arrays, that is, realizing a many-to-one synchronous control effect.

[0045] In some embodiments, the master device ready state information is sent using a reserved message mechanism. In order to resolve the order of publishing and subscription, the publisher (e.g., the master device) can publish a message with a reserved feature (i.e., a reserved message). When the publisher publishes this message, the communication bus unit will retain the message (only the most recent message), so that when a new subscriber (e.g., a slave device) subscribes to this message, the subscriber can receive the most recent message.

[0046] In some embodiments, it also includes: after the master device notifies the slave device that networking is complete, starting a disconnection monitoring mechanism between the master device and the slave device.

[0047] In this embodiment, by establishing a connection between networked devices and monitoring device online and offline messages, it is possible to know in real time whether each microphone array is online, so that the disconnected microphone array can be discovered in time, so as to re-network the disconnected microphone array as soon as possible and avoid the impact of the microphone array disconnection on the conference effect.

[0048] In some embodiments, the present application provides a microphone array system, including multiple microphone arrays, configured in a conference space, characterized in that: The microphone array in the conference space is configured to: in response to the active broadcast of the configuration terminal, send feedback information back to the configuration terminal; determine multiple microphone arrays for networking from the microphone arrays in the conference space according to the networking information sent by the configuration terminal; wherein the networking information includes master-slave device relationship information, which is used to determine the master device and the slave device in the multiple microphone arrays; The master device is configured to: initialize and send master device ready status information to the slave device; The slave device is configured to: perform initialization and, after receiving the ready state information of the master device, send the slave device ready state information to the master device, so as to complete the networking of the multiple microphone arrays.

[0049] The embodiment of the present application discovers the microphone arrays already installed in the conference space by actively broadcasting the configuration terminal, further selects multiple microphone arrays for networking to meet the conference needs by configuring the networking information of the terminal, and configures the master-slave relationship between the multiple microphone arrays by configuring the master-slave device relationship information set in the configuration terminal, thereby realizing the decentralization of multiple microphone arrays and solving the problem of dependence on central devices.

[0050] In some embodiments, the networking information further includes identification information of multiple microphone arrays; wherein, The slave device is further configured to: perform initialization and send slave device identification information to the master device after receiving the ready state information of the master device; The master device is further configured to: perform slave device legitimacy verification on the slave device according to the identification information of the plurality of microphone arrays and the slave device identification information, and mark the slave device that has passed the legitimacy verification as ready; The master device is further configured to: when all the slave devices are marked as ready, the master device notifies the slave devices that networking is completed.

[0051] This embodiment sets the identification information of multiple microphone arrays in the networking information, so that after receiving the identification information of the slave device uploaded by the slave device during the networking process, it can check whether the slave device identification information exists in the identification information of the multiple microphone arrays. If so, it indicates that the slave device is a legal device. If not, it indicates that the slave device is an illegal device, thereby realizing the legitimacy verification of the slave devices participating in the networking.

[0052] In some embodiments, multiple microphone arrays respectively include a communication bus unit, and the initialization of the master device includes: initializing a first data structure required by the master device; publishing a first preset message topic for the slave device to subscribe to the first preset message topic through the communication bus unit of the master device.

[0053] This embodiment implements subscription of messages from the slave microphone array to the master microphone array, thereby enabling synchronous control of multiple networked microphone arrays through the master microphone array.

[0054] In some embodiments, the initialization of the slave device includes: initializing a second data structure required by the slave device; publishing a second preset message topic for the master device to subscribe to the second preset message topic through the communication bus unit of the master device.

[0055] This embodiment implements the subscription of the master microphone array to the messages of the slave microphone array, so that the control instructions can be synchronized to the master microphone array through any slave microphone array, thereby realizing synchronous control of multiple networked microphone arrays.

[0056] In some embodiments, the master device ready state information is sent using a reserved message mechanism. In order to resolve the order of publishing and subscribing, the publisher can publish a message with a reserved feature (i.e., a reserved message). When the publisher publishes this message, the communication bus unit will retain the message (only the most recent message is retained), so that when a new subscriber subscribes to this message, the subscriber can receive the most recent message.

[0057] In some embodiments, the master device is further configured to: after the master device notifies the slave device that networking is complete, start a disconnection monitoring mechanism between the master device and the slave device.

[0058] The microphone array system of the embodiment of the present application can be used to execute the microphone array networking method of the embodiment of the present application, and accordingly achieve the technical effect achieved by the microphone array networking method of the embodiment of the present application, which will not be repeated here. In the embodiment of the present application, the relevant functional modules can be implemented by a hardware processor.

[0059] like Figure 5The figure shows a connection diagram of the configuration software in this application and multiple microphone arrays, including the configuration software, microphone array A, microphone array B and microphone array C. The three microphone arrays are respectively connected to the configuration software so that the configuration software can complete the networking control of the three microphone arrays.

[0060] The configuration software discovers all microphone arrays in the conference space LAN in an agreed manner, connects to each microphone array, and then communicates in an agreed message format. Exemplarily, the following steps are included: 1. Device discovery: that is, discovering all microphone arrays that support networking in the local area network. The microphone array uses passive broadcasting to announce its existence. The configuration software periodically issues active broadcast requests to find the microphone array in the local area network. When the microphone array receives the query broadcast request, it returns its own device information (for example, device ID).

[0061] 2. The configuration software connects the communication bus unit in the corresponding microphone array according to the device information returned by the microphone array.

[0062] 3. When networking devices through the configuration software, the networking information is sent to multiple microphone arrays that need to be networked. The networking information includes the master-slave relationship of the devices and the set of devices in the group.

[0063] The master device (i.e., the master microphone array) can be selected by the configuration personnel in the configuration software. For example, the configuration software defaults the last device added as the master device; or if the various devices (microphone arrays) in the network have different processing performance, the device with better performance can be selected as the master device.

[0064] For example, multiple microphone arrays required for building a certain scene are selected from the discovered microphone arrays through the configuration software to form a group (in the on-site implementation, several arrays that meet the needs of the current conference scene may be selected according to the spatial installation conditions of the ceiling microphone). The above-mentioned group device set is the device ID of all microphone arrays in this group. For example, if devices A / B / C form a group, then these three devices can know each other based on the networking information.

[0065] In some embodiments, a communication bus unit is configured in the microphone array to achieve connection with configuration software and other microphone arrays.

[0066] like Figure 6As shown, it is a schematic diagram of the connection between the microphone arrays and the configuration software in this application. It includes configuration software, device A, device B and device C. In this embodiment, device A is taken as the main device as an example. Device A is provided with a communication bus unit and a networking unit, a receiving arbitration unit and an execution unit connected to the communication bus unit. The configuration software, device B and device C are connected to device A via the communication bus unit.

[0067] Exemplarily, the communication bus unit is responsible for network connection access, connection management, connection activity detection, supports external connection and internal connection, and the configuration software and slave devices in the group are connected to this bus unit. The communication bus unit is also responsible for message routing (i.e., messages from one location to another), supporting one-to-one and one-to-many methods. The communication bus unit is also used to use the publish-subscribe method for message interaction.

[0068] In some embodiments, the communication between the master device and the slave device adopts a publish-subscribe mode. As shown in FIG. 7, a schematic diagram of a one-to-many publish-subscribe mode between the master device and the slave device in the present application is shown. Among them, the master device is the publisher and the slave device is the subscriber. The master device publishes messages through message topics, and the slave device obtains messages from the master device by subscribing to message topics. The message topics published by the master device include master / ready / notify. When a device configures its own role as a master device according to the networking information, after completing the role setting (for example, initializing the data structure required for subsequent decision-making as a master device, that is, the first data structure. The data structure includes, for example, but is not limited to, the timer required for window mechanism decision-making, the linked list for storing dynamic data, etc.), this topic message is published.

[0069] like Figure 8 This is a schematic diagram of the many-to-one publish-subscribe model used between the master device and the slave device in this application. Among them, the master device is the publisher and the slave device is the subscriber. The slave device publishes messages through the message topic, and the master device obtains messages from the slave device by subscribing to the message topic. The message topics published by the slave device include slave / ready / notify. When a device configures its own role as a slave device according to the networking information, it also needs to initialize some data structures required for its own processing (that is, the second data structure. For example, the second data structure includes but is not limited to the type of control message and the structure composed of specific values) and complete the subscription to the master / ready / notify topic. If a message of this topic is received at this time, it is necessary to publish its own continued message through slave / ready / notify. The master device is used to receive the message topic slave / command / notify of all slave device instructions.

[0070] In some embodiments, in order to resolve the order of publishing and subscribing, the publisher can publish a message with a retention feature (i.e., a retained message). When the publisher publishes this message, the communication bus unit will retain the message (only the most recent message), so that when a new subscriber subscribes to this message, the subscriber can receive the most recent message.

[0071] like Figure 9 As shown, it is a schematic diagram of the connection relationship between multiple microphone arrays in this application. It includes device A, device B and device C, and these three devices respectively include a networking unit and a communication bus unit. In this embodiment, device A is taken as the master device, and device B and device C are taken as slave devices. The networking unit of each device is connected to its own communication bus unit respectively. In addition, the networking unit of device B is also connected to the communication bus unit of device A, and the networking unit of device C is also connected to the communication bus unit of device A.

[0072] Each device has a networking unit, which is connected to its own communication bus unit. It conducts networking and monitors the networking status based on the received networking information. The networking process is as follows: 1. The configuration software sends networking information to multiple devices selected for networking (for example, multiple microphone arrays). Multiple devices used for networking refer to all devices that need to form a network. The configuration software can discover all devices in the current LAN. When configuring a scene, select device A / B / C and then send networking information to A / B / C respectively.

[0073] Exemplarily, the networking information includes master-slave relationship information, telling whether it should be a master device or a slave device, and if it is a slave device, it may also include the network address of the master device and the unique device identifier.

[0074] 2. After the networking unit in the master device (for example, device A) completes the necessary initialization operations, it publishes its ready status (retain message), and the communication bus retains this status (one-to-many).

[0075] For the master device, the necessary initialization operation means that the master device needs to subscribe to the slave / ready / notify and slave / command / notify topics of the slave device and some data structures required for decision-making processing as a master device.

[0076] 3. The networking unit of the slave device accesses the communication bus of the master device and informs its own identity information (eg, device ID) upon receiving the ready status of the master device.

[0077] 4. The master device checks the identity legitimacy of the connected slave devices in turn and marks the slave devices as ready. For example, the master device knows which slave devices belong to this group. After connecting to the communication bus unit of the master device, the slave device will send the slave device ID. The master device will check whether the slave device ID is a device in the set to verify the legitimacy of the slave device.

[0078] 5. The master device checks whether all slave devices are ready. If all slave devices in the group are ready, it notifies all slave devices and the networking is completed. The internal monitoring device offline mechanism is also enabled.

[0079] In some embodiments, the master-slave relationship of devices can be dynamically modified as the scenario requires. Thanks to the existence of a communication bus unit in all devices, when a device switches from a master device to a slave device, it only needs to establish a new connection to the target master device.

[0080] For example, a conference room may be relatively large (and support partitions), and sometimes three microphone arrays may be needed to work together for a large meeting. Sometimes, only two microphone arrays may be needed to work together for a small meeting. That is to say, device A may be the master device in scenario 1, but may be the slave device in scenario 2. The switching of device roles can be modified by configuring software or operated by central control (there are many implementation methods, which are not limited in this application).

[0081] In some embodiments, regarding the receiving arbitration unit: the receiving arbitration unit of the master device receives the control instructions of all microphone arrays after networking (many to one) through a specific message topic, and selects the final instruction through a time window mechanism; the time window is divided into a sampling window and a non-sampling window (idle period), and only the instructions within the sampling window will be used for arbitration election. Fig.10 It is a schematic diagram showing the receiving arbitration unit in the present application sampling instructions through a practice window.

[0082] When the receiving arbitration unit of the master device receives the instruction sent by the first device (whether it is the master device or the slave device), the current sampling process starts immediately, and all possible instructions are collected within the set time period (the set time period can be determined based on experience or experiments, and this application does not limit this. For example, the sampling window for remote control signals can be 300ms), and the result arbitration is performed at the end of the sampling window. Different instructions require different arbitration methods.

[0083] For example, for time-sensitive instructions, the first-come-first-served principle can be selected; time-sensitive instructions can be some instructions controlled by infrared, such as volume increase and decrease, mute, sound amplification switch, etc.

[0084] For example, for data that changes frequently, the average value can be used. You only need to define different types of message topics to achieve simultaneous processing of different instructions. For example, each microphone array can output the speaker's position determined by itself. When multiple microphone arrays work at the same time, the speaker's position information needs to be aggregated to select the most accurate speaker position. Here, the decision will be made based on the energy in each speaker's position information.

[0085] In some embodiments, the message subject type is used to receive messages from the slave device and the master device (equivalent to providing a logical channel to transmit messages), and the instruction type is used to classify these messages. For example, message a is volume increase or decrease, and message b is the speaker location information. Different messages are judged by different arbitration units, and the judgment methods are also different.

[0086] In some embodiments, regarding the execution unit: after the receiving arbitration unit of the master device decides the final result, it publishes the result through the control message topic (one-to-many), so that the execution units of all devices can receive the final result and execute it, and finally all devices remain in synchronization.

[0087] In some embodiments, an embodiment of the present application provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the microphone array networking method described in any embodiment of the present application.

[0088] In some embodiments, an embodiment of the present application provides a computer-readable storage medium on which a computer program / instruction is stored, characterized in that when the computer program / instruction is executed by a processor, the steps of the microphone array networking method described in any embodiment of the present application are implemented.

[0089] In some embodiments, an embodiment of the present application provides a computer program product, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the steps of the microphone array networking method described in any embodiment of the present application are implemented.

[0090] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of actions combined, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0091] Fig.11 is a schematic diagram of the hardware structure of an electronic device for executing a microphone array networking method provided by another embodiment of the present application, such as Fig.11 As shown, the device includes: One or more processors 1110 and memory 1120, Fig.11 A processor 1110 is taken as an example.

[0092] The device for executing the microphone array networking method may further include: an input device 1130 and an output device 1140 .

[0093] The processor 1110, the memory 1120, the input device 1130 and the output device 1140 may be connected via a bus or other means. Fig.11 The example of connecting through bus is taken in the following.

[0094] The memory 1120 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the microphone array networking method in the embodiment of the present application. The processor 1110 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 1120, that is, the microphone array networking method in the above method embodiment is implemented.

[0095] The memory 1120 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created according to the use of the microphone array networking device, etc. In addition, the memory 1120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 1120 may optionally include a memory remotely arranged relative to the processor 1110, and these remote memories may be connected to the microphone array networking device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0096] The input device 1130 can receive input digital or character information and generate signals related to user settings and function control of the microphone array networking device. The output device 1140 can include display devices such as a display screen.

[0097] The one or more modules are stored in the memory 1120, and when executed by the one or more processors 1110, the microphone array networking method in any of the above method embodiments is executed.

[0098] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0099] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by their mobile communication functions and their main purpose is to provide voice and data communications. These terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.

[0100] (2) Ultra-mobile personal computer devices: These devices fall into the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, such as iPad.

[0101] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

[0102] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0103] (5) Other electronic devices with data interaction functions.

[0104] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0105] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A microphone array networking method for multiple microphone arrays, the method comprising: In response to the active broadcast of the configuration terminal, the microphone array in the conference space sends feedback information back to the configuration terminal; Determine, according to the networking information sent by the configuration terminal, a plurality of microphone arrays for networking from the microphone arrays in the conference space; wherein the networking information includes master-slave device relationship information, which is used to determine the master device and the slave device in the plurality of microphone arrays; The master device is initialized and sends master device ready status information to the slave device; The slave device is initialized and, after receiving the ready state information of the master device, sends the slave device ready state information to the master device to complete the networking of the multiple microphone arrays.

2. The method according to claim 1, characterized in that The networking information also includes identification information of multiple microphone arrays; the method also includes: The slave device is initialized and sends slave device identification information to the master device after receiving the ready state information of the master device; The master device performs slave device legitimacy verification on the slave device according to the identification information of the plurality of microphone arrays and the slave device identification information, and marks the slave device that passes the legitimacy verification as ready; When the slave devices are all marked as ready, the master device notifies the slave devices that networking is complete.

3. The method according to claim 1, characterized in that The plurality of microphone arrays respectively include a communication bus unit, and the master device initializing includes: Initialize a first data structure required by the master device; A first preset message topic is published, so that the slave device can subscribe to the first preset message topic through the communication bus unit of the master device.

4. The method according to claim 3, characterized in that The slave device initialization includes: Initialize a second data structure required by the slave device; A second preset message topic is published, so that the master device can subscribe to the second preset message topic through the communication bus unit of the master device.

5. The method according to claim 4, characterized in that The master device ready state information is sent using a reserved message mechanism.

6. The method according to claim 1, characterized in that Also includes: After the master device notifies the slave device that networking is complete, a disconnection monitoring mechanism between the master device and the slave device is started.

7. A microphone array system, comprising a plurality of microphone arrays, arranged in a conference space, characterized in that: The microphone array in the conference space is configured to: in response to the active broadcast of the configuration terminal, send feedback information back to the configuration terminal; determine multiple microphone arrays for networking from the microphone arrays in the conference space according to the networking information sent by the configuration terminal; wherein the networking information includes master-slave device relationship information, which is used to determine the master device and the slave device in the multiple microphone arrays; The master device is configured to: initialize and send master device ready status information to the slave device; The slave device is configured to: perform initialization and, after receiving the ready state information of the master device, send the slave device ready state information to the master device, so as to complete the networking of the multiple microphone arrays.

8. The system according to claim 7, characterized in that The networking information also includes identification information of multiple microphone arrays; wherein, The slave device is further configured to: perform initialization and send slave device identification information to the master device after receiving the ready state information of the master device; The master device is further configured to: perform slave device legitimacy verification on the slave device according to the identification information of the plurality of microphone arrays and the slave device identification information, and mark the slave device that has passed the legitimacy verification as ready; The master device is further configured to: when all the slave devices are marked as ready, the master device notifies the slave devices that networking is completed.

9. An electronic device, comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the method described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

11. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

Images