Multi-device instruction awareness systems, methods, and apparatus, ad hoc networking methods and apparatus

Abstract

The application relates to the technical field of Internet of Things, and provides a multi-device instruction sensing system, method and device, and a self-organizing network method and device. The system comprises an edge gateway, intelligent devices joined in an intranet through the edge gateway; the edge gateway is used for determining a master device based on the service functions and hardware configurations of an initialized intelligent device and at least one intelligent device in the case that the service functions and hardware configurations of the initialized intelligent device are sent by the initialized intelligent device; the master device is used for determining a master device and a slave device; the master device is used for determining a target sound pickup device in the case that the master device or any one of the slave devices is woken up by a voice instruction of a user, acquiring audio collected by the target sound pickup device and comprising the voice instruction of the user, and responding to the voice instruction in the audio to execute an operation indicated by the voice instruction or control an execution device corresponding to the voice instruction to execute the operation indicated by the voice instruction.

Description

Technical Field

[0001] This application relates to the field of Internet of Things (IoT) technology, specifically to a multi-device command sensing system, method, and apparatus, and a self-organizing network method and apparatus. Background Technology

[0002] Currently, different brands or series of smart home devices correspond to different control platforms, and the control commands are different and cannot be used interchangeably. Using a single terminal device to control smart home devices of different brands or series requires a large adaptation cost and the command perception effect is poor. Summary of the Invention

[0003] This application provides a multi-device command sensing system, method, and apparatus, as well as a self-organizing network method and apparatus, to solve the technical problem of poor command sensing performance.

[0004] In a first aspect, embodiments of this application provide a multi-device command sensing system, including: an edge gateway, and at least one smart device connected to an intranet through the edge gateway;

[0005] The edge gateway is used to determine the master control device from the initial smart device and the at least one smart device based on the service functions and hardware configuration of the initial smart device and the at least one smart device when it receives the service functions and hardware configuration sent by the initial smart device.

[0006] The master control device is used to determine the master device and slave device from at least one smart device, and / or, in the case of a new smart device added to the intranet, to determine the master device and slave device from the newly added smart device and the at least one smart device;

[0007] The master device is used to determine a target sound pickup device from the master device and the slave device when the master device or any slave device is woken up by the user's voice command, acquire the audio including the user's voice command collected by the target sound pickup device, and execute the operation indicated by the voice command or control the execution device corresponding to the voice command to execute the operation indicated by the voice command in response to the voice command in the audio.

[0008] In one embodiment, the master device is specifically used to send a pickup instruction to a smart device with pickup function in the slave device when the master device or any slave device is woken up by the user's voice command. The pickup instruction is used to instruct the smart device with pickup function to collect the audio segment corresponding to the user's voice command and report the audio quality data of the audio segment.

[0009] The master device is further used to determine the target audio pickup device from the master device and slave devices based on the audio quality data reported by each smart device with audio pickup function.

[0010] In one embodiment, the master device is further configured to: calculate the sound pickup score of each slave device for the user's voice command based on the audio quality data reported by each smart device with sound pickup function, using a discrimination scoring algorithm, and determine the smart device with the highest sound pickup score among the master device and slave devices as the target sound pickup device;

[0011] The discrimination scoring algorithm is expressed as follows:

[0012]

[0013] Where a0 represents the base score for the number of microphones on the current smart device, a n b represents the signal-to-noise ratio weighting factor for the current smart device's audio pickup. n The background noise impact factor is represented by L, which represents the physical distance between the user and the current smart device; x represents the audio signal-to-noise ratio of the current smart device; s represents the background noise impact factor; and j represents the background noise decibel of the current smart device.

[0014] In one embodiment, the edge gateway is also used to periodically determine the master control device from the at least one smart device.

[0015] In one embodiment, the edge gateway is specifically used to calculate the computing power score of each smart device through a computing power algorithm, and to determine the smart device with the highest computing power score as the main control device;

[0016] The computing power scoring algorithm is expressed as follows:

[0017]

[0018] Where x is the computing power of the logic computing chip of the current smart device, a1 is the weighting coefficient of the logic computing chip of the current smart device, y is the computing power of the parallel computing chip of the current smart device, a2 is the weighting coefficient of the parallel computing chip of the current smart device, z is the computing power of the neural network computing chip of the current smart device, and a3 is the weighting coefficient of the neural network computing chip of the current smart device.

[0019] In a second aspect, embodiments of this application provide a multi-device command sensing method based on the multi-device command sensing system described in the first aspect, applied to a master device, the method comprising:

[0020] When any one of the at least one smart devices with a sound pickup function is woken up by a user's voice command, the target sound pickup device is determined.

[0021] Acquire the audio of the user's voice commands collected by the target sound pickup device;

[0022] In response to a voice command in the audio, execute the operation indicated by the voice command or control the execution device corresponding to the voice command to execute the operation indicated by the voice command.

[0023] In one embodiment, determining the target pickup device includes:

[0024] Send a pickup command to the smart device with pickup function in the slave device. The pickup command is used to instruct the smart device with pickup function to collect the audio segment corresponding to the user's voice command and report the audio quality data of the audio segment.

[0025] Based on the audio quality data reported by each smart device with sound pickup function, the target sound pickup device is determined from the master device and the slave device.

[0026] In one embodiment, determining the target audio pickup device from the master device and slave devices based on the audio quality data reported by each smart device with audio pickup capabilities includes:

[0027] Based on the audio quality data reported by each smart device with sound pickup function, the sound pickup score of each slave device for the user's voice command is calculated by a discrimination scoring algorithm, and the smart device with the highest sound pickup score among the master device and slave devices is determined as the target sound pickup device.

[0028] The discrimination scoring algorithm is expressed as follows:

[0029]

[0030] Where a0 represents the base score for the number of microphones on the current smart device, a n b represents the signal-to-noise ratio weighting factor for the current smart device's audio pickup. n The background noise impact factor is represented by L, which represents the physical distance between the user and the current smart device; x represents the audio signal-to-noise ratio of the current smart device; s represents the background noise impact factor; and j represents the background noise decibel of the current smart device.

[0031] Thirdly, embodiments of this application provide a self-organizing network method based on the multi-device command sensing system described in the first aspect, applied to an edge gateway, the method comprising:

[0032] Receive the service functions and hardware configuration sent by the initialized smart device;

[0033] Based on the initialized smart device and the business functions and hardware configuration of at least one smart device, a master control device is determined from the initialized smart device and the at least one smart device.

[0034] In one embodiment, determining the master control device from the initialized smart device and the at least one smart device includes:

[0035] The computing power score of each smart device is calculated using a computing power algorithm, and the smart device with the highest computing power score is identified as the main control device.

[0036] The computing power scoring algorithm is expressed as follows:

[0037]

[0038] Where x is the computing power of the logic computing chip of the current smart device, a1 is the weighting coefficient of the logic computing chip of the current smart device, y is the computing power of the parallel computing chip of the current smart device, a2 is the weighting coefficient of the parallel computing chip of the current smart device, z is the computing power of the neural network computing chip of the current smart device, and a3 is the weighting coefficient of the neural network computing chip of the current smart device.

[0039] In one embodiment, the method further includes:

[0040] The master control device is periodically determined from the at least one smart device.

[0041] Fourthly, embodiments of this application provide a multi-device command sensing device based on the multi-device command sensing system described in the first aspect, the device comprising:

[0042] The first determining module is used to determine the target sound pickup device when any one of the at least one smart devices with sound pickup function is woken up by the user's voice command.

[0043] The audio acquisition module is used to acquire the audio of the user's voice commands collected by the target sound pickup device;

[0044] The instruction response module is used to respond to a voice instruction in the audio, execute the operation indicated by the voice instruction, or control the execution device corresponding to the voice instruction to execute the operation indicated by the voice instruction.

[0045] Fifthly, embodiments of this application provide a self-organizing network device based on the multi-device command sensing system described in the first aspect, the device comprising:

[0046] The first receiving module is used to receive the service functions and hardware configurations sent by the initialized smart device;

[0047] The second determining module is used to determine the master control device from the initialized smart device and the at least one smart device based on the business functions and hardware configuration of the initialized smart device and at least one smart device.

[0048] Sixthly, embodiments of this application provide a master device, including a memory, a transceiver, and a processor;

[0049] A memory for storing computer programs; a transceiver for sending and receiving data under the control of the processor; and a processor for reading the computer programs from the memory and performing the following operations:

[0050] In the case where any one of the smart devices with sound pickup function is woken up by the user's voice command, the target sound pickup device is determined.

[0051] Acquire the audio of the user's voice commands collected by the target sound pickup device;

[0052] In response to a voice command in the audio, execute the operation indicated by the voice command or control the execution device corresponding to the voice command to execute the operation indicated by the voice command.

[0053] In a seventh aspect, embodiments of this application provide an edge gateway, including a memory, a transceiver, and a processor;

[0054] A memory for storing computer programs; a transceiver for sending and receiving data under the control of the processor; and a processor for reading the computer programs from the memory and performing the following operations:

[0055] Receive the service functions and hardware configuration sent by the initialized smart device;

[0056] Based on the initialized smart device and the business functions and hardware configuration of at least one smart device, a master control device is determined from the initialized smart device and the at least one smart device.

[0057] The multi-device command sensing system, method, and apparatus, as well as the self-organizing network method and apparatus provided in this application, realize automated self-organizing networks among smart devices through edge gateways. Users do not need to perform complex network configuration operations. Based on the business functions and hardware configurations of each smart device, a master control device is determined from the connected smart devices to realize internal management and control of the local area network and master-slave device selection. Control of all smart devices can be easily realized within the local area network, effectively improving the user's voice command sensing effect. Moreover, during the self-organizing process, the initialized smart devices can participate in the selection of the master control device by reporting their own business functions and hardware configurations, or realize dynamic capability registration on the master control device. Smart devices that have completed capability registration on the master control device can realize voice command control and various information queries, realize seamless access to multiple types of smart devices, and realize dynamic compatibility of smart devices, enabling collaboration and flexible sensing of multiple smart devices, and improving the command sensing effect. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a schematic diagram of the structure of the multi-device command sensing system provided in the embodiments of this application;

[0060] Figure 2 This is a flowchart illustrating the multi-device command sensing method provided in an embodiment of this application;

[0061] Figure 3 This is a flowchart illustrating the self-organizing network method provided in the embodiments of this application;

[0062] Figure 4 This is the multi-device command sensing device provided in the embodiments of this application;

[0063] Figure 5 This is a schematic diagram of the self-organizing network device provided in the embodiments of this application;

[0064] Figure 6 This is a schematic diagram of the structure of a smart device according to an embodiment of this application;

[0065] Figure 7 This is a schematic diagram of the structure of an edge gateway according to an embodiment of this application. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0067] Figure 1 This is a schematic diagram of the structure of the multi-device command sensing system provided in the embodiments of this application, such as... Figure 1 As shown, the multi-device command sensing system 100 includes: an edge gateway 110, and at least one smart device 120 that has joined the intranet through the edge gateway;

[0068] The edge gateway is used to determine the master control device from the initial smart device and the at least one smart device based on the service functions and hardware configuration of the initial smart device and the at least one smart device when it receives the service functions and hardware configuration sent by the initial smart device.

[0069] The master control device is used to determine the master device and slave device from at least one smart device, and / or, in the case of a new smart device added to the intranet, to determine the master device and slave device from the newly added smart device and the at least one smart device;

[0070] The master device is used to determine a target sound pickup device from the master device and the slave device when the master device or any slave device is woken up by the user's voice command, acquire the audio including the user's voice command collected by the target sound pickup device, and execute the operation indicated by the voice command or control the execution device corresponding to the voice command to execute the operation indicated by the voice command in response to the voice command in the audio.

[0071] Specifically, to overcome the shortcomings of related technologies where intelligent voice devices require network connectivity for device management, even simple device management necessitates cloud access, leading to inconvenience, and the requirement for network configuration for voice interaction among various terminal devices, resulting in significant learning costs for users, this application embodiment utilizes an edge gateway to achieve automated self-organizing networks between intelligent devices. Communication occurs within the local area network, eliminating the need for complex network configuration operations. Furthermore, during user interaction with intelligent devices via voice commands, voice recognition and business logic execution are completed entirely within the local area network environment, without requiring data to leave the network.

[0072] The multi-device instruction sensing system in this application embodiment has the ability to autonomously elect a master control device. By receiving information reported by each of the attached smart devices, it selects a master control device. The smart hardware terminal that meets the requirements of the master control device will be selected as the master control device and used to complete the scheduling of computing resources and the execution of services within the service network.

[0073] The main control equipment must have functions such as resource coordination and sub-equipment management.

[0074] Specifically, to overcome the shortcomings of related technologies where controlling different brands or series of smart home devices with a single terminal device requires significant adaptation costs and has poor command perception, this application embodiment determines a master control device from the connected smart devices based on the business functions and hardware configurations of each smart device. This master control device is used to achieve internal control within the local area network and master / slave device selection, easily enabling control of all smart devices within the local area network and effectively improving the user's voice command perception. Furthermore, during the self-organizing process, the initialized smart devices can participate in the selection of the master control device by reporting their own business functions and hardware configurations, or dynamically register their capabilities on the master control device. Smart devices that have completed capability registration on the master control device can achieve voice command control and various information queries.

[0075] Through the self-organizing network, each intelligent device can report its own business functions and hardware configuration (such as computing power configuration and acoustic configuration) to the edge gateway. Based on the business functions and hardware configuration of each intelligent device, the edge gateway selects the master control device and the other intelligent devices as the controlled devices.

[0076] Optionally, after initialization, the smart device automatically starts Wi-Fi scanning. After scanning for the default Wi-Fi, it forms a self-organizing network and connects to the local area network environment of the edge gateway to realize the networking and interconnection between devices. After the smart device connects to the local area network, it can report its own business functions and hardware configuration to the edge gateway, which will then determine whether to select it as the master control device to realize the internal management and control of the local area network and the selection of master and slave devices.

[0077] Optionally, after the master control device is selected, the newly selected master control device can send task instructions to each controlled device, and each controlled device can report functions according to the standard protocol.

[0078] Optionally, the edge gateway can periodically select the master device, for example, every 2 minutes.

[0079] Optionally, the hardware configuration may include computing power configuration and acoustic configuration, etc.

[0080] Specifically, after the master control device is selected, the newly selected master control device can select master and slave devices.

[0081] Optionally, the multi-device command awareness system supports the dynamic addition of smart terminals to the intranet and can recommend a master device based on the computing power and functionality of newly added smart terminals. This intelligent recommendation method can enhance the computing power reserves of internal network services and accelerate user voice parsing and task execution as much as possible.

[0082] When a smart terminal prepares to join the intranet, it initiates a Wi-Fi scan to determine if a home service intranet exists. After confirming the smart terminal's intention to join the intranet, the master control device determines whether a change is needed. The master device must possess the following characteristics: a microphone pickup component, and CPU and memory meeting the minimum specifications defined for the edge gateway's functions. If the master control device determines a change is necessary, it performs the change. After the change, the master device sends task instructions to the slave devices. The slave devices then report relevant functionalities, including business functions and hardware configuration information, to allow the master device to identify smart devices with microphone pickup capabilities or select a target microphone.

[0083] Optionally, if there are multiple smart devices that meet the main device characteristics requirements (such as having a microphone pickup component, and CPU and memory meeting the minimum indicators corresponding to the edge gateway's defined functions), one of them can be arbitrarily selected as the main device, or the one with the largest memory can be selected as the main device, or the one with the best CPU performance can be selected as the main device, or the one with the best combined memory and CPU performance can be selected as the main device, or the computing power of each smart device that meets the conditions can be calculated (for example, calculated with reference to the computing power scoring algorithm used by the main control device, or a similar scoring algorithm that weights the CPU and memory) and the one with the best computing power can be selected after comparison. This application embodiment does not limit this.

[0084] Optionally, a smart device selected as the master device may also be selected as the master device if it has a microphone pickup component and its CPU and memory meet the minimum requirements corresponding to the edge gateway's defined functions.

[0085] Optionally, the smart device selected as the master device may be excluded from being selected as the master device by default.

[0086] In one embodiment, smart devices need to report their own capabilities when forming a network, and the main control device is selected and managed.

[0087] Specifically, after the smart devices complete the network, they report their own business capabilities and hardware configurations. This can be achieved by broadcasting based on Wi-Fi 6.0 to synchronize information such as smart hardware functions, hardware computing power, and front-end acoustics to the main device and / or the main control device. The main device performs end-side voice recognition, semantic understanding, and task execution, while the main control device selects the main device and manages the local area network.

[0088] For example, taking Table 1 as an example, smart devices need to publish a message with the subject xxx / [di] / deviceCapabilityReport to report hardware functions.

[0089] Publish a message with the subject xxx / [di] / deviceHardwareConfiguration to report hardware configuration. Publish a message with the subject xxx / [di] / deviceAcousticsConfiguration to report front-end acoustic configuration.

[0090] Table 1

[0091]

[0092]

[0093] Specifically, to overcome the shortcomings of related technologies where multiple microphones exist in a user's home space, and users often need to locate a single device for microphone pickup when interacting with smart devices at home, which is extremely difficult and results in a situation where one call can trigger a multitude of responses, and the use of different wake words for differentiation is also rather rigid, making it difficult to achieve multi-terminal collaboration and perception, this application's embodiment, when a smart device with microphone functionality is awakened by a user's voice command, allows the main device to select a target microphone from multiple smart devices and collect the audio of the user's voice command only through the target microphone. Then, in response to the voice command in the audio, the main device executes the operation indicated by the voice command or controls the execution device corresponding to the voice command to execute the operation indicated by the voice command. This avoids the "one call, many responses" problem and achieves multi-terminal collaboration and flexible perception.

[0094] For example, after a device reports its own sound pickup capability to the master device, the master device will assume that any smart device with sound pickup capability can perform sound pickup. This multi-device command perception system allows the master device to select the appropriate target sound pickup device based on factors such as the distance to the speaker (user), the sound pickup signal-to-noise ratio, and the intensity of background noise when multiple smart devices are present. This enables intelligent sound pickup and related suggestions. For instance, if a user instructs via voice to listen to a specific song, the system can suggest a listening location before playing the song.

[0095] For example, if a user's voice command is: "Play the chorus of song A," then the smart device (such as the main device or the target audio pickup device) can output the audio: "Okay, I will play the chorus of song A for you right away; to make it easier for you to listen, please turn down the TV volume and sit near the speakers at the dining table." This informs the user that there is background noise in the current space, which will affect the listening experience, and recommends a better listening position.

[0096] Alternatively, the smart device can be a smart terminal, a smart speaker, or other smart home device.

[0097] To achieve intelligent voice pickup, task execution, and related suggestions, the multi-device command sensing system also includes:

[0098] Offline voice processing middleware is used to convert user commands into audio and text information. During the conversion process, it will prioritize matching based on hot words from various fields uploaded by the user.

[0099] The Natural Language Processing (NLP) module is used to filter sensitive words based on the text information corresponding to the audio of the user's command, then prioritize matching question-answer pairs, and finally perform NLP parsing. The parsed skill domains and parsing results are then transmitted to the smart terminal (such as the main device).

[0100] The device networking module, integrated in various smart devices, is used to initiate networking requests or requests to join an existing service network.

[0101] The device wake-up discrimination module is used to coordinate with various smart terminals. When discerning the sound pickup (i.e., selecting the target sound pickup device), it can coordinate the sound pickup devices based on factors such as the environmental signal-to-noise ratio, sound source distance, and background noise level, so as to give full play to the resources of multiple sound pickup terminals in the collaborative system and ensure the sound pickup effect.

[0102] Optionally, the edge gateway is also used to implement security authentication, key updates, and network connectivity functions between the edge gateway and its downstream devices.

[0103] It should be noted that this solution can be used for smart gateways, routers, home control systems, and other smart devices that support the Andlink protocol, especially smart devices involving network connectivity, to achieve proactive smart device discovery and seamless network configuration.

[0104] This application's embodiments enable device interconnection without the need for a specific router / gateway. The device to be configured interacts with the user through already configured network devices, requiring no specific router / gateway support. Capabilities are provided as components, allowing smart device manufacturers to quickly achieve seamless device configuration through component porting. Cross-brand device interconnection and configuration are supported. A unified access protocol enables unified access, unified accounts, and unified device management. By breaking down access barriers between brands, intelligent interaction based on the Internet of Things is achieved.

[0105] The multi-device command sensing system provided in this application embodiment realizes automated self-organizing network among smart devices through an edge gateway. Users do not need to perform complex network configuration operations. Based on the business functions and hardware configurations of each smart device, a master control device is determined from the connected smart devices to realize internal management and control of the local area network and master-slave device selection. It can easily realize the control of all smart devices within the local area network, effectively improving the user's voice command sensing effect. Moreover, during the self-organizing process, the smart devices that are initialized can participate in the selection of the master control device by reporting their own business functions and hardware configurations, or realize dynamic capability registration on the master control device. The smart devices that have completed capability registration on the master control device can realize voice command control and various information queries, realize seamless access to multiple types of smart devices, realize dynamic compatibility of smart devices, realize the collaboration and flexible sensing of multiple smart devices, and improve the command sensing effect.

[0106] In one embodiment, the master device is specifically used to send a pickup instruction to a smart device with pickup function in the slave device when the master device or any slave device is woken up by the user's voice command. The pickup instruction is used to instruct the smart device with pickup function to collect the audio segment corresponding to the user's voice command and report the audio quality data of the audio segment.

[0107] The master device is further used to determine the target audio pickup device from the master device and slave devices based on the audio quality data reported by each smart device with audio pickup function.

[0108] Specifically, after a smart device reports its voice pickup capability to the master device, the master device assumes that any smart device with voice pickup capability can perform voice pickup. When multiple smart devices with voice pickup capabilities exist on the local area network, the system can first test the voice pickup performance of each device on the user's voice commands, and then select the smart device with the best performance as the target voice pickup device.

[0109] Specifically, when multiple smart devices with audio pickup capabilities exist within a local area network, the master device can send audio pickup commands to these devices. Upon receiving the audio pickup command, the smart device can briefly (e.g., within 2 seconds) collect the audio segment corresponding to the user's voice command and report the audio quality data of the audio segment. The master device then obtains the audio quality data reported by each slave device and determines the target audio pickup device with the best audio quality from among the master device and slave devices.

[0110] It should be noted that the user's position will not shift significantly during a single session. To reduce the computational load on the device side, the main device determines the target microphone based on the aforementioned process N seconds before each conversation begins (e.g., the first 2 seconds or the first 1.5 seconds, which is not limited in this embodiment). This target microphone will be retained in the current session, and the target microphone will be recalculated after the start of a new conversation to ensure the collaborative microphone effect of the system.

[0111] Specifically, when the master device discovers multiple slave devices with audio pickup capabilities within the local area network through the self-organizing network, after the master device or a slave device is woken up (when the user's wake-up voice or visual signal exceeds a threshold, it is considered to be woken up), it can send audio pickup (also known as a wake-up command) to each smart device (slave device) with audio pickup capability via broadcast. Each smart device (master device and / or slave device) with audio pickup capability (master device and / or slave device) briefly collects audio and reports audio quality data. Based on the audio quality data reported by each smart device with audio pickup capability, the master device determines the best audio pickup smart device and identifies the target audio pickup device.

[0112] Optionally, the audio quality data may include detection data such as the distance between the smart device currently picking up the sound and the user, the signal-to-noise ratio, and the background volume of the smart device currently picking up the sound.

[0113] Optionally, when a user interacts with a smart device via voice, the user gives voice instructions, or when a voice device interacts with a smart device via voice, the voice device uploads its own voice stream; when the smart device picks up the sound, it will cancel the echo of the user's voice instructions or the voice stream uploaded by the voice device.

[0114] In one embodiment, the master device is further configured to: calculate the sound pickup score of each slave device for the user's voice command based on the audio quality data reported by each smart device with sound pickup function, using a discrimination scoring algorithm, and determine the smart device with the highest sound pickup score among the master device and slave devices as the target sound pickup device;

[0115] The discrimination scoring algorithm is expressed as follows:

[0116]

[0117] Where a0 represents the base score for the number of microphones on the current smart device, a n b represents the signal-to-noise ratio weighting factor for the current smart device's audio pickup. n The background noise impact factor is represented by L, which represents the physical distance between the user and the current smart device; x represents the audio signal-to-noise ratio of the current smart device; s represents the background noise impact factor; and j represents the background noise decibel of the current smart device.

[0118] Specifically, for each smart device, the discrimination score algorithm is expressed as follows:

[0119]

[0120] Where a0 represents the base score for the number of microphones on the current smart device, a n b represents the signal-to-noise ratio weighting factor for the current smart device's audio pickup. n The background noise impact factor is represented by L, which represents the physical distance between the user and the current smart device; x represents the audio signal-to-noise ratio of the current smart device; s represents the background noise impact factor; and j represents the background noise decibel of the current smart device.

[0121] In this embodiment of the application, by weighting the signal-to-noise ratio and background noise, a discrimination score of the smart device with sound pickup function is obtained. The sound pickup terminal with the highest score is selected as the final sound pickup smart device, that is, as the target sound pickup device to pick up sound in the current session.

[0122] Optionally, the smart device selected as the master device can further participate in the selection of the target sound pickup device together with the slave devices. For example, if the master device's discrimination score is higher than the discrimination scores of all other slave devices, then the master device can be used as the target sound pickup device to pick up sound in the current session.

[0123] In one embodiment, the edge gateway is also used to periodically determine the master control device from the at least one smart device.

[0124] Optionally, the edge gateway can periodically select the master device, for example, every 2 minutes.

[0125] In one embodiment, the edge gateway is specifically used to calculate the computing power score of each smart device through a computing power algorithm, and to determine the smart device with the highest computing power score as the main control device;

[0126] The computing power scoring algorithm is expressed as follows:

[0127]

[0128] Where x is the computing power of the logic computing chip of the current smart device, a1 is the weighting coefficient of the logic computing chip of the current smart device, y is the computing power of the parallel computing chip of the current smart device, a2 is the weighting coefficient of the parallel computing chip of the current smart device, z is the computing power of the neural network computing chip of the current smart device, and a3 is the weighting coefficient of the neural network computing chip of the current smart device.

[0129] Specifically, if the edge gateway finds that multiple smart devices meet the requirements to be the master control device, it can compete for hardware computing power among the multiple smart devices. Based on the computing power score, the smart device with the highest computing power score is selected as the master control device.

[0130] For each smart device, its computing power score algorithm is expressed as follows:

[0131]

[0132] Where x is the computing power of the logic computing chip of the current smart device, a1 is the weighting coefficient of the logic computing chip of the current smart device, y is the computing power of the parallel computing chip of the current smart device, a2 is the weighting coefficient of the parallel computing chip of the current smart device, z is the computing power of the neural network computing chip of the current smart device, and a3 is the weighting coefficient of the neural network computing chip of the current smart device.

[0133] Figure 2 This is a flowchart illustrating the multi-device command sensing method provided in this application embodiment. This method is implemented based on the multi-device command sensing system provided in this application embodiment. Figure 2 As shown, the execution subject of this method is the master device in the current intranet, and the method includes:

[0134] Step 200: If any one of the at least one smart devices with sound pickup function is woken up by the user's voice command, determine the target sound pickup device;

[0135] Step 210: Obtain the audio of the user's voice commands collected by the target sound pickup device;

[0136] Step 220: In response to the voice command in the audio, execute the operation indicated by the voice command or control the execution device corresponding to the voice command to execute the operation indicated by the voice command.

[0137] Specifically, to overcome the shortcomings of related technologies where multiple microphones exist in a user's home space, and users often need to locate a single device for microphone pickup when interacting with smart devices at home, which is extremely difficult and results in a situation where one call can trigger a multitude of responses, and the use of different wake words for differentiation is also rather rigid, making it difficult to achieve multi-terminal collaboration and perception, this application's embodiment, when a smart device with microphone functionality is awakened by a user's voice command, allows the main device to select a target microphone from multiple smart devices and collect the audio of the user's voice command only through the target microphone. Then, in response to the voice command in the audio, the main device executes the operation indicated by the voice command or controls the execution device corresponding to the voice command to execute the operation indicated by the voice command. This avoids the "one call, many responses" problem and achieves multi-terminal collaboration and flexible perception.

[0138] For example, after a device reports its own sound pickup capability to the master device, the master device will assume that any smart device with sound pickup capability can perform sound pickup. This multi-device command perception system allows the master device to select the appropriate target sound pickup device based on factors such as the distance to the speaker (user), the sound pickup signal-to-noise ratio, and the intensity of background noise when multiple smart devices are present. This enables intelligent sound pickup and related suggestions. For instance, if a user instructs via voice to listen to a specific song, the system can suggest a listening location before playing the song.

[0139] For example, if a user's voice command is: "Play the chorus of song A," then the smart device (such as the main device or the target audio pickup device) can output audio: "Okay, I will play the chorus of song A for you right away; to make it easier for you to listen, please turn down the TV volume and sit near the speakers at the dining table."

[0140] This means informing users that there is background noise in the current space, which will affect the listening experience of the song, and recommending a better listening location.

[0141] Alternatively, the smart device can be a smart terminal, a smart speaker, or other smart home device.

[0142] It should be noted that this solution can be used for smart gateways, routers, home control systems, and other smart devices that support the Andlink protocol, especially smart devices involving network connectivity, to achieve proactive smart device discovery and seamless network configuration.

[0143] This application's embodiments enable device interconnection without the need for a specific router / gateway. The device to be configured interacts with the user through already configured network devices, requiring no specific router / gateway support. Capabilities are provided as components, allowing smart device manufacturers to quickly achieve seamless device configuration through component porting. Cross-brand device interconnection and configuration are supported. A unified access protocol enables unified access, unified accounts, and unified device management. By breaking down access barriers between brands, intelligent interaction based on the Internet of Things is achieved.

[0144] The multi-device command perception method provided in this application embodiment realizes automated self-organizing network among smart devices through an edge gateway. Users do not need to perform complex network configuration operations. Based on the business functions and hardware configurations of each smart device, a master control device is determined from the connected smart devices to realize internal management and control of the local area network and master-slave device selection. Control of all smart devices can be easily realized within the local area network, effectively improving the user's voice command perception effect. Moreover, during the self-organizing process, the initialized smart devices can participate in the selection of the master control device by reporting their own business functions and hardware configurations, or realize dynamic capability registration on the master control device. Smart devices that have completed capability registration on the master control device can realize voice command control and various information queries, realize seamless access to multiple types of smart devices, realize dynamic compatibility of smart devices, realize the collaboration and flexible perception of multiple smart devices, and improve the command perception effect.

[0145] In one embodiment, determining the target pickup device includes:

[0146] Send a pickup command to the smart device with pickup function in the slave device. The pickup command is used to instruct the smart device with pickup function to collect the audio segment corresponding to the user's voice command and report the audio quality data of the audio segment.

[0147] Based on the audio quality data reported by each smart device with sound pickup function, the target sound pickup device is determined from the master device and the slave device.

[0148] Specifically, after a smart device reports its voice pickup capability to the master device, the master device assumes that any smart device with voice pickup capability can perform voice pickup. When multiple smart devices with voice pickup capabilities exist on the local area network, the system can first test the voice pickup performance of each device on the user's voice commands, and then select the smart device with the best performance as the target voice pickup device.

[0149] Specifically, when multiple smart devices with audio pickup capabilities exist within a local area network, the master device can send audio pickup commands to these devices. Upon receiving the audio pickup command, the smart device can briefly (e.g., within 2 seconds) collect the audio segment corresponding to the user's voice command and report the audio quality data of the audio segment. The master device then obtains the audio quality data reported by each slave device and determines the target audio pickup device with the best audio quality from among the master device and slave devices.

[0150] It should be noted that the user's position will not shift significantly during a single session. To reduce the computational load on the device side, the main device determines the target microphone based on the aforementioned process N seconds before each conversation begins (e.g., the first 2 seconds or the first 1.5 seconds, which is not limited in this embodiment). This target microphone will be retained in the current session, and the target microphone will be recalculated after the start of a new conversation to ensure the collaborative microphone effect of the system.

[0151] Specifically, when the master device discovers multiple slave devices with audio pickup capabilities within the local area network through the self-organizing network, the master device or a slave device is woken up (when the user's wake-up voice or visual signal exceeds a threshold, it is considered to be woken up). It can then send audio pickup (also known as a wake-up command) to each smart device (slave device) with audio pickup capabilities via broadcast. Each smart device with audio pickup capabilities will briefly collect audio data and report audio quality data. Based on the audio quality data reported by each smart device with audio pickup capabilities, the master device will determine the best audio pickup smart device and identify the target audio pickup device.

[0152] Optionally, the audio quality data may include detection data such as the distance between the smart device currently picking up the sound and the user, the signal-to-noise ratio, and the background volume of the smart device currently picking up the sound.

[0153] Optionally, when a user interacts with a smart device via voice, the user gives voice instructions, or when a voice device interacts with a smart device via voice, the voice device uploads its own voice stream; when the smart device picks up the sound, it will cancel the echo of the user's voice instructions or the voice stream uploaded by the voice device.

[0154] In one embodiment, determining the target audio pickup device from the master device and slave devices based on the audio quality data reported by each smart device with audio pickup capabilities includes:

[0155] Based on the audio quality data reported by each smart device with sound pickup function, the sound pickup score of each slave device for the user's voice command is calculated by a discrimination scoring algorithm, and the smart device with the highest sound pickup score among the master device and slave devices is determined as the target sound pickup device.

[0156] The discrimination scoring algorithm is expressed as follows:

[0157]

[0158] Where a0 represents the base score for the number of microphones on the current smart device, a n b represents the signal-to-noise ratio weighting factor for the current smart device's audio pickup. nThe background noise impact factor is represented by L, which represents the physical distance between the user and the current smart device; x represents the audio signal-to-noise ratio of the current smart device; s represents the background noise impact factor; and j represents the background noise decibel of the current smart device.

[0159] Specifically, for each smart device, the discrimination score algorithm is expressed as follows:

[0160]

[0161] Where a0 represents the base score for the number of microphones on the current smart device, a n b represents the signal-to-noise ratio weighting factor for the current smart device's audio pickup. n The background noise impact factor is represented by L, which represents the physical distance between the user and the current smart device; x represents the audio signal-to-noise ratio of the current smart device; s represents the background noise impact factor; and j represents the background noise decibel of the current smart device.

[0162] In this embodiment of the application, by weighting the signal-to-noise ratio and background noise, a discrimination score of the smart device with sound pickup function is obtained. The sound pickup terminal with the highest score is selected as the final sound pickup smart device, that is, as the target sound pickup device to pick up sound in the current session.

[0163] Figure 3 This is a flowchart illustrating the self-organizing network method provided in this application embodiment. This method is implemented based on the multi-device command sensing system provided in this application embodiment, such as... Figure 3 As shown, the execution entity of this method is an edge gateway, and the method includes:

[0164] Step 300: Receive the service functions and hardware configuration sent by the initialized smart device;

[0165] Step 310: Based on the initialized smart device and the business functions and hardware configuration of at least one smart device, determine the master control device from the initialized smart device and the at least one smart device.

[0166] Specifically, to overcome the shortcomings of related technologies where intelligent voice devices require network connectivity for device management, even simple device management necessitates cloud access, leading to inconvenience, and the requirement for network configuration for voice interaction among various terminal devices, resulting in significant learning costs for users, this application embodiment utilizes an edge gateway to achieve automated self-organizing networks between intelligent devices. Communication occurs within the local area network, eliminating the need for complex network configuration operations. Furthermore, during user interaction with intelligent devices via voice commands, voice recognition and business logic execution are completed entirely within the local area network environment, without requiring data to leave the network.

[0167] The multi-device instruction sensing system in this application embodiment has the ability to autonomously elect a master control device. By receiving information reported by each of the attached smart devices, it selects a master control device. The smart hardware terminal that meets the requirements of the master control device will be selected as the master control device and used to complete the scheduling of computing resources and the execution of services within the service network.

[0168] The main control equipment must have functions such as resource coordination and sub-equipment management.

[0169] Specifically, to overcome the shortcomings of related technologies where controlling different brands or series of smart home devices with a single terminal device requires significant adaptation costs and has poor command perception, this application embodiment determines a master control device from the connected smart devices based on the business functions and hardware configurations of each smart device. This master control device is used to achieve internal control within the local area network and master / slave device selection, easily enabling control of all smart devices within the local area network and effectively improving the user's voice command perception. Furthermore, during the self-organizing process, the initialized smart devices can participate in the selection of the master control device by reporting their own business functions and hardware configurations, or dynamically register their capabilities on the master control device. Smart devices that have completed capability registration on the master control device can achieve voice command control and various information queries.

[0170] Through the self-organizing network, each intelligent device can report its own business functions and hardware configuration (such as computing power configuration and acoustic configuration) to the edge gateway. Based on the business functions and hardware configuration of each intelligent device, the edge gateway selects the master control device and the other intelligent devices as the controlled devices.

[0171] Optionally, after initialization, the smart device automatically starts Wi-Fi scanning. After scanning for the default Wi-Fi, it forms a self-organizing network and connects to the local area network environment of the edge gateway to realize the networking and interconnection between devices. After the smart device connects to the local area network, it can report its own business functions and hardware configuration to the edge gateway, which will then determine whether to select it as the master control device to realize the internal management and control of the local area network and the selection of master and slave devices.

[0172] Optionally, after the master control device is selected, the newly selected master control device can send task instructions to each controlled device, and each controlled device can report functions according to the standard protocol.

[0173] Optionally, the edge gateway can periodically select the master device, for example, every 2 minutes.

[0174] Optionally, the hardware configuration may include computing power configuration and acoustic configuration, etc.

[0175] Specifically, after the master control device is selected, the newly selected master control device can select master and slave devices.

[0176] Optionally, the multi-device command awareness system supports the dynamic addition of smart terminals to the intranet and can recommend a master device based on the computing power and functionality of newly added smart terminals. This intelligent recommendation method can enhance the computing power reserves of internal network services and accelerate user voice parsing and task execution as much as possible.

[0177] When a smart terminal prepares to join the intranet, it initiates a Wi-Fi scan to determine if a home service intranet exists. After confirming the smart terminal's intention to join the intranet, the master control device determines whether a change is needed. The master device must possess the following characteristics: a microphone pickup component, and CPU and memory meeting the minimum specifications defined for the edge gateway's functions. If the master control device determines a change is necessary, it performs the change. After the change, the master device sends task instructions to the slave devices. The slave devices then report relevant functionalities, including business functions and hardware configuration information, to allow the master device to identify smart devices with microphone pickup capabilities or select a target microphone.

[0178] In one embodiment, smart devices need to report their own capabilities when forming a network, and the main control device is selected and managed.

[0179] Specifically, after the smart devices complete the network, they report their own business capabilities and hardware configurations. This can be achieved by broadcasting based on Wi-Fi 6.0 to synchronize information such as smart hardware functions, hardware computing power, and front-end acoustics to the main device and / or the main control device. The main device performs end-side voice recognition, semantic understanding, and task execution, while the main control device selects the main device and manages the local area network.

[0180] For example, taking Table 1 as an example, smart devices need to publish a message with the subject xxx / [di] / deviceCapabilityReport to report hardware functions.

[0181] Publish a message with the subject xxx / [di] / deviceHardwareConfiguration to report hardware configuration. Publish a message with the subject xxx / [di] / deviceAcousticsConfiguration to report front-end acoustic configuration.

[0182] Table 1

[0183]

[0184] The self-organizing network method provided in this application embodiment realizes automated self-organizing networks among smart devices through an edge gateway. Users do not need to perform complex network configuration operations. Based on the business functions and hardware configurations of each smart device, a master control device is determined from the connected smart devices to realize internal management and control of the local area network and master-slave device selection. Control of all smart devices can be easily realized within the local area network, effectively improving the user's voice command perception effect. Moreover, during the self-organizing network process, the initialized smart devices can participate in the selection of the master control device by reporting their own business functions and hardware configurations, or realize dynamic capability registration on the master control device. Smart devices that have completed capability registration on the master control device can realize voice command control and various information queries, realize seamless access to multiple types of smart devices, realize dynamic compatibility of smart devices, realize the collaboration and flexible perception of multiple smart devices, and improve the command perception effect.

[0185] In one embodiment, determining the master control device from the initialized smart device and the at least one smart device includes:

[0186] The computing power score of each smart device is calculated using a computing power algorithm, and the smart device with the highest computing power score is identified as the main control device.

[0187] The computing power scoring algorithm is expressed as follows:

[0188]

[0189] Where x is the computing power of the logic computing chip of the current smart device, a1 is the weighting coefficient of the logic computing chip of the current smart device, y is the computing power of the parallel computing chip of the current smart device, a2 is the weighting coefficient of the parallel computing chip of the current smart device, z is the computing power of the neural network computing chip of the current smart device, and a3 is the weighting coefficient of the neural network computing chip of the current smart device.

[0190] Specifically, if the edge gateway finds that multiple smart devices meet the requirements to be the master control device, it can compete for hardware computing power among the multiple smart devices. Based on the computing power score, the smart device with the highest computing power score is selected as the master control device.

[0191] For each smart device, its computing power score algorithm is expressed as follows:

[0192]

[0193] Where x is the computing power of the logic computing chip of the current smart device, a1 is the weighting coefficient of the logic computing chip of the current smart device, y is the computing power of the parallel computing chip of the current smart device, a2 is the weighting coefficient of the parallel computing chip of the current smart device, z is the computing power of the neural network computing chip of the current smart device, and a3 is the weighting coefficient of the neural network computing chip of the current smart device.

[0194] In one embodiment, the method further includes:

[0195] The master control device is periodically determined from the at least one smart device.

[0196] Optionally, the edge gateway selects a master control device every 2 minutes. The master control device must have functions such as resource coordination and sub-device management.

[0197] The multi-device command sensing device provided in the embodiments of this application is described below. The multi-device command sensing device described below can be referred to in correspondence with the multi-device command sensing method described above.

[0198] Figure 4 This application provides a multi-device command sensing device 400, which is implemented based on the multi-device command sensing system provided in this application. Figure 4 As shown, the device 400 includes:

[0199] The first determining module 410 is used to determine the target sound pickup device when any one of the at least one smart devices with sound pickup function is woken up by a user's voice command.

[0200] The audio acquisition module 420 is used to acquire the audio of the user's voice commands collected by the target sound pickup device;

[0201] The instruction response module 430 is used to respond to a voice instruction in the audio, execute the operation indicated by the voice instruction, or control the execution device corresponding to the voice instruction to execute the operation indicated by the voice instruction.

[0202] It should be noted that the multi-device command sensing device 400 can implement various embodiments of the multi-device command sensing method provided in this application and achieve the same technical effect, which will not be elaborated here.

[0203] The self-organizing network device provided in the embodiments of this application is described below. The self-organizing network device described below can be referred to in correspondence with the self-organizing network method described above.

[0204] Figure 5This is a schematic diagram of the self-organizing network device provided in an embodiment of this application. The device 500 is implemented based on the multi-device command sensing system provided in an embodiment of this application, such as... Figure 5 As shown, the device 500 includes:

[0205] The first receiving module 510 is used to receive the service functions and hardware configuration sent by the initialized smart device;

[0206] The second determining module 520 is used to determine the master control device from the initialized smart device and the at least one smart device based on the business functions and hardware configuration of the initialized smart device and at least one smart device.

[0207] It should be noted that the self-organizing network device 500 can implement various embodiments of the self-organizing network method provided in the embodiments of this application and achieve the same technical effect, which will not be described in detail here.

[0208] The terminal involved in the embodiments of this application may be a device that provides voice and / or data connectivity to a user, a handheld device with wireless connectivity, or other processing devices connected to a wireless modem. The name of the terminal device may differ in different systems; for example, in a 5G system, the terminal device may be called a User Equipment (UE).

[0209] The network device involved in the embodiments of this application can be a base station, which may include multiple cells providing services to terminals. Depending on the specific application, a base station may also be called an access point, or a device in an access network that communicates with wireless terminal devices through one or more sectors on the air interface, or other names.

[0210] Figure 6 This is a schematic diagram of the structure of a smart device according to an embodiment of this application, with reference to... Figure 6 This application also provides a smart device, which may include: a memory 610, a transceiver 620, and a processor 630;

[0211] The memory 610 is used to store computer programs; the transceiver 620 is used to send and receive data under the control of the processor 630; the processor 630 is used to read the computer program in the memory 610 and perform the following operations:

[0212] When any one of the at least one smart devices with a sound pickup function is woken up by a user's voice command, the target sound pickup device is determined.

[0213] Acquire the audio of the user's voice commands collected by the target sound pickup device;

[0214] In response to a voice command in the audio, execute the operation indicated by the voice command or control the execution device corresponding to the voice command to execute the operation indicated by the voice command.

[0215] Among them, Figure 6 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 630) and memory (memory 610). The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 620 can be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium. For different user equipment, the user interface 640 can also be an interface capable of connecting external or internal devices as needed.

[0216] The processor 630 is responsible for managing the bus architecture and general processing, while the memory 610 can store the data used by the processor 630 when performing operations.

[0217] The processor 630 executes any of the methods described in the embodiments of this application according to the obtained executable instructions by calling a computer program stored in the memory 610. The processor and the memory may also be physically separated.

[0218] Figure 7 This is a schematic diagram of the edge gateway according to an embodiment of this application, with reference to... Figure 7 This application embodiment also provides an edge gateway, which may include: a memory 710, a transceiver 720, and a processor 730;

[0219] The memory 710 is used to store computer programs; the transceiver 720 is used to send and receive data under the control of the processor 730; the processor 730 is used to read the computer program in the memory 710 and perform the following operations:

[0220] Receive the service functions and hardware configuration sent by the initialized smart device;

[0221] Based on the initialized smart device and the business functions and hardware configuration of at least one smart device, a master control device is determined from the initialized smart device and the at least one smart device.

[0222] Among them, Figure 7In this context, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 730) and memory (memory 710). The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 720 may be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium. The processor 730 is responsible for managing the bus architecture and general processing, and the memory 710 may store data used by the processor 730 during operation.

[0223] It should be noted that the smart device and edge gateway provided in this application embodiment can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0224] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the steps of the methods provided in the above embodiments, such as:

[0225] When any one of the at least one smart devices with a sound pickup function is woken up by a user's voice command, the target sound pickup device is determined.

[0226] Acquire the audio of the user's voice commands collected by the target sound pickup device;

[0227] In response to a voice command in the audio, execute the operation indicated by the voice command or control the execution device corresponding to the voice command to execute the operation indicated by the voice command.

[0228] or

[0229] Receive the service functions and hardware configuration sent by the initialized smart device;

[0230] Based on the initialized smart device and the business functions and hardware configuration of at least one smart device, a master control device is determined from the initialized smart device and the at least one smart device.

[0231] On the other hand, embodiments of this application also provide a processor-readable storage medium storing a computer program for causing a processor to perform the steps of the methods provided in the above embodiments, such as including:

[0232] When any one of the at least one smart devices with a sound pickup function is woken up by a user's voice command, the target sound pickup device is determined.

[0233] Acquire the audio of the user's voice commands collected by the target sound pickup device;

[0234] In response to a voice command in the audio, execute the operation indicated by the voice command or control the execution device corresponding to the voice command to execute the operation indicated by the voice command.

[0235] or

[0236] Receive the service functions and hardware configuration sent by the initialized smart device;

[0237] Based on the initialized smart device and the business functions and hardware configuration of at least one smart device, a master control device is determined from the initialized smart device and the at least one smart device.

[0238] The processor-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).

[0239] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0240] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0241] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A multi-device command sensing system, characterized in that, include: An edge gateway, through which at least one intelligent device joins the intranet; The edge gateway is used to determine the master control device from the initial smart device and the at least one smart device based on the service functions and hardware configuration of the initial smart device and the at least one smart device when it receives the service functions and hardware configuration sent by the initial smart device. The master control device is used to determine the master device and slave device from at least one smart device, and / or, in the case of a new smart device added to the intranet, to determine the master device and slave device from the newly added smart device and the at least one smart device; The master device is used to determine the target sound pickup device from the master device and the slave device when the master device or any slave device is woken up by the user's voice command, acquire the audio including the user's voice command collected by the target sound pickup device, and execute the operation indicated by the voice command or control the execution device corresponding to the voice command to execute the operation indicated by the voice command in response to the voice command in the audio. The master device is further configured to: based on the audio quality data reported by each smart device with sound pickup function, calculate the signal-to-noise ratio and background noise by weighting the calculation through a discrimination scoring algorithm, obtain the sound pickup score of each slave device for the user's voice command, and determine the smart device with the highest sound pickup score among the master device and slave devices as the target sound pickup device.

2. The multi-device command sensing system according to claim 1, characterized in that, The master device is specifically used to send a pickup command to a smart device with pickup function in the slave device when the master device or any slave device is woken up by the user's voice command. The pickup command is used to instruct the smart device with pickup function to collect the audio segment corresponding to the user's voice command and report the audio quality data of the audio segment. The master device is further used to determine the target audio pickup device from the master device and slave devices based on the audio quality data reported by each smart device with audio pickup function.

3. The multi-device command sensing system according to claim 1, characterized in that, The edge gateway is also used to periodically determine the master control device from the at least one smart device.

4. The multi-device command sensing system according to claim 1 or 3, characterized in that, The edge gateway is specifically used to calculate the computing power score of each smart device through a computing power algorithm, and to determine the smart device with the highest computing power score as the main control device; The computing power scoring algorithm is expressed as follows: ； in, The computing power of the logic computing chips in current smart devices, These are the weighting coefficients for the logic computing chips in current smart devices. The computing power of parallel computing chips in current smart devices, These are the weighting coefficients for the parallel computing chips in current smart devices. The computing power of neural network computing chips in current smart devices, These are the weighting coefficients for the neural network computing chips in current smart devices.

5. A multi-device command sensing method based on the multi-device command sensing system according to any one of claims 1-4, characterized in that, Applied to a master device, the method includes: When any one of the at least one smart devices with a sound pickup function is woken up by a user's voice command, the target sound pickup device is determined. Acquire the audio of the user's voice commands collected by the target sound pickup device; In response to a voice command in the audio, execute the operation indicated by the voice command or control the execution device corresponding to the voice command to execute the operation indicated by the voice command.

6. The multi-device command sensing method according to claim 5, characterized in that, The target pickup device includes: Send a pickup command to the smart device with pickup function in the slave device. The pickup command is used to instruct the smart device with pickup function to collect the audio segment corresponding to the user's voice command and report the audio quality data of the audio segment. Based on the audio quality data reported by each smart device with sound pickup function, the target sound pickup device is determined from the master device and the slave device.

7. A self-organizing network method based on the multi-device command sensing system according to any one of claims 1-4, characterized in that, Applied to an edge gateway, the method includes: Receive the service functions and hardware configuration sent by the initialized smart device; Based on the initialized smart device and the business functions and hardware configuration of at least one smart device, a master control device is determined from the initialized smart device and the at least one smart device.

8. The self-organizing network method according to claim 7, characterized in that, Determining the master control device from the initialized smart device and the at least one smart device includes: The computing power score of each smart device is calculated using a computing power algorithm, and the smart device with the highest computing power score is identified as the main control device. The computing power scoring algorithm is expressed as follows: ； in, The computing power of the logic computing chips in current smart devices, These are the weighting coefficients for the logic computing chips in current smart devices. The computing power of parallel computing chips in current smart devices, These are the weighting coefficients for the parallel computing chips in current smart devices. The computing power of neural network computing chips in current smart devices, These are the weighting coefficients for the neural network computing chips in current smart devices.

9. The self-organizing network method according to claim 7, characterized in that, The method further includes: The master control device is periodically determined from the at least one smart device.

10. A multi-device command sensing device based on the multi-device command sensing system according to any one of claims 1-4, characterized in that, The device includes: The first determining module is used to determine the target sound pickup device when any one of the at least one smart devices with sound pickup function is woken up by the user's voice command. The audio acquisition module is used to acquire the audio of the user's voice commands collected by the target sound pickup device; The instruction response module is used to respond to a voice instruction in the audio, execute the operation indicated by the voice instruction, or control the execution device corresponding to the voice instruction to execute the operation indicated by the voice instruction.

11. A self-organizing network device based on the multi-device command sensing system according to any one of claims 1-4, characterized in that, The device includes: The first receiving module is used to receive the service functions and hardware configurations sent by the initialized smart device; The second determining module is used to determine the master control device from the initialized smart device and the at least one smart device based on the business functions and hardware configuration of the initialized smart device and at least one smart device.

12. A master device based on the multi-device command sensing system according to any one of claims 1-4, characterized in that, Includes memory, transceiver, and processor; A memory for storing computer programs; a transceiver for sending and receiving data under the control of the processor; and a processor for reading the computer programs from the memory and performing the following operations: In the case where any one of the smart devices with sound pickup function is woken up by the user's voice command, the target sound pickup device is determined. Acquire the audio of the user's voice commands collected by the target sound pickup device; In response to a voice command in the audio, execute the operation indicated by the voice command or control the execution device corresponding to the voice command to execute the operation indicated by the voice command; Based on the audio quality data reported by each smart device with sound pickup function, a discrimination scoring algorithm is used to calculate the signal-to-noise ratio and background noise weighted by the algorithm. The sound pickup score of each slave device for the user's voice command is calculated, and the smart device with the highest sound pickup score among the master device and slave devices is determined as the target sound pickup device.

13. An edge gateway based on the multi-device command sensing system according to any one of claims 1-4, characterized in that, Includes memory, transceiver, and processor; A memory for storing computer programs; a transceiver for sending and receiving data under the control of the processor; and a processor for reading the computer programs from the memory and performing the following operations: Receive the service functions and hardware configuration sent by the initialized smart device; Based on the initialized smart device and the business functions and hardware configuration of at least one smart device, a master control device is determined from the initialized smart device and the at least one smart device; Based on the audio quality data reported by each smart device with sound pickup function, a discrimination scoring algorithm is used to calculate the signal-to-noise ratio and background noise weighted by the algorithm. The sound pickup score of each slave device for the user's voice command is calculated, and the smart device with the highest sound pickup score among the master device and slave devices is determined as the target sound pickup device.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the multi-device instruction sensing method according to any one of claims 5 to 6, or the steps of the self-organizing network method according to any one of claims 7 to 9.

Images