Off-line voice cross-device control method, device, equipment, medium and product

Abstract

The invention relates to the technical field of off-line interaction, and provides an off-line voice cross-device control method and device, equipment, a medium and a product, the method is applied to voice equipment, and the method specifically comprises the following steps: in an off-line state, obtaining a user voice instruction; a controlled device and a control instruction are determined according to the user voice instruction; the control instruction is sent to the controlled device, and an execution result is fed back to the user according to the state of the controlled device within the preset time; through the above mode, on the premise that the equipment is not networked, cross-machine control of other equipment can be realized through offline voice recognition and offline communication technologies, more application scenes are adapted, and more convenience and security are provided for users.

Description

Technical Field

[0001] The present application relates to the field of offline interaction technology, and in particular to a method, apparatus, device, medium and product for cross-device control of offline voice. Background Art

[0002] In order to better serve users, smart home can provide users with the service of Internet of Everything, which means connecting various home devices together through the Internet to achieve interconnection and intelligent management between devices. In addition, users can also issue control instructions to smart home devices through voice commands, which is more in line with user habits.

[0003] However, the Internet of Everything requires that smart home devices be connected to the Internet, and voice commands also need to be recognized online, which limits the application scenarios. Devices cannot provide services to users when they are offline, which reduces convenience and security. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes an offline voice cross-device control method, which can realize cross-device control of other devices through offline voice recognition and offline communication technology.

[0005] The application also proposes an offline voice cross-device control device, an electronic device, a storage medium and a program product.

[0006] According to the offline voice cross-device control method of the first aspect embodiment of the present application, it includes: obtaining user voice commands in an offline state; determining the controlled device and control commands according to the user voice commands; sending the control commands to the controlled device, and feeding back the execution results to the user according to the state of the controlled device within a preset time.

[0007] According to the offline voice cross-device control method of the embodiment of the present application, user voice commands can be obtained through offline voice recognition technology, and control commands and feedback execution results can be transmitted between devices through offline communication technology. It is not affected by the network and can reduce dependence on external services; enhance the reliability and adaptability of device interaction.

[0008] According to an embodiment of the present application, sending a control instruction to a controlled device includes: sending the control instruction to the controlled device through a preset management frame of broadcast information.

[0009] According to one embodiment of the present application, the preset management frame includes a frame header and an information element; the frame header includes basic information and a control field of the frame, which is used to indicate the sending and receiving of the frame; the information element includes configuration information, which is used to implement function configuration and parameter configuration.

[0010] According to an embodiment of the present application, sending a control instruction to a controlled device includes: sending the control instruction to the controlled device via a direct communication method.

[0011] According to one embodiment of the present application, the execution result is fed back to the user according to the state of the controlled device within a preset time, including: if a reply message from the controlled device is received within the preset time, and it is determined according to the state of the controlled device that the execution of the control instruction is completed, the result of successful execution is fed back to the user; if a reply message from the controlled device is not received within the preset time, or it is determined according to the state of the controlled device that the control instruction cannot be executed, the result of failed execution is fed back to the user.

[0012] According to one embodiment of the present application, the preset time is dynamically updated.

[0013] According to one embodiment of the present application, before sending a control instruction to a controlled device and feeding back the execution result to a user based on the state of the controlled device within a preset time, it also includes: obtaining the loop time of the last communication with the controlled device; determining the allocation weight of the loop time; and determining the preset time this time by weighted averaging based on the loop time and the allocation weight.

[0014] According to one embodiment of the present application, a controlled device and a control instruction are determined according to a user voice instruction, including: inputting the user voice instruction into an offline voice recognition model to obtain the controlled device and the control instruction output by the offline voice recognition model; wherein the offline voice recognition model is pre-set with a mapping relationship between the user voice and the intention representation, and the offline voice recognition model is used to output the controlled device and the control instruction corresponding to the user voice instruction according to the mapping relationship.

[0015] According to an embodiment of the present application, the controlled device is a non-voice device.

[0016] According to the offline voice cross-device control method of the second aspect of the embodiment of the present application, applied to non-voice devices, the offline voice cross-device control method includes: receiving a control instruction sent by a voice device; wherein the control instruction is obtained by the voice device in an offline state according to the user's voice instruction; executing a function according to the control instruction, and feeding back the local status to the voice device within a preset time.

[0017] According to the offline voice cross-device control device of the third aspect embodiment of the present application, it is deployed on a voice device, and the offline voice cross-device control device includes: a voice acquisition module, which is used to acquire user voice commands in an offline state; an instruction analysis module, which is used to determine the controlled device and control instructions according to the user voice instructions; a transmission module, which is used to send the control instructions to the controlled device and feedback the execution result to the user according to the state of the controlled device within a preset time.

[0018] According to the offline voice cross-device control system of the fourth aspect of the present application, it includes a voice device and a controlled device; the voice device is used to obtain user voice commands in an offline state, determine the controlled device and control commands according to the user voice commands, and send the control commands to the controlled device; and feedback the execution results to the user according to the state of the controlled device within a preset time; the controlled device is used to receive the control commands sent by the voice device, execute functions according to the control commands, and feedback the local state to the voice device within a preset time.

[0019] According to the electronic device of the fifth aspect embodiment of the present application, it includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned offline voice cross-device control methods.

[0020] According to the non-transitory computer-readable storage medium of the sixth aspect embodiment of the present application, a computer program is stored thereon, and when the computer program is executed by a processor, it implements any of the above-mentioned cross-device control methods for offline voice.

[0021] According to the computer program product of the seventh aspect embodiment of the present application, it includes a computer program, and when the computer program is executed by a processor, it implements any of the above-mentioned offline voice cross-device control methods.

[0022] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects: The user's voice commands are obtained through offline voice recognition technology, which is not affected by the network and reduces the dependence on external services. In addition, the voice device and the controlled device transmit control commands and feedback execution results through offline communication technology, which enhances reliability and adaptability. Furthermore, the preset time for receiving feedback information is dynamically updated, which can adapt to more application scenarios; Furthermore, offline voice recognition and offline device communication can process information data locally on the device without going through the server, reducing the risk of data being intercepted or leaked and protecting user privacy. Furthermore, offline device communication includes broadcast communication and direct communication, both of which can simplify network configuration, facilitate deployment, and achieve faster data transmission and response speed; Furthermore, the preset time can be dynamically updated according to the loop time of the last communication to achieve dynamic adjustment of the threshold, which can significantly improve the flexibility, performance and user experience of the system.

[0023] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 This is one of the flow charts of the offline voice cross-device control method provided in the embodiment of the present application.

[0026] Figure 2 It is a data format diagram of the device control protocol provided in the embodiment of the present application.

[0027] Figure 3 It is a schematic diagram of the data structure of the 802.11 management frame provided in an embodiment of the present application.

[0028] Figure 4 It is a flowchart of the broadcast communication provided in the embodiment of the present application.

[0029] Figure 5 It is an interactive schematic diagram of broadcast communication provided in an embodiment of the present application.

[0030] Figure 6 It is a flow chart of the direct communication provided in the embodiment of the present application.

[0031] Figure 7 It is an interactive schematic diagram of the direct communication provided in an embodiment of the present application.

[0032] Figure 8 It is a structural diagram of the cross-device control device for offline voice provided in an embodiment of the present application.

[0033] Fig. 9 This is the second flow chart of the offline voice cross-device control method provided in the embodiment of the present application.

[0034] Fig.10 It is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The following is a further detailed description of the implementation of the present application in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but cannot be used to limit the scope of the present application.

[0036] In the description of the embodiments of the present application, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the embodiments of the present application. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0037] In the description of the embodiments of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0038] In the embodiments of the present application, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0039] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0040] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located, and with the authorization given by the owner of the corresponding device.

[0041] In related technologies, after the intelligent voice device is connected to the Internet, it can control other non-voice devices through online voice recognition and the Internet, realizing cross-machine control. The premise of cross-machine control is that the voice device and the non-voice device are connected to the Internet, and the voice device and the non-voice device can communicate with each other through server forwarding.

[0042] The premise for realizing cross-machine communication between intelligent voice devices and non-voice devices is that all devices are first connected to a router, access the Internet, and communicate through cloud services. When a voice device (such as a voice air conditioner) transmits the conversation audio to the cloud service, the cloud service performs voice recognition and determines other devices (such as washing machines) that need to be controlled based on the intent (such as turning off the washing machine). The corresponding intent "turn off the washing machine" is sent to the cloud service, and the cloud service generates a control instruction and sends it to the non-voice device that needs to be controlled. The non-voice device receives the corresponding control instruction and executes it, realizing cross-machine control.

[0043] Another implementation is to control through the LAN, but the control range is limited to the same LAN. Devices outside the LAN and devices not connected to the network cannot be controlled. For example, voice device A can control non-voice device B in the same LAN, but cannot control non-voice device C and non-voice device D.

[0044] Therefore, in the related technologies, networking is required to realize cross-device control, which reduces convenience and safety. Based on this, the embodiment of the present application provides an offline voice cross-device control method, which can realize cross-device control of other devices through offline voice recognition and offline communication technology.

[0045] See also Figure 1 , Figure 1 This is one of the flow charts of the offline voice cross-device control method provided in the embodiment of the present application. In this embodiment, the offline voice cross-device control method may include steps 110 to 130, and each step is specifically as follows: Step 110: In an offline state, obtain user voice commands.

[0046] The offline voice cross-device control method of this embodiment can be applied to intelligent voice devices, which refer to intelligent devices with integrated voice acquisition modules, including but not limited to smart phones, smart tablets, laptops, smart car systems, smart home devices, smart wearable devices, etc.

[0047] Among them, smart home devices include but are not limited to smart speakers, smart light bulbs, smart sockets, smart refrigerators, smart air conditioners, smart kitchen equipment, smart washing machines and dryers and other home appliances.

[0048] In this embodiment, the user's voice can be acquired through the voice acquisition module, which can convert the sound signal into an electrical signal and perform preliminary processing for subsequent voice recognition. The voice acquisition module can be a microphone array.

[0049] Step 120: Determine the controlled device and the control instruction according to the user voice instruction.

[0050] In this embodiment, offline voice recognition is adopted. Specifically, an offline voice recognition model can be built into the intelligent voice device, and the offline voice recognition model can realize offline natural language understanding. The instruction word set in the offline voice recognition model can be expanded from hundreds of words to hundreds of thousands of words, so that the user can freely speak the control instruction.

[0051] In some embodiments, the user voice command may be input into an offline voice recognition model to obtain a controlled device and a control command output by the offline voice recognition model.

[0052] Among them, the offline speech recognition model is pre-set with a mapping relationship between the user's speech and the intention expression, and the offline speech recognition model is used to output the controlled device and the control instruction corresponding to the user's voice instruction according to the mapping relationship.

[0053] It can be understood that the offline speech recognition model is a type of speech recognition that runs locally on the device and can convert the user's voice into text or execute specific commands without a network connection. The offline speech recognition model can use the voice acquisition module to collect signals and capture the user's voice signal through a microphone. The user's voice signal is then preprocessed, such as noise removal, echo cancellation, noise reduction, etc., to improve the quality of the voice signal. At the same time, sampling and quantization are performed to convert the continuous analog signal into a discrete digital signal. After that, feature extraction is performed to convert the voice signal into a representative feature vector, which can capture key information in the voice signal, such as pitch, timbre, and syllables. Finally, semantic and intention skinning is performed to match the extracted feature vector with the words in the predefined dictionary to obtain the corresponding intention representation, so as to obtain the controlled device and control instructions according to the intention.

[0054] In this embodiment, offline voice recognition model can realize offline voice intent mapping. Through offline voice intent mapping, the natural language processing (NLP) technology implemented on the local device allows the device to understand and execute the user's voice instructions without a network connection. Therefore, the intelligent voice device of this embodiment can achieve fast recognition response, and the data is processed locally without uploading to the cloud, which enhances the protection of user privacy and is suitable for a variety of application scenarios.

[0055] In some embodiments, the offline speech recognition model can support hundreds of thousands of offline voice commands. During the training phase of the offline speech recognition model, the output of ASR (Automatic Speech Recognition) can be divided into multiple semantic slots to form a mapping between ASR and intent and device, such as the following format, which includes ASR information, intent, and control device: { "intent": "close", "asr": "Turn off the washing machine / don't want to wash clothes anymore", "device": "washing_machine", } During the model training or annotation stage, the offline speech algorithm undergoes a large amount of corpus collection and template training, which can improve the accuracy and adaptability of recognition, thereby achieving the above mapping.

[0056] Step 130: Send the control instruction to the controlled device, and feedback the execution result to the user according to the state of the controlled device within a preset time.

[0057] In this embodiment, after the controlled device and the control instruction are determined, the intelligent voice device can send the control instruction to the controlled device through a communication method without networking.

[0058] The controlled device is a device that can have a receiving module and can perform a specific function. In some embodiments, the controlled device can also be an intelligent device, a voice device, or an intelligent voice device.

[0059] When the intelligent voice device sends a control command, it starts timing at the same time. The purpose of timing is to detect the state of the controlled device within the preset time and feedback the execution result of the command to the user based on the state of the controlled device.

[0060] Optionally, the execution results generally include two: successful execution and failed execution. The feedback of the execution results can be designed according to the actual situation of the intelligent voice device. Optionally, the form of feedback execution results includes but is not limited to voice broadcast, indicator light display and / or screen display light.

[0061] It should be noted that in order for the controlled device to parse the control instructions sent by the intelligent voice device, the intelligent voice device and the controlled device need to implement the control and parsing results through an agreed device control protocol.

[0062] Please refer to Figure 2 , Figure 2 It is a data format diagram of the device control protocol provided in the embodiment of the present application.

[0063] The data structure of the device control protocol may generally include header information (Header), content information (Data) and other information. The header information may further include MSG Label, Device Type, MSG Type and MSG Length, and the content information may further include MSG Body.

[0064] Specifically, MSG Label is used to distinguish whether it is a protocol, such as "DBCA". Device Type indicates the device type, such as 0D for a washing machine. MSG Type indicates the message type, such as 1A for control and 2A for control result. MSGBody indicates the message body, which is the part that carries the actual data in the information transmission. MSG Length indicates the message length, that is, the length of MSGBody. For example, if the content of MSG Body is "010203", the content of MSG Length is "03".

[0065] The information may include verification information, such as CRC. CRC check is used to ensure data consistency, and can be calculated using algorithms such as CRC16. For example, the CRC value calculated by "DBCA0D1A03010203" is 2E96. The control result of the controlled device is passed through the instruction DBCA0D2A03010204, indicating that the shutdown is successful.

[0066] Through the above method, this embodiment can obtain user voice commands through offline voice recognition technology, which is not affected by the network and reduces dependence on external services; the voice device and the controlled device can also transmit control commands and feedback execution results through offline communication technology to enhance reliability and adaptability.

[0067] Before the intelligent voice device sends a control command to the controlled device, it first needs to discover the controlled device without being connected to the Internet, establish an offline communication connection with the controlled device, and then exchange data through a communication protocol.

[0068] Based on any of the above embodiments, the step of sending the control instruction to the controlled device may specifically include: Step 130 - 1 : Send the control instruction to the controlled device via a preset management frame of the broadcast information.

[0069] In this embodiment, the intelligent voice device and the controlled device can perform broadcast communication by means of preset management frames.

[0070] Specifically, a management frame refers to a special frame used to manage and maintain the connection and communication between network devices in a network communication protocol. In this embodiment, a preset management frame used to transmit control instructions is obtained by specially designing the management frame.

[0071] Optionally, the preset management frame may include a frame header and information elements, wherein the frame header (MAC Header) may include basic information and a control field of the frame, and the frame header may be used to indicate the sending and receiving of the frame.

[0072] Information elements may include configuration information, and information elements may be used to implement function configuration and parameter configuration.

[0073] In some embodiments, the frame header may include a frame control field, a duration / identifier, an address field, and a sequence control field, among other things.

[0074] In some embodiments, the information element may include the type of the information element, the length of the field, specific data related to the type of the information element, and the like.

[0075] Exemplarily, the broadcast information may be sent through 802.11 management frames (probe, action frames, etc.), thereby enabling direct communication between devices.

[0076] See also Figure 3 , Figure 3 It is a schematic diagram of the data structure of the 802.11 management frame provided in an embodiment of the present application.

[0077] The 802.11 management frame may include a MAC Header, Elements, and FCS. The MAC Header may further include Frame Control, Duration, DA, SA, BSS ID, and Sequence Control. The Elements may include a Frame Body.

[0078] The Frame Body area can be customized to send data information. For example, the command "DBCA0D2A03010204" to control a washing machine is broadcast to a nearby controlled device (such as a washing machine). The controlled device is in a listening state. After receiving the broadcast information, it parses the command according to the protocol to confirm whether a response command is required and the content of the response command. After executing the command, the command execution result is returned to the intelligent voice device. The intelligent voice device receives the broadcast command of the controlled device and parses the command. Other nearby devices can receive related commands, but do not need to process them.

[0079] See also Figure 4-Figure 5 , Figure 4 is a flowchart of broadcast communication provided by an embodiment of the present application, The voice device generates a control instruction according to the text recognition result and intention and the agreed home appliance control protocol; the voice device broadcasts the control instruction through the 802.11 management frame on channel X, receives the broadcast data, and starts timing; the non-voice device (i.e., the controlled device mentioned above) receives the 802.11 management frame on channel X and parses the control instruction according to the home appliance control protocol; the non-voice device that needs to execute the control instruction broadcasts the instruction execution result to the voice device through the 802.11 management frame; the voice device receives the instruction executed by the non-voice device within the timeout period (i.e., the preset time mentioned above), and broadcasts the corresponding audio according to the execution status; if the voice device does not receive the instruction executed by the non-voice device within the timeout period, it broadcasts the timeout prompt audio.

[0080] See also Figure 5 , Figure 5 It is an interactive schematic diagram of broadcast communication provided in an embodiment of the present application.

[0081] The interactive subjects involved in this embodiment include voice device A, non-voice device B and non-voice device C. Voice device A sends a command to control device B by broadcasting. At this time, both non-voice device B and non-voice device C will receive the control command, but only non-voice device B will command the command and return the control result B. Non-voice device C can also receive the control command, but does not respond.

[0082] Similarly, voice device A sends a command to control device C via broadcast. At this time, non-voice device B and non-voice device C will receive the control command, but only non-voice device C will execute the command and return the control result C. Non-voice device B can also receive the control command, but does not respond.

[0083] It should be noted that the broadcast communication involved in the above embodiments may be in the form of WiFi broadcasting, but does not require connection to a router and the Internet. It only uses WiFi technology to achieve point-to-point connection between devices.

[0084] Based on any of the above embodiments, the step of sending the control instruction to the controlled device may specifically include: Step 130 - 2 : Send the control instruction to the controlled device via direct communication.

[0085] See also Figure 6 , Figure 6 It is a flow chart of the direct communication provided in the embodiment of the present application.

[0086] In this embodiment, direct communication between devices can be achieved through WiFi direct connection (WiFi direct).

[0087] The voice device generates control instructions according to the text recognition results and intentions and the agreed home appliance control protocol; the voice device sends the control instructions to the device to be controlled through direct communication and starts timing; the controlled device (non-voice device) receives the control instructions through direct communication and parses the control instructions according to the home appliance control protocol; the non-voice device that needs to execute the control instructions sends the instruction execution results to the voice device through direct communication; if the voice device receives the instructions executed by the non-voice device within the timeout period, it will broadcast the corresponding audio according to the execution status; if the voice device does not receive the instructions executed by the non-voice device within the timeout period, it will broadcast the timeout prompt audio.

[0088] Among them, the voice device acts as the Group Owner (GO) and the non-voice device acts as the Group Client (GC) to realize a one-to-many networking mode. For example, the softAP+STA mode can be adopted, that is, the voice device GO works in the softAP mode, and the non-voice device GC works in the STA mode. All voice devices are connected to the voice device to realize a one-to-many communication network.

[0089] SoftAP+STA mode refers to an operating mode of a WiFi device, in which the device acts as a wireless access point (SoftAccessPoint, SoftAP) and a station (Station, STA) at the same time. This mode enables the device to be part of the network (STA mode) and provide network access services (SoftAP mode) at the same time.

[0090] See also Figure 7 , Figure 7 It is an interactive schematic diagram of the direct communication provided in an embodiment of the present application.

[0091] The interaction subjects involved in this embodiment include voice device A, non-voice device B and non-voice device C. Voice device A uses softAP mode as GO, and non-voice device B and non-voice device C use STA mode as GC.

[0092] Specifically, when it is necessary to control non-voice device B, voice device A sends a command to control device B to non-voice device B. After non-voice device B executes the command, it feeds back control result B to voice device A. When it is necessary to control non-voice device C, voice device A sends a command to control device C to non-voice device C. After non-voice device C executes the command, it feeds back control result C to voice device A.

[0093] Through the above-mentioned embodiments, the offline voice recognition and offline device communication of the present application can process information data locally on the device without going through the server, reducing the risk of data interception or leakage and protecting user privacy; and offline device communication includes broadcast communication and direct communication, both of which can simplify network configuration, facilitate deployment, and achieve faster data transmission and response speed; the voice devices and controlled devices in the present application do not need to be connected to the Internet, so that the voice device can control the devices in the whole house and realize cross-machine control.

[0094] Based on any of the above embodiments, the step of feeding back the execution result to the user according to the state of the controlled device within a preset time may specifically include: Step 130 - 3 : If the response information of the controlled device is received within the preset time, and it is determined that the execution of the control instruction is completed according to the state of the controlled device, the result of successful execution is fed back to the user.

[0095] Step 130 - 4 : If no reply information is received from the controlled device within the preset time, or if it is determined based on the state of the controlled device that the control instruction cannot be executed, a result of execution failure is fed back to the user.

[0096] In this embodiment, the intelligent voice device can also monitor the state of the controlled device within a preset time to determine the execution status of the control instruction and feedback the execution result to the user.

[0097] In some embodiments, the preset time is usually a preset fixed value, and in order to cover more scenarios, it is set to a larger value. In this embodiment, a method for dynamically updating the preset time is proposed, that is, the preset time in the embodiment of the present application can be dynamically updated. By dynamically updating the preset time, more application scenarios can be adapted.

[0098] Based on any of the above embodiments, the steps before sending the control instruction to the controlled device and feeding back the execution result to the user according to the state of the controlled device within a preset time may further include: Step 130-01: Obtain the loop time of the last communication with the controlled device.

[0099] Step 130 - 02 : Determine the allocation weight of the loop time.

[0100] Step 130-03: Determine the preset time for this time by taking a weighted average according to the loop time and the allocation weight.

[0101] In this embodiment, the preset time can be dynamically updated in a weighted average manner according to the loop time of the last communication with the controlled device.

[0102] All devices set an initial preset time T. After the voice device sends a broadcast message, the voice device receives replies from other devices and dynamically updates the preset time T based on the loop time.

[0103] Specifically, assuming that the initial timeout is T0, the voice device A sends a broadcast message, the controlled device B receives the broadcast message and replies to the voice device A, and the time from the voice device A sending the message to receiving the reply from the controlled device B is RTT0, then the dynamic update method is as follows: T0=RTT0; T1=(1-α) RTT0+α RTT1; RTD0=T0 / 2; RTD1=(1-β) RTD0+β (RTT1-RTT0); RTO0=T0+RTD0 / 2; Among them, RTD (Round-Trip Delay) indicates the total time required to send data from the sender to the receiver and then return from the receiver to the sender. RTD is usually used to measure the network communication delay from one node to another. RTO (Retransmission TimeOut) refers to the time the sender waits for the receiver to confirm the information after sending a data packet. If the confirmation information is not received within RTO, the sender will consider the data packet lost and resend it. α and β are pre-set weighting parameters. RTT0 and RTD0 indicate the time of the last information transmission, RTT1 and RTD1 indicate the time of this information transmission, RTO0 indicates the retransmission timeout, and T1 indicates the preset time for this time.

[0104] Through the above method, the embodiment of the present application can dynamically update the preset time according to the loop time of the last communication, realize dynamic adjustment of the threshold, and significantly improve the flexibility, performance and user experience of the system.

[0105] In some embodiments, the intelligent voice device can also act as a controlled device, receiving control instructions sent by other intelligent voice devices, and feeding back the results of the control instructions to other intelligent voice devices.

[0106] In some embodiments, the controlled device may also be a non-voice device.

[0107] On the other hand, an embodiment of the present application also provides a cross-device control device for offline voice. The cross-device control device for offline voice provided by the present application is described below. The cross-device control device for offline voice described below and the cross-device control method for offline voice described above can be referenced to each other.

[0108] See also Figure 8 , Figure 8 8 is a schematic diagram of the structure of the cross-device control device for offline voice provided in an embodiment of the present application. In this embodiment, the cross-device control device for offline voice is deployed on a voice device and may include a voice acquisition module 810 , an instruction analysis module 820 and a transmission module 830 .

[0109] The voice acquisition module 810 is used to acquire user voice commands in an offline state.

[0110] The command analysis module 820 is used to determine the controlled device and the control command according to the user voice command.

[0111] The transmission module 830 is used to send the control instruction to the controlled device and feed back the execution result to the user according to the state of the controlled device within a preset time; wherein the preset time is dynamically updated.

[0112] Based on any of the above embodiments, the transmission module 830 may be specifically used for: The control command is sent to the controlled device through a preset management frame of the broadcast information.

[0113] Based on any of the above embodiments, the preset management frame may include a frame header and an information element; the frame header may include basic information and a control field of the frame, used to indicate the sending and receiving of the frame; the information element may include configuration information, used to implement function configuration and parameter configuration.

[0114] Based on any of the above embodiments, the transmission module 830 may be specifically used for: The control instruction is sent to the controlled device through a direct communication method. Based on any of the above embodiments, the transmission module 830 can also be used for: If a reply message from the controlled device is received within the preset time, and the control instruction is determined to be executed based on the status of the controlled device, the result of successful execution is fed back to the user; if a reply message from the controlled device is not received within the preset time, or the control instruction is determined to fail to be executed based on the status of the controlled device, the result of failed execution is fed back to the user.

[0115] Based on any of the above embodiments, the preset time is dynamically updated.

[0116] Based on any of the above embodiments, the cross-device control apparatus for offline voice may further include a preset time update module, and the preset time update module may be specifically used for: Obtain the loop time of the last communication with the controlled device; determine the allocation weight of the loop time; and determine the preset time of this time by weighted average based on the loop time and the allocation weight.

[0117] Based on any of the above embodiments, the root voice acquisition module 810 can be specifically used for: The user voice command is input into the offline voice recognition model to obtain the controlled device and control command output by the offline voice recognition model; wherein the offline voice recognition model is pre-set with a mapping relationship between the user voice and the intention expression, and the offline voice recognition model is used to output the controlled device and control command corresponding to the user voice command according to the mapping relationship.

[0118] Based on any of the above embodiments, the controlled device may be a non-voice device.

[0119] In summary, the embodiment of the present application provides an offline voice cross-device control device, including: a voice acquisition module, used to acquire user voice commands in an offline state; an instruction analysis module, used to determine the controlled device and control instructions according to the user voice commands; a transmission module, used to send the control instructions to the controlled device, and feedback the execution results to the user according to the state of the controlled device within a preset time. Through the above method, the embodiment of the present application can acquire user voice commands through offline voice recognition technology, is not affected by the network, and reduces dependence on external services; control instructions and feedback execution results can also be transmitted between voice devices and controlled devices through offline communication technology, thereby enhancing reliability and adaptability; further, the preset time for receiving feedback information is dynamically updated to adapt to more application scenarios.

[0120] In another aspect, the present application also provides an offline voice cross-device control method, which is applied to non-voice devices. Fig. 9 , Fig. 9 This is the second flow chart of the offline voice cross-device control method provided in the embodiment of the present application.

[0121] In this embodiment, the offline voice cross-device control method may include steps 910 to 920, each of which is specifically as follows: Step 910: receiving a control instruction sent by a voice device; wherein the control instruction is obtained by the voice device in an offline state according to a user voice instruction.

[0122] Step 920: Execute the function according to the control instruction and feedback the local status to the voice device within a preset time.

[0123] The cross-device control method of offline voice in this embodiment can be referred to in correspondence with the various embodiments described above. This embodiment obtains user voice instructions through offline voice recognition technology, is not affected by the network, and reduces dependence on external services; and the voice device and the controlled device transmit control instructions and feedback execution results through offline communication technology to enhance reliability and adaptability; and the preset time for receiving feedback information is dynamically updated to adapt to more application scenarios; further, offline voice recognition and offline device communication can process information data locally on the device without going through the server, reducing the risk of data being intercepted or leaked, and protecting user privacy; further, offline device communication includes broadcast communication and direct communication, both of which can simplify network configuration, are easy to deploy, and achieve faster data transmission and response speed; further, the preset time can be dynamically updated according to the loop time of the last communication, and the threshold can be dynamically adjusted, which can significantly improve the flexibility, performance and user experience of the system.

[0124] On the other hand, an embodiment of the present application also provides an offline voice cross-device control device, which is deployed on a non-voice device.

[0125] The cross-device control apparatus of offline voice may include a control instruction acquisition module and an execution and status feedback module.

[0126] The control instruction acquisition module is used to receive the control instruction sent by the voice device; wherein the control instruction is obtained by the voice device according to the user's voice instruction in an offline state.

[0127] The execution and status feedback module is used to execute functions according to control instructions and feedback the local status to the voice device within a preset time.

[0128] In another aspect, the embodiment of the present application further provides an offline voice cross-device control system. The offline voice cross-device control system may include a voice device and a controlled device.

[0129] Specifically, the voice device is used to obtain user voice commands in an offline state, determine the controlled device and control commands based on the user voice commands, send the control commands to the controlled device; and feedback the execution results to the user based on the state of the controlled device within a preset time.

[0130] Specifically, the controlled device can be used to receive the control instructions sent by the voice device, execute functions according to the control instructions, and feedback the local status to the voice device within a preset time.

[0131] In another aspect, the present application also provides an electronic device. Fig.10 , Fig.10 1 is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present application, and the electronic device may include: a processor 1010, a communication interface 1020, a memory 1030, and a communication bus 1040, wherein the processor 1010, the communication interface 1020, and the memory 1030 communicate with each other through the communication bus 1040. The processor 1010 may call the logic instructions in the memory 1030 to execute the following method: In an offline state, obtain user voice commands; determine the controlled device and control commands according to the user voice commands; send the control commands to the controlled device, and feedback the execution results to the user according to the state of the controlled device within a preset time; Alternatively, a control instruction sent by a voice device is received; wherein the control instruction is obtained by the voice device in an offline state according to a user's voice instruction; a function is executed according to the control instruction, and the local status is fed back to the voice device within a preset time.

[0132] In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0133] In another aspect, an embodiment of the present application discloses a computer program product, wherein the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program includes program instructions. When the computer program instructions are executed by a computer, the computer can perform the methods provided by the above-mentioned method embodiments, for example, including: In an offline state, obtain user voice commands; determine the controlled device and control commands according to the user voice commands; send the control commands to the controlled device, and feedback the execution results to the user according to the state of the controlled device within a preset time; Alternatively, a control instruction sent by a voice device is received; wherein the control instruction is obtained by the voice device in an offline state according to a user's voice instruction; a function is executed according to the control instruction, and the local status is fed back to the voice device within a preset time.

[0134] In another aspect, an embodiment of the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method provided in each of the above embodiments is implemented, for example, including: In an offline state, obtain user voice commands; determine the controlled device and control commands according to the user voice commands; send the control commands to the controlled device, and feedback the execution results to the user according to the state of the controlled device within a preset time; Alternatively, a control instruction sent by a voice device is received; wherein the control instruction is obtained by the voice device in an offline state according to a user's voice instruction; a function is executed according to the control instruction, and the local status is fed back to the voice device within a preset time.

[0135] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0136] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0137] Finally, it should be noted that the above implementation modes are only used to illustrate the present application, rather than to limit the present application. Although the present application is described in detail with reference to the embodiments, a person skilled in the art should understand that various combinations, modifications or equivalent substitutions of the technical solutions of the present application do not depart from the spirit and scope of the technical solutions of the present application, and should be included in the scope of the claims of the present application.

Claims

1. An offline voice cross-device control method, applied to voice devices, characterized in that: The cross-device control method of offline voice includes: In offline state, obtain user voice commands; Determining a controlled device and a control instruction according to the user voice instruction; The control instruction is sent to the controlled device, and the execution result is fed back to the user according to the state of the controlled device within a preset time.

2. The offline voice cross-device control method according to claim 1, characterized in that: The sending the control instruction to the controlled device includes: The control instruction is sent to the controlled device via a preset management frame of broadcast information.

3. The offline voice cross-device control method according to claim 1, characterized in that: The preset management frame includes a frame header and an information element; The frame header includes basic information of the frame and a control field, which is used to indicate the sending and receiving of the frame; The information elements include configuration information, which is used to implement function configuration and parameter configuration.

4. The offline voice cross-device control method according to claim 1, characterized in that: The sending the control instruction to the controlled device includes: The control instruction is sent to the controlled device through direct communication.

5. The offline voice cross-device control method according to any one of claims 1 to 4, characterized in that: Feedback of the execution result to the user according to the state of the controlled device within a preset time includes: If a reply message from the controlled device is received within the preset time, and it is determined that the execution of the control instruction is completed according to the state of the controlled device, a successful execution result is fed back to the user; If no reply information from the controlled device is received within the preset time, or if it is determined according to the state of the controlled device that the control instruction cannot be executed, a result of execution failure is fed back to the user.

6. The offline voice cross-device control method according to any one of claims 1 to 4, characterized in that: The preset time is updated dynamically.

7. The offline voice cross-device control method according to any one of claims 1 to 4, characterized in that: Before sending the control instruction to the controlled device and feeding back the execution result to the user according to the state of the controlled device within a preset time, the method further includes: Obtaining the loop time of the last communication with the controlled device; determining a distribution weight of the loop time; The preset time is determined based on the loop time and the allocation weight by means of weighted average.

8. The offline voice cross-device control method according to any one of claims 1 to 4, characterized in that: The determining the controlled device and the control instruction according to the user voice instruction includes: Inputting the user voice command into an offline voice recognition model to obtain a controlled device and a control command output by the offline voice recognition model; The offline speech recognition model is pre-set with a mapping relationship between user speech and intention expression, and the offline speech recognition model is used to output a controlled device and a control instruction corresponding to the user speech instruction according to the mapping relationship.

9. The offline voice cross-device control method according to any one of claims 1 to 4, characterized in that: The controlled device is a non-voice device.

10. An offline voice cross-device control method, applied to non-voice devices, characterized in that: The cross-device control method of offline voice includes: Receive a control instruction sent by a voice device; wherein the control instruction is obtained by the voice device in an offline state according to a user voice instruction; Execute functions according to the control instructions and feedback the local status to the voice device within a preset time.

11. An offline voice cross-device control device, characterized in that: Deployed on a voice device, the cross-device control device for offline voice includes: The voice acquisition module is used to obtain user voice commands in an offline state; An instruction analysis module, used to determine the controlled device and the control instruction according to the user voice instruction; The transmission module is used to send the control instruction to the controlled device and feed back the execution result to the user according to the state of the controlled device within a preset time.

12. An offline voice cross-device control system, characterized in that: Including voice equipment and controlled equipment; The voice device is used to obtain a user voice command in an offline state, determine a controlled device and a control command according to the user voice command, and send the control command to the controlled device; And feedback the execution result to the user according to the state of the controlled device within the preset time; The controlled device is used to receive the control instruction sent by the voice device, execute the function according to the control instruction, and feedback the local status to the voice device within a preset time.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the cross-device control method of offline voice is implemented as described in any one of claims 1 to 10.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the cross-device control method of offline voice is implemented as described in any one of claims 1 to 10.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the cross-device control method of offline voice is implemented as described in any one of claims 1 to 10.

Images