Voice control method and electronic device

Abstract

The application provides a voice control method and an electronic device. Voice information for controlling the device is obtained. The voice information is recognized based on an offline light model to obtain a recognition result, and the device is controlled to execute based on a first non-terminal state operation instruction. The first non-terminal state operation instruction and the voice information are sent to the cloud to enable the cloud to recognize the voice information based on an online model to obtain a second non-terminal state operation instruction and a second terminal state operation instruction that need to be executed before the second terminal state operation instruction, and determine a corresponding first control instruction based on the first non-terminal state operation instruction and the second non-terminal state operation instruction. In the case where the first control instruction and the second terminal state operation instruction are obtained within a preset time length, the device is controlled based on the first control instruction and the second terminal state operation instruction, which can ensure control accuracy while improving the response speed of the device.

Description

Technical Field

[0001] This application relates to the field of equipment control, and more particularly to a voice control method and electronic device. Background Technology

[0002] In the process of intelligent device control, voice control has become a widely used interaction method due to its convenience and naturalness, and is widely used in smart homes, industrial control, and vehicle systems. Traditional voice control mostly relies on cloud processing, where devices upload voice commands, and the cloud recognizes and returns instructions. However, this method is significantly constrained by network limitations; when the network is poor, the upload and feedback of instructions are delayed, affecting the device's response speed and reducing the user experience. Some devices use local offline voice processing, deploying models locally, which can improve response speed. However, limited by local computing and storage resources, offline models have low complexity and poor recognition accuracy for complex and ambiguous voice commands, making them prone to misoperation. Summary of the Invention

[0003] This application provides a voice control method and electronic device that can improve the response speed of the device while ensuring control accuracy.

[0004] Firstly, a voice control method is provided, including:

[0005] Acquire voice information for controlling the device;

[0006] The speech information is recognized based on an offline lightweight model to obtain a recognition result, which includes: a first non-final state operation instruction that needs to be executed before the first final state operation instruction is executed;

[0007] The device is controlled to execute based on the first non-final state operation command;

[0008] The first non-final state operation instruction and the voice information are sent to the cloud so that the cloud can recognize the voice information based on an online model, obtain the second non-final state operation instruction and the second final state operation instruction that need to be executed before the second final state operation instruction is executed, and determine the corresponding first control instruction based on the first non-final state operation instruction and the second non-final state operation instruction.

[0009] If the first control command and the second final state operation command are obtained within a preset time period, the device is controlled based on the first control command and the second final state operation command.

[0010] The method provided in this application embodiment recognizes the voice information based on an offline lightweight model to obtain a recognition result. The recognition result includes: a first non-final operation instruction that needs to be executed before the first final operation instruction is executed; the device is controlled to execute based on the first non-final operation instruction, which can quickly execute non-final operations through the offline lightweight model, directly improving the response speed. By sending the first non-final operation instruction and the voice information to the cloud, the cloud recognizes the voice information based on an online model to obtain a second non-final operation instruction and a second final operation instruction that need to be executed before the second final operation instruction is executed, and determines the corresponding first control instruction based on the first non-final operation instruction and the second non-final operation instruction; if the first control instruction and the second final operation instruction are obtained within a preset time, the device is controlled based on the first control instruction and the second final operation instruction, which can accurately verify the final operation through the cloud online model, thereby ensuring control accuracy.

[0011] In some embodiments, the first control instruction includes: a confirmation instruction or a correction instruction. When the first non-final state operation instruction and the second non-final state operation instruction are the same, the first control instruction is a confirmation instruction. When the first non-final state operation instruction and the second non-final state operation instruction are different, the first control instruction is a correction instruction.

[0012] Controlling the device based on the first control command and the second final state operation command includes:

[0013] When the first control command is a confirmation command, the device is controlled based on the second final state operation command;

[0014] When the first control instruction is a correction instruction, the operation performed by the device based on the first non-final state operation instruction is corrected based on the correction instruction, and after correction, the device is controlled based on the second final state operation instruction.

[0015] The method provided in this application embodiment directly controls the device based on the second final state operation command when the first control command is a confirmation command, which simplifies the operation process; when the first control command is a correction command, the operation already performed by the device is first corrected based on the correction command, and then the device is controlled based on the second final state operation command, which ensures the accuracy of the final state of the device.

[0016] In some embodiments, the method further includes:

[0017] Obtain the correction urgency for correcting the recognition results of the offline lightweight model;

[0018] When the urgency level is less than the urgency level threshold, the device is controlled based on the first non-final state operation instruction and the first final state operation instruction, and the first non-final state operation instruction, the first final state operation instruction, and the voice information are sent to the cloud. The cloud then recognizes the voice information based on the online model to obtain the second non-final state operation instruction and the second final state operation instruction. The cloud determines the corresponding first control instruction based on the second non-final state operation instruction, the second final state operation instruction, the first non-final state operation instruction, and the first final state operation instruction. If the first control instruction is obtained, the device is controlled based on the first control instruction.

[0019] Sending the first non-final state operation command and the voice information to the cloud includes: sending the first non-final state operation command and the voice information to the cloud when the correction urgency is greater than or equal to the correction urgency threshold.

[0020] The method provided in this application obtains the correction urgency of the recognition results of the offline lightweight model and determines the control method based on a comparison between the correction urgency and a correction urgency threshold. When the correction urgency is less than the threshold, the device is first controlled based on the first non-final state operation command and the first final state operation command, and then information is sent to the cloud to obtain more accurate commands. When the correction urgency is greater than or equal to the threshold, the information is directly sent to the cloud. This dynamic adjustment method improves the efficiency and rationality of control. It avoids device operation delays caused by over-reliance on cloud feedback when the correction urgency is low, and also ensures that accurate commands can be obtained in a timely manner when correction is needed.

[0021] In some embodiments, obtaining the urgency of correcting the recognition results of the offline lightweight model includes:

[0022] Obtain the offline recognition confidence score of the offline lightweight model, the difference between the target state corresponding to the first final state operation command and the current state of the device, and the environmental risk factor; calculate the correction urgency for correcting the recognition result of the voice information based on the offline recognition confidence score, the difference, and the environmental risk factor; or,

[0023] Obtain the offline recognition confidence of the offline lightweight model, the difference between the target state corresponding to the first final state operation instruction and the current state of the device, and the user habit factor. Calculate the correction urgency for correcting the recognition result of the offline lightweight model based on the offline recognition confidence, the difference, and the user habit factor.

[0024] The method provided in this application, through comprehensive consideration of multiple factors, makes the calculation of the urgency of correction more comprehensive and accurate, and can better reflect the actual situation. In scenarios with high environmental risks, the introduction of environmental risk factors can trigger correction operations more promptly; while consideration of user habit factors can better meet personalized needs.

[0025] In some embodiments, the method further includes:

[0026] Obtain historical correction information for correcting the recognition results of the offline lightweight model;

[0027] The correction frequency is obtained based on the historical correction information;

[0028] If the correction frequency is greater than or equal to the correction frequency threshold, reduce the correction urgency threshold;

[0029] If the correction frequency is less than the correction frequency threshold, the correction urgency threshold is increased.

[0030] The method provided in this application reduces the urgency threshold for corrections when the correction frequency is high, making it easier for the system to trigger correction operations; conversely, it increases the urgency threshold when the correction frequency is low, avoiding unnecessary corrections. This adaptive adjustment improves system stability and efficiency. Furthermore, by adjusting based on the user's historical usage, the system can better adapt to the user's operating habits, reducing user interference from unnecessary correction prompts and enhancing the user experience.

[0031] In some embodiments, the method further includes:

[0032] User type is identified based on the aforementioned voice information;

[0033] The offline identification confidence level is adjusted based on the user type.

[0034] The method provided in this application can gradually learn its voice features, improve the confidence of offline recognition, reduce misrecognition, and further improve the reliability of voice control and user experience.

[0035] In some embodiments, the method further includes:

[0036] If the first control instruction and the second final state operation instruction are not obtained within the preset time period, a progressive adjustment instruction between the first non-final state operation instruction and the first final state operation instruction is obtained based on the first non-final state operation instruction and the first final state operation instruction. If the device is controlled to complete execution based on the first final state operation instruction, the device is controlled to execute based on the progressive adjustment instruction.

[0037] If the device is controlled to complete execution based on the progressive adjustment instruction and the first control instruction and the second final state operation instruction are not obtained, the device is controlled based on the first final state operation instruction.

[0038] After the device is controlled to complete the execution based on the progressive adjustment instruction and the first control instruction and the second final state operation instruction are obtained, the device is controlled based on the first control instruction and the second final state operation instruction.

[0039] The method provided in this application avoids prolonged interruptions in device operation due to waiting for cloud feedback, ensuring the continuity of device operation. If the first control command and the second final state operation command are subsequently obtained, control is then performed based on these commands. This approach considers both the accuracy of cloud feedback and the timeliness of operation, improving the system's adaptability to situations such as network latency.

[0040] In some embodiments, the method further includes:

[0041] If the first control instruction and the second final state operation instruction are not obtained within the preset time period, the first non-final state operation instruction and the voice information are sent to the target device so that the target device can recognize the voice information and obtain the third non-final state operation instruction and the third final state operation instruction to be executed before the third final state operation instruction is executed. The target device determines the corresponding second control instruction based on the first non-final state operation instruction and the third non-final state operation instruction.

[0042] The device is controlled based on the second control command and the third final state operation command.

[0043] The method provided in this application utilizes the computing and recognition capabilities of other devices to achieve collaborative work among multiple devices, thereby improving the fault tolerance and reliability of the system. When a problem is reported from the cloud, control commands can be obtained through the target device to ensure that the device can continue to be effectively controlled, thus avoiding the loss of control of the device due to a single control path failure.

[0044] In some embodiments, the method further includes:

[0045] When the device is not connected to the network, obtain the execution risk information of the first final state operation instruction;

[0046] When the offline recognition confidence of the offline lightweight model is greater than the confidence threshold and the execution risk information meets the requirements, the device is controlled based on the first final state operation instruction;

[0047] If the offline recognition confidence of the offline lightweight model is less than the confidence threshold and the execution risk information does not meet the requirements, a prompt message is output. The prompt message is used to confirm the first final state operation instruction, and if the confirmation of the first final state operation instruction is obtained, the device is controlled based on the first final state operation instruction.

[0048] The method provided in this application embodiment allows the device to remain available even when offline, meeting the user's basic control needs in a network-free environment. When the offline identification confidence is low and the execution risk information does not meet the requirements, a prompt message is output to confirm the first final state operation command. After obtaining confirmation, the device is controlled based on the command. This safety control mechanism avoids the risk of device operation caused by misidentification and protects the safety of the device and the user.

[0049] Secondly, embodiments of this application provide a voice control device, including:

[0050] The first acquisition module is used to acquire voice information for controlling the device;

[0051] The first recognition module is used to recognize the voice information based on an offline lightweight model to obtain a recognition result, the recognition result including: a first non-final state operation instruction to be executed before the first final state operation instruction is executed;

[0052] The first execution module is used to control the device to perform execution based on the first non-final state operation instruction;

[0053] The first sending module is used to send the first non-final state operation instruction and the voice information to the cloud, so that the cloud can recognize the voice information based on an online model, obtain the second non-final state operation instruction and the second final state operation instruction that need to be executed before the second final state operation instruction is executed, and determine the corresponding first control instruction based on the first non-final state operation instruction and the second non-final state operation instruction.

[0054] The first control module is used to control the device based on the first control instruction and the second final state operation instruction when it receives the first control instruction and the second final state operation instruction within a preset time period.

[0055] Thirdly, embodiments of this application provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any one of the first aspects.

[0056] Fourthly, embodiments of this application provide an apparatus, including: the electronic device described in the third aspect.

[0057] Fifthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a display voice control device, causes the display voice control device to perform the voice control method of the first aspect.

[0058] In a sixth aspect, a computer program product is provided, comprising: a computer program that, when executed by an electronic device, causes the electronic device to perform the voice control method of the first aspect.

[0059] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0060] Figure 1 A schematic diagram illustrating the implementation process of a voice control method provided in this application embodiment;

[0061] Figure 2 A flowchart illustrating the process of determining a correction frequency threshold is provided for an embodiment of this application;

[0062] Figure 3 An interactive schematic diagram of a voice control method provided in an embodiment of this application;

[0063] Figure 4 This is a schematic diagram of the structure of a voice control device provided in an embodiment of this application;

[0064] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0065] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0066] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0067] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0068] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0069] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0070] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0071] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0072] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0073] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once detected," or "in response to detection."

[0074] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0075] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0076] To address the problems in related technologies, this application provides a voice control method applicable to electronic devices. These electronic devices can be controllers, smart home devices, industrial equipment, in-vehicle devices, mobile terminals, etc. Smart home devices can include air conditioners, refrigerators, televisions, smart door locks, heating devices, etc.

[0077] Figure 1 This is a schematic diagram illustrating the implementation process of a voice control method provided in an embodiment of this application, as shown below. Figure 1 As shown, it includes:

[0078] Step S101: Obtain voice information for controlling the device.

[0079] In this embodiment of the application, the voice information is a natural language command issued by the user to control the device (such as "set the air conditioner temperature to 26 degrees").

[0080] In this embodiment, the device has a built-in microphone that captures the user's voice and uses a noise reduction algorithm (such as beamforming) to filter out ambient noise and generate a clear voice signal. For example, if the user says "turn on the air conditioner and set it to 26 degrees," the microphone captures the voice and performs preprocessing to obtain the voice information.

[0081] Step S102: Recognize the voice information based on the offline lightweight model to obtain the recognition result. The recognition result includes: the first non-final state operation instruction that needs to be executed before the first final state operation instruction is executed.

[0082] In this embodiment, the offline lightweight model is a lightweight speech recognition model deployed locally on the device. It features low computational resource consumption and fast response speed, but its recognition accuracy and generalization ability are limited. The first non-final state operation instruction is an intermediate step instruction that must be completed before executing the final state operation (such as "start the air conditioner compressor"), which is reversible and will not cause serious consequences if executed incorrectly. The first final state operation instruction is the operation instruction corresponding to the user's final intention (such as "set the air conditioner temperature to 26 degrees"), which requires high-precision recognition to avoid false triggering.

[0083] In this application embodiment, the non-final state operation and final state operation of different devices are different. For example, taking an air conditioner as an example, the non-final state operation may include: starting the fan / sweeping the air, and the final state operation may be temperature / mode setting. Taking a TV as an example, the non-final state operation may include: turning on the backlight / loading the interface, and the final state operation may include: channel switching / volume locking. Taking a smart door lock as an example, the non-final state operation may include: clicking pre-power supply, and the final state operation may include the lock tongue action.

[0084] In some embodiments, non-final state operations and final state operations can be defined by software, which allows for adaptation to multiple devices. In some embodiments, non-final state operations and final state operations can be distinguished by physical circuits. For example, non-final state operations are connected to low-power circuits (such as fan relays), and final state operations are connected to high-safety circuits (such as compressor controllers). Physical circuit isolation can prevent erroneous operations at the hardware level.

[0085] In this embodiment, the offline lightweight model performs feature extraction and pattern matching on the speech, and outputs a recognition result containing a first non-final state operation instruction.

[0086] Following the example above, the offline lightweight model recognizes the following voice information: "Start compressor" (first non-final state operation command) and "Set to 26 degrees" (first final state operation command).

[0087] Step S103: Control the device to execute based on the first non-final state operation instruction.

[0088] In this embodiment of the application, the device can be directly controlled to perform operations based on the first non-final state operation instruction.

[0089] Continuing with the example above, if the first non-final state operation command is to start the compressor, then the compressor controlling the air conditioner will start running, and the user will immediately perceive the device response.

[0090] Step S104: Send the first non-final state operation instruction and the voice information to the cloud, so that the cloud can recognize the voice information based on the online model, obtain the second non-final state operation instruction and the second final state operation instruction to be executed before the second final state operation instruction is executed, and determine the corresponding first control instruction based on the first non-final state operation instruction and the second non-final state operation instruction.

[0091] In this embodiment, the online model is a complex speech recognition model deployed in the cloud. Relying on big data and powerful computing power, it can process complex speech and generate high-precision recognition results. The second non-final state operation instruction is an intermediate step instruction of the cloud model recognition, used for comparison and verification with the local first non-final state operation instruction. The second final state operation instruction is the final operation instruction of the cloud model recognition, used to correct possible recognition errors locally. The first control instruction is a confirmation / correction instruction (such as "confirm compressor start" or "correct to heating mode") generated by the cloud based on the comparison result between the local first non-final state operation instruction and the cloud's second non-final state operation instruction.

[0092] In this embodiment, the device can upload the original speech and local recognition results to the cloud via an encrypted channel to ensure data security. An online model performs deep analysis of the speech (such as LSTM or Transformer) to generate a second non-final state operation command and a second final state operation command. Continuing the example above, the model identifies "start compressor" (second non-final state operation command) and "cooling mode 26 degrees" (second final state operation command). The cloud can compare the first non-final state operation command and the second non-final state command; if they match, a confirmation command is generated; if they do not match, a correction command is generated.

[0093] Continuing with the example above, if the first non-final state operation command and the second non-final state operation command are consistent, a "confirm compressor start" command is generated; if the first non-final state operation command and the second non-final state operation command are inconsistent, a "correction command" is generated in the cloud.

[0094] Step S105: If the first control command and the second final state operation command are obtained within a preset time period, the device is controlled based on the first control command and the second final state operation command.

[0095] In this embodiment of the application, the preset duration is an allowable cloud response time threshold (e.g., 1 second). After the timeout, the device can enter the default mode or trigger a backup scheme. If the timeout is not exceeded, the device is controlled based on the first control command and the second final state operation command.

[0096] Following the example above, the device receives "confirmation of compressor start" and "cooling to 26 degrees" within 1 second. At this time, the temperature is adjusted and the compressor is kept running.

[0097] The method provided in this application embodiment recognizes the voice information based on an offline lightweight model to obtain a recognition result. The recognition result includes: a first non-final operation instruction that needs to be executed before the first final operation instruction is executed; the device is controlled to execute based on the first non-final operation instruction, which can quickly execute non-final operations through the offline lightweight model, directly improving the response speed. By sending the first non-final operation instruction and the voice information to the cloud, the cloud recognizes the voice information based on an online model to obtain a second non-final operation instruction and a second final operation instruction that need to be executed before the second final operation instruction is executed, and determines the corresponding first control instruction based on the first non-final operation instruction and the second non-final operation instruction; if the first control instruction and the second final operation instruction are obtained within a preset time, the device is controlled based on the first control instruction and the second final operation instruction, which can accurately verify the final operation through the cloud online model, thereby ensuring control accuracy.

[0098] The method provided in this application is applicable to scenarios that are latency-sensitive (such as smart homes and in-vehicle systems) or have high security requirements (such as industrial control and medical devices).

[0099] In some embodiments, the first control instruction includes: a confirmation instruction or a correction instruction. When the first non-final state operation instruction and the second non-final state operation instruction are the same, the first control instruction is a confirmation instruction. When the first non-final state operation instruction and the second non-final state operation instruction are different, the first control instruction is a correction instruction.

[0100] In this embodiment, the confirmation instruction is a cloud-generated instruction when the local first non-final state operation instruction matches the cloud second non-final state operation instruction. This notifies the device that the local operation is correct and the final state operation (such as setting the temperature) can be executed directly. The correction instruction is a cloud-generated instruction when the local first non-final state operation instruction and the cloud second non-final state operation instruction are inconsistent. This instruction is used to notify the device to correct the intermediate operation already executed (such as changing "heating" to "cooling") before executing the final state operation.

[0101] In this embodiment of the application, step S105 can be implemented through the following steps:

[0102] Step S1051: If the first control command is a confirmation command, control the device based on the second final state operation command.

[0103] In this embodiment, the device directly executes the second final state operation instruction returned from the cloud.

[0104] Step S1052: If the first control instruction is a correction instruction, the operation performed by the device based on the first non-final state operation instruction is corrected based on the correction instruction, and after correction, the device is controlled based on the second final state operation instruction.

[0105] In this embodiment, the device first executes a correction command (such as "turn off heating and start cooling"). After the correction is completed, the second final state operation command is executed. For example, the device executes "turn off heating and start cooling" based on the correction command, and then directly sets the temperature to 26 degrees.

[0106] The method provided in this application embodiment directly controls the device based on the second final state operation command when the first control command is a confirmation command, which simplifies the operation process; when the first control command is a correction command, the operation already performed by the device is first corrected based on the correction command, and then the device is controlled based on the second final state operation command, which ensures the accuracy of the final state of the device.

[0107] In some embodiments, prior to step S104, the method further includes:

[0108] Step S1: Obtain the urgency of correcting the recognition results of the offline lightweight model.

[0109] In this embodiment, the urgency level is used to measure the urgency of the impact of offline lightweight model recognition error on device control. It is typically determined by a comprehensive evaluation of factors such as offline recognition confidence, the difference between the target state corresponding to the first final state operation command and the current state of the device, and environmental risk factors. The urgency level can be calculated using preset rules (such as a weighting table) or a machine learning model (such as a risk assessment network), outputting a value between 0 and 1 (1 representing the highest urgency). In some embodiments, the urgency level can also be calculated using offline recognition confidence, the difference between the target state corresponding to the first final state operation command and the current state of the device, and user habit factors.

[0110] Step S2: If the urgency level is less than the urgency level threshold, control the device based on the first non-final state operation instruction and the first final state operation instruction, and send the first non-final state operation instruction, the first final state operation instruction, and the voice information to the cloud. The cloud then recognizes the voice information based on the online model to obtain the second non-final state operation instruction and the second final state operation instruction. The cloud determines the corresponding first control instruction based on the second non-final state operation instruction, the second final state operation instruction, the first non-final state operation instruction, and the first final state operation instruction. If the first control instruction is obtained, control the device based on the first control instruction.

[0111] In this embodiment, the urgency threshold is a preset critical value used to determine whether local execution needs to be immediately interrupted and cloud-based correction should be waited for. The urgency threshold is set based on device security requirements, user tolerance, and network latency tolerance. For example, the urgency threshold is set to 0.8. When the urgency is ≥0.8, the cloud-based correction process is triggered. When the urgency is less than 0.8, the device can be controlled based on the first non-final state operation command and the first final state operation command.

[0112] In this embodiment of the application, when the correction urgency is less than the correction urgency threshold, the device controls the device based on the first non-final state operation instruction and the first final state operation instruction, and the device can execute immediately.

[0113] In this embodiment of the application, if the first control instruction is a correction instruction, the device is corrected; if the first control instruction is a confirmation instruction, the execution is completed.

[0114] Step S104 can be achieved through the following steps:

[0115] Step S1041: If the modified urgency is greater than or equal to the modified urgency threshold, the first non-final state operation command and the voice information are sent to the cloud.

[0116] The method provided in this application obtains the correction urgency of the recognition results of the offline lightweight model and determines the control method based on a comparison between the correction urgency and a correction urgency threshold. When the correction urgency is less than the threshold, the device is first controlled based on the first non-final state operation command and the first final state operation command, and then information is sent to the cloud to obtain more accurate commands. When the correction urgency is greater than or equal to the threshold, the information is directly sent to the cloud. This dynamic adjustment method improves the efficiency and rationality of control. It avoids device operation delays caused by over-reliance on cloud feedback when the correction urgency is low, and also ensures that accurate commands can be obtained in a timely manner when correction is needed.

[0117] In some embodiments, step S1 can be implemented through the following steps:

[0118] Step S11: Obtain the offline recognition confidence of the offline lightweight model, the difference between the target state corresponding to the first final state operation instruction and the current state of the device, and the environmental risk factor.

[0119] In this embodiment, the offline recognition confidence level is an assessment of the reliability of the recognition result by the offline lightweight model, typically represented by a probability value (0-1). A higher value indicates stronger certainty regarding the recognition result. The target state is the state the device is expected to reach according to the first final state operation command (e.g., an air conditioner temperature of 26°C). The current state of the device is its actual state before executing the command (e.g., the current air conditioner temperature and operating mode). The environmental risk factor reflects the tolerance of the current environment to recognition errors. Environmental risks can include physical risks, user risks, and equipment risks. Physical risks include factors such as high temperature and high pressure that may cause equipment damage or safety hazards. User risks include the user's tolerance for errors in sensitive scenarios (e.g., sleep or medical conditions). Equipment risks include the sensitivity of the equipment type (industrial equipment vs. consumer electronics) to errors. The environmental risk factor can be calculated using preset rules (e.g., a risk level table) or environmental sensor data (e.g., temperature and humidity).

[0120] In this embodiment of the application, the offline recognition confidence level is obtained based on the recognition results of the offline lightweight model.

[0121] In this embodiment, the target state (e.g., temperature 26℃) can be parsed from the first final state operation instruction, and the current state (e.g., temperature 30℃) can be obtained from the device sensors. The absolute difference (e.g., |30℃-26℃|=4℃) or relative difference (e.g., 4℃ / 30℃≈13.3%) is calculated, and the difference is divided into three levels: low, medium, and high according to preset rules (e.g., difference ≤ 2℃ is low, 2℃ < difference ≤ 5℃ is medium, and difference > 5℃ is high). Environmental data can be obtained through sensors (e.g., temperature sensor, human infrared sensor) or user context (e.g., time, scene mode), and the risk factor is mapped to a value between 0 and 1 based on the environmental data.

[0122] Step S12: Calculate the correction urgency for correcting the recognition result of the voice information based on the offline recognition confidence, the difference, and the environmental risk factor.

[0123] In this embodiment, the urgency level is a quantification of the urgency at which the identification results need to be corrected, based on a comprehensive assessment of offline identification confidence, state differences, and environmental risk factors.

[0124] In this embodiment of the application, the urgency level can be calculated using the following formula:

[0125] Ec=α*(1-Coffline)+β*ΔS+γ*Henv;

[0126] Where Ec represents the urgency level, Coffline represents the offline identification confidence level, ΔS represents the difference between the current state and the target state, and Henv represents the environmental risk factor; α, β, and γ are the weighting coefficients for device types. Different devices can be assigned different weighting coefficients. For example, the α, β, and γ values ​​for an air conditioner are 0.6, 0.3, and 0.1, while for a smart door lock they are 0.2, 0.1, and 0.7.

[0127] The method provided in this application embodiment can calculate the urgency of correction by integrating multi-dimensional information.

[0128] In some embodiments, when environmental risk factors are difficult to quantify, step S1 can be achieved through the following steps:

[0129] Step S13: Obtain the offline recognition confidence of the offline lightweight model, the difference between the target state corresponding to the first final state operation instruction and the current state of the device, and the user habit factor.

[0130] In this embodiment of the application, the user habit factor is a parameter that reflects the user's tolerance for the current instruction based on the user's historical behavior. The user habit factor may include: operation frequency, time pattern, and device preference, etc.

[0131] In this embodiment of the application, a value of 0-1 can be generated by statistical analysis of user historical data or machine learning models (such as cluster analysis) (1 indicates that it is completely in line with convention, and 0 indicates that it is completely incompatible).

[0132] For example, record user operation commands, times, device status, and other data for the past 30 days, calculate the frequency of the user executing the current command (e.g., "set to 26℃" accounts for 60% of the air conditioner commands in the past 30 days), check whether the current time is in line with the user's habits, compare the user's tolerance for similar devices (e.g., being more strict about adjusting the air conditioner temperature than the lights), and use a weighted formula to calculate the user habit factor.

[0133] Step S104: Calculate the correction urgency for correcting the recognition results of the offline lightweight model based on the offline recognition confidence, the difference, and the user habit factor.

[0134] In this embodiment, the urgency level can be calculated using the following formula:

[0135] Ec=α*(1-Coffline)+β*ΔS+μ*Huser;

[0136] Here, Huser is the user habit factor, and u is the weight corresponding to the user habit factor.

[0137] The method provided in this application, by comprehensively considering multiple factors, makes the calculation of the urgency of correction more comprehensive and accurate, and can better reflect the actual situation. In scenarios with high environmental risks, the introduction of environmental risk factors can trigger correction operations more promptly; while the consideration of user habit factors can better meet personalized needs.

[0138] In some embodiments, the corresponding weights can also be dynamically adjusted through federated learning.

[0139] In some embodiments, a correction frequency threshold is determined before step S1. Figure 2 This application provides a flowchart illustrating a process for determining a correction frequency threshold, as shown in the embodiments below. Figure 2 As shown, the method further includes:

[0140] Step S11: Obtain historical correction information for correcting the recognition results of the offline lightweight model.

[0141] In this embodiment of the application, the historical correction information records detailed data on the correction of the offline lightweight model recognition results over a period of time. The historical correction information may include the number of corrections.

[0142] In this embodiment of the application, detailed information (such as time, reason, and result) of each correction can be recorded in a local or cloud database, and historical correction information can be queried according to a time range.

[0143] Step S12: Obtain the correction frequency based on the historical correction information.

[0144] In this embodiment, the correction frequency is the number of times a correction is triggered per unit time, used to quantify the correction requirements of the model. The correction frequency can be obtained by dividing the number of corrections by the time window.

[0145] For example, if 15 corrections were triggered in the past 30 days, the correction frequency is 0.5 times / day.

[0146] Step S13: If the correction frequency is greater than or equal to the correction frequency threshold, reduce the correction urgency threshold.

[0147] In this embodiment, the correction frequency threshold is a preset correction frequency critical value, used to determine whether the system needs to adjust the correction urgency threshold. The correction frequency threshold can be set based on historical data statistics (such as average correction frequency) or user experience requirements (such as allowing a maximum of 1 correction per day).

[0148] In this embodiment, if the correction frequency is greater than or equal to the correction frequency threshold, it indicates that the correction demand is high and the urgency threshold needs to be lowered so that identification can be performed through the cloud, thereby improving control accuracy and reducing corrections.

[0149] Step S14: If the correction frequency is less than the correction frequency threshold, increase the correction urgency threshold.

[0150] In this embodiment of the application, if the correction frequency is less than the correction frequency threshold, it indicates that the correction requirement is low. The urgency threshold can be increased so that identification can be performed locally, reducing the need for cloud-based identification.

[0151] The method provided in this application reduces the urgency threshold for corrections when the correction frequency is high, making it easier for the system to trigger correction operations; conversely, it increases the urgency threshold when the correction frequency is low, avoiding unnecessary corrections. This adaptive adjustment improves system stability and efficiency. Furthermore, by adjusting based on the user's historical usage, the system can better adapt to the user's operating habits, reducing user interference from unnecessary correction prompts and enhancing the user experience.

[0152] In some embodiments, prior to step S11 or step S13, the method further includes:

[0153] Step S21: Identify the user type based on the voice information.

[0154] In this embodiment of the application, the user type is a user category classified according to the characteristics of the voice information, such as adult, child, and elderly.

[0155] In this embodiment of the application, speech signal processing algorithms (such as MFCC, Mel spectrum) can be used to extract features such as pitch, speech rate, and volume, and user identity can be classified through machine learning models.

[0156] Step S22: Adjust the offline identification confidence level based on the user type.

[0157] In this embodiment, a confidence offset can be preset for different user types. For example, the confidence of child users can be reduced, while the confidence of adults can be increased. Different confidence offsets can be set for different user types. After the user type is determined, the offline recognition confidence can be adjusted based on the confidence offset.

[0158] For example, the user type is "child", the original confidence level is 0.75, and the adjusted confidence level is 0.75 - 0.15 = 0.60.

[0159] The method provided in this application embodiment may lead to model misidentification due to unclear pronunciation in children or slow speech in the elderly. By reducing the confidence level of offline recognition, the urgency of correction can be reduced, thereby improving the accuracy of recognition through the cloud.

[0160] In some embodiments, after step S104, the method further includes:

[0161] Step S106: If the first control instruction and the second final state operation instruction are not obtained within the preset time period, a progressive adjustment instruction between the first non-final state operation instruction and the first final state operation instruction is obtained based on the first non-final state operation instruction and the first final state operation instruction. If the device is controlled to complete execution based on the first final state operation instruction, the device is controlled to execute based on the progressive adjustment instruction.

[0162] In this embodiment of the application, if the first control instruction and the second final state operation instruction are not obtained within the preset time period, a gradual adjustment process is executed.

[0163] In this embodiment, a first non-final state operation instruction and a first final state operation instruction can be input into the neural network model to obtain a progressive adjustment instruction. The progressive adjustment instruction can be a progressive adjustment instruction set divided by time or steps.

[0164] For example, if the current temperature is 20°C and the target temperature is 25°C, then a progressive instruction sequence is generated:

[0165] Step 1: Heat to 21℃ (for 1 minute);

[0166] Step 2: Heat to 22℃ (for 1 minute); ...

[0168] Step 4: Raise the temperature to 24℃.

[0169] In this embodiment of the application, the device can be executed based on a progressive adjustment instruction.

[0170] Step S107: If the device is controlled to complete execution based on the progressive adjustment instruction and the first control instruction and the second final state operation instruction are not obtained, the device is controlled based on the first final state operation instruction.

[0171] In this embodiment of the application, if the gradual adjustment has been completed and there are no new instructions, the first final state operation instruction is confirmed as the final target.

[0172] Step S108: After the device is controlled to complete the execution based on the progressive adjustment instruction and the first control instruction and the second final state operation instruction are obtained, the device is controlled based on the first control instruction and the second final state operation instruction.

[0173] The method provided in this application avoids abrupt changes in device state and sudden operational jumps through gradual adjustments, improving user experience and preventing prolonged interruptions in device operation due to waiting for cloud feedback, thus ensuring the continuity of device operation. If the first control command and the second final state operation command are subsequently obtained, control is then performed based on these commands, taking into account both the accuracy of cloud feedback and the timeliness of operation, thereby improving the system's adaptability to situations such as network latency.

[0174] In some embodiments, after step S104, the method further includes:

[0175] Step S109: If the first control command and the second final state operation command are not obtained within the preset time period, the first non-final state operation command and the voice information are sent to the target device so that the target device can recognize the voice information and obtain the third non-final state operation command and the third final state operation command to be executed before the third final state operation command is executed. The target device determines the corresponding second control command based on the first non-final state operation command and the third non-final state operation command.

[0176] Step S110: Control the device based on the second control command and the third final state operation command.

[0177] The method provided in this application utilizes the computing and recognition capabilities of other devices to achieve collaborative work among multiple devices, improving the fault tolerance and reliability of the system. When a problem is reported from the cloud, control commands can be obtained through the target device to ensure that the device can continue to be effectively controlled, avoiding device loss of control due to failure of a single control path. It can also solve the problem of insufficient computing power of low-end devices.

[0178] In some embodiments, after step S102, the method further includes:

[0179] Step S111: If the device is not connected to the network, obtain the execution risk information of the first final state operation instruction.

[0180] In this embodiment, the device cannot communicate with the cloud server via Wi-Fi, 4G / 5G, or other networks when it is not connected to the internet, and it only relies on local computing resources (such as embedded chips and local storage) to operate. The execution risk information is an assessment of the potential negative consequences after the device executes the first final state operation command, including security, compliance, and device wear and tear. This execution risk information can be pre-set tag information.

[0181] In this embodiment of the application, execution risk information tags corresponding to each final state operation instruction can be pre-established. After obtaining the first final state operation instruction, execution risk information can be obtained based on the correspondence.

[0182] In some embodiments, the first final state operation instruction can be input into the risk information prediction model to obtain execution risk information, which can be represented numerically.

[0183] Step S112: If the offline recognition confidence of the offline lightweight model is greater than the confidence threshold and the execution risk information meets the requirements, control the device based on the first final state operation instruction.

[0184] In this embodiment, conditions corresponding to the execution risk information can be set, and the system can determine whether the requirements are met by making a judgment. In some embodiments, an execution risk information threshold can be set. If the execution risk information is less than the execution risk information threshold, it is considered low risk, and the requirements are determined to be met.

[0185] In this embodiment of the application, when the offline recognition confidence of the offline lightweight model is greater than the confidence threshold and the execution risk information meets the requirements, the device is controlled based on the first final state operation instruction.

[0186] For example, if the first final state operation instruction is "adjust to 25°C", the offline recognition confidence of the offline light model is 0.95, which is greater than 0.8, and the risk is "low risk", then the air conditioner will immediately adjust the temperature.

[0187] Step S113: If the offline recognition confidence of the offline lightweight model is less than the confidence threshold and the execution risk information does not meet the requirements, output a prompt message. The prompt message is used to confirm the first final state operation instruction. If the first final state operation instruction is confirmed, control the device based on the first final state operation instruction.

[0188] In this embodiment of the application, the prompt message is a confirmation request output to the user when the model confidence is insufficient or the risk is too high. The confirmation request can be displayed to the user or read aloud.

[0189] Example: "If the offline recognition confidence of the offline lightweight model is less than the confidence threshold and the execution risk information does not meet the requirements, the output prompt message is: 'Do you confirm adjusting the temperature to 30℃?' If the user confirms (e.g., presses the 'Yes' button or says 'Confirm'), the device is controlled based on the first final state operation instruction."

[0190] The method provided in this application embodiment allows the device to remain available even when offline, meeting the user's basic control needs in a network-free environment. When the offline identification confidence is low and the execution risk information does not meet the requirements, a prompt message is output to confirm the first final state operation command. After obtaining confirmation, the device is controlled based on the command. This safety control mechanism avoids the risk of device operation caused by misidentification and protects the safety of the device and the user.

[0191] Based on the foregoing embodiments, this application provides a voice control method. Figure 3 This is an interactive schematic diagram of a voice control method provided in an embodiment of this application, such as... Figure 3 As shown, it includes:

[0192] Step S301: The user sends a voice message;

[0193] Step S302: The device uses an offline lightweight model for recognition;

[0194] Step S303: The device performs a non-final state operation.

[0195] Step S304: The device uploads voice information and non-final state operations to the cloud.

[0196] Step S305: The cloud performs verification through an online model.

[0197] In this embodiment of the application, if the identification of non-final state operations in the cloud is different from the identification of non-final state operations on the device side, a correction instruction is generated and step S306 is executed.

[0198] Step S306: Send correction instructions and final state operation instructions.

[0199] In step S307, the device cancels the non-final state operation, performs the correction, and executes the final state operation command.

[0200] In some embodiments, if the identification of a non-final state operation in the cloud is the same as the identification of a non-final state operation on the device side, a confirmation command is sent, and the device side completes the final state operation.

[0201] The method provided in this application decomposes the operation into a non-final state (reversible basic action) and a final state (irreversible core operation). The device only executes the non-final state part, and an operation window is reserved for correction. Through online correction, the control accuracy is improved.

[0202] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0203] According to the foregoing embodiments, this application provides a voice control device. The various modules and units included in the device can be implemented by a processor in a computer device; of course, they can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0204] Figure 4 This is a schematic diagram of the structure of a voice control device provided in an embodiment of this application, as shown below. Figure 4 As shown, the voice control device 400 includes:

[0205] The first acquisition module 401 is used to acquire voice information for controlling the device;

[0206] The first recognition module 402 is used to recognize the voice information based on an offline lightweight model to obtain a recognition result, the recognition result including: a first non-final state operation instruction to be executed before the first final state operation instruction is executed;

[0207] The first execution module 403 is used to control the device to perform execution based on the first non-final state operation instruction;

[0208] The first sending module 404 is used to send the first non-final state operation instruction and the voice information to the cloud, so that the cloud can recognize the voice information based on an online model, obtain the second non-final state operation instruction and the second final state operation instruction to be executed before the second final state operation instruction is executed, and determine the corresponding first control instruction based on the first non-final state operation instruction and the second non-final state operation instruction.

[0209] The first control module 405 is used to control the device based on the first control instruction and the second final state operation instruction when it receives the first control instruction and the second final state operation instruction within a preset time period.

[0210] In some embodiments, the first control instruction includes: a confirmation instruction or a correction instruction. When the first non-final state operation instruction and the second non-final state operation instruction are the same, the first control instruction is a confirmation instruction. When the first non-final state operation instruction and the second non-final state operation instruction are different, the first control instruction is a correction instruction.

[0211] The first control module includes:

[0212] A first control unit is configured to control the device based on the second final state operation command when the first control command is a confirmation command;

[0213] The second control unit is configured to, when the first control instruction is a correction instruction, correct the operation performed by the device based on the first non-final state operation instruction based on the correction instruction, and control the device based on the second final state operation instruction after correction.

[0214] In some embodiments, the voice control device 400 further includes:

[0215] The first determining module is used to obtain the correction urgency of correcting the recognition results of the offline lightweight model;

[0216] The second control module is configured to control the device based on the first non-final state operation instruction and the first final state operation instruction when the correction urgency is less than the correction urgency threshold, and to send the first non-final state operation instruction, the first final state operation instruction, and the voice information to the cloud, so that the cloud can recognize the voice information based on the online model to obtain the second non-final state operation instruction and the second final state operation instruction. The cloud determines the corresponding first control instruction based on the second non-final state operation instruction, the second final state operation instruction, the first non-final state operation instruction, and the first final state operation instruction. When the first control instruction is obtained, the cloud controls the device based on the first control instruction.

[0217] The first sending module includes:

[0218] The first sending unit is configured to send the first non-final state operation command and the voice information to the cloud when the correction urgency is greater than or equal to the correction urgency threshold.

[0219] In some embodiments, the first determining module includes:

[0220] The first determining unit is configured to acquire the offline recognition confidence level of the offline lightweight model, the difference between the target state corresponding to the first final state operation instruction and the current state of the device, and environmental risk factors; and calculate the correction urgency for correcting the recognition result of the voice information based on the offline recognition confidence level, the difference, and the environmental risk factors; or,

[0221] The second determining unit is used to obtain the offline recognition confidence of the offline lightweight model, the difference between the target state corresponding to the first final state operation instruction and the current state of the device, and the user habit factor, and to calculate the correction urgency of correcting the recognition result of the offline lightweight model based on the offline recognition confidence, the difference, and the user habit factor.

[0222] In some embodiments, the voice control device 400 further includes:

[0223] The second acquisition module is used to acquire historical correction information for correcting the recognition results of the offline lightweight model;

[0224] The module is used to obtain the correction frequency based on the historical correction information;

[0225] A reduction module is used to reduce the correction urgency threshold when the correction frequency is greater than or equal to the correction frequency threshold.

[0226] An amplification module is used to increase the correction urgency threshold when the correction frequency is less than the correction frequency threshold.

[0227] In some embodiments, the voice control device 400 further includes:

[0228] The second determining module is used to identify the user type based on the voice information;

[0229] An adjustment module is used to adjust the offline identification confidence level based on the user type.

[0230] In some embodiments, the voice control device 400 further includes:

[0231] The second identification module is used to obtain a progressive adjustment instruction between the first non-final operation instruction and the first final operation instruction based on the first non-final operation instruction and the first final operation instruction when the first control instruction and the second final operation instruction are not obtained within the preset time period, and to control the device to execute based on the progressive adjustment instruction when the device is controlled to complete execution based on the first final operation instruction.

[0232] The third control module is used to control the device based on the first final state operation instruction when the device has completed execution based on the progressive adjustment instruction and the first control instruction and the second final state operation instruction have not been obtained.

[0233] The fourth control module is used to control the device to complete the execution based on the progressive adjustment instruction and obtain the first control instruction and the second final state operation instruction, and to control the device based on the first control instruction and the second final state operation instruction.

[0234] In some embodiments, the voice control device 400 further includes:

[0235] The second identification module is used to send the first non-final operation instruction and the voice information to the target device when the first control instruction and the second final operation instruction are not obtained within the preset time period, so that the target device can identify the voice information and obtain the third non-final operation instruction and the third final operation instruction to be executed before the third final operation instruction is executed. The target device determines the corresponding second control instruction based on the first non-final operation instruction and the third non-final operation instruction.

[0236] The fifth control module is used to control the device based on the second control command and the third final state operation command.

[0237] In some embodiments, the voice control device 400 further includes:

[0238] The third determining module is used to determine the execution risk information of the first final state operation instruction when the device is not connected to the network.

[0239] The sixth control module is used to control the device based on the first final state operation command when the offline recognition confidence of the offline lightweight model is greater than the confidence threshold and the execution risk information meets the requirements.

[0240] The prompting module is used to output prompting information when the offline recognition confidence of the offline lightweight model is less than the confidence threshold and the execution risk information does not meet the requirements. The prompting information is used to confirm the first final state operation instruction, and when the confirmation of the first final state operation instruction is obtained, the device is controlled based on the first final state operation instruction.

[0241] In addition, the voice control device described above can be a software unit, a hardware unit, or a combination of software and hardware. It can also be integrated into the electronic device as an independent accessory, or exist as an independent terminal device.

[0242] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0243] This application provides an electronic device. Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 3 in this embodiment may include: at least one processor 30 ( Figure 5 Only one processor 30, memory 31, and computer program 32 stored in memory 31 and executable on at least one processor 30 are shown. When the processor 30 executes the computer program 32, it implements the steps in any of the above method embodiments, or the processor 30 executes the computer program 32 to implement the functions of each module / unit in the above device embodiments.

[0244] For example, computer program 32 may be divided into one or more modules / units, one or more of which are stored in memory 31 and executed by processor 30 to complete this application. One or more modules / units may be a series of computer program 32 instruction segments capable of performing a specific function, which describe the execution process of computer program 32 in electronic device 3.

[0245] This application provides another type of freezer, including the electronic equipment described in the above embodiments.

[0246] This application also provides a computer-readable storage medium storing a computer program 32, which, when executed by a processor 30, implements the steps described in the above-described method embodiments.

[0247] This application provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0248] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program 32 instructing related hardware. The computer program 32 can be stored in a computer-readable storage medium, and when executed by the processor 30, it can implement the steps of the various method embodiments described above. The computer program 32 includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code to a terminal, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0249] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0250] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0251] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0252] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0253] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A voice control method, characterized in that, include: Acquire voice information for controlling the device; The speech information is recognized based on an offline lightweight model to obtain a recognition result. The recognition result includes a first non-final state operation instruction that needs to be executed before the first final state operation instruction is executed, wherein the non-final state operation instruction is an intermediate step instruction that must be completed before the final state operation is executed. The device is controlled to execute based on the first non-final state operation command; The first non-final state operation instruction and the voice information are sent to the cloud, so that the cloud can recognize the voice information based on an online model to obtain the second non-final state operation instruction and the second final state operation instruction that need to be executed before the second final state operation instruction is executed. Based on the first non-final state operation instruction and the second non-final state operation instruction, the cloud determines the corresponding first control instruction. The first control instruction is a confirmation instruction or correction instruction generated by the cloud based on the comparison result between the first non-final state operation instruction and the second non-final state operation instruction. If the first control command and the second final state operation command are obtained within a preset time period, the device is controlled based on the first control command and the second final state operation command.

2. The method according to claim 1, characterized in that, The first control instruction includes: a confirmation instruction or a correction instruction. When the first non-final state operation instruction and the second non-final state operation instruction are the same, the first control instruction is a confirmation instruction. When the first non-final state operation instruction and the second non-final state operation instruction are different, the first control instruction is a correction instruction. Controlling the device based on the first control command and the second final state operation command includes: When the first control command is a confirmation command, the device is controlled based on the second final state operation command; When the first control instruction is a correction instruction, the operation performed by the device based on the first non-final state operation instruction is corrected based on the correction instruction, and after correction, the device is controlled based on the second final state operation instruction.

3. The method according to claim 2, characterized in that, The method further includes: Obtain the correction urgency of correcting the recognition results of the offline lightweight model, wherein the correction urgency is used to measure the urgency of the impact of the error of the recognition results of the offline lightweight model on device control; If the urgency level is less than the urgency level threshold, the device is controlled based on the first non-final state operation command and the first final state operation command. The first non-final state operation command, the first final state operation command, and the voice information are sent to the cloud. The cloud then uses the online model to recognize the voice information, obtaining the second non-final state operation command and the second final state operation command. The cloud then determines the corresponding first control command based on the second non-final state operation command, the second final state operation command, the first non-final state operation command, and the first final state operation command. Upon receiving the first control command, the device is controlled based on the first control command; Sending the first non-final state operation command and the voice information to the cloud includes: sending the first non-final state operation command and the voice information to the cloud when the correction urgency is greater than or equal to the correction urgency threshold.

4. The method according to claim 3, characterized in that, The urgency of obtaining the correction urgency for correcting the recognition results of the offline lightweight model includes: Obtain the offline recognition confidence score of the offline lightweight model, the difference between the target state corresponding to the first final state operation command and the current state of the device, and the environmental risk factor; calculate the correction urgency for correcting the recognition result of the voice information based on the offline recognition confidence score, the difference, and the environmental risk factor; or, Obtain the offline recognition confidence of the offline lightweight model, the difference between the target state corresponding to the first final state operation instruction and the current state of the device, and the user habit factor. Calculate the correction urgency for correcting the recognition result of the offline lightweight model based on the offline recognition confidence, the difference, and the user habit factor.

5. The method according to claim 3, characterized in that, The method further includes: Obtain historical correction information for correcting the recognition results of the offline lightweight model; The correction frequency is obtained based on the historical correction information; If the correction frequency is greater than or equal to the correction frequency threshold, reduce the correction urgency threshold; If the correction frequency is less than the correction frequency threshold, the correction urgency threshold is increased.

6. The method according to claim 4, characterized in that, The method further includes: User type is identified based on the aforementioned voice information; The offline identification confidence level is adjusted based on the user type.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: If the first control instruction and the second final state operation instruction are not obtained within the preset time period, a progressive adjustment instruction between the first non-final state operation instruction and the first final state operation instruction is obtained based on the first non-final state operation instruction and the first final state operation instruction. If the device is controlled to complete execution based on the first final state operation instruction, the device is controlled to execute based on the progressive adjustment instruction. The progressive adjustment instruction is a progressive adjustment instruction divided by time or steps. If the device is controlled to complete execution based on the progressive adjustment instruction and the first control instruction and the second final state operation instruction are not obtained, the device is controlled based on the first final state operation instruction. After the device is controlled to complete the execution based on the progressive adjustment instruction and the first control instruction and the second final state operation instruction are obtained, the device is controlled based on the first control instruction and the second final state operation instruction.

8. The method according to any one of claims 1 to 6, characterized in that, The method further includes: If the first control command and the second final state operation command are not obtained within the preset time period, the first non-final state operation command and the voice information are sent to the target device so that the target device can recognize the voice information and obtain the third non-final state operation command and the third final state operation command to be executed before the third final state operation command is executed. The target device determines the corresponding second control command based on the first non-final state operation command and the third non-final state operation command. The second control command is a confirmation command or correction command generated by the target device based on the comparison result between the first non-final state operation command and the third non-final state operation command. The device is controlled based on the second control command and the third final state operation command.

9. The method according to claim 1, characterized in that, The method further includes: When the device is not connected to the network, obtain the execution risk information of the first final state operation instruction; When the offline recognition confidence of the offline lightweight model is greater than the confidence threshold and the execution risk information meets the requirements, the device is controlled based on the first final state operation instruction; If the offline recognition confidence of the offline lightweight model is less than the confidence threshold and the execution risk information does not meet the requirements, a prompt message is output. The prompt message is used to confirm the first final state operation instruction, and if the confirmation of the first final state operation instruction is obtained, the device is controlled based on the first final state operation instruction.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 9.

Images