Voice task scheduling method, device, system, and storage medium

Abstract

The present application relates to the field of data processing technology, and discloses a method and device, system, and storage medium for scheduling voice tasks, which are applied to terminal air conditioners. The method includes: performing audio classification on received voice tasks to obtain voice classification tasks; wherein the language classification tasks include non-wake-up tasks; performing network classification on non-wake-up tasks to obtain offline interaction tasks and online interaction tasks; obtaining the network connection status and terminal load status of the terminal air conditioner; and according to the network connection status and terminal load status, unloading online interaction tasks or offline interaction tasks to the collaborative device end for task processing, or the terminal air conditioner performs task processing locally. The method performs flexible task scheduling on voice tasks, effectively meeting the requirements of low-latency voice wake-up and high-reliability voice execution.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and for example, to a method and device, system, and storage medium for scheduling voice tasks. Background Art

[0002] With the rapid development of Mobile Edge Computing (MEC), edge smart homes integrated with MEC have become a new trend in the smart home appliance industry. The increasing number of voice-activated air conditioners and the increasing complexity of voice tasks are placing higher demands on them, such as low-latency wake-up and highly reliable voice execution in voice interaction scenarios.

[0003] In order to meet the low-latency wake-up and high-reliability voice execution requirements of voice air conditioners in voice interaction scenarios, the relevant technology (publication number CN111597025A) offloads tasks to different edge servers for task processing.

[0004] During the implementation of the embodiments of the present disclosure, it was found that at least the following problems exist in the related art:

[0005] Related technologies only process voice tasks locally or offload them to the edge for processing, which can alleviate the pressure of voice air conditioning in processing voice tasks, but cannot perform flexible task scheduling based on the task status of the voice tasks, making it difficult to meet the requirements of low-latency voice wake-up and high-reliability voice execution.

[0006] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0007] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0008] The embodiments of the present disclosure provide a method, device, system, and storage medium for scheduling voice tasks to perform flexible task scheduling and meet the requirements of low-latency voice wake-up and high-reliability voice execution.

[0009] In some embodiments, the method is applied to a terminal air conditioner, and the method includes: performing audio classification on the received voice task to obtain a voice classification task; wherein the language classification task includes a non-wake-up task; performing networking classification on the non-wake-up task to obtain an offline interaction task and an online interaction task; obtaining the network connection status and the terminal load status of the terminal air conditioner; according to the network connection status and the terminal load status, unloading the online interaction task or the offline interaction task to the collaborative device end for task processing, or the terminal air conditioner performs task processing locally.

[0010] In some embodiments, the collaborative device end includes an edge device end and a movable device end. According to the network connection status and the terminal load status, the online interaction task or the offline interaction task is unloaded to the collaborative device end for task processing, or the terminal air conditioner performs task processing locally, including: when the network connection status indicates that the network is connected, the online interaction task is unloaded to the edge device end for task processing; when the network connection status indicates that the network is disconnected and the terminal load status indicates overload, the offline interaction task is forwarded to the movable device end for task processing; when the network connection status indicates that the network is disconnected and the terminal load status indicates normal load, the terminal air conditioner performs task processing of the offline interaction task locally.

[0011] In some embodiments, the collaborative device end also includes a cloud side, the edge device end is configured with a first heavyweight voice processing model, and the cloud side is configured with a second heavyweight voice processing model, and the online interaction task is offloaded to the edge device end for task processing, including: obtaining the edge load status of the edge device end; when the edge load status indicates overload, unloading the online interaction task to the cloud side for task processing through the second heavyweight voice processing model; or, when the edge load status indicates normal load, unloading the online interaction task edge device end for task processing through the first heavyweight voice processing model.

[0012] In some embodiments, the collaborative device end includes a mobile device end, the mobile device end is configured with a lightweight speech recognition model, the language classification task also includes a wake-up task, and the method also includes: parsing the wake-up task to obtain a parsing result; when the parsing result includes wake-up audio, responding to the wake-up task according to the delay strategy; when the parsing result does not include wake-up audio, forwarding the parsing result to the mobile device end to respond to the wake-up task through the lightweight speech recognition model.

[0013] In some embodiments, the wake-up task is responded to according to the delay strategy, including: if the terminal air conditioner locally responds to the wake-up task within the delay period, the terminal air conditioner locally performs a wake-up response; if the terminal air conditioner locally does not respond to the wake-up task within the delay period, it is unloaded to the mobile device to respond to the task through a lightweight speech recognition model.

[0014] In some embodiments, the mobile device is configured with a microphone array, which is used to collect wake-up features included in the wake-up task; unloading to a lightweight speech recognition model configured on the mobile device to respond to the task includes: unloading to the mobile device to collect the wake-up features included in the wake-up task through the microphone array; when the microphone array collects the wake-up features, triggering the lightweight speech recognition model to respond to the wake-up task.

[0015] In some embodiments, it also includes: obtaining the task processing results of the online interactive task and the offline interactive task; sending the task processing results to the cloud, so that the cloud can dynamically configure the air-conditioning parameters according to the task processing results and environmental data and human body data; wherein, the environmental data and human body data are collected by the mobile device through a multimodal sensor and sent to the cloud.

[0016] In some embodiments, the voice task scheduling device includes a processor and a memory storing program instructions, and the processor is configured to execute the aforementioned voice task scheduling method when running the program instructions.

[0017] In some embodiments, the system is applied to a scheduling device for voice tasks, and the system includes: a terminal air conditioner; a collaborative device end, which is communicatively connected to the terminal air conditioner, including an edge device end, a movable device end, and a cloud.

[0018] In some embodiments, the storage medium stores program instructions, which, when executed, enable a computer to execute the aforementioned method for scheduling voice tasks.

[0019] The voice task scheduling method, device, system, and storage medium provided by the embodiments of the present disclosure can achieve the following technical effects:

[0020] The terminal air conditioner first performs audio classification on the received voice tasks to obtain voice classification tasks including non-wake-up tasks. Then the non-wake-up tasks are classified online to obtain offline interactive tasks and online interactive tasks. Finally, the terminal air conditioner unloads the online interactive tasks or offline interactive tasks to the collaborative device end for task processing according to the network connection status and the terminal load status, or the terminal air conditioner performs task processing locally. The embodiment of the present disclosure realizes a flexible task scheduling strategy by reasonably classifying voice tasks and unloading the classified offline or online interactive tasks, effectively meeting the requirements of low-latency voice wake-up and high-reliability voice execution, which is conducive to improving the user experience under voice interaction.

[0021] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] One or more embodiments are exemplarily described by corresponding drawings. These exemplary descriptions and drawings do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements. The drawings do not constitute a scale limitation. In addition,

[0023] Figure 1 This is an architectural diagram of a voice task scheduling system provided by an embodiment of the present disclosure;

[0024] Figure 2 is a schematic diagram of a first voice task scheduling method provided by an embodiment of the present disclosure;

[0025] Figure 3 is a schematic diagram of a second voice task scheduling method provided by an embodiment of the present disclosure;

[0026] Figure 4 is a schematic diagram of a third voice task scheduling method provided by an embodiment of the present disclosure;

[0027] Figure 5 is an application diagram of an embodiment of the present disclosure;

[0028] Figure 6 Schematic diagram of a voice task scheduling device provided by an embodiment of the present disclosure.

[0029] Reference numerals:

[0030] 100: terminal air conditioner; 200: mobile device end; 300: edge device end;

[0031] 400: cloud; 70: voice task scheduling device;

[0032] 700: processor; 701: memory; 702: communication interface; 703: bus. DETAILED DESCRIPTION

[0033] In order to be able to understand the features and technical content of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The accompanying drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the sake of convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0034] In the description of the embodiments of the present disclosure and the accompanying drawings, the terms "first," "second," and the like are used to distinguish similar items and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate to describe the embodiments of the present disclosure herein. In addition, the terms "including," "having," and any variations thereof are intended to cover non-exclusive inclusions.

[0035] Unless otherwise stated, the term "plurality" means two or more.

[0036] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B means: A or B.

[0037] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0038] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0039] Figure 1 FIG. 1 is a schematic diagram of a scheduling system for voice tasks according to an embodiment of the present disclosure. Figure 1 As shown, the scheduling system includes a terminal air conditioner 100, a mobile device 200, an edge device 300, and a cloud 400. The terminal air conditioner 100 is in communication with the mobile device 200, the edge device 300, and the cloud 400, and the edge device 300 is in communication with the cloud 400.

[0040] The terminal air conditioner 100 represents an intelligent voice air conditioner with a voice interaction function.

[0041] The mobile device end 200 represents a mobile service device that has both mobility and edge computing capabilities. For example, a drone or a home service robot. Specifically, the home service robot can be a sweeping robot equipped with a microphone array and an edge computing box. The mobile device end 200 is equipped with a microphone array, which is used to collect the wake-up features included in the wake-up task. It should be noted that the mobile device end 200 also has sensor integration capabilities. As an example, the mobile device end 200 is also equipped with a multimodal sensor, which is used to collect environmental data and human body data in real time. The environmental data includes the ambient temperature and humidity of the space where the terminal air conditioner is located and the number of users in the space. The human body data includes human thermal comfort data and user preference data. The human body thermal comfort data includes body temperature, the relative position of the user and the terminal air conditioner, the user's movement speed and movement direction, and the user's preference data includes a historical temperature curve that reflects the user's historical habits.

[0042] The cloud 400 may be a single server, a server cluster consisting of several servers, or a cloud computing service center, which is not limited in the embodiments of the present disclosure.

[0043] The edge device is equipped with a first-class heavyweight speech processing model, the cloud is equipped with a second-class heavyweight speech processing model, and the mobile device is equipped with a lightweight speech recognition model.

[0044] In the disclosed embodiments, the speech processing model represents a model that performs deep processing on text or instructions and generates air conditioning control parameters. As an example, when the model input is a wake-up task, the speech processing model performs a wake-up response on the model input and performs logical processing, and outputs the logical processing result as the model. As another example, when the model input is an instruction task, the speech processing model performs text conversion on the model input and generates a control intent, and outputs the control intent as the model. A heavyweight speech processing model represents a speech processing model with a large number of parameters, a large amount of computation, and a high memory usage.

[0045] In the disclosed embodiments, the speech recognition model represents a model that performs audio recognition on an audio signal to generate structured text or executable control instructions. As an example, when the model input is a wake-up audio (e.g., Xiaoyou Xiaoyou, Hello, air conditioner, etc.), the speech recognition model performs audio recognition on the model input, determines specific keywords, and generates structured text corresponding to the specific keywords. As another example, when the model input is a basic instruction (e.g., instruction words such as "turn on the air conditioner" or "turn on in cooling mode"), the speech recognition model performs audio recognition on the model input and generates executable control instructions. A heavyweight speech recognition model refers to a speech recognition model with a large number of parameters, a large amount of computation, and a high memory usage.

[0046] It's important to note that the model inputs and outputs of speech processing models differ from those of speech recognition models. Furthermore, the data sources, real-time requirements, and model complexity of speech processing models and speech recognition models differ. The former relies on audio data, sensor data, and historical data, while the latter relies solely on audio data. The former requires millisecond-level real-time performance, while the latter requires sub-second performance. The former's model complexity is lower than the latter's.

[0047] It should be understood that Figure 1 The number of terminal air conditioners, mobile device terminals and edge device terminals is only indicative. According to actual needs, there can be any number of terminal air conditioners, mobile device terminals and edge device terminals. For example, one terminal air conditioner can correspond to multiple mobile device terminals, and one terminal air conditioner can also correspond to multiple edge device terminals.

[0048] It should be noted that the voice task scheduling method provided in the embodiment of the present disclosure is generally jointly executed by the mobile terminal air conditioner, the mobile device side, the edge device side and the cloud side.

[0049] Based on the architecture of the scheduling system for the above voice tasks, combined with Figure 2 As shown, the embodiment of the present disclosure provides a method for scheduling voice tasks, including:

[0050] S01: The terminal air conditioner performs audio classification on the received voice task to obtain a voice classification task, wherein the language classification task includes a non-wake-up task.

[0051] In this step, the terminal air conditioner performs audio classification on the received voice task to obtain the voice classification task, including: the terminal air conditioner extracts the audio features of the voice task; when the audio features match the first audio features corresponding to the non-wake-up task, the terminal air conditioner determines that the voice classification task after the audio classification is a non-wake-up task; when the audio features match the second audio features corresponding to the wake-up task, the terminal air conditioner determines that the voice classification task after the audio classification is a wake-up task.

[0052] S02: The terminal air conditioner classifies the non-wake-up tasks online to obtain offline interaction tasks and online interaction tasks.

[0053] S03, the terminal air conditioner obtains the network connection status and terminal load status of the terminal air conditioner.

[0054] S04, the terminal air conditioner offloads the online interactive task or the offline interactive task to the collaborative device end for task processing according to the network connection status and the terminal load status, or the terminal air conditioner processes the task locally.

[0055] Using the voice task scheduling method provided by the embodiment of the present disclosure, the terminal air conditioner first performs audio classification on the received voice tasks to obtain voice classification tasks including non-wake-up tasks. Then, the non-wake-up tasks are classified online to obtain offline interactive tasks and online interactive tasks. Finally, the terminal air conditioner unloads the online interactive tasks or offline interactive tasks to the collaborative device end for task processing according to the network connection status and the terminal load status, or the terminal air conditioner performs task processing locally. The embodiment of the present disclosure realizes a flexible task scheduling strategy by reasonably classifying voice tasks and unloading the classified offline or online interactive tasks, which effectively meets the requirements of low-latency voice wake-up and high-reliability voice execution, and is conducive to improving the user experience under voice interaction.

[0056] Optionally, the collaborative device side includes an edge device side and a mobile device side. The terminal air conditioner offloads online interactive tasks or offline interactive tasks to the collaborative device side for task processing based on the network connection status and terminal load status, or the terminal air conditioner performs task processing locally, including:

[0057] When the network connection status indicates that the network is connected, the terminal air conditioner offloads the online interaction task to the edge device for task processing.

[0058] When the network connection status indicates that the network is disconnected and the terminal load status indicates that the terminal is overloaded, the terminal air conditioner forwards the offline interactive task to the mobile device for task processing.

[0059] When the network connection status indicates that the network is disconnected and the terminal load status indicates a normal load, the terminal air conditioner locally performs task processing of the offline interactive task.

[0060] In this way, after the terminal air conditioner obtains its network connection status and terminal load status, if the network connection status indicates a network connection, the online interactive task is directly unloaded to the edge device to perform the task. If the network connection status indicates a network disconnection, the terminal load status is further judged. When the terminal load status indicates an overload, it indicates that the terminal air conditioner currently does not have the ability to process the task. At this time, the terminal air conditioner forwards the offline interactive task to the mobile device to perform the task. When the terminal load status indicates a normal load, the terminal device performs task processing of the offline interactive task locally. In this way, the embodiment of the present disclosure can flexibly unload offline or online interactive tasks according to the network connection status of the terminal air conditioner and its terminal load status, which is conducive to improving the reliability of voice execution and enhancing the user experience under voice interaction.

[0061] Optionally, the collaborative device side also includes the cloud. The terminal air conditioner offloads the online interaction tasks to the edge device side for task processing, including:

[0062] The terminal air conditioner obtains the edge load status of the edge device.

[0063] When the edge load status indicates overload, the terminal air conditioner offloads the online interaction task to the cloud to be processed by the second heavyweight voice processing model. Or,

[0064] When the edge load status indicates a normal load, the terminal air conditioner offloads the online interactive task to the edge device side so that the task can be processed through the first heavyweight voice processing model.

[0065] In this way, when the network connection status indicates a network connection, the terminal air conditioner obtains the edge load status of the edge device side. When the edge load status indicates an overload, it means that the edge device side currently does not have the ability to process the task, and the terminal air conditioner directly unloads the online interactive task to the cloud to process the task through its second heavyweight voice processing model. When the edge load status indicates a normal load, the terminal air conditioner directly unloads the online interactive task to the edge device side to process the task through its first heavyweight voice processing model. The embodiment of the present disclosure configures heavyweight voice processing models on the edge device side and the cloud side respectively and introduces a second-order task offloading strategy to ensure that the online interactive tasks in non-wake-up tasks can be reliably executed.

[0066] Combine Figure 3 As shown, the embodiment of the present disclosure also provides a method for scheduling voice tasks, including:

[0067] S11: The terminal air conditioner performs audio classification on the received voice task to obtain a voice classification task. The language classification task includes a non-wake-up task and a wake-up task.

[0068] In step S12, the terminal air conditioner classifies the non-wake-up tasks into online categories to obtain offline interaction tasks and online interaction tasks.

[0069] S13, the terminal air conditioner obtains the network connection status and terminal load status of the terminal air conditioner.

[0070] S14, the terminal air conditioner offloads the online interactive task or the offline interactive task to the collaborative device end for task processing according to the network connection status and the terminal load status, or the terminal air conditioner processes the task locally.

[0071] S15, the terminal air conditioner analyzes the wake-up task and obtains the analysis result.

[0072] S16: When the analysis result includes the wake-up audio, the terminal air conditioner responds to the wake-up task according to the delay strategy.

[0073] S17, when the analysis result does not include the wake-up audio, the terminal air conditioner forwards the analysis result to the mobile device to respond to the wake-up task through the lightweight speech recognition model.

[0074] Using the voice task scheduling method provided by the embodiment of the present disclosure, the terminal air conditioner first performs audio classification on the received voice tasks to obtain voice classification tasks including non-wake-up tasks and wake-up tasks. Then, the terminal air conditioner performs network classification on the non-wake-up tasks to obtain offline interactive tasks and online interactive tasks. The terminal air conditioner then unloads the online interactive tasks or offline interactive tasks to the collaborative device end for task processing based on the network connection status and the terminal load status, or the terminal air conditioner performs task processing locally. The embodiment of the present disclosure realizes a flexible task scheduling strategy by reasonably classifying voice tasks and unloading the classified offline or online interactive tasks.

[0075] In addition, since the wake-up task requires the terminal air conditioner to respond or execute within the specified delay, the terminal air conditioner parses the wake-up task after obtaining the voice classification task to obtain the analysis result. If the analysis result includes wake-up audio, the wake-up task is responded to according to the delayed response strategy to perform the wake-up response normally. When the analysis result does not include wake-up audio, the terminal air conditioner forwards the analysis result to the mobile device side, and responds to the wake-up task through the lightweight speech recognition model configured therein to perform the wake-up response normally. The terminal air conditioner provided in the embodiment of the present disclosure can ensure the reliability of the wake-up operation in cooperation with the mobile device side, and more efficiently meet the requirements of low-latency voice wake-up and high-reliability voice execution.

[0076] Optionally, the terminal air conditioner responds to the wake-up task according to a delay strategy, including:

[0077] If the terminal air conditioner locally responds to the wake-up task within the delay time, the terminal air conditioner locally performs a wake-up response.

[0078] If the terminal air conditioner does not respond to the wake-up task locally within the delay time, the terminal air conditioner is offloaded to the mobile device to respond to the task through a lightweight speech recognition model.

[0079] In this way, since the wake-up task requires the terminal air conditioner to respond or execute within the specified delay, when the terminal air conditioner obtains the analysis result including the wake-up audio, the terminal air conditioner continues to determine whether the terminal air conditioner responds to the wake-up task locally within the delay period. If the terminal air conditioner locally determines to respond, the wake-up response is directly performed locally. If the terminal air conditioner does not respond locally within the delay period, it is directly offloaded to the mobile device to respond to the task through the lightweight speech recognition model configured there. In this way, the reliability of the wake-up task response is guaranteed.

[0080] Optionally, the terminal air conditioner is unloaded to a mobile device to respond to the task through a lightweight speech recognition model, including: the terminal air conditioner is unloaded to a mobile device to collect the wake-up features contained in the wake-up task through a microphone array; when the microphone array collects the wake-up features, the terminal air conditioner triggers the lightweight speech recognition model to respond to the wake-up task.

[0081] In this way, by configuring a microphone array on the mobile device, it is possible to take into account both wake-up sensitivity and false wake-up, thereby ensuring the reliability of voice task execution.

[0082] Combine Figure 4 As shown, the embodiment of the present disclosure also provides a method for scheduling voice tasks, including:

[0083] S21: The terminal air conditioner performs audio classification on the received voice task to obtain a voice classification task, wherein the language classification task includes a non-wake-up task.

[0084] In step S22, the terminal air conditioner classifies the non-wake-up tasks into an online classification to obtain offline interaction tasks and online interaction tasks.

[0085] S23, the terminal air conditioner obtains the network connection status and terminal load status of the terminal air conditioner.

[0086] S24, the terminal air conditioner offloads the online interactive task or the offline interactive task to the collaborative device end for task processing according to the network connection status and the terminal load status, or the terminal air conditioner processes the task locally.

[0087] S25, the terminal air conditioner obtains the task processing results of the online interactive task and the offline interactive task.

[0088] In step S26, the terminal air conditioner sends the task processing result to the cloud, so that the cloud can dynamically configure the air conditioner parameters based on the task processing result and the environmental data and human body data. The environmental data and human body data are collected by the mobile device through multimodal sensors and sent to the cloud.

[0089] Using the voice task scheduling method provided by the embodiment of the present disclosure, the terminal air conditioner first performs audio classification on the received voice tasks to obtain voice classification tasks including non-wake-up tasks. Then, the non-wake-up tasks are classified online to obtain offline interactive tasks and online interactive tasks. Finally, the terminal air conditioner unloads the online interactive tasks or offline interactive tasks to the collaborative device end for task processing according to the network connection status and the terminal load status, or the terminal air conditioner performs task processing locally. The embodiment of the present disclosure realizes a flexible task scheduling strategy by reasonably classifying voice tasks and unloading the classified offline or online interactive tasks. After the terminal air conditioner sends the task processing results to the cloud, the cloud dynamically configures the air conditioning parameters according to the task processing results and the environmental data and human body data sent by the movable device end to realize adaptive control of the terminal air conditioner and meet the user's flexible service needs.

[0090] Optionally, the cloud dynamically configures air-conditioning parameters based on the task processing results, environmental data, and human body data, including: the cloud inputs the task processing results, environmental data, and human body data into an LSTM (Long Short-Term Memory) model for model training, obtains the model output, and determines the model output as the air-conditioning parameters of the terminal air conditioner.

[0091] In practical applications, the mobile device is a drone. Figure 5 As shown, the method for scheduling voice tasks specifically performs the following steps:

[0092] S31: The terminal air conditioner performs audio classification on the received voice task to obtain a voice classification task.

[0093] S32, the terminal air conditioner determines whether the voice classification task is a wake-up task, if not, executes S33, if so, executes S41.

[0094] In step S33, the terminal air conditioner classifies the non-wake-up tasks into an online category to obtain offline interaction tasks and online interaction tasks.

[0095] S34, the terminal air conditioner determines whether it is connected to the network, if so, executes S35, otherwise executes S38.

[0096] S35: The terminal air conditioner obtains the edge load status of the edge device and then executes S36 or S37.

[0097] S36, when the edge load status indicates overload, the terminal air conditioner offloads the online interaction task to the cloud so that the task is processed by the second heavyweight voice processing model.

[0098] S37, when the edge load status indicates a normal load, the terminal air conditioner offloads the online interaction task to the edge device end so that the task is processed by the first heavyweight voice processing model.

[0099] S38, the terminal air conditioner determines whether the terminal load status is overloaded, if so, execute S39, otherwise execute S40.

[0100] S39, the terminal air conditioner forwards the offline interaction task to the drone for task processing.

[0101] S40, the terminal air conditioner locally performs task processing of the offline interactive task.

[0102] S41: The terminal air conditioner analyzes the wake-up task and obtains the analysis result, and then executes S42 or S43.

[0103] S42, the terminal air conditioner determines whether the analysis result includes the wake-up audio, and if so, executes S43, otherwise executes S45.

[0104] S43, the terminal air conditioner determines whether the terminal air conditioner locally responds to the wake-up task within the delay time. If so, execute S44, otherwise execute S45.

[0105] S44, the terminal air conditioner performs a local wake-up response.

[0106] S45, the terminal air conditioning is offloaded to the UAV to respond to the task through a lightweight speech recognition model.

[0107] S46: The terminal air conditioner obtains the task processing results of the online interactive task and the offline interactive task, and sends them to the cloud.

[0108] S47, the cloud dynamically configures air conditioning parameters based on task processing results, environmental data, and human body data.

[0109] Combine Figure 6 As shown, an embodiment of the present disclosure provides a voice task scheduling device 70, comprising a processor 700 and a memory 701. Optionally, the voice task scheduling device 70 may further comprise a communication interface 702 and a bus 703. The processor 700, the communication interface 702, and the memory 701 may communicate with each other via the bus 703. The communication interface 702 may be used for information transmission. The processor 700 may invoke logic instructions in the memory 701 to execute the voice task scheduling method of the above embodiment.

[0110] In addition, the logic instructions in the memory 701 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0111] Memory 701, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of the present disclosure. Processor 700 executes the program instructions / modules stored in memory 701 to execute functional applications and data processing, thereby implementing the voice task scheduling method in the above-mentioned embodiments.

[0112] The memory 701 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data generated based on the use of the terminal device. Furthermore, the memory 701 may include high-speed random access memory and non-volatile memory.

[0113] The disclosed embodiment also provides a voice task scheduling system, which is applied to the above-mentioned voice task scheduling device, and the above-mentioned voice task scheduling system includes: a terminal air conditioner and a collaborative device end. The collaborative device end is communicatively connected with the terminal air conditioner, and the collaborative device end includes an edge device end, a movable device end, and a cloud. The above-mentioned voice task scheduling device can be installed in the terminal air conditioner. The installation relationship described here is not limited to placement inside the terminal air conditioner, but also includes installation connections with other components of the voice task scheduling system, including but not limited to physical connections, electrical connections, or signal transmission connections. It can be understood by those skilled in the art that the above-mentioned voice task scheduling device 70 can be adapted to a feasible system body, thereby realizing other feasible embodiments.

[0114] An embodiment of the present disclosure provides a storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the above-mentioned method for scheduling voice tasks.

[0115] The technical solutions of the embodiments of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present disclosure. The aforementioned storage medium may be a non-transitory storage medium, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, among other media capable of storing program code.

[0116] The above description and accompanying drawings sufficiently illustrate the embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, process, and other changes. The embodiments represent only possible variations. Unless expressly required, individual components and functions are optional, and the order of operations may vary. Portions and features of some embodiments may be included in or substituted for portions and features of other embodiments. When used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Without further limitation, an element defined by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, or apparatus that includes the element. Each embodiment herein may focus on its differences from other embodiments, and similar portions between the various embodiments may be referenced across them. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can be found in the description of the method part.

[0117] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. The technicians will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0118] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices and equipment) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units may be merely a logical functional division. In actual implementation, other divisions may be used, such as combining or integrating multiple units or components into another system, or omitting or disabling some features. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be through interfaces, indirect couplings or communication connections between devices or units, and may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of these units may be selected to implement the embodiments according to actual needs. Furthermore, the functional units in the disclosed embodiments may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0119] The flowcharts and block diagrams in the accompanying drawings show the possible implementation architectures, functions and operations of the systems, methods and computer program products according to the embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of the code, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in an order different from that disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for scheduling voice tasks, characterized in that: Applied to terminal air conditioners, the collaborative device side includes an edge device side, a cloud side, and a mobile device side. The edge device side is configured with a first-class heavyweight voice processing model, and the cloud side is configured with a second-class heavyweight voice processing model. The method includes: Performing audio classification on the received speech task to obtain a speech classification task; wherein the language classification task includes a non-wake-up task; Classify non-wake-up tasks online to obtain offline interaction tasks and online interaction tasks; Obtain the network connection status and terminal load status of the terminal air conditioner; Based on the network connection status and terminal load status, online or offline interactive tasks are offloaded to the collaborative device for task processing, or the terminal air conditioner processes the tasks locally; Among them, according to the network connection status and terminal load status, online interactive tasks or offline interactive tasks are offloaded to the collaborative device end for task processing, or the terminal air conditioner performs task processing locally, including: In the case where the network connection status indicates a network connection, obtaining an edge load status at the edge device end; When the edge load status indicates overload, the online interaction task is offloaded to the cloud for task processing by the second heavyweight speech processing model; or, when the edge load status indicates normal load, the online interaction task is offloaded to the edge device for task processing by the first heavyweight speech processing model; When the network connection status indicates that the network is disconnected and the terminal load status indicates that the terminal is overloaded, the offline interactive task is forwarded to the mobile device for task processing; When the network connection status indicates that the network is disconnected and the terminal load status indicates a normal load, the terminal air conditioner locally performs task processing of the offline interactive task.

2. The method according to claim 1, characterized in that The collaborative device side includes a mobile device side, the mobile device side is configured with a lightweight speech recognition model, the language classification task also includes a wake-up task, and the method further includes: Parse the wake-up task and obtain the parsing results; If the parsing result includes the wake-up audio, respond to the wake-up task according to the delay strategy; If the parsing result does not include the wake-up audio, the parsing result is forwarded to the mobile device to respond to the wake-up task through the lightweight speech recognition model.

3. The method according to claim 2, characterized in that Respond to wakeup tasks according to the delay strategy, including: If the terminal air conditioner responds to the wake-up task locally within the delay time, the terminal air conditioner responds locally to the wake-up task; If the terminal air conditioner does not respond to the wake-up task locally within the delay time, it is offloaded to the mobile device to respond to the task through a lightweight speech recognition model.

4. The method according to claim 3, characterized in that The mobile device is equipped with a microphone array for collecting the wake-up features included in the wake-up task; Offload to a lightweight speech recognition model configured on a mobile device to respond to tasks, including: Offloading to a mobile device to collect the wake-up features included in the wake-up task through a microphone array; When the microphone array collects the wake-up features, the lightweight speech recognition model is triggered to respond to the wake-up task.

5. The method according to any one of claims 1 to 4, characterized in that Also includes: Obtain the task processing results of online interactive tasks and offline interactive tasks; The task processing results are sent to the cloud so that the cloud can dynamically configure the air-conditioning parameters based on the task processing results and environmental data and human body data; wherein, the environmental data and human body data are collected by the mobile device through multimodal sensors and sent to the cloud.

6. A voice task scheduling device, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the method for scheduling voice tasks according to any one of claims 1 to 5 when running the program instructions.

7. A voice task scheduling system, characterized in that: The system for scheduling voice tasks according to claim 6 includes: Terminal air conditioning; The collaborative device side communicates with the terminal air conditioner, including the edge device side, the mobile device side, and the cloud.

8. A storage medium storing program instructions, characterized in that: When the program instructions are executed, the computer is configured to execute the method for scheduling voice tasks according to any one of claims 1 to 5.

Images