Voice task scheduling method, device and system, and storage medium

Abstract

The invention relates to the technical field of data processing, and discloses a voice task scheduling method, device and system and a storage medium, which are applied to a terminal air conditioner, and the method comprises the following steps: carrying out audio classification on a received voice task to obtain a voice classification task; wherein the language classification task comprises a non-awakening task; performing networking classification on the non-awakening tasks to obtain offline interaction tasks and online interaction tasks; obtaining a network connection state and a terminal load state of the terminal air conditioner; and according to the network connection state and the terminal load state, the online interaction task or the offline interaction task is unloaded to the collaborative equipment end for task processing, or the terminal air conditioner locally carries out task processing. According to the method, flexible task scheduling is carried out on the voice task, and the requirements of voice low-delay wakeup and high-reliability voice execution are effectively met.

Description

Technical Field

[0001] This application relates to the technical field of data processing, for example, to a method and apparatus, system, and storage medium for scheduling voice tasks. Background Art

[0002] Currently, with the rapid development of MEC (Mobile Edge Computing), edge smart homes combined with MEC have become a new direction in the smart home appliance industry. Due to the increasing number of voice air conditioners and the gradually complex voice tasks, higher requirements are put forward for low-latency wake-up and high-reliability voice execution of voice air conditioners in voice interaction scenarios.

[0003] To meet the requirements of low-latency wake-up and high-reliability voice execution of voice air conditioners in voice interaction scenarios, related technologies (publication number: CN111597025A) offload tasks to different edge servers for task processing.

[0004] In the process of implementing the embodiments of the present disclosure, it is found that at least the following problems exist in related technologies: Related technologies only process voice tasks locally or offload them to the edge for processing, which can relieve the pressure of voice air conditioners in processing voice tasks, but cannot perform elastic task scheduling according to the task situation of voice tasks, and it is difficult to meet the requirements of low-latency wake-up and high-reliability voice execution.

[0005] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0006] To provide a basic understanding of some aspects of the disclosed embodiments, a simple summary is given below. The summary is not a comprehensive review, nor is it intended to identify key / important elements or delineate the scope of protection of these embodiments, but rather serves as a preamble to the following detailed description.

[0007] Embodiments of the present disclosure provide a method, apparatus, system, and storage medium for scheduling voice tasks to perform elastic task scheduling and meet the requirements of low-latency wake-up and high-reliability voice execution.

[0008] In some embodiments, the method is applied to a terminal air conditioner, and the method includes: performing audio classification on a received voice task to obtain a voice classification task; wherein, the language classification task includes a non-wake-up task; performing network classification on the non-wake-up task to obtain an offline interaction task and an online interaction task; obtaining the network connection status and the terminal load status of the terminal air conditioner; according to the network connection status and the terminal load status, offloading the online interaction task or the offline interaction task to a collaborative device side for task processing, or, performing task processing locally on the terminal air conditioner.

[0009] In some embodiments, the collaborative device side includes an edge device side and a mobile device side. According to the network connection status and the terminal load status, offloading the online interaction task or the offline interaction task to the collaborative device side for task processing, or, performing task processing locally on the terminal air conditioner, includes: when the network connection status indicates a network connection, offloading the online interaction task to the edge device side for task processing; when the network connection status indicates a network disconnection and the terminal load status indicates an overload, forwarding the offline interaction task to the mobile device side for task processing; when the network connection status indicates a network disconnection and the terminal load status indicates a normal load, the terminal air conditioner locally executes the task processing of the offline interaction task.

[0010] In some embodiments, the collaborative device side further includes a cloud. The edge device side is configured with a first heavyweight voice processing model, and the cloud is configured with a second heavyweight voice processing model. Offloading the online interaction task to the edge device side for task processing includes: obtaining the edge load status of the edge device side; when the edge load status indicates an overload, offloading the online interaction task to the cloud to perform task processing through the second heavyweight voice processing model; or, when the edge load status indicates a normal load, offloading the online interaction task to the edge device side to perform task processing through the first heavyweight voice processing model.

[0011] In some embodiments, the collaborative device side includes a mobile device side. The mobile device side is configured with a lightweight voice recognition model. The language classification task further includes a wake-up task. The method further includes: parsing the wake-up task to obtain a parsing result; when the parsing result includes a wake-up audio, responding to the wake-up task according to a delay strategy; when the parsing result does not include a wake-up audio, forwarding the parsing result to the mobile device side to respond to the wake-up task through the lightweight voice recognition model.

[0012] In some embodiments, responding to the wake-up task according to a delay strategy includes: when the terminal air conditioner locally responds to the wake-up task within the delay duration, the terminal air conditioner locally performs a wake-up response; when the terminal air conditioner does not locally respond to the wake-up task within the delay duration, offloading it to the mobile device side to perform a task response through the lightweight voice recognition model.

[0013] In some embodiments, the mobile device is configured with a microphone array, which is used to collect the wake-up features included in the wake-up task; and unload the wake-up task to a lightweight speech recognition model configured on the mobile device for task response, including: unloading to the mobile device to collect the wake-up features included in the wake-up task through the microphone array; and triggering the lightweight speech recognition model to respond to the wake-up task when the wake-up features are collected by the microphone array.

[0014] In some embodiments, it further includes: obtaining the task processing results of the online interaction task and the offline interaction task respectively; sending the task processing results to the cloud so that the cloud dynamically configures the air conditioner parameters according to the task processing results, environmental data, and human body data; wherein the environmental data and human body data are collected by the mobile device through a multi-modal sensor and sent to the cloud.

[0015] In some embodiments, the scheduling device for the voice task includes a processor and a memory storing program instructions, and the processor is configured to execute the scheduling method for the voice task as described above when running the program instructions.

[0016] In some embodiments, the system is applied to a scheduling device for voice tasks, and the system includes: a terminal air conditioner; a collaborative device end, communicatively connected to the terminal air conditioner, including an edge device end, a mobile device end, and a cloud.

[0017] In some embodiments, the storage medium stores program instructions, and when the program instructions are running, they are used to cause a computer to execute the scheduling method for the voice task as described above.

[0018] The scheduling method, device, system, and storage medium for voice tasks provided by the embodiments of the present disclosure can achieve the following technical effects: The terminal air conditioner first performs audio classification on the received voice task to obtain a voice classification task including non-wake-up tasks. Then, it performs online classification on the non-wake-up tasks to obtain offline interaction tasks and online interaction tasks. Finally, the terminal air conditioner unloads the online interaction tasks or offline interaction tasks to the collaborative device end for task processing according to the network connection status and the terminal load status, or the terminal air conditioner performs task processing locally. The embodiments of the present disclosure realize an elastic task scheduling strategy by reasonably classifying voice tasks and unloading the classified offline or online interaction tasks, effectively meeting the requirements of low-latency wake-up of voice and high-reliability voice execution, and being beneficial to improving the user experience under voice interaction.

[0019] The above general description and the following description are only exemplary and explanatory, and are not used to limit this application. Description of the Drawings

[0020] One or more embodiments are exemplarily illustrated by corresponding accompanying drawings. These exemplary illustrations and the accompanying drawings do not constitute a limitation on the embodiments. Elements with the same reference numerals in the accompanying drawings are shown as similar elements. The accompanying drawings do not constitute a scale limitation, and wherein: Figure 1 is an architecture diagram of a scheduling system for voice tasks provided by an embodiment of the present disclosure; Figure 2 is a schematic diagram of a first scheduling method for voice tasks provided by an embodiment of the present disclosure; Figure 3 is a schematic diagram of a second scheduling method for voice tasks provided by an embodiment of the present disclosure; Figure 4 is a schematic diagram of a third scheduling method for voice tasks provided by an embodiment of the present disclosure; Figure 5 is an application schematic diagram of an embodiment of the present disclosure; Figure 6 is a schematic diagram of a scheduling device for a voice task provided by an embodiment of the present disclosure.

[0021] Reference numerals: 100: Terminal air conditioner; 200: Mobile device end; 300: Edge device end; 400: Cloud; 70: Scheduling device for voice tasks; 700: Processor; 701: Memory; 702: Communication interface; 703; Bus. Detailed implementation manners

[0022] In order to be able to understand the features and technical content of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings. The attached accompanying drawings are for reference and illustration only, and are not used to limit the embodiments of the present disclosure. In the following technical description, for the sake of explanation, multiple details are provided to provide a full understanding of the disclosed embodiments. However, one or more embodiments can still be implemented without these details. In other cases, well-known structures and devices can be shown in a simplified manner.

[0023] The terms "first", "second", etc. in the specification of the embodiments of the present disclosure and the above accompanying drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so as to implement the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion.

[0024] Unless otherwise specified, the term "plurality" means two or more.

[0025] In the embodiments of the present disclosure, the character " / " indicates an "or" relationship between the preceding and following objects. For example, A / B means: A or B.

[0026] The term "and / or" is a description of the association relationship of an object, indicating that there can be three relationships. For example, A and / or B means: A or B, or, the three relationships of A and B.

[0027] The term "corresponding" can refer to an association relationship or a binding relationship. A corresponding to B means that there is an association relationship or a binding relationship between A and B.

[0028] Figure 1 It is a schematic diagram of the scheduling system for the voice task in the embodiments of the present disclosure. As Figure 1 shown, the scheduling system includes a terminal air conditioner 100, a mobile device terminal 200, an edge device terminal 300, and a cloud 400. The terminal air conditioner 100 is communicatively connected to the mobile device terminal 200, the edge device terminal 300, and the cloud 400. The edge device terminal 300 is communicatively connected to the cloud 400.

[0029] The terminal air conditioner 100 represents an intelligent voice air conditioner with voice interaction function.

[0030] The mobile device terminal 200 represents a mobile service device with both mobility and edge computing capabilities. For example, a drone, a home service robot. Among them, the home service robot can specifically be a floor cleaning robot configured with a microphone array and an edge computing box. The mobile device terminal 200 is configured with a microphone array, and the microphone array is used to collect the wake-up features included in the wake-up task. It should be noted that the mobile device terminal 200 also has sensor integration capabilities. As an example, the mobile device terminal 200 is also configured with a multi-modal sensor, and the multi-modal sensor is used to collect environmental data and human body data in real time. Among them, the environmental data includes the environmental temperature and humidity of the space where the terminal air conditioner is located and the number of users in the space. The human body data includes human thermal comfort data and user preference data. The human thermal comfort data such as body temperature, the relative position of the user and the terminal air conditioner, the moving speed and moving direction of the user. The user preference data includes a historical temperature curve used to reflect the user's historical habits.

[0031] The cloud 400 can be a server, or a server cluster composed of several servers, or can also be a cloud computing service center. The embodiments of the present disclosure do not limit this.

[0032] Among them, the edge device terminal is configured with a first heavyweight voice processing model. The cloud is configured with a second heavyweight voice processing model. The mobile device terminal is configured with a lightweight voice recognition model.

[0033] In the disclosed embodiments, the speech processing model represents a model that deeply processes text or instructions and generates air conditioner control parameters. As an example, when the model input is a wake-up task, the speech processing model performs a wake-up response on the model input and conducts logical processing, and uses the logical processing result as the model output. As another example, when the model input is an instruction task, the speech processing model performs text conversion on the model input and generates a control intention, and uses the control intention as the model output. A heavyweight speech processing model represents a speech processing model with a large number of parameters, a large amount of computation, and high memory occupancy.

[0034] In the disclosed embodiments, the speech recognition model represents a model that performs audio recognition on an audio signal to generate structured text or executable control instructions. As an example, when the model input is a wake-up audio (e.g., "Xiaoyou Xiaoyou", "Hello, air conditioner", etc.), the speech recognition model performs audio recognition on the model input, determines specific keywords, and generates structured text corresponding to the specific keywords. As another example, when the model input is a basic instruction (e.g., instruction words such as "turn on the air conditioner", "turn on according to the cooling mode", etc.), the speech recognition model performs audio recognition on the model input and generates an executable control instruction. A heavyweight speech recognition model represents a speech recognition model with a large number of parameters, a large amount of computation, and high memory occupancy.

[0035] It should be noted that the model inputs and model outputs of the speech processing model and the speech recognition model are different. In addition, the data sources, real-time requirements, and model complexities of the speech processing model and the speech recognition model are different. The data source of the former is audio data, sensor data, and historical data, while the data source of the latter is audio data. The real-time requirement of the former is in milliseconds, while the real-time requirement of the latter is in sub-seconds. The model complexity of the former is lower than that of the latter.

[0036] It should be understood that Figure 1 the numbers of the terminal air conditioners, mobile device terminals, and edge device terminals in

[0037] It should be noted that the scheduling method for speech tasks provided in the disclosed embodiments of the present disclosure is generally jointly executed by the mobile terminal air conditioner, mobile device terminals, edge device terminals, and the cloud.

[0038] Based on the architecture of the above-mentioned speech task scheduling system, combined with Figure 2 as shown in S01. The terminal air conditioner classifies the received voice task by audio to obtain a voice classification task. Among them, the language classification task includes a non-awakening task.

[0039] In this step, the terminal air conditioner classifies the received voice task by audio to obtain a voice classification task, including: the terminal air conditioner extracts the audio features of the voice task; when the audio features match the first audio features corresponding to the non-awakening task, the terminal air conditioner determines that the voice classification task after audio classification is a non-awakening task; when the audio features match the second audio features corresponding to the awakening task, the terminal air conditioner determines that the voice classification task after audio classification is an awakening task.

[0040] S02. The terminal air conditioner classifies the non-awakening task by network to obtain an offline interaction task and an online interaction task.

[0041] S03. The terminal air conditioner obtains the network connection status and the terminal load status of the terminal air conditioner.

[0042] S04. The terminal air conditioner unloads the online interaction task or the offline interaction task to the collaborative device side for task processing according to the network connection status and the terminal load status, or the terminal air conditioner performs task processing locally.

[0043] By using the voice task scheduling method provided in the embodiments of the present disclosure, the terminal air conditioner first classifies the received voice task by audio to obtain a voice classification task including a non-awakening task. Then, it classifies the non-awakening task by network to obtain an offline interaction task and an online interaction task. Finally, the terminal air conditioner unloads the online interaction task or the offline interaction task to the collaborative device side for task processing according to the network connection status and the terminal load status, or the terminal air conditioner performs task processing locally. The embodiments of the present disclosure realize an elastic task scheduling strategy by reasonably classifying voice tasks and unloading the classified offline or online interaction tasks, effectively meeting the requirements of low-latency voice wake-up and high-reliability voice execution, and are beneficial to improving the user experience under voice interaction.

[0044] Optionally, the collaborative device side includes an edge device side and a mobile device side. The terminal air conditioner unloads the online interaction task or the offline interaction task to the collaborative device side for task processing according to the network connection status and the terminal load status, or the terminal air conditioner performs task processing locally, including: When the network connection status indicates network connection, the terminal air conditioner unloads the online interaction task to the edge device side for task processing.

[0045] When the network connection status indicates network disconnection and the terminal load status indicates overloading, the terminal air conditioner forwards the offline interaction task to the mobile device side for task processing.

[0046] When the network connection status indicates a network disconnection and the terminal load status indicates normal load, the terminal air conditioner locally executes the task processing of the offline interaction task.

[0047] In this way, after the terminal air conditioner obtains its network connection status and terminal load status, if the network connection status indicates a network connection, it directly offloads the online interaction task to the edge device side for task processing. If the network connection status indicates a network disconnection, it further determines the terminal load status. When the terminal load status indicates overloading, it means that the terminal air conditioner currently does not have the ability to process tasks. At this time, the terminal air conditioner forwards the offline interaction task to the mobile device side for task processing. When the terminal load status indicates normal load, the terminal device locally performs the task processing of the offline interaction task. In this way, the embodiments of the present disclosure can elastically offload offline or online interaction tasks according to the network connection situation of the terminal air conditioner and its terminal load status, which is beneficial to improving the reliability of voice execution and enhancing the user experience under voice interaction.

[0048] Optionally, the collaborative device side further includes the cloud. The terminal air conditioner offloading the online interaction task to the edge device side for task processing includes: The terminal air conditioner obtains the edge load status of the edge device side.

[0049] In the case where the edge load status indicates overloading, the terminal air conditioner offloads the online interaction task to the cloud to perform task processing through the second heavyweight voice processing model. Or, In the case where the edge load status indicates normal load, the terminal air conditioner offloads the online interaction task to the edge device side to perform task processing through the first heavyweight voice processing model.

[0050] In this way, when the network connection status indicates a network connection, the terminal air conditioner obtains the edge load status of the edge device side. When the edge load status indicates overloading, it means that the edge device side currently does not have the ability to process tasks, and the terminal air conditioner directly offloads the online interaction task to the cloud to perform task processing through its second heavyweight voice processing model. When the edge load status indicates normal load, the terminal air conditioner directly offloads the online interaction task to the edge device side to perform task processing through its first heavyweight voice processing model. The embodiments of the present disclosure configure heavyweight voice processing models at the edge device side and the cloud respectively and introduce a second-order task offloading strategy to ensure the reliable execution of online interaction tasks in non-awakening tasks.

[0051] Combined with Figure 3 As shown, the embodiments of the present disclosure also provide a scheduling method for voice tasks, including: S11, the terminal air conditioner classifies the received voice task by audio to obtain a voice classification task. Among them, the language classification task includes non-awakening tasks and awakening tasks.

[0052] S12, the terminal air conditioner classifies the non-wake-up tasks for networking to obtain offline interaction tasks and online interaction tasks.

[0053] S13, the terminal air conditioner obtains the network connection status and the terminal load status of the terminal air conditioner.

[0054] S14, the terminal air conditioner unloads the online interaction task or the offline interaction task to the collaborative device side for task processing according to the network connection status and the terminal load status, or the terminal air conditioner performs task processing locally.

[0055] S15, the terminal air conditioner parses the wake-up task to obtain a parsing result.

[0056] S16, when the parsing result includes a wake-up audio, the terminal air conditioner responds to the wake-up task according to a delay strategy.

[0057] S17, when the parsing result does not include a wake-up audio, the terminal air conditioner forwards the parsing result to the mobile device side to respond to the wake-up task through a lightweight speech recognition model.

[0058] Using the scheduling method for voice tasks provided by the embodiments of the present disclosure, the terminal air conditioner first classifies the received voice tasks for audio to obtain voice classification tasks including non-wake-up tasks and wake-up tasks. Then, the terminal air conditioner classifies the non-wake-up tasks for networking to obtain offline interaction tasks and online interaction tasks. The terminal air conditioner then unloads the online interaction task or the offline interaction task to the collaborative device side for task processing according to the network connection status and the terminal load status, or the terminal air conditioner performs task processing locally. The embodiments of the present disclosure realize an elastic task scheduling strategy by reasonably classifying voice tasks and unloading the classified offline or online interaction tasks.

[0059] In addition, since the wake-up task requires the terminal air conditioner to respond or execute within a specified delay, the terminal air conditioner parses the wake-up task after obtaining the voice classification task to obtain a parsing result. If the parsing result includes a wake-up audio, the wake-up task is responded to according to the delay response strategy to perform a normal wake-up response. When the parsing result does not include a wake-up audio, the terminal air conditioner forwards the parsing result to the mobile device side, and the wake-up task is responded to through the configured lightweight speech recognition model to perform a normal wake-up response. The terminal air conditioner provided in the embodiments of the present disclosure can ensure the reliability of the wake-up operation with the cooperation of the mobile device side, and more efficiently meet the requirements of voice low-latency wake-up and high-reliability voice execution.

[0060] Optionally, the terminal air conditioner responds to the wake-up task according to a delay strategy, including: When the terminal air conditioner locally responds to the wake-up task within the delay duration, the terminal air conditioner performs a wake-up response locally.

[0061] When the terminal air conditioner fails to respond to the wake-up task locally within the delay duration, the terminal air conditioner is unloaded to the mobile device side to respond to the task through a lightweight speech recognition model.

[0062] In this way, since the wake-up task requires the terminal air conditioner to respond or execute within the specified delay, when the terminal air conditioner obtains the parsing result including the wake-up audio, the terminal air conditioner continues to determine whether the terminal air conditioner responds to the wake-up task locally within the delay duration. If the terminal air conditioner determines to respond locally, it directly performs a wake-up response locally. If the terminal air conditioner does not respond locally within the delay duration, it is directly unloaded to the mobile device side to respond to the task through the configured lightweight speech recognition model. In this way, the reliability of the wake-up task response is ensured.

[0063] Optionally, the terminal air conditioner is unloaded to the mobile device side to respond to the task through a lightweight speech recognition model, including: the terminal air conditioner is unloaded to the mobile device side to collect the wake-up features included in the wake-up task through a microphone array; when the microphone array collects the wake-up features, the terminal air conditioner triggers the lightweight speech recognition model to respond to the wake-up task.

[0064] In this way, by configuring a microphone array on the mobile device side, the wake-up sensitivity and false wake-up can be taken into account, and the reliability of the voice task execution can be ensured.

[0065] Combined Figure 4 As shown, the embodiments of the present disclosure also provide a scheduling method for voice tasks, including: S21, the terminal air conditioner classifies the received voice task to obtain a voice classification task. Among them, the language classification task includes non-wake-up tasks.

[0066] S22, the terminal air conditioner classifies the non-wake-up task through the network to obtain an offline interaction task and an online interaction task.

[0067] S23, the terminal air conditioner obtains the network connection status and the terminal load status of the terminal air conditioner.

[0068] S24, the terminal air conditioner unloads the online interaction task or the offline interaction task to the collaborative device side for task processing according to the network connection status and the terminal load status, or the terminal air conditioner performs task processing locally.

[0069] S25, the terminal air conditioner obtains the task processing results of the online interaction task and the offline interaction task respectively.

[0070] S26, the terminal air conditioner sends the task processing result to the cloud so that the cloud dynamically configures the air conditioner parameters according to the task processing result, environmental data, and human body data. Among them, the environmental data and human body data are collected by the mobile device side through multi-modal sensors and sent to the cloud.

[0071] Using the scheduling method for voice tasks provided by the embodiments of the present disclosure, the terminal air conditioner first performs audio classification on the received voice tasks to obtain voice classification tasks including non-wake-up tasks. Then, it performs network classification on the non-wake-up tasks to obtain offline interaction tasks and online interaction tasks. Finally, the terminal air conditioner unloads the online interaction tasks or offline interaction tasks to the collaborative device side for task processing according to the network connection status and the terminal load status, or the terminal air conditioner performs task processing locally. By reasonably classifying the voice tasks and unloading the classified offline or online interaction tasks, the embodiments of the present disclosure implement an elastic task scheduling strategy. After the terminal air conditioner sends the task processing results to the cloud, the cloud dynamically configures the air conditioner parameters according to the task processing results and the environmental data and human body data sent by the mobile device side to achieve adaptive control of the terminal air conditioner and meet the elastic service requirements of users.

[0072] Optionally, the cloud dynamically configures the air conditioner parameters according to the task processing results, environmental data, and human body data, including: the cloud inputs the task processing results, environmental data, and human body data into an LSTM (Long Short-Term Memory) model for model training, obtains the model output, and determines that the model output is the air conditioner parameters of the terminal air conditioner.

[0073] In practical applications, the mobile device side is a drone. As Figure 5 shown, the scheduling method for voice tasks specifically performs the following steps: S31, the terminal air conditioner performs audio classification on the received voice tasks to obtain voice classification tasks.

[0074] S32, the terminal air conditioner determines whether the voice classification task is a wake-up task. If not, execute S33; if so, execute S41.

[0075] S33, the terminal air conditioner performs network classification on the non-wake-up tasks to obtain offline interaction tasks and online interaction tasks.

[0076] S34, the terminal air conditioner determines whether it is connected to the network. If so, execute S35; otherwise, execute S38.

[0077] S35, the terminal air conditioner obtains the edge load status of the edge device side. Execute S36 or S37.

[0078] S36, in the case where the edge load status indicates overloading, the terminal air conditioner unloads the online interaction tasks to the cloud for task processing through the second heavyweight voice processing model.

[0079] S37. When the edge load status indicates normal load, the terminal air conditioner unloads the online interaction task to the edge device side for task processing through the first heavyweight voice processing model.

[0080] S38. The terminal air conditioner determines whether the terminal load status is overloaded. If so, execute S39; otherwise, execute S40.

[0081] S39. The terminal air conditioner forwards the offline interaction task to the drone for task processing.

[0082] S40. The terminal air conditioner locally executes the task processing of the offline interaction task.

[0083] S41. The terminal air conditioner parses the wake-up task to obtain the parsing result, and then executes S42 or S43.

[0084] S42. The terminal air conditioner determines whether the parsing result includes the wake-up audio. If so, execute S43; otherwise, execute S45.

[0085] S43. The terminal air conditioner determines whether the terminal air conditioner locally responds to the wake-up task within the delay duration. If so, execute S44; otherwise, execute S45.

[0086] S44. The terminal air conditioner locally makes a wake-up response.

[0087] S45. The terminal air conditioner unloads to the drone to perform task response through the lightweight speech recognition model.

[0088] S46. The terminal air conditioner obtains the task processing results of the online interaction task and the offline interaction task respectively, and sends them to the cloud.

[0089] S47. The cloud dynamically configures the air conditioner parameters according to the task processing results, environmental data, and human body data.

[0090] Combined Figure 6 As shown in , an embodiment of the present disclosure provides a scheduling device 70 for voice tasks, including a processor 700 and a memory 701. Optionally, the above scheduling device 70 for voice tasks may further include a communication interface 702 and a bus 703. Among them, the processor 700, the communication interface 702, and the memory 701 can complete mutual communication through the bus 703. The communication interface 702 can be used for information transmission. The processor 700 can call the logical instructions in the memory 701 to execute the scheduling method for voice tasks in the above embodiment.

[0091] In addition, when the logical instructions in the above memory 701 are implemented in the form of a software functional unit and sold or used as an independent product, they can be stored in a computer-readable storage medium.

[0092] The memory 701, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as the program instructions / modules corresponding to the methods in the embodiments of the present disclosure. The processor 700 executes functional applications and data processing by running the program instructions / modules stored in the memory 701, that is, implements the scheduling method for voice tasks in the above embodiments.

[0093] The memory 701 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 701 may include a high-speed random access memory and may also include a non-volatile memory.

[0094] The embodiments of the present disclosure also provide a scheduling system for voice tasks, which is applied to the above scheduling device for voice tasks. The above scheduling system for voice tasks includes: a terminal air conditioner and a collaborative device end. The collaborative device end is communicatively connected to the terminal air conditioner, and the collaborative device end includes an edge device end, a mobile device end, and a cloud end. The above scheduling device for voice tasks can be installed in the terminal air conditioner. The installation relationship described here is not limited to being placed inside the terminal air conditioner, but also includes installation connections with other components of the scheduling system for voice tasks, including but not limited to physical connections, electrical connections, or signal transmission connections, etc. Those skilled in the art can understand that the above scheduling device 70 can be adapted to a feasible system main body, and thus implement other feasible embodiments.

[0095] The embodiments of the present disclosure provide a storage medium storing computer-executable instructions, and the computer-executable instructions are set to execute the above scheduling method for voice tasks.

[0096] The technical solution of the embodiments of the present disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present disclosure. The foregoing storage medium may be a non-transitory storage medium, such as: a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc, etc., which are various media that can store program codes.

[0097] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure, enabling those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, process, and other changes. Embodiments merely represent possible variations. Unless explicitly required, individual components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. When used in this application, the term "comprise" and its variants "comprises" and / or "comprising", etc. refer to the presence of the stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups thereof. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, or device comprising the element. Herein, what each embodiment focuses on may be the differences from other embodiments, and the same or similar parts among the embodiments may be referred to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method parts disclosed in the embodiments, the relevant parts may refer to the description of the method parts.

[0098] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner may depend on the specific application and design constraints of the technical solution. The technician can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the embodiments of the present disclosure. The technician can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0099] In the embodiments disclosed herein, the disclosed methods, products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units can be merely a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Additionally, the couplings or direct couplings or communication connections shown or discussed between each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to implement this embodiment. Additionally, in the embodiments of the present disclosure, the various functional units can be integrated in one processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0100] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the block can occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, which can depend on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks can also occur in a different order than that disclosed in the description. Sometimes, there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, which can depend on the functions involved. Each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for scheduling voice tasks, characterized in that: Applied to terminal air conditioners, the methods include: Performing audio classification on the received speech task to obtain a speech classification task; wherein the language classification task includes a non-wake-up task; Classify the non-wake-up tasks online to obtain offline interaction tasks and online interaction tasks; Obtain the network connection status and terminal load status of the terminal air conditioner; According to the network connection status and terminal load status, the online interaction tasks or offline interaction tasks are offloaded to the collaborative device for task processing, or the terminal air conditioner processes the tasks locally.

2. The method according to claim 1, characterized in that The collaborative device side includes the edge device side and the mobile device side. According to the network connection status and the terminal load status, the online interaction tasks or offline interaction tasks are offloaded to the collaborative device side for task processing, or the terminal air conditioner performs task processing locally, including: When the network connection status indicates that the network is connected, the online interaction task is offloaded to the edge device for task processing; When the network connection status indicates that the network is disconnected and the terminal load status indicates that the terminal is overloaded, the offline interaction task is forwarded to the mobile device for task processing; When the network connection status indicates that the network is disconnected and the terminal load status indicates a normal load, the terminal air conditioner locally performs task processing of the offline interactive task.

3. The method according to claim 2, characterized in that The collaborative device side also includes the cloud side. The edge device side is configured with a first heavyweight voice processing model, and the cloud side is configured with a second heavyweight voice processing model. The online interaction tasks are offloaded to the edge device side for task processing, including: Get the edge load status of the edge device; When the edge load status indicates overload, offloading the online interaction task to the cloud for task processing by a second heavyweight speech processing model; or, When the edge load status indicates a normal load, the online interaction task edge device is offloaded to perform task processing through the first heavyweight speech processing model.

4. The method according to claim 1, characterized in that: The collaborative device end includes a mobile device end, the mobile device end is configured with a lightweight speech recognition model, the language classification task also includes a wake-up task, and the method further includes: Analyze the wake-up task and obtain the analysis result; When the analysis result includes the wake-up audio, respond to the wake-up task according to the delay strategy; When the parsing result does not include the wake-up audio, the parsing result is forwarded to the mobile device to respond to the wake-up task through the lightweight speech recognition model.

5. The method according to claim 4, characterized in that Respond to the wake-up task according to the delay strategy, including: If the terminal air conditioner responds to the wake-up task locally within the delay time, the terminal air conditioner responds to the wake-up locally; If the terminal air conditioner does not respond to the wake-up task locally within the delay time, it is offloaded to the mobile device to respond to the task through a lightweight speech recognition model.

6. The method according to claim 5, characterized in that The mobile device is equipped with a microphone array, which is used to collect wake-up features included in the wake-up task; Offload to the lightweight speech recognition model configured on the mobile device to respond to tasks, including: Offloading to a mobile device to collect the wake-up features included in the wake-up task through a microphone array; When the microphone array collects the wake-up features, the lightweight speech recognition model is triggered to respond to the wake-up task.

7. The method according to any one of claims 1 to 6, characterized in that: Also includes: Obtaining the task processing results of the online interactive task and the offline interactive task; The task processing results are sent to the cloud so that the cloud can dynamically configure the air-conditioning parameters according to the task processing results and the environmental data and human body data; wherein the environmental data and human body data are collected by the mobile device through multimodal sensors and sent to the cloud.

8. A voice task scheduling device, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the method for scheduling voice tasks according to any one of claims 1 to 7 when running the program instructions.

9. A voice task scheduling system, characterized in that: A scheduling device for voice tasks, the system comprising: Terminal air conditioning; The collaborative device side is connected to the terminal air conditioner through communication, including the edge device side, the movable device side, and the cloud.

10. A storage medium storing program instructions, characterized in that: When the program instructions are executed, the computer is used to execute the method for scheduling voice tasks as claimed in any one of claims 1 to 7.

Images