Voice interaction method and device of refrigeration equipment, refrigeration equipment and electronic equipment

Abstract

The invention discloses a voice interaction method and device of refrigeration equipment, the refrigeration equipment and electronic equipment, and belongs to the technical field of refrigeration equipment. The method comprises the following steps: acquiring voice interaction data of a target user to refrigeration equipment; target identity information corresponding to the target user is determined in a user library based on the voice interaction data, and the user library comprises identity information of multiple users; processing the voice interaction data based on the target identity information to obtain a first processing result; inputting the first processing result into a multi-task processing model to obtain a task embedding vector output by the multi-task processing model; and inputting the task embedding vector into a language model to obtain an interaction result corresponding to the voice interaction data. According to the method, voice identity recognition and multi-task recognition are combined, the accuracy of voice recognition and interaction is high, and the interaction effect on complex tasks is good.

Description

Technical Field

[0001] This application belongs to the field of refrigeration equipment technology, and particularly relates to a voice interaction method, device, refrigeration equipment, and electronic equipment for refrigeration equipment. Background Technology

[0002] Voice interaction has become one of the important ways for users to control smart home appliances. Currently, smart home appliances receive voice commands through built-in microphones and perform voice recognition and semantic analysis through cloud or local algorithms to respond to users' voice commands.

[0003] However, in practical applications, the accuracy of voice recognition in smart home appliances is often difficult to guarantee, and they can only support basic commands such as simple on / off control and mode switching, and cannot handle complex tasks. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a voice interaction method, device, refrigeration equipment, and electronic equipment for refrigeration devices, which features high accuracy in voice recognition and interaction, and good interaction performance for complex tasks.

[0005] In a first aspect, this application provides a voice interaction method for a refrigeration device, the method comprising: Acquire voice interaction data of target users with refrigeration equipment; Based on the voice interaction data, the target identity information corresponding to the target user is determined in the user database, which includes the identity information of multiple users; Based on the target identity information, the voice interaction data is processed to obtain a first processing result; The first processing result is input into the multi-task processing model to obtain the task embedding vector output by the multi-task processing model; The task is embedded into a vector input language model to obtain the interaction result corresponding to the voice interaction data.

[0006] According to the voice interaction method for refrigeration equipment provided in this application embodiment, the target identity information corresponding to the target user is determined based on the voice interaction data. Personalized voice recognition can be performed based on the target identity information to obtain a first processing result, thereby effectively improving the accuracy of voice recognition and interaction. Furthermore, a multi-task processing model is used to perform task recognition and extraction on the first processing result to obtain a task embedding vector that includes multiple task semantic features, thereby enabling the language model to handle complex tasks in the voice interaction data. This method combines voice identity recognition and multi-task recognition, resulting in high accuracy of voice recognition and interaction and good interaction effect for complex tasks.

[0007] According to one embodiment of this application, determining the target identity information corresponding to the target user in the user database based on the voice interaction data includes: Feature extraction is performed on the voice interaction data to obtain the target feature vector; Based on the target feature vector, the target identity features of the voice interaction data are obtained by combining the self-attention mechanism. Based on the target identity features, the target identity information corresponding to the target user is determined in the user database.

[0008] According to one embodiment of this application, the multi-task processing model outputs the task embedding vector through the following steps; Obtain the task start label and task end label of the first processing result; Based on the spatiotemporal attributes of the task start label and the task end label, the text identifier fields corresponding to the task start label and the task end label are retrieved in a loop. The task embedding vector is determined based on the text identifier field.

[0009] According to one embodiment of this application, the step of processing the voice interaction data based on the target identity information to obtain a first processing result includes: Based on the target identity information, feature extraction is performed on the voice interaction data to obtain the first voice feature; Rotational position encoding is performed on the first speech feature to obtain the rotational position encoding vector of the speech interaction data; The rotation position encoding vector is decoded to obtain the first processing result.

[0010] According to one embodiment of this application, the rotation position encoding vector includes at least one of the time-domain position information, frequency-domain position information, and angular position information of the voice interaction data.

[0011] According to one embodiment of this application, the language model includes a first sub-model and a second sub-model, wherein the response speed of the first sub-model is greater than that of the second sub-model, and the processing range of the second sub-model is greater than that of the first sub-model.

[0012] According to one embodiment of this application, obtaining the voice interaction data of the target user with the refrigeration device includes: Acquire the initial voice interaction data of the target user with the refrigeration equipment; The initial voice interaction data is subjected to noise reduction and purification processing to obtain intermediate voice interaction data; The intermediate voice interaction data is enhanced with features to obtain the voice interaction data.

[0013] Secondly, this application provides a voice interaction device for a refrigeration device, the device comprising: The acquisition module is used to acquire voice interaction data of the target user with the refrigeration equipment; The first processing module is used to determine the target identity information corresponding to the target user in the user database based on the voice interaction data. The user database includes the identity information of multiple users. The second processing module is used to process the voice interaction data based on the target identity information to obtain a first processing result; The third processing module is used to input the first processing result into the multi-task processing model to obtain the task embedding vector output by the multi-task processing model. The fourth processing module is used to embed the task into the vector input language model to obtain the interaction result corresponding to the voice interaction data.

[0014] According to the voice interaction device for refrigeration equipment provided in the embodiments of this application, the target identity information corresponding to the target user is determined based on the voice interaction data. Personalized voice recognition can be performed based on the target identity information to obtain a first processing result, thereby effectively improving the accuracy of voice recognition and interaction. Furthermore, the first processing result is also used to perform task recognition and extraction through a multi-task processing model to obtain a task embedding vector that includes multiple task semantic features, so that the language model can handle complex tasks in the voice interaction data. This device combines voice identity recognition and multi-task recognition, resulting in high accuracy of voice recognition and interaction and good interaction effect for complex tasks.

[0015] Thirdly, this application provides a refrigeration device, which includes: The voice interaction device for the refrigeration equipment described in the second aspect above.

[0016] According to the refrigeration device provided in the embodiments of this application, the target identity information corresponding to the target user is determined based on voice interaction data. Personalized voice recognition can be performed based on the target identity information to obtain a first processing result, thereby effectively improving the accuracy of voice recognition and interaction. Furthermore, the first processing result is also used to perform task recognition and extraction through a multi-task processing model to obtain a task embedding vector that includes multiple task semantic features, so that the language model can handle complex tasks in voice interaction data. This refrigeration device combines voice identity recognition and multi-task recognition, resulting in high accuracy of voice recognition and interaction and good interaction effect for complex tasks.

[0017] Fourthly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the voice interaction method of the cooling device as described in the first aspect above.

[0018] Fifthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the voice interaction method for the refrigeration device as described in the first aspect above.

[0019] In a sixth aspect, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the voice interaction method for the cooling device as described in the first aspect.

[0020] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the voice interaction method for the refrigeration device as described in the first aspect above.

[0021] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0022] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is one of the flowcharts illustrating the voice interaction method for a refrigeration device provided in the embodiments of this application; Figure 2 This is a second schematic flowchart of the voice interaction method for the refrigeration equipment provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the voice interaction device of the refrigeration equipment provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0024] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0025] It should be noted that all actions involving the acquisition of signals, information, or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with authorization from the owner of the relevant device.

[0026] It should be noted that the refrigeration equipment in the embodiments of this application can be understood as a broad refrigeration storage device, including but not limited to refrigerators, freezers, display cases, beverage cabinets, wine cabinets, refrigerated display cases, and refrigerated vending machines, etc. Refrigeration equipment has diverse structural forms and a wide range of applications.

[0027] It should be noted that the text identifier field, text token, and text word in the embodiments of this application are semantically identical and have the same meaning.

[0028] It should be noted that the language model in this embodiment includes a first sub-model and a second sub-model. Both the first sub-model and the second sub-model can be small models, thereby reducing the computing resources required for the language model and enabling the language model to be adapted to small devices.

[0029] The following description, in conjunction with the accompanying drawings, details the voice interaction method, voice interaction device, refrigeration equipment, electronic equipment, and readable storage medium for refrigeration equipment provided in this application, through specific embodiments and application scenarios.

[0030] The voice interaction method for a refrigeration device provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the voice interaction method of the refrigeration device. The following describes the voice interaction method for a refrigeration device provided in this application embodiment using an electronic device as the execution subject.

[0031] like Figure 1 As shown, the voice interaction method of the refrigeration device includes steps 110, 120 and 130.

[0032] Step 110: Obtain the voice interaction data of the target user with the refrigeration equipment.

[0033] The target user refers to the user who interacts with the refrigeration equipment via voice. Refrigeration equipment is a device that uses refrigeration technology to lower the ambient temperature or maintain a specific low temperature. Common types include refrigerators and air conditioners.

[0034] Voice interaction data is the data generated when the target user communicates with the cooling device through voice input. Voice interaction data includes the target user's instructions and can represent the target user's intention to operate the cooling device.

[0035] Interactive data refers to the data that users input into the refrigeration equipment, which can represent the user's intention to operate the refrigeration equipment. Interactive data includes at least voice interactive data, which can be represented as the data generated by the user communicating with the refrigeration equipment.

[0036] In this step, a corresponding voice acquisition device can be set up on the cooling equipment to obtain voice interaction data. For example, voice interaction data can be obtained through a microphone or voice sensor.

[0037] Step 120: Based on the voice interaction data, determine the target identity information corresponding to the target user in the user database. The user database includes the identity information of multiple users.

[0038] The user database is a database that stores and manages the identity information of multiple users. The user database may include data such as the user's name, voiceprint characteristics, and preference settings, which are used to identify and distinguish different users of the refrigeration equipment.

[0039] Target identity information refers to the specific identity data corresponding to the target user matched from the user database. Target identity information may include the target user's personal information, historical interaction records, and unique identification information such as voiceprint features.

[0040] In practice, the speech recognition model can be personalized for training or fine-tuning based on the voiceprint features in the user database to improve the accuracy and response speed of speech recognition and interaction.

[0041] In this step, the target user's voice interaction data can reflect the target user's voice characteristics, i.e., voiceprint characteristics. Based on the target user's voice interaction data, the target user's identity information can be determined in the user database, thereby improving the accuracy and response speed of voice recognition and interaction.

[0042] Step 130: Based on the target identity information, process the voice interaction data to obtain the first processing result.

[0043] The processing of voice interaction data can be achieved by extracting features from the voice interaction data based on the target identity information and performing encoding and decoding processing to obtain the voice features of the voice interaction data, which is the first processing result.

[0044] In this step, the voice interaction data is processed in a targeted manner based on the target user's identity information to obtain the first processing result. This can more accurately identify the target user's pronunciation characteristics and voice patterns, thereby improving the accuracy and response speed of voice recognition and interaction.

[0045] Step 140: Input the first processing result into the multi-task processing model to obtain the task embedding vector output by the multi-task processing model.

[0046] Among them, the multi-task processing model can perform task identification and extraction on the first processing result, thereby obtaining the task embedding vector of the voice interaction data.

[0047] Task embedding vectors are low-dimensional vector representations output by multi-task processing models that encode the semantic features of different tasks in voice interaction data.

[0048] In this step, a multi-task processing model can be used to identify and extract tasks from voice interaction data containing complex tasks, resulting in task embedding vectors that include semantic features of multiple tasks. Through these task embedding vectors, the language model can more accurately understand the complex tasks in the voice interaction data, thereby obtaining interaction results that meet the user's requirements.

[0049] Step 150: Embed the task into the vector input language model to obtain the interaction results corresponding to the voice interaction data.

[0050] Among them, a language model is an AI model that can understand and generate natural language. The interaction result is the output content generated by the language model based on the task embedding vector of the input, including feedback on instructions in the voice interaction data, such as text replies or operation feedback.

[0051] In this step, the interaction results are generated by the language model based on task embedding vectors. The task embedding vectors include semantic features of multiple tasks, which enables the language model to understand complex tasks in the voice interaction data, such as compound tasks like "Based on the amount of food taken out of the refrigerator today, what other food do I need to replenish today?", thus providing more accurate interaction results.

[0052] In related technologies, by setting up voice acquisition devices in smart home appliances (such as refrigerators), recognizing the collected user voice data, and then inputting it into a language model, smart home appliances can be made to have a certain interactive response capability and make relevant feedback to the user's voice commands. However, in practical applications, the accuracy of voice recognition of smart home appliances is often difficult to guarantee, and they can only support basic commands such as simple on / off control and mode switching, and cannot handle complex tasks.

[0053] In this embodiment, after obtaining the voice interaction data of the target user, identity recognition can be performed based on the existing user database. The target user's identity information in the user database is determined by the voice features of the voice interaction data. Voice recognition and subsequent processing are performed based on the target identity information to obtain the first processing result, which can effectively improve the accuracy of voice recognition and interaction. Furthermore, a multi-task processing model is used to identify and extract tasks from the first processing result to obtain the task embedding vector of the voice interaction data. The task embedding vector includes the semantic features of multiple tasks, which enables the language model to understand complex tasks in the voice interaction data, such as composite tasks like "Based on the amount of food taken out of the refrigerator today, what other food do I need to replenish today?"

[0054] According to the voice interaction method for refrigeration equipment provided in this application embodiment, the target identity information corresponding to the target user is determined based on the voice interaction data. Personalized voice recognition can be performed based on the target identity information to obtain a first processing result, thereby effectively improving the accuracy of voice recognition and interaction. Furthermore, a multi-task processing model is used to perform task recognition and extraction on the first processing result to obtain a task embedding vector that includes multiple task semantic features, thereby enabling the language model to handle complex tasks in the voice interaction data. This method combines voice identity recognition and multi-task recognition, resulting in high accuracy of voice recognition and interaction and good interaction effect for complex tasks.

[0055] In some embodiments, based on voice interaction data, determining the target identity information corresponding to the target user in the user database includes: Feature extraction is performed on the voice interaction data to obtain the target feature vector; Based on the target feature vector, the target identity features of the voice interaction data are obtained by combining the self-attention mechanism. Based on the target identity characteristics, the target identity information corresponding to the target user is determined in the user database.

[0056] The target feature vector is a vectorized representation obtained by extracting features from the voice interaction data. The target feature vector contains key characteristics of the voice signal, such as spectrum or intonation. The target feature vector can characterize the voice features of different users.

[0057] Self-attention mechanisms can automatically calculate the correlation weights between features at different positions in the input sequence, thereby highlighting important features.

[0058] Target identity features are user-discriminative feature representations obtained after processing by a self-attention mechanism. Target identity features enhance the voice features related to user identity in voice interaction data.

[0059] In practice, CNN-nD multi-channel, multi-size convolution operations can be used on the voice interaction data to obtain the target feature vector; through self-attention mechanism and multi-speaker embedding calculation of spatial and temporal convolution, the most effective and feature weighted target identity features can be obtained; then the target identity features in a single channel are extracted and the decoding task is completed, the target identity features are mapped to the vector calculation space and the identity information corresponding to the target user is retrieved in the user database, that is, the target identity information. The retrieval process may include specific calculation normalization and identity information score calculation.

[0060] It should be noted that, based on the target identity features, if a user matching the target identity features is found in the user database, the user's identity information can be identified as the target identity information. If no user matching the target identity features is found in the user database, the user can be prompted to enter a new user account, thereby entering the user's identity information into the user database for use in subsequent interaction scenarios.

[0061] In this embodiment, feature extraction is performed on the voice interaction data, and the target identity features of the voice interaction data are obtained by combining the self-attention mechanism. This enables accurate and rapid identity matching, thereby improving the accuracy and response speed of voice recognition and interaction.

[0062] In some embodiments, the multi-task processing model outputs task embedding vectors through the following steps; The task start label and task end label that obtain the first processing result; Based on the spatiotemporal attributes of the task start label and the task end label, the text identifier fields corresponding to the task start label and the task end label are retrieved in a loop. The task embedding vector is determined based on the text identifier field.

[0063] The task start label is used to mark the starting position of a specific task in the multi-task processing flow. The task start label can instruct the multi-task processing model to start processing the relevant task content from the label.

[0064] The task end label is an identifier that marks the end point of task processing. It is used in conjunction with the task start label to define the boundary of a specific task processing.

[0065] Spatiotemporal attributes refer to the characteristic information of task start labels and task end labels in the time dimension, spatial dimension, or spatiotemporal dimension. For example, the temporal position of a speech segment or the spatial coordinate data of an object referred to by the speech segment.

[0066] The text identifier field can be a text sequence with semantic information extracted from the voice interaction data. The text identifier field carries key task information in the voice interaction data.

[0067] In actual execution, taking time series as an example, we can first obtain the task time series start tag; determine whether the task time series start tag has time series information and obtain text tokens, that is, text identifier fields; then obtain the task time series end tag and loop until all text identifier fields are obtained; merge the text identifier fields and text tokens that are not spatiotemporal or time series data into multi-task embedded tokens information with sequence end text tags (End Of Sequence Token), that is, task embedding vector.

[0068] In this embodiment, the multi-task processing model obtains all text identifier fields corresponding to the voice interaction data based on the spatiotemporal attributes of the task start label and the task end label. It can accurately extract multiple tasks contained in the voice interaction data. The task embedding vector obtained through the text identifier fields enables the language model to accurately understand the complex voice commands of the target user and obtain accurate interaction results.

[0069] In some embodiments, the voice interaction data is processed based on the target identity information to obtain a first processing result, including: Based on the target identity information, feature extraction is performed on the voice interaction data to obtain the first voice feature; The first speech feature is rotated and position encoded to obtain the rotational position encoded vector of the speech interaction data. The rotation position encoding vector is decoded to obtain the first processing result.

[0070] Among them, the first speech feature is a key acoustic feature extracted from the speech interaction data. The first speech feature includes, but is not limited to, information such as spectrum and pitch, which can reflect the unique attributes of the target user's speech.

[0071] Rotational Position Encoding (RoPE) vectors are feature representations processed by Rotational Position Embedding (RoPE) technology. RoPE can encode positional information into word embedding vectors through rotation matrices, thereby introducing relative positional information when calculating query and key vectors. RoPE vectors introduce positional information in the time and frequency domains, enhancing the spatiotemporal correlation of features.

[0072] Decoding is the process of processing the rotation position encoded vector through operations such as attention mechanisms and normalization, ultimately converting it into understandable text or instructions.

[0073] In practice, features can be extracted from speech data using a feature extractor. Then, the resulting first speech features are aligned using multimodal RoPE to ensure that the validity of the obtained speech features is aligned with the position embedding vector in the time and frequency domains. This allows the language model to better understand and process the positional information in the first speech features, providing clear distinctions and connections for subsequent speech transcription. Multimodal RoPE alignment can use speech recognition architectures such as Kaldi, Conformer, Whisper, or WaveNet.

[0074] In addition, the following process can be used to decode the rotation position encoding vector: first, the rotation position encoding vector is used to obtain features through a convolutional network, then RoPE is used to obtain position selection information such as time, frequency domain and angle; then, feature weight information is obtained through a self-attention mechanism; then, normalization is performed through residual connection and layer normalization (ADD&NORM); the self-attention mechanism, residual connection and layer normalization are repeated, and finally the normalization exponent (Softmax) function is used to obtain the first processing result.

[0075] In this embodiment, by extracting features from the voice interaction data and performing rotational position encoding and decoding, a first processing result containing unique voice features of the target user's voice can be obtained. Furthermore, rotational position encoding embeds positional information into the first processing result, enabling the language model to better understand and process the positional information in the first processing result.

[0076] In some embodiments, the rotation position encoding vector includes at least one of the temporal position information, frequency position information, and angular position information of the voice interaction data.

[0077] Among them, the temporal location information represents the relative positional characteristics of the voice interaction data on the time axis, which is used to identify the temporal order relationship of each sampling point in the voice segment.

[0078] Frequency domain location information reflects the distribution characteristics of voice interaction data at different frequency components. Frequency domain location information can reflect the spectral composition and energy distribution of sound.

[0079] Angular position information is a directional parameter obtained through polar coordinate transformation, used to describe the directional characteristics of voice interaction data in phase space.

[0080] In this embodiment, the rotation position encoding vector may include temporal position information, frequency domain position information, and angular position information. Through the rotation position encoding vector, the language model can better understand and help process the position information in the first processing result.

[0081] In some embodiments, the language model, i.e. the large language model, includes a first sub-model and a second sub-model. The response speed of the first sub-model is greater than that of the second sub-model, and the processing range of the second sub-model is greater than that of the first sub-model.

[0082] The first sub-model is the vertical domain model, which typically employs a dedicated network structure or quantization technique and is suitable for vertical scenarios.

[0083] The second sub-model is a general model with a wider processing range, enabling semantic understanding and reasoning for a wider variety of tasks.

[0084] In practice, the first sub-model is suitable for vertical scenarios. The first sub-model can be based on DeepSeek-761B distilled LLama model, etc. The parameters of the basic model are learned and adapted to the vertical domain by distillation method. This improves the prediction efficiency and accuracy of the vertical domain, and reduces the learning cost and computing resources.

[0085] The second sub-model is suitable for general scenarios. The second sub-model can adopt reinforcement learning training strategies, such as fine-tuning and distilling the model based on DeepSeek-V3 / R1 or the open-source visual language model VLM, to obtain an efficient model.

[0086] In this embodiment, the language model is divided into a first sub-model and a second sub-model, which are responsible for quick response to simple tasks and deep thinking for complex tasks, respectively. Through the first sub-model and the second sub-model, the accuracy of interaction and response speed can be improved.

[0087] In some embodiments, acquiring voice interaction data of a target user with a cooling device includes: Acquire initial voice interaction data of the target user with the refrigeration equipment; The initial voice interaction data is denoised and purified to obtain intermediate voice interaction data. Feature enhancement is performed on the intermediate voice interaction data to obtain the voice interaction data.

[0088] The initial voice interaction data is the initial voice signal that the user directly sends to the cooling device, which includes human voice commands and interference components such as environmental noise.

[0089] The intermediate voice interaction data is a voice signal that has been processed by noise reduction and purification, and most of the background noise has been removed, but further optimization is still needed.

[0090] Feature enhancement is a process that uses prompting algorithms to enhance and prompt speech data with multiple tasks, multiple speakers, and multiple channels, which helps in the feature extraction of subsequent speech interaction data.

[0091] In practice, speech noise reduction models based on CNN-nD or multi-channel convolutional neural networks, such as deep neural networks (DNN) or long short-term memory (LSTM), can be used to separate speech noise and extract pure speech data.

[0092] In this embodiment, by denoising, refining, and enhancing the features of the user's initial voice interaction data, purer and more feature-rich voice interaction data can be obtained, thereby improving the accuracy of subsequent voice recognition and interaction.

[0093] The following is a specific example.

[0094] In this embodiment, the voice interaction method of the refrigeration equipment is achieved through... Figure 2 The process shown is implemented.

[0095] Step 1: Complete the data collection for the interaction between refrigeration equipment such as refrigerators and users, including internal and external environmental information, human-computer interaction, multi-turn dialogue, intelligent agents, multiple intelligent agents, personal intelligent assistants, data transmission, data generation tools, data uploading, and other software and hardware tasks of intelligent terminals. The focus is on voice interaction data. The sampling rate of voice interaction data includes, but is not limited to, 16bit / 8bit / 32bit. Voice interaction data includes effective voice based on models or data driven, and voice data with or without wake words.

[0096] Step 2: Complete the task of collecting text content from the voice interaction data of the cooling equipment, such as... Figure 2 As shown in 102-1 to 102-3.

[0097] Step 2.1: Interactive voice data is transcribed into text, such as... Figure 2As shown in 102-1, considering the noise, slow and inaccurate speech-to-text conversion, and poor efficiency in voice interaction, this paper proposes a method for separating audio and video data to obtain voice data. Based on this, a CNN-nD or multi-channel convolutional neural network-based speech denoising model is used to separate speech noise and extract clean voice interaction data. This data is then further processed through voice activity detection (Voice Activity Detection). Activity Detection (VAD) is further refined; a prompting algorithm is applied to the speech data, namely, enhanced prompting for speech data with multiple tasks, multiple speakers, and multiple channels; through a speaker discriminator, the speech features of the target user are quickly obtained. If the speech interaction data contains already recorded features, the process directly proceeds to the speech feature extraction and encoding steps, reducing the memory resource consumption of speech data feature extraction, thereby reducing speech training data, training costs, and computing power; features are extracted from the speech interaction data, and then the obtained first speech features are aligned using multimodal RoPE to ensure that the validity of the obtained speech features is aligned with the position embedding vector in the time and frequency domains, so that the language model can better understand and help process the position information in the first speech features, providing clear feature distinctions and connections for subsequent speech transcription. Multimodal RoPE alignment can adopt speech recognition architectures such as Kaldi, Conformer, Whisper, or WaveNet.

[0098] Step 2.2: Complete the multi-channel multi-speaker generator task, such as... Figure 2 As shown in Figure 102-2, CNN-nD multi-channel, multi-size convolution operations can be used on voice interaction data to obtain target feature vectors; through self-attention mechanism and multi-speaker embedding calculation of spatial and temporal convolution, the most effective target identity features with important feature weights are obtained; then, the target identity features in a single channel are extracted and the decoding task is completed, the target identity features are mapped to the vector calculation space, and the identity information corresponding to the target user is retrieved in the user database, that is, the target identity information. The retrieval process may include specific calculation normalization and identity information score calculation.

[0099] Step 2.3: Construct and train the multi-speaker decoder task, such as... Figure 2 As shown in 102-3, the rotation position encoding vector is first processed by a convolutional network to obtain features. Then, RoPE is used to obtain position selection information such as time, frequency domain and angle. Next, the feature weight information is obtained through a self-attention mechanism. Then, normalization is performed through residual connection and layer normalization (ADD&NORM). The self-attention mechanism and ADD&NORM are repeated. Finally, the normalization exponent (Softmax) function is used to obtain the first processing result.

[0100] Step 3: Multi-task embedding and task generation, completing the embedding and generation of various tasks, such as those for a refrigeration device. Figure 2 As shown in 103-1.

[0101] Step 3.1: Construct and train a multi-task processing model, such as... Figure 2 As shown in 103-2, first obtain the task time sequence start tag; determine whether the task time sequence start tag has time sequence information and obtain text tokens, that is, text identifier fields; then obtain the task time sequence end tag, and repeat until all text identifier fields are obtained; merge the text identifier fields and text tokens that are not spatiotemporal or time sequence data into multi-task embedding tokens information with EOS text tags, that is, task embedding vectors.

[0102] Step 4: Complete the language modeling task, including the first and second sub-models, such as... Figure 2 As shown in 104-1 and 104-2.

[0103] Step 4.1: Complete the first sub-model task, such as using DeepSeek-761B distillation LLama model as the base model, learn the parameters of the base model and use the distillation method to adapt it to the vertical domain. This improves the prediction efficiency and accuracy of the vertical domain and reduces the learning cost and computing resources.

[0104] Step 4.2: Complete the second sub-model task, such as fine-tuning and distilling the model based on DeepSeek-V3 / R1 or the open-source visual language model VLM to obtain an efficient model.

[0105] Step 5: Complete the engineering deployment and application of the pre-trained language model, such as... Figure 2 As shown in 105, it includes tasks such as language model interface configuration, deployment, data protection, model fine-tuning, and interface interaction, realizing a visual and quantifiable multi-task application service interface.

[0106] Step 6: Complete the text-to-speech task, such as... Figure 2 As shown in 106, a customized speech synthesis model is constructed using a natural language instructional guidance method.

[0107] Step 7: Presentation of interactive results, including voice broadcasting, text and image presentation tasks, such as... Figure 2 As shown in 107, the speech and text output processing is as follows.

[0108] Step 7.1: For the speech output data, the decoding task is completed by the transformer deep network model of the full attention module. First, the feature vectors or expressions of the features themselves and between features are unified based on the full attention mechanism. Then, convolution operation, rectified linear unit (ReLU) calculation and layer normalization (LayerNorm) are performed to complete the decoding task and generate the corresponding speech data.

[0109] Step 7.2: Output the text data from 105.

[0110] Various interactive services and modes are shown in Table 1.

[0111] Table 1

[0112] The voice interaction method for refrigeration equipment provided in this application can be executed by a voice interaction device for the refrigeration equipment. This application uses the voice interaction device of the refrigeration equipment executing the voice interaction method as an example to illustrate the voice interaction device for the refrigeration equipment provided in this application.

[0113] This application also provides a voice interaction device for a refrigeration equipment.

[0114] like Figure 3 As shown, the voice interaction device of the refrigeration equipment includes: an acquisition module 210, a first processing module 220, a second processing module 230, a third processing module 240, and a fourth processing module 250.

[0115] The acquisition module 210 is used to acquire voice interaction data of the target user with the refrigeration equipment; The first processing module 220 is used to determine the target identity information corresponding to the target user in the user database based on the voice interaction data. The user database includes the identity information of multiple users. The second processing module 230 is used to process the voice interaction data based on the target identity information to obtain the first processing result; The third processing module 240 is used to input the first processing result into the multi-task processing model to obtain the task embedding vector output by the multi-task processing model. The fourth processing module 250 is used to embed the task into the vector input language model to obtain the interaction result corresponding to the voice interaction data.

[0116] According to the voice interaction device for refrigeration equipment provided in the embodiments of this application, the target identity information corresponding to the target user is determined based on the voice interaction data. Personalized voice recognition can be performed based on the target identity information to obtain a first processing result, thereby effectively improving the accuracy of voice recognition and interaction. Furthermore, the first processing result is also used to perform task recognition and extraction through a multi-task processing model to obtain a task embedding vector that includes multiple task semantic features, so that the language model can handle complex tasks in the voice interaction data. This device combines voice identity recognition and multi-task recognition, resulting in high accuracy of voice recognition and interaction and good interaction effect for complex tasks.

[0117] In some embodiments, the first processing module 220 is used to determine the target identity information corresponding to the target user in the user database based on voice interaction data, including: Feature extraction is performed on the voice interaction data to obtain the target feature vector; Based on the target feature vector, the target identity features of the voice interaction data are obtained by combining the self-attention mechanism. Based on the target identity characteristics, the target identity information corresponding to the target user is determined in the user database.

[0118] In some embodiments, the multi-task processing model outputs task embedding vectors through the following steps; The task start label and task end label that obtain the first processing result; Based on the spatiotemporal attributes of the task start label and the task end label, the text identifier fields corresponding to the task start label and the task end label are retrieved in a loop. The task embedding vector is determined based on the text identifier field.

[0119] In some embodiments, the second processing module 230 is used to process the voice interaction data based on the target identity information to obtain a first processing result, including: Based on the target identity information, feature extraction is performed on the voice interaction data to obtain the first voice feature; The first speech feature is rotated and position encoded to obtain the rotational position encoded vector of the speech interaction data. The rotation position encoding vector is decoded to obtain the first processing result.

[0120] In some embodiments, the rotation position encoding vector includes at least one of the temporal position information, frequency position information, and angular position information of the voice interaction data.

[0121] In some embodiments, the language model includes a first sub-model and a second sub-model, wherein the response speed of the first sub-model is greater than that of the second sub-model, and the processing range of the second sub-model is greater than that of the first sub-model.

[0122] In some embodiments, the acquisition module 210 is used to acquire voice interaction data of the target user with the cooling device, including: Acquire initial voice interaction data of the target user with the refrigeration equipment; The initial voice interaction data is denoised and purified to obtain intermediate voice interaction data. Feature enhancement is performed on the intermediate voice interaction data to obtain the voice interaction data.

[0123] This application also provides a refrigeration device, which includes a voice interaction device as described above.

[0124] According to the refrigeration device provided in the embodiments of this application, the target identity information corresponding to the target user is determined based on voice interaction data. Personalized voice recognition can be performed based on the target identity information to obtain a first processing result, thereby effectively improving the accuracy of voice recognition and interaction. Furthermore, the first processing result is also used to perform task recognition and extraction through a multi-task processing model to obtain a task embedding vector that includes multiple task semantic features, so that the language model can handle complex tasks in voice interaction data. This refrigeration device combines voice identity recognition and multi-task recognition, resulting in high accuracy of voice recognition and interaction and good interaction effect for complex tasks.

[0125] In some embodiments, such as Figure 4 As shown, this application embodiment also provides an electronic device 400, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the program is executed by the processor 801, it implements the various processes of the above-described voice interaction method embodiment of the cooling device and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0126] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0127] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described voice interaction method embodiment for the cooling device and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0128] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0129] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the voice interaction method of the above-mentioned refrigeration device.

[0130] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0131] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described voice interaction method embodiment for the cooling device, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0132] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0133] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0134] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0135] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0136] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0137] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A voice interaction method for a refrigeration device, characterized in that, include: Acquire voice interaction data of target users with refrigeration equipment; Based on the voice interaction data, the target identity information corresponding to the target user is determined in the user database, which includes the identity information of multiple users; Based on the target identity information, the voice interaction data is processed to obtain a first processing result; The first processing result is input into the multi-task processing model to obtain the task embedding vector output by the multi-task processing model; The task is embedded into a vector input language model to obtain the interaction result corresponding to the voice interaction data.

2. The voice interaction method for refrigeration equipment according to claim 1, characterized in that, The step of determining the target identity information corresponding to the target user in the user database based on the voice interaction data includes: Feature extraction is performed on the voice interaction data to obtain the target feature vector; Based on the target feature vector, the target identity features of the voice interaction data are obtained by combining the self-attention mechanism. Based on the target identity features, the target identity information corresponding to the target user is determined in the user database.

3. The voice interaction method for refrigeration equipment according to claim 1, characterized in that, The multi-task processing model outputs the task embedding vector through the following steps; Obtain the task start label and task end label of the first processing result; Based on the spatiotemporal attributes of the task start label and the task end label, the text identifier fields corresponding to the task start label and the task end label are retrieved in a loop. The task embedding vector is determined based on the text identifier field.

4. The voice interaction method for refrigeration equipment according to claim 1, characterized in that, The step of processing the voice interaction data based on the target identity information to obtain a first processing result includes: Based on the target identity information, feature extraction is performed on the voice interaction data to obtain the first voice feature; Rotational position encoding is performed on the first speech feature to obtain the rotational position encoding vector of the speech interaction data; The rotation position encoding vector is decoded to obtain the first processing result.

5. The voice interaction method for refrigeration equipment according to claim 4, characterized in that, The rotation position encoding vector includes at least one of the time-domain position information, frequency-domain position information, and angular position information of the voice interaction data.

6. The voice interaction method for a refrigeration device according to any one of claims 1-5, characterized in that, The language model includes a first sub-model and a second sub-model. The response speed of the first sub-model is greater than that of the second sub-model, and the processing range of the second sub-model is greater than that of the first sub-model.

7. The voice interaction method for a refrigeration device according to any one of claims 1-5, characterized in that, The acquisition of the target user's voice interaction data with the refrigeration equipment includes: Acquire the initial voice interaction data of the target user with the refrigeration equipment; The initial voice interaction data is subjected to noise reduction and purification processing to obtain intermediate voice interaction data; The intermediate voice interaction data is enhanced with features to obtain the voice interaction data.

8. A voice interaction device for a refrigeration equipment, characterized in that, include: The acquisition module is used to acquire voice interaction data of the target user with the refrigeration equipment; The first processing module is used to determine the target identity information corresponding to the target user in the user database based on the voice interaction data. The user database includes the identity information of multiple users. The second processing module is used to process the voice interaction data based on the target identity information to obtain a first processing result; The third processing module is used to input the first processing result into the multi-task processing model to obtain the task embedding vector output by the multi-task processing model. The fourth processing module is used to embed the task into the vector input language model to obtain the interaction result corresponding to the voice interaction data.

9. A refrigeration device, characterized in that, include: The voice interaction device for the refrigeration equipment as described in claim 8.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the voice interaction method of the refrigeration device as described in any one of claims 1-7.

Images