Air conditioner and control method thereof

Abstract

The embodiment of the invention discloses an air conditioner and a control method thereof, relates to the technical field of air conditioners, and aims to improve the accuracy of voice processing. The air conditioner comprises a voice collecting device used for collecting voice input by a user to the air conditioner; the controller is configured to acquire voice input to the air conditioner by a user through the voice acquisition equipment; inputting the voice into an encoder to obtain output characteristics of the encoder; the output features are input into a decoder, the output features are processed through the decoder by means of attention information extracted by an attention module coupled between the encoder and the decoder, and an air conditioner operation instruction output by the decoder is obtained; and controlling operation of the air conditioner based on the air conditioner operation instruction.

Description

Technical Field

[0001] This application relates to the field of air conditioner technology, and in particular to an air conditioner and its control method. Background Technology

[0002] With the development of smart air conditioning technology, air conditioners, as key devices for regulating the indoor environment, play a vital role in improving the efficiency and comfort of activities in the area.

[0003] Existing air conditioners can automatically adjust their operating parameters based on preset conditions by integrating sensors and automated control logic. However, this method has limitations in handling complex user voice commands and responding quickly to environmental changes. Especially in multi-user environments, a uniform temperature setting is difficult to meet the comfort needs of everyone.

[0004] Therefore, how to accurately analyze users' voices and adjust the operating parameters of air conditioners to better adapt to personalized needs in diverse scenarios has become an urgent problem to be solved. Summary of the Invention

[0005] This application provides an air conditioner and its control method for improving the accuracy of voice processing.

[0006] To achieve the above objectives, this application adopts the following technical solution.

[0007] In a first aspect, embodiments of this application provide an air conditioner, including: a voice acquisition device for acquiring voice input from a user to the air conditioner; and a controller configured to: acquire the voice input from the user to the air conditioner via the voice acquisition device; input the voice into an encoder to obtain the output features of the encoder; input the output features into a decoder; process the output features using attention information extracted by an attention module coupled between the encoder and the decoder to obtain an air conditioner operation command output by the decoder; and control the operation of the air conditioner based on the air conditioner operation command.

[0008] The technical solution provided in this application provides at least the following beneficial effects: This technical solution extracts features from speech using an encoder, avoiding the problem of obtaining inaccurate or ambiguous speech features by directly analyzing the user's speech input to the air conditioner, thus improving the accuracy of speech processing; then, the obtained output features are processed through an attention module to extract attention information, and the output features are processed using the determined attention information to determine the air conditioner operation command, thereby controlling the air conditioner to operate based on the operating parameters indicated by the air conditioner operation command, making the air conditioner's operating parameters more in line with user needs, realizing intelligent and personalized adjustment of the air conditioner's operating parameters, and improving the user experience and comfort.

[0009] In some embodiments, the encoder includes multiple encoder modules, the decoder includes multiple decoder modules, and the attention module includes multiple attention sub-modules, each attention sub-module being hopped between an encoder module and a decoder module; wherein, the input of the subsequent encoder module in the multiple encoder modules is connected to the output of the preceding encoder module; and the input of the subsequent decoder module in the multiple decoder modules is connected to the output of the preceding decoder module.

[0010] In some embodiments, the encoder module includes a fusion convolution submodule and a downsampling submodule connected in sequence; wherein, the fusion convolution submodule is used to perform channel-dimensional decomposition on the information input to the encoder module to obtain a first decomposition feature focusing on global information and a second decomposition feature focusing on local information; based on the first decomposition feature and the second decomposition feature, local features and global features of speech are obtained; the local features and global features are concatenated and the concatenated features are output; the concatenated features are input to the downsampling submodule; the downsampling submodule is used to perform average pooling downsampling processing on the concatenated features to obtain downsampled features.

[0011] In some embodiments, obtaining local features of downsampled features based on first decomposition features and second decomposition features includes: performing standard convolution on the first decomposition features to obtain first processed features, performing standard convolution on the second decomposition features to obtain second processed features; concatenating the first processed features and second processed features to obtain third processed features; and performing batch normalization and activation function operations on the third processed features sequentially to obtain local features.

[0012] In some embodiments, the global features of the downsampled features are obtained based on the first decomposition features and the second decomposition features, including: performing dilated convolution on the first decomposition features to obtain the fourth processed features, and performing standard convolution on the second decomposition features to obtain the fifth processed features; concatenating the fourth processed features and the fifth processed features to obtain the sixth processed features; and performing batch normalization and activation function operations on the sixth processed features in sequence to obtain the global features.

[0013] In some embodiments, an intermediate module exists between the last encoder in the plurality of encoder modules and the first decoder module in the plurality of decoder modules; the controller is configured to input the output features into the decoder, specifically configured to: input the output features into the intermediate module, and obtain intermediate features by performing standard convolution processing, batch normalization operation and activation function operation on the output features in sequence through the intermediate module, thereby obtaining the intermediate features output by the intermediate module; and input the intermediate features into the decoder.

[0014] Secondly, embodiments of this application provide a method for controlling an air conditioner, comprising: acquiring voice input from a user to the air conditioner; inputting the voice into an encoder to obtain the output features of the encoder; inputting the output features into a decoder, and processing the output features using attention information extracted by an attention module coupled between the encoder and the decoder to obtain an air conditioner operation command output by the decoder; and controlling the operation of the air conditioner based on the air conditioner operation command.

[0015] Thirdly, embodiments of this application provide a controller, including: one or more processors; one or more memories; wherein the one or more memories are used to store computer program code, the computer program code including computer speech, and when the one or more processors execute the computer speech, the controller executes the method provided in the second aspect.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium including computer speech, which, when executed on a computer, causes the computer to perform the methods provided in the second aspect and possible implementations.

[0017] Fifthly, embodiments of the present invention provide a computer program product that can be directly loaded into a memory and contains software code. After being loaded and executed by a computer, the computer program product can implement the methods provided in the second aspect and possible implementations.

[0018] It should be noted that the aforementioned computer speech can be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium can be packaged together with the controller's processor, or it can be packaged separately from the controller's processor; this application does not impose any limitations on this.

[0019] The beneficial effects described in aspects two through five of this application can be referred to the analysis of the beneficial effects of aspect one, and will not be repeated here. Attached Figure Description

[0020] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of the present invention and do not constitute a limitation on the technical solutions of the present invention.

[0021] Figure 1 This is a schematic diagram of the structure of an air conditioner provided in an embodiment of this application;

[0022] Figure 2 A hardware configuration block diagram of an air conditioner provided in an embodiment of this application;

[0023] Figure 3This is a schematic flowchart of a control method for an air conditioner provided in an embodiment of this application;

[0024] Figure 4 A schematic diagram of the processing flow of a fused convolutional submodule provided in an embodiment of this application;

[0025] Figure 5 This application provides a schematic diagram of the structure of an integrated attention multi-domain fusion network as an embodiment of the present application.

[0026] Figure 6 A schematic diagram of the processing flow of an attention module provided in an embodiment of this application;

[0027] Figure 7 This is a schematic flowchart of another air conditioner control method provided in an embodiment of this application;

[0028] Figure 8 A schematic diagram of the processing flow of an intermediate module provided in an embodiment of this application;

[0029] Figure 9 This is a schematic flowchart of another air conditioner control method provided in an embodiment of this application;

[0030] Figure 10 This is a schematic flowchart of another air conditioner control method provided in an embodiment of this application;

[0031] Figure 11 A schematic flowchart illustrating a model training method provided in an embodiment of this application;

[0032] Figure 12 This is a schematic diagram of another air conditioner provided in an embodiment of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0035] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0036] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "connected" and "linked" should be interpreted broadly, for example, as a fixed connection, a detachable connection, or an integral connection. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, when describing pipelines, the terms "connected" and "linked" as used in this application have the meaning of establishing electrical connection. The specific meaning needs to be understood in conjunction with the context.

[0037] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0038] To facilitate understanding, we will first provide a brief introduction and explanation of some terms or basic concepts of technology involved in the embodiments of the present invention.

[0039] As mentioned above, traditional air conditioners often rely on manual control or basic automation, making it difficult to adapt to the personalized needs of different users in diverse meeting scenarios. With the development of smart technology, modern air conditioners, through the integration of sensors and automated control logic, can automatically adjust the indoor environment based on preset conditions. However, these systems have limitations in handling complex user voices and responding quickly to environmental changes, especially in multi-user environments where a uniform temperature setting is difficult to meet the comfort needs of everyone.

[0040] To address this challenge, natural language processing (NLP) technology has enabled air conditioners to understand and respond to users' natural language speech, promoting the personalized and intelligent development of environmental regulation. However, in the process of applying NLP technology, technical challenges such as fuzzy speech parsing still exist.

[0041] Based on this, this application provides an air conditioner that processes user-inputted voice through an encoder and a decoder to obtain air conditioner operation commands output by the decoder, thereby controlling the operation of the air conditioner based on these commands. This makes the operation of the air conditioner more aligned with user needs and improves the user experience.

[0042] The embodiments provided in this application will now be described in detail with reference to the accompanying drawings.

[0043] Figure 1 This is a schematic diagram of the structure of an air conditioner provided as an example of an embodiment of this application, such as... Figure 1 As shown, the air conditioner 1 includes an outdoor unit 2.

[0044] It should be noted that the air conditioner can be a multi-split air conditioner, a central air conditioner, etc., and this application does not limit the type of air conditioner.

[0045] As a feasible implementation method, the air conditioner 1 and the remote control 4 are connected by communication.

[0046] As a feasible implementation method, outdoor unit 2 is usually installed outdoors for heat exchange in the indoor environment.

[0047] In some embodiments, the air conditioner 1 includes an indoor unit 3.

[0048] As a feasible implementation method, indoor unit 3, taking indoor wall-mounted units as an example, is typically installed on indoor walls (such as in bedrooms, living rooms, conference rooms, etc.). Another example is indoor floor-standing units (…). Figure 1 (Not shown in the image) is also a type of indoor unit.

[0049] It should be noted that indoor unit 3 may include at least one indoor unit, and this application does not limit the number of indoor units of the air conditioner.

[0050] In some embodiments, the air conditioner 1 includes a voice acquisition device 5 ( Figure 1 (Not shown in the image), used to collect voice input from users in the area where air conditioner 1 is located.

[0051] For example, the voice acquisition device 5 can be a microphone, which can be set in any part of the air conditioner 1, or the voice acquisition device 5 can be set independently of the air conditioner 1 and communicate with the air conditioner 1 through a voice interface. This application does not limit the type and setting method of the voice acquisition device 5.

[0052] Figure 2 This is a hardware configuration block diagram of an air conditioner provided according to an exemplary embodiment of this application. For example... Figure 2 As shown, the air conditioner 1 may also include a controller 1000.

[0053] In some embodiments, the controller 1000 is the control center of the air conditioner 1, which connects various parts of the air conditioner 1 through various interfaces and lines, and executes various programs of the air conditioner 1 by running or executing programs stored in the memory and calling data stored in the memory.

[0054] In some embodiments, the controller 1000 may include one or more processing units 1001.

[0055] In the embodiments shown in this application, controller 1000 refers to a device that can generate operation control signals based on voice operation codes and timing signals to instruct air conditioner 1 to execute control methods. Exemplarily, controller 1000 can be a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof. Controller 1000 can also be other devices with processing functions, such as circuits, devices, or software modules; this application embodiment does not impose any limitations on this.

[0056] In addition, the controller 1000 can be used to control the various components inside the air conditioner 1 so that each component can operate to achieve the predetermined functions of the air conditioner 1.

[0057] In some embodiments, such as Figure 2 As shown, the controller 1000 includes a communicator 1002.

[0058] In some embodiments, the communicator 1002 is used to establish communication connections with other network entities, such as establishing communication connections with terminal devices. The communicator 1002 may include a radio frequency (RF) module, a cellular module, a wireless fidelity (WIFI) module, and a GPS module, etc. Taking an RF module as an example, the RF module can be used for signal reception and transmission; specifically, it sends received information to the controller 1000 for processing; additionally, it transmits signals generated by the controller 1000. Typically, the RF circuit may include, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc.

[0059] In some embodiments, such as Figure 2 As shown, the controller 1000 also includes a memory 1003.

[0060] In some embodiments, the memory 1003 may be used to store software programs and data. The controller 1000 executes various functions of the air conditioner 1 and data processing by running the software programs or data stored in the memory 1003. The memory 1003 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The memory 1003 stores an operating system that enables the air conditioner 1 to run. In this application, the memory 1003 may store the operating system and various application programs, and may also store code that executes the control method of the air conditioner 1 provided in the embodiments of this application.

[0061] Those skilled in the art will understand that Figure 2 The hardware structure shown does not constitute a limitation on the air conditioner 1. The air conditioner 1 may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0062] As one feasible implementation method, the air conditioner 1 communicates with the remote controller 4 through the communicator 1002.

[0063] In some embodiments, the remote controller 4 has the function of communicating with the controller 1000, for example, using infrared or other communication methods. Users can set various operating parameters of the air conditioner 1 using the remote controller 4.

[0064] Figure 3 This is a schematic flowchart of a control method for an air conditioner provided in an embodiment of this application, as shown below. Figure 3 As shown, the method includes the following steps:

[0065] S101, The controller acquires the user's voice input to the air conditioner.

[0066] As a feasible approach, a voice acquisition device can be installed in the area where the air conditioner is located, and the controller can acquire at least one voice message from at least one user in that area through the voice acquisition device.

[0067] It should be understood that the area where the air conditioner is located can be a bedroom, conference room, workshop, office, or other areas where the air conditioner may be located, and the user can be any user in the area where the air conditioner is located. This application does not limit the type of area where the air conditioner is located or the user.

[0068] It should be noted that in practical applications, the voice input from users to the air conditioner is generally divided into explicit voice (such as "switching to cooling mode" or "setting the temperature to 26℃") and vague voice (such as "too hot", "too cold", "a little hot", "a little cold", "too humid", "too dry"). After receiving explicit voice, the controller can adjust accordingly; after receiving vague voice, the controller needs to determine the current environmental conditions and the air conditioner's own settings.

[0069] Meanwhile, since at least one voice recording acquired by the voice acquisition device may include invalid voice (e.g., "What did I eat today?" or "What did I wear today?"), or, in areas with many people where the air conditioner is located, there may be interfering voice, at least one voice recording needs to be processed after it is acquired. The processing process includes, but is not limited to: eliminating interfering voice, classifying the source of the voice, and parsing ambiguous voice.

[0070] Furthermore, the user's voice input to the air conditioner is determined from at least one of the processed voice recordings.

[0071] As a feasible implementation method, a preset voice library can be set in the memory. This preset voice library stores voices that can be used as user input to the air conditioner. After the air conditioner controller receives at least one voice from the voice acquisition device, it identifies the voice contained in the voice and compares each voice with the voices in the preset voice library one by one to determine whether at least one voice contains any voice from the preset voice library. If it does, the voice is used as the user input to the air conditioner; if it does not, it means that the voice cannot be executed by the air conditioner, and the voice is not used as the user input to the air conditioner. The system continues to collect user voices.

[0072] As a feasible implementation method, the preset voice library can include sensory voice, action voice, adjustment amplitude voice, and parameter type voice.

[0073] The system includes: **Feeling Voice**, which uses voice keywords to express the user's subjective comfort level, including at least one of "hot," "cold," "large," "small," "dry," and "humid"; **Action Voice**, which uses voice keywords to express how the user wants the air conditioner to operate, including at least one of "raise," "lower," "increase," and "decrease"; **Adjustment Range Voice**, which uses voice keywords to express the user's desired adjustment range for the air conditioner's operating parameters, including at least one of "most," "too much," "good," "a little," and "true"; and **Parameter Type Voice**, which uses voice keywords to express the type of operating parameter the user wants to adjust, including at least one of "temperature," "humidity," "fan speed," and "volume."

[0074] Specifically, when a user in the area where the air conditioner is located says "It's too hot today," the controller receives the message and determines that the voice keywords include "too" and "hot." Based on these keywords, the controller then interprets "It's too hot today" as the user's voice input to the air conditioner.

[0075] S102, The controller inputs voice into the encoder to obtain the encoder's output characteristics.

[0076] The encoder includes multiple encoder modules. The input of the next encoder module is connected to the output of the previous encoder module. Each encoder module includes a fusion convolution submodule and a downsampling submodule connected in sequence.

[0077] It should be noted that when processing features of different scales within the same dataset, using a fixed-size convolutional kernel may hinder the learning efficiency and performance of an integrated attention-based multi-domain fusion network. Since each convolutional kernel can only perceive local information from the input data, it cannot capture sufficiently broad contextual information when processing large-scale global information, potentially leading to information loss. Therefore, the encoder in this application includes one or more fusion convolutional submodules to achieve a broader extraction of speech feature information.

[0078] As a feasible approach, the fusion convolution submodule is used to decompose the downsampled features along the channel dimension to obtain the first decomposed features that focus on global information and the second decomposed features that focus on local information.

[0079] Among them, the fusion convolution submodule can be a multi-unite convolution (MUC) module. In the multi-unite convolution module, the propagation of local and global information is a cross-fusion, which allows both information parts to learn all channel information, ensuring that the local and global perspectives are not processed in isolation, and that each part can benefit from comprehensive channel information.

[0080] As a feasible implementation, the fused convolutional submodule can consist of three 3×3 convolutions and one dilated convolution, while also using batch normalization (BN) and activation functions (leaky rectified linear unit, Leaky ReLU) (or LR).

[0081] It should be noted that, without adding additional parameters or computational cost, dilated convolution significantly increases the receptive field of network layers by inserting zero padding into standard convolution kernels. Even shallower network layers can observe a wider input region and can better capture global contextual information.

[0082] Figure 4 This is a schematic diagram of the processing flow of a fused convolutional submodule provided in an embodiment of this application, as shown below. Figure 4 As shown, the information from the input encoder module is input into the fusion convolution submodule of the encoder. The fusion convolution submodule decomposes the information from the input encoder module into two parts: one part focuses on local information (tensor local, TL), and the other part focuses on global information (tensor global, TG). The calculation methods of local information TL and global information TG are shown in formula (1):

[0083] TL, TG = split(T) in ) Formula (1)

[0084] Here, `split` is a decomposition operation based on the channel dimension.

[0085] Furthermore, TG is processed by a 3×3 convolution (Conv) and an atrous convolution (Atrous) to obtain the first decomposition features (TG1, TG2), and TL is processed by two 3×3 convolutions (Conv) to obtain the second decomposition features (TL1, TL2).

[0086] As a feasible implementation method, the first decomposition feature (TG1, TG2) and the second decomposition feature (TL1, TL2) can be represented by vectors.

[0087] Furthermore, based on the first and second decomposition features, the local and global features of the information are obtained.

[0088] Specifically, TG2 from the first decomposition feature and TL1 from the second decomposition feature are concatenated (Concat) to form a single vector. Then, batch normalization (BN) and activation function LR (BN-LR) are performed to obtain the local feature T of the information. local By performing a similarity operation between TG1 in the first decomposition feature and TL2 in the second decomposition feature, the global feature T of the information is obtained. global .

[0089] As a feasible implementation method, T local The method for obtaining it is shown in formula (2):

[0090] T local=δ(BN(Concat(Conv(TL1),Conv(TG2)))) Formula (2)

[0091] Where δ is the activation function LeakyReLU, and Conv is a regular convolution.

[0092] T global The method for obtaining it is shown in formula (3):

[0093] T global =δ(BN(Concat(Conv(TL2),Atrous(TG1)))) Formula (3)

[0094] Atrous is a dilated convolution.

[0095] Furthermore, the local and global features are concatenated and the concatenated features are output. The concatenation method is shown in formula (4):

[0096] T out =T local +T global Formula (4)

[0097] Among them, T out These are the features after splicing.

[0098] Furthermore, the spliced ​​features are input into the downsampling submodule.

[0099] As a feasible implementation method, the downsampling submodule is used to perform average pooling downsampling on the concatenated features to obtain downsampled features.

[0100] It should be noted that by performing average pooling downsampling on the concatenated features, not only can the computational load on the encoder be reduced, but the encoder's ability to process and understand speech can also be enhanced. In this way, a larger receptive field can be achieved in the early stages of encoder learning, capturing the global features of the user's speech input from the air conditioner, while also taking into account local features.

[0101] S103. The controller inputs the output features into the decoder. The decoder processes the output features using the attention information extracted by the attention module coupled between the encoder and the decoder, and obtains the air conditioning operation command output by the decoder.

[0102] In some embodiments, a new high perceptual loss (HPL) or feature reconstruction loss can be introduced into the decoder to measure the difference between the operating parameters indicated by the predicted air conditioning operation command and the target operating parameters, so as to ensure that the operating parameters indicated by the predicted air conditioning operation command can more accurately meet the comfort requirements of the area.

[0103] As a feasible implementation, the decoder can adopt a high perceptual base model (HPBM) and use a pre-trained (visual geometry group19, VGG19) convolutional neural network as the high perceptual base model.

[0104] As a feasible implementation method, both the encoder and decoder belong to the integrated-attention multi-unite network (IAMU). Figure 5 A schematic diagram of an integrated attention multi-domain fusion network provided in an embodiment of this application is shown below. Figure 5 As shown, the controller inputs the encoder's output features into the decoder, and the decoder processes the output features using the attention information extracted by the attention module to obtain the air conditioning operation command output by the decoder.

[0105] The decoder includes multiple decoder modules. The attention module is coupled between the encoder and the decoder. The attention module includes multiple attention sub-modules. Each attention sub-module is connected between an encoder module and a decoder module. The input of the next decoder module is connected to the output of the previous decoder module.

[0106] It should be noted that the attention module can enhance the ability to capture key information in multi-domain fusion network speech. Whether it is adjusting the air conditioner's operating temperature, fan speed, or mode, it can be quickly transmitted to the decoder through jump connections, thereby ensuring the accuracy and real-time performance of the air conditioner's operating commands.

[0107] Figure 6 This is a schematic diagram of the processing flow of an attention module provided in an embodiment of this application, such as... Figure 6 As shown, given the speech features z output by an encoder i The attention module first constructs dual-scale global channel and local spatial information, and then integrates the speech features z... iAfter layer normalization (LN), the channel dimension is expanded using a 1×1 regular convolution, and the speech features z are then... i It is divided into two branches, and each branch is processed by a standard convolution (3×3 convolution).

[0108] The Gaussian error linear unit (GELU) activation function and adaptive average pooling (AAP) are used to obtain the average information in the vector space; another branch uses a 5×5 convolution, GELU activation function, and adaptive global pooling (AGP) to obtain the top information of the speech features in the spatial dimension. The two branches are concatenated according to channels to restore the original dimension. This process can be represented as:

[0109] z 3× ,z 5× =Divide(conv) DE (LN(z i ))) Formula (5)

[0110] z spatial =conv DS [AAP(δ(conv 3× (z 3× )),AMP(δ(conv 5× (z 5× )))] Formula (6)

[0111] Where Divide means dividing the speech features into two parts equally according to the channel dimension, DE means dimension expansion (implemented using 1×1 convolution), DS means dimension shrinkage (implemented using 1×1 convolution), convi× means 3×3 or 5×5 convolution, and [,] means Concat operation.

[0112] It should be noted that pooling is a technique used to reduce the dimensionality of input data and effectively perform downsampling operations. Pooling provides non-linear properties, which allows the model to capture and represent more complex patterns and relationships in the data. This non-linearity requires a deeper convolutional layer to achieve. Since pooling downsampling is relatively shallow in Integrative Attention Multi-Domain Fusion Networks, it provides a non-linear processing method, which not only reduces the computational load but also enhances the Integrative Attention Multi-Domain Fusion Networks' ability to process and understand data.

[0113] Furthermore, the attention module constructs local channel and global spatial information, and uses a projection function (PF) to directly map the speech features onto a unified dimension for flattening, resulting in z. spatial-channel , i.e. z sc

[0114] z sc =PF(z) spatial ) Formula (7)

[0115] in, PF is the projection function, which aims to integrate information across channels. It is implemented using k×k convolution, where the size of k represents the interaction range of the channels and can be proportional to the number of channels C.

[0116] Suppose that the integrated attention multi-domain fusion network uses 5 attention sub-modules. From the high layer to the low layer of the model, as the number of channels gradually decreases, the convolution kernels of the attention modules are set to 11, 9, 7, 5 and 3 respectively to adaptively set the interaction range.

[0117] Furthermore, the Sigmoid function is used to assign weights to each convolutional kernel, which are then multiplied with the original speech features to obtain the output feature z containing attention information. out The formula is expressed as:

[0118] z out =σ(z) sc )°z i Formula (8)

[0119] Where σ is the Sigmod function, and ° represents element-wise multiplication (i.e., Hadamard product).

[0120] Furthermore, the output features are upsampled and information is reconstructed by the upsampling submodule in the decoder to obtain the speech features corresponding to the voice input from the air conditioner. Then, based on the user demand information corresponding to the voice input from the air conditioner represented by the speech features, the air conditioning operation instructions of the air conditioner are determined.

[0121] It should be noted that through a series of processing of speech features by this attention module, complementary spatial and channel information is fused, enabling the integrated attention multi-domain fusion network to pay more attention to the key parts of the user's speech input to the air conditioner, thereby improving the feature identification capability.

[0122] Table 1 shows a correspondence table. As shown in Table 1, the correspondence table can include multiple voice features, multiple user demand information, and multiple air conditioner operation commands, and there is a one-to-one correspondence between the multiple voice features, multiple user demand information, and multiple air conditioner operation commands.

[0123] Table 1

[0124] speech features User needs information Air conditioning operation instructions Users feel hot Lower temperature "Lower operating temperature" The user felt cold Increase temperature "Increase operating temperature" The wind speed is too high Reduce wind speed "Reduce operating wind speed" The wind speed is too low Increase wind speed Increase operating wind speed ... ... ...

[0125] It should be noted that Table 1 only illustrates the correspondence between four voice features, user demand information, and air conditioner operation commands as examples. In specific implementations, the voice features may include more or fewer, and the corresponding user demand information and air conditioner operation commands may also include more or fewer. This application does not limit this.

[0126] S104. The controller controls the operation of the air conditioner based on the air conditioner operation command.

[0127] In some embodiments, when the air conditioner operation command is a voice command indicating the adjustment range, taking the operating temperature as an example, if the air conditioner's operating temperature is to reach the air conditioner operating temperature indicated by the air conditioner operation command, the current operating temperature needs to be lowered or raised. The adjustment range can be 1°C, 2°C or other possible ranges, which are not limited in this application.

[0128] For example, if the current operating temperature of the air conditioner is 28℃, the adjustment range is 1℃, and the voice feature corresponding to the user's voice input to the air conditioner output by the encoder is that the user feels hot, then the operating parameter indicated by the air conditioner operation command is determined to be an operating temperature of 27℃; if the current operating fan speed of the air conditioner is at level one, the adjustment range is at level one, and the voice feature corresponding to the user's voice input to the air conditioner output by the encoder is that the user feels the fan speed is too low, then the operating parameter indicated by the air conditioner operation command is determined to be an operating fan speed of level two.

[0129] In some embodiments, if the air conditioner operation command is a parameter-type voice command, for example, if the operating parameter indicated by the air conditioner operation command is an operating temperature of 26°C, then the controller controls the air conditioner to operate at 26°C; if the operating parameter indicated by the air conditioner operation command is an operating fan speed of level two, then the air conditioner operates at level two fan speed; if the operating parameter indicated by the air conditioner operation command is a dehumidification level of level three, then the air conditioner operates at level three dehumidification, satisfying the user's voice input corresponding to the user's needs; if the operating parameter indicated by the air conditioner operation command is a cooling mode, then the air conditioner operates in cooling mode, satisfying the user's voice input corresponding to the user's needs.

[0130] The technical solution provided in this application provides at least the following beneficial effects: This technical solution extracts features from speech using an encoder, avoiding the problem of obtaining inaccurate or ambiguous speech features by directly analyzing the user's speech input to the air conditioner, thus improving the accuracy of speech processing; then, the obtained output features are processed by an attention module to extract attention information, and the output features are processed using the determined attention information to determine the air conditioner operation command, thereby controlling the air conditioner to operate based on the operating parameters indicated by the air conditioner operation command, making the air conditioner's operating parameters more in line with user needs, realizing intelligent and personalized adjustment of the air conditioner's operating parameters, and improving the user experience and comfort.

[0131] In some embodiments, the process of the user's voice input to the air conditioner by the above-mentioned integrated attention multi-domain fusion network can also be represented by formula (9):

[0132] y=IAB([MUC(x'),↑(MUC(x'))]) Formula (9)

[0133] Where y represents the prediction result of the integrated attention multi-domain fusion network or whether the current speech is the user's input to the air conditioner, IAB is the attention module, MUC is the multi-domain fusion convolutional module, and ↑ is the decoder.

[0134] It should be noted that the integrated attention multi-domain fusion network, through this structured design, not only improves the ability to understand speech, but also enhances the flexibility and accuracy of the air conditioner's response. Within the area where the air conditioner is located, it can automatically adjust the air conditioning settings based on real-time feedback from people in the area or preset activity schedules to ensure comfort and efficiency during activities.

[0135] In some embodiments, such as Figure 5 As shown, there is an intermediate module (Centre) between the last encoder in the multiple encoder modules and the first decoder module in the multiple decoder modules. The "inputting the output features into the decoder" step S103 can be specifically achieved through... Figure 7 The flowchart shown is implemented as follows: Figure 7 This is a schematic flowchart of another air conditioner control method provided in an embodiment of this application, as shown below. Figure 7 As shown, the method includes the following steps:

[0136] S201. Input the output features into the intermediate module. The intermediate module performs standard convolution, batch normalization and activation function operations on the output features in sequence to obtain intermediate features, and obtains the intermediate features output by the intermediate module.

[0137] As a feasible implementation, after the output features undergo one MUC and pooling downsampling, the number of channels in the integrated attention multi-domain fusion network doubles, while the spatial size is reduced to half. Figure 8 This is a schematic diagram of the processing flow of an intermediate module provided in an embodiment of this application, such as... Figure 8 As shown, the controller sends the encoder's output feature Tout to the intermediate module. After standard convolution processing (one layer of 3×3 convolution), batch normalization operation, and ReLU activation function operation, the output feature Tout is used to obtain the intermediate feature output by the intermediate module.

[0138] S202, The controller sends the intermediate features to the decoder.

[0139] Specifically, the controller inputs the intermediate features into the decoder, which then processes the output features using the attention information extracted by the attention module coupled between the encoder and decoder, to obtain the air conditioning operation command output by the decoder.

[0140] In some embodiments, the processing of downsampled features by the above-mentioned fusion convolution submodule can also be achieved through, for example... Figure 9 The flowchart shown is implemented as follows: Figure 9 Another air conditioner control method provided in this application embodiment is used to acquire local features, such as... Figure 9 As shown, the method includes the following steps:

[0141] S301. Perform standard convolution on the first decomposed feature to obtain the first processed feature, and perform standard convolution on the second decomposed feature to obtain the second processed feature.

[0142] Specifically, the first decomposed feature is processed by a 3×3 convolution (Conv) to obtain the first processed feature TG1; the first decomposed feature is processed by a 3×3 convolution (Conv) to obtain the second processed feature TL2.

[0143] S302. The first and second processing features are concatenated to obtain the third processing feature.

[0144] Specifically, TG1 and TL2 are concatenated to obtain the third processing feature.

[0145] S303. Perform batch normalization and activation function operations on the third processing features in sequence to obtain local features.

[0146] Specifically, the third-processed feature is sequentially subjected to batch normalization and LeakyReLU activation function operations to obtain the local feature T of the downsampled feature. local The calculation method is as shown in formula (2) above, and will not be repeated here.

[0147] In some embodiments, the processing of downsampled features by the above-mentioned fusion convolution submodule can also be achieved through, for example... Figure 10 The flowchart shown is implemented as follows: Figure 10 Another air conditioner control method provided in this application embodiment is used to obtain global features, such as... Figure 10 As shown, the method includes the following steps:

[0148] S401. Perform dilated convolution on the first decomposition feature to obtain the fourth processed feature, and perform standard convolution on the second decomposition feature to obtain the fifth processed feature.

[0149] Specifically, the first decomposed feature is processed by an atrous convolution to obtain the fourth processed feature TG2, and the second decomposed feature is processed by a 3×3 convolution to obtain the fifth processed feature TL1.

[0150] S402. The fourth and fifth processing features are concatenated to obtain the sixth processing feature.

[0151] Specifically, TG2 and TL1 are concatenated to obtain the sixth processing feature.

[0152] S403. Perform batch normalization and activation function operations on the sixth processing feature in sequence to obtain the global feature.

[0153] Specifically, the sixth processed feature is sequentially subjected to batch normalization and LeakyReLU activation function operations to obtain the global feature T of the downsampled feature. global The calculation method is as shown in the above formula (3), and will not be repeated here.

[0154] Furthermore, the downsampled local features T local and global features T global Perform splicing and output the spliced ​​features. For details on the splicing method, please refer to the above formula (5), which will not be repeated here.

[0155] It should be noted that since the initial integrated attention multi-domain fusion network has not been trained iteratively with a large amount of data, the output results may be inaccurate. Therefore, before using the integrated attention multi-domain fusion network, the initial integrated attention multi-domain fusion network needs to be trained. Figure 11 This is a flowchart illustrating a model training method provided in an embodiment of this application, as shown below. Figure 11 As shown, the method includes the following steps:

[0156] S501. Obtain multiple sample voices and the corresponding sample air conditioner operation instructions for each sample voice.

[0157] It should be noted that, in order to avoid invalid speech increasing the cost of model training, the multiple speech samples obtained need to be processed to obtain multiple processed sample speech samples.

[0158] The processing includes, but is not limited to: excluding speech with interference, classifying the speaker of the speech, and parsing ambiguous speech.

[0159] S502. Input multiple sample speech into the initial integrated attention multi-domain fusion network to obtain air conditioning operation instructions corresponding to the multiple sample speech.

[0160] S503. Based on the predicted air conditioning operation instructions and sample air conditioning operation instructions corresponding to multiple sample speech, determine the loss function corresponding to multiple sample speech.

[0161] As a feasible approach, the loss function corresponding to multiple sample speech can be determined based on the first feature information of the predicted air conditioning operation command and the second feature information of the sample air conditioning operation command.

[0162] Specifically, the predicted air conditioner operation command is input into the feature extraction network to obtain the first feature information output by each network layer of the feature extraction network.

[0163] Among them, the feature extraction network is used to extract feature information of the air conditioner operation instructions.

[0164] Specifically, the predicted air conditioning operation command is input into each network layer of the feature extraction network to obtain the first feature information output by each network layer of the feature extraction network.

[0165] Furthermore, the sample air conditioner operation command is input into the feature extraction network to obtain the second feature information output by each network layer of the feature extraction network.

[0166] Specifically, the sample air conditioner operation command is input into each network layer of the feature extraction network to obtain the second feature information output by each network layer of the feature extraction network.

[0167] Furthermore, based on the first feature information and the second feature information, the loss function corresponding to multiple sample speech is determined.

[0168] Specifically, the loss function for multiple sample speech is shown in Equation (10):

[0169]

[0170] Among them, y i This indicates the sample air conditioning operation command. This represents the predicted air conditioning operation command, where i represents the i-th layer of the feature extraction network, and C... iN is the number of channels corresponding to the speech features. i The resolution corresponding to the speech features (since the speech feature vector is a specific number, N) i The default value is 1). φ represents the fitting coefficients to prevent excessive loss from affecting the model's optimization process. HPBM (y i ) represents the first feature information output by the i-th layer of the feature extraction network. This represents the second feature information output by the i-th layer of the feature extraction network.

[0171] S504. The initial integrated attention multi-domain fusion network is iteratively trained based on the loss function corresponding to multiple sample speech until the iteration stopping condition is met, thus obtaining the trained integrated attention multi-domain fusion network.

[0172] As a feasible approach, once the loss functions corresponding to multiple sample speech are determined, the initial integrated attention multi-domain fusion network is iteratively trained based on the loss functions corresponding to multiple sample speech until the iteration stopping condition is met, thus obtaining the trained integrated attention multi-domain fusion network.

[0173] It should be noted that the iteration stopping condition can be the number of training iterations reaching a preset number, the training duration reaching a preset duration, or other possible iteration stopping conditions. This application does not limit the iteration stopping condition.

[0174] Furthermore, after the integrated attention multi-domain fusion network is trained, the user's voice input to the air conditioner is fed into the encoder in the integrated attention multi-domain fusion network. The decoder processes the output features of the encoder to obtain the air conditioner operation command output by the decoder, and then controls the operation of the air conditioner based on the air conditioner operation command.

[0175] In this embodiment of the invention, electronic products can be divided into functional modules according to the above method examples. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0176] When dividing each function into modules according to its corresponding function. Figure 12 This is a schematic diagram of another air conditioner provided in an embodiment of this application, as shown below. Figure 12 As shown, the air conditioner 1 may include: an acquisition module 201, an input / output module 202, and a processing module 203.

[0177] The acquisition module 201 is used to acquire the user's voice input to the air conditioner through the voice acquisition device.

[0178] The input / output module 202 is used to input voice into the encoder to obtain the output characteristics of the encoder.

[0179] The input / output module 202 is also used to input the output features into the intermediate module, and the intermediate module sequentially performs standard convolution processing, batch normalization operation and activation function operation on the output features to obtain intermediate features, and obtains the intermediate features output by the intermediate module.

[0180] As one feasible implementation, the input / output module 202 is also used to input the intermediate features into the decoder.

[0181] In some embodiments, the air conditioner 1 may further include a storage module 204 and a communication module 205.

[0182] In some embodiments, storage module 204 may be a memory. Communication module 205 may be a transceiver, transceiver circuit, or communication interface, etc.

[0183] In this configuration, when the processing module 203 is a processor, the storage module 204 is a memory, and the communication module 205 is a transceiver, the processor, transceiver, and memory can be connected via a bus. The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc.

[0184] This invention also provides a computer-readable storage medium, which includes computer-executed speech, such that when the computer-executed speech is run on a computer, the computer performs the method provided in the above embodiments.

[0185] This invention also provides a computer program product that can be directly loaded into a memory and contains software code. After being loaded and executed by a computer, the computer program product can implement the methods provided in the above embodiments.

[0186] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more voice or code messages on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of computer programs from one place to another. Storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0187] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0188] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely exemplary; for instance, the division of modules or units is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed between each other may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate; components shown as units may be one physical unit or multiple physical units, i.e., they may be located in one place or distributed in multiple different places. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0189] Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several voice commands to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0190] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An air conditioner, characterized in that, include: A voice acquisition device is used to acquire the voice input by the user to the air conditioner; The controller is configured as follows: The voice input from the user to the air conditioner is acquired through the voice acquisition device; The voice input encoder is used to obtain the output features of the encoder. The output features are input into the decoder, and the decoder processes the output features using the attention information extracted by the attention module coupled between the encoder and the decoder to obtain the air conditioner operation command output by the decoder. The operation of the air conditioner is controlled based on the air conditioner operation command.

2. The air conditioner according to claim 1, characterized in that, The encoder includes multiple encoder modules, the decoder includes multiple decoder modules, and the attention module includes multiple attention sub-modules. Each attention sub-module is connected in a skip connection between one encoder module and one decoder module. The input of the next encoder module is connected to the output of the previous encoder module, and the input of the next decoder module is connected to the output of the previous decoder module.

3. The air conditioner according to claim 2, characterized in that, The encoder module includes a fusion convolution submodule and a downsampling submodule connected in sequence. The fusion convolution submodule performs channel-dimensional decomposition on the information input to the encoder module to obtain a first decomposition feature focusing on global information and a second decomposition feature focusing on local information. Based on the first and second decomposition features, local and global features of the speech are obtained. The local and global features are concatenated and the concatenated features are output. The concatenated features are input to the downsampling submodule. The downsampling submodule performs average pooling downsampling on the concatenated features to obtain downsampled features.

4. The air conditioner according to claim 3, characterized in that, The process of obtaining local features of the speech based on the first decomposition features and the second decomposition features includes: The first decomposed feature is subjected to standard convolution to obtain the first processed feature, and the second decomposed feature is subjected to standard convolution to obtain the second processed feature. The first processing feature and the second processing feature are concatenated to obtain the third processing feature; The third processed feature is subjected to batch normalization and activation function operations in sequence to obtain the local feature.

5. The air conditioner according to claim 3, characterized in that, The process of obtaining global features of the speech based on the first decomposition features and the second decomposition features includes: The first decomposed feature is subjected to dilated convolution to obtain the fourth processed feature, and the second decomposed feature is subjected to standard convolution to obtain the fifth processed feature. The fourth and fifth processing features are concatenated to obtain the sixth processing feature; The sixth processed feature is subjected to batch normalization and activation function operations in sequence to obtain the global feature.

6. The air conditioner according to claim 2, characterized in that, There is an intermediate module between the last encoder in the plurality of encoder modules and the first decoder module in the plurality of decoder modules; The controller is configured to input the output features into the decoder, specifically configured as follows: The output features are input into the intermediate module, and the intermediate module sequentially performs standard convolution, batch normalization and activation function operations on the output features to obtain intermediate features, which are then output by the intermediate module. The intermediate features are then input into the decoder.

7. A control method for an air conditioner, characterized in that, include: Acquire the user's voice input to the air conditioner; The voice input encoder is used to obtain the output features of the encoder. The output features are input into the decoder, and the decoder processes the output features using the attention information extracted by the attention module coupled between the encoder and the decoder to obtain the air conditioner operation command output by the decoder. The operation of the air conditioner is controlled based on the air conditioner operation command.

8. The method according to claim 7, characterized in that, The encoder includes multiple encoder modules, the decoder includes multiple decoder modules, and the attention module includes multiple attention sub-modules. Each attention sub-module is connected in a skip connection between one encoder module and one decoder module. The input of the next encoder module is connected to the output of the previous encoder module, and the input of the next decoder module is connected to the output of the previous decoder module.

9. The method according to claim 8, characterized in that, The encoder module includes a fusion convolution submodule and a downsampling submodule connected in sequence. The fusion convolution submodule performs channel-dimensional decomposition on the information input to the encoder module to obtain a first decomposition feature focusing on global information and a second decomposition feature focusing on local information. Based on the first and second decomposition features, local and global features of the speech are obtained. The local and global features are concatenated and the concatenated features are output. The concatenated features are input to the downsampling submodule. The downsampling submodule performs average pooling downsampling on the concatenated features to obtain downsampled features.

10. The method according to claim 8, characterized in that, There is an intermediate module between the last encoder in the plurality of encoder modules and the first decoder module in the plurality of decoder modules; The step of inputting the output features into the decoder includes: The output features are input into the intermediate module, and the intermediate module sequentially performs standard convolution, batch normalization and activation function operations on the output features to obtain intermediate features, which are then output by the intermediate module. The intermediate features are then input into the decoder.

Images