Home customer control method and system based on large model, and storage medium

Abstract

The invention provides a home customer control method and system based on a large model and a storage medium, and the method comprises the steps: inputting a home customer control sample into the large model for model training until the large model converges; performing model compression on the converged large model to obtain a pre-trained large model, and obtaining a user home customer control instruction; inputting the user home guest control instruction into a pre-trained large model for guest control analysis to obtain target home equipment and equipment guest control parameters, and generating a guest control adjustment instruction according to the equipment guest control parameters; and controlling the target home equipment to perform parameter adjustment according to the guest control adjustment instruction. According to the embodiment of the invention, the home customer control sample is input into the large model for model training, so that the converged large model can better understand the semantics and intention in the natural language instruction of the user, and has higher adaptability to fuzzy and complex instructions and different expression modes, thereby remarkably improving the accuracy of understanding the complex customer control instruction, and improving the user experience. And the accuracy of home customer control is improved.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to a home customer control method, system and storage medium based on a large model. Background Art

[0002] With the rapid development of information technology and the gradual improvement of network technology, intelligent information services are entering households, creating smart homes, improving people's living standards and quality of life. Smart homes, as the main way to achieve smart homes, connect various home appliances with users through the Internet, the Internet of Things and home guest control technology to achieve a home environment with intelligent management and control.

[0003] In the existing home guest control process, home guest control is generally performed based on a preset instruction set, which may result in complex instructions not being accurately recognized and executed, thereby reducing the accuracy of home guest control. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a home customer control method, system and storage medium based on a large model to solve the problem of low accuracy of home customer control in the prior art.

[0005] The embodiment of the present invention is implemented as follows: a home customer control method based on a large model, the method comprising:

[0006] Acquire home customer control samples, and input the home customer control samples into the large model for model training until the large model converges;

[0007] Performing model compression on the converged large model to obtain the pre-trained large model, and acquiring the user's home control instructions;

[0008] Inputting the user's home customer control command into the pre-trained large model for customer control analysis, obtaining target home equipment and equipment customer control parameters, and generating customer control adjustment instructions according to the equipment customer control parameters;

[0009] The target home appliance is controlled to adjust parameters according to the customer control adjustment instruction.

[0010] Preferably, controlling the target home appliance to adjust parameters according to the customer control adjustment instruction includes:

[0011] Acquire the home environment of the user corresponding to the user home guest control command;

[0012] If there is one target home appliance in the home environment, the customer control adjustment instruction is sent to the target home appliance to control the target home appliance to adjust parameters;

[0013] If the number of the target home devices in the home environment is greater than 1, obtaining the location information of the user and the target home devices to obtain the user coordinates and the device coordinates;

[0014] The distance between the device coordinates and the user coordinates is calculated to obtain the interval distance, and the customer control adjustment instruction is sent to the target home device corresponding to the minimum interval distance to control the target home device corresponding to the minimum interval distance to adjust parameters.

[0015] Preferably, if the number of the target home devices in the home environment is greater than 1, the method further includes:

[0016] Capturing an image of the user to obtain a facial image of the user, and acquiring an eye image in the facial image of the user;

[0017] Performing gaze point recognition on the eye image to obtain a user gaze point, and determining a user gaze area according to the user gaze point;

[0018] If there is one of the target home appliances in the user's gaze area, the customer control adjustment instruction is sent to the target home appliance to control the target home appliance to adjust parameters;

[0019] If the number of the target home devices in the user's gaze area is greater than 1, calculating the gaze distance between the device coordinates and the user's gaze point;

[0020] Sending the customer control adjustment instruction to the target home device corresponding to the minimum gaze distance to control the target home device corresponding to the minimum gaze distance to adjust parameters;

[0021] If the target home appliance does not exist in the user's gaze area, the customer control adjustment instruction is sent to the target home appliance corresponding to the minimum interval distance to control the target home appliance corresponding to the minimum interval distance to adjust parameters.

[0022] Preferably, the large model after convergence is compressed to obtain the pre-trained large model, including:

[0023] Obtaining parameter weights and weight distribution of the large model after convergence, and determining quantization parameters according to the parameter weights and the weight distribution;

[0024] The floating-point number parameters in the converged large model are quantized and converted according to the quantization parameters to obtain the pre-trained large model.

[0025] Preferably, after the floating-point number parameters in the converged large model are quantized according to the quantization parameter, the method further includes:

[0026] Obtaining neuron evaluation parameters, and obtaining the parameter values ​​to be evaluated of the neurons in the large model after quantization conversion according to the neuron evaluation parameters;

[0027] Sorting the neurons according to the parameter values ​​to be evaluated to obtain an evaluation sorting list, and determining neurons to be pruned among the neurons according to the evaluation sorting list;

[0028] The neurons to be pruned are deleted to obtain the pre-trained large model.

[0029] Preferably, the home customer control samples are input into the large model for model training until the large model converges, including:

[0030] Segmenting the home customer control sample according to the large model to obtain segmentation samples, and encoding the segmentation template to obtain word embedding information;

[0031] Performing vector conversion on the word embedding information to obtain an embedding vector, and performing position encoding on the embedding vector to obtain a position vector;

[0032] Processing the position vector using a multi-head self-attention mechanism to obtain a self-attention feature, and encoding the self-attention feature to obtain a home guest control feature;

[0033] Decoding the home customer control features to obtain customer control prediction parameters and prediction target devices, and determining model loss according to the customer control prediction parameters and the prediction target devices;

[0034] Parameters of the large model are updated according to the model loss until the large model converges.

[0035] Preferably, determining the model loss according to the customer-controlled prediction parameter and the prediction target device includes:

[0036] Obtaining standard customer control parameters and standard equipment corresponding to the home customer control sample, and calculating parameter similarity between the standard customer control parameters and the customer control prediction parameters;

[0037] Determining a first loss according to the parameter similarity, and calculating the pronunciation similarity between the standard device and the predicted target device;

[0038] A second loss is determined according to the pronunciation similarity, and a weighted operation is performed on the first loss and the second loss to obtain the model loss.

[0039] Another object of an embodiment of the present invention is to provide a home guest control system based on a large model, the system comprising:

[0040] A model training module, used for obtaining home customer control samples and inputting the home customer control samples into a large model for model training until the large model converges;

[0041] A model compression module, used to compress the converged large model to obtain the pre-trained large model and obtain the user's home control instructions;

[0042] A customer control analysis module, used to input the user's home customer control command into the pre-trained large model for customer control analysis, obtain target home equipment and equipment customer control parameters, and generate a customer control adjustment command according to the equipment customer control parameters;

[0043] The parameter adjustment module is used to control the target home appliance to adjust the parameters according to the customer control adjustment instruction.

[0044] Preferably, the parameter adjustment module is further used to: obtain the home environment of the user corresponding to the user home guest control instruction;

[0045] If there is one target home appliance in the home environment, the customer control adjustment instruction is sent to the target home appliance to control the target home appliance to adjust parameters;

[0046] If the number of the target home devices in the home environment is greater than 1, obtaining the location information of the user and the target home devices to obtain the user coordinates and the device coordinates;

[0047] The distance between the device coordinates and the user coordinates is calculated to obtain the interval distance, and the customer control adjustment instruction is sent to the target home device corresponding to the minimum interval distance to control the target home device corresponding to the minimum interval distance to adjust parameters.

[0048] The embodiment of the present invention inputs home customer control samples into a large model for model training, so that the converged large model can better understand the semantics and intentions in the user's natural language instructions, and has stronger adaptability to ambiguous, complex instructions and different expressions, thereby significantly improving the accuracy of understanding complex customer control instructions and improving the accuracy of home customer control. By compressing the converged large model, the reasoning speed of the large model is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of a home customer control method based on a large model provided by a first embodiment of the present invention;

[0050] Figure 2 is a structural schematic diagram of a home guest control system based on a large model provided by a second embodiment of the present invention;

[0051] Figure 3 It is a schematic diagram of the structure of a terminal device provided in the third embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0053] In order to illustrate the technical solution of the present invention, a specific embodiment is provided below for illustration.

[0054] Embodiment 1

[0055] See also Figure 1 , is a flow chart of a home customer control method based on a large model provided in a first embodiment of the present invention. The home customer control method based on a large model can be applied to any device or system. The home customer control method based on a large model includes the following steps:

[0056] Step S10, obtaining home customer control samples, and inputting the home customer control samples into the large model for model training until the large model converges;

[0057] Among them, a large amount of natural language command data related to home (hotel) customer control is collected, including multiple expressions in different scenarios (such as lighting control, temperature adjustment, electrical equipment operation, etc.), and corresponding control action data. For example, collect instructions such as "brighten the living room lights", "I want to increase the bedroom temperature by 2 degrees", "turn on the TV in the room" and corresponding equipment control parameter change data. The collected natural language command data related to home (hotel) customer control is labeled to obtain home customer control samples. In this step, a deep learning framework, such as TensorFlow or PyTorch, is used to train the end-side intent understanding large model based on the home customer control samples. The model architecture can adopt a Transformer-based neural network structure.

[0058] Optionally, the home customer control samples are input into a large model for model training until the large model converges, including:

[0059] Segmenting the home customer control sample according to the large model to obtain a segmentation sample, and encoding the segmentation template to obtain word embedding information; wherein, by segmenting the home customer control sample, the word embedding processing operation is effectively facilitated, and by performing word embedding processing on the segmentation template, the encoding effect of the segmentation template is achieved;

[0060] The word embedding information is vectorized to obtain an embedding vector, and the embedding vector is positionally encoded to obtain a position vector; since the Transformer neural network has no concept of time steps, position information needs to be added to help the large model understand the sequence order. Position encoding can use mathematical formulas (such as sine and cosine functions) to add position information to the embedding vector;

[0061] The position vector is processed by a multi-head self-attention mechanism to obtain a self-attention feature, and the self-attention feature is feature encoded to obtain a home customer control feature; wherein the multi-head self-attention mechanism is adopted in the large model, so that the large model can pay attention to other words in the input sequence when processing each word, thereby capturing the association and dependency relationship between words;

[0062] Decoding the home customer control features to obtain customer control prediction parameters and prediction target devices, and determining model loss according to the customer control prediction parameters and the prediction target devices;

[0063] The parameters of the large model are updated according to the model loss until the large model converges; wherein, during the large model training process, the mapping relationship between the command and the control intention is learned by processing the word vector representation, semantic encoding, attention mechanism, etc. of the natural language command. During the training process, the model parameters are continuously adjusted using optimization algorithms such as the cross entropy loss function to improve the accuracy of the large model's understanding of the command intention.

[0064] Further, determining the model loss according to the customer-controlled prediction parameter and the prediction target device includes:

[0065] Obtaining standard customer control parameters and standard equipment corresponding to the home customer control sample, and calculating parameter similarity between the standard customer control parameters and the customer control prediction parameters;

[0066] Determine a first loss according to the parameter similarity, and calculate the pronunciation similarity between the standard device and the predicted target device. In this step, obtain the phoneme strings of the standard device and the predicted target device, calculate the similarity of the phoneme strings between the standard device and the predicted target device, and obtain the pronunciation similarity;

[0067] The second loss is determined according to the pronunciation similarity, and the first loss and the second loss are weighted to obtain the model loss; wherein the parameter similarity, the pronunciation similarity and the loss query table are matched to obtain the first loss and the second loss, and the loss query table stores the correspondence between different parameter similarities and the corresponding first loss, and the correspondence between different pronunciation similarities and the corresponding second loss.

[0068] Step S20, compressing the converged large model to obtain the pre-trained large model, and obtaining the user's home control instructions;

[0069] In order to adapt to the computing resources and storage limitations of the end-side devices, the large model after convergence is compressed and optimized. Quantization technology is used to convert floating-point parameters in the model into low-precision data types, such as int8 type, to reduce the model storage occupancy. At the same time, the pruning algorithm is used to remove connections or neurons in the model that contribute less to intent understanding, further simplifying the model structure and improving the reasoning speed of the large model.

[0070] In this step, the compressed and optimized pre-trained large model is deployed to home (hotel) guest control devices, such as smart speakers, smart control panels, etc. The end-side device has a certain computing capability, such as an ARM architecture processor, which can run model inference tasks. The end-side device collects the user's voice commands through the built-in microphone, or receives the user's home guest control commands input by the user through the touch panel. The collected user home guest control commands are recognized and converted into text form.

[0071] Optionally, compressing the converged large model to obtain the pre-trained large model includes:

[0072] Obtaining the parameter weights and weight distribution of the large model after convergence, and determining the quantization parameters according to the parameter weights and the weight distribution; wherein the parameter weights and the weight distribution are combined to obtain a weight combination, and the weight combination is matched with a quantization query table to obtain the quantization parameters, and the quantization query table stores the corresponding relationship between different weight combinations and corresponding quantization parameters;

[0073] The floating-point parameters in the converged large model are quantized and converted according to the quantization parameters to obtain the pre-trained large model; wherein, by quantizing the floating-point parameters, the floating-point parameters are effectively converted into low-precision data types, which effectively simplifies the large model.

[0074] Further, after the floating point number parameters in the converged large model are quantized according to the quantization parameter, the method further includes:

[0075] Obtain neuron evaluation parameters, and obtain the parameter values ​​to be evaluated of the neurons in the large model after quantization conversion according to the neuron evaluation parameters; wherein the neuron evaluation parameters can be set according to needs, and the parameter values ​​corresponding to the neurons in the large model after quantization conversion on the neuron evaluation parameters are respectively obtained to obtain the parameter values ​​to be evaluated;

[0076] The neurons are sorted according to the parameter values ​​to be evaluated to obtain an evaluation sort list, and neurons to be pruned among the neurons are determined according to the evaluation sort list; wherein neurons with lower parameter values ​​to be evaluated (lower ranking) in the evaluation sort list are determined as neurons to be pruned;

[0077] The neurons to be pruned are deleted to obtain the pre-trained large model; wherein, by deleting the neurons to be pruned, the connections or neurons that contribute less to the understanding of intent in the large model are effectively removed.

[0078] Step S30, inputting the user's home customer control command into the pre-trained large model for customer control analysis, obtaining target home equipment and equipment customer control parameters, and generating a customer control adjustment command according to the equipment customer control parameters;

[0079] The converted user home customer control command is input into the pre-trained large model, and the large model performs customer control analysis to parse out the user's control intention and target home equipment. For example, it determines whether the user wants to control the brightness of the light, adjust the temperature of the air conditioner, or operate other electrical equipment, and determines the specific equipment customer control parameters.

[0080] Step S40, controlling the target home appliance to adjust parameters according to the customer control adjustment instruction;

[0081] Among them, the terminal side device sends the customer control adjustment instructions to the target home device through the communication protocol with the smart device inside the home (hotel), such as ZigBee, Wi-Fi or Bluetooth, to realize the control operation of the home (hotel) equipment.

[0082] Optionally, controlling the target home appliance to adjust parameters according to the customer control adjustment instruction includes:

[0083] Acquire the home environment of the user corresponding to the user home guest control command;

[0084] If there is one target home appliance in the home environment, the customer control adjustment instruction is sent to the target home appliance to control the target home appliance to adjust parameters;

[0085] If the number of the target home devices in the home environment is greater than 1, obtaining the location information of the user and the target home devices to obtain the user coordinates and the device coordinates;

[0086] The distance between the device coordinates and the user coordinates is calculated to obtain the interval distance, and the customer control adjustment instruction is sent to the target home device corresponding to the minimum interval distance to control the target home device corresponding to the minimum interval distance to adjust parameters; wherein, when there are multiple target home devices in the home environment, the device that needs to be home customer controlled is determined based on the interval distance between each target home device and the user, thereby improving the accuracy of home customer control.

[0087] Furthermore, if the number of the target home devices in the home environment is greater than 1, the method further includes:

[0088] Capturing an image of the user to obtain a facial image of the user, and acquiring an eye image from the facial image of the user;

[0089] Performing gaze point recognition on the eye image to obtain a user gaze point, and determining a user gaze area according to the user gaze point;

[0090] If there is one of the target home appliances in the user's gaze area, the customer control adjustment instruction is sent to the target home appliance to control the target home appliance to adjust parameters;

[0091] If the number of the target home devices in the user's gaze area is greater than 1, calculating the gaze distance between the device coordinates and the user's gaze point;

[0092] The customer control adjustment instruction is sent to the target home device corresponding to the minimum gaze distance to control the target home device corresponding to the minimum gaze distance to adjust the parameters; wherein, when there are multiple target home devices in the user's gaze area, the device that needs to be controlled by the home customer is determined based on the gaze distance between each target home device and the user's gaze point, thereby improving the accuracy of home customer control

[0093] If the target home appliance does not exist in the user's gaze area, the customer control adjustment instruction is sent to the target home appliance corresponding to the minimum interval distance to control the target home appliance corresponding to the minimum interval distance to adjust parameters.

[0094] During operation, the end-side device records the user's command data and the corresponding control results and device status change information. The local data forms a small-scale data set for local update and optimization of the large model. Periodically or when certain conditions are met (such as the amount of local data reaches a certain threshold), the end-side device uses the locally recorded data to fine-tune the large model. Through the self-learning mechanism, the large model can gradually adapt to the command habits of specific users and changes in the home (hotel) environment, further improving the accuracy and personalization of intent understanding. On the premise of ensuring data privacy and security, the end-side device can synchronize the locally updated model parameters or a small amount of valuable local data to the cloud server. The cloud server can integrate the data of multiple end-side devices for more comprehensive model training and optimization, and push the optimized model parameters or general model updates to other end-side devices to achieve the collaborative evolution of the global model.

[0095] In this embodiment, by inputting home customer control samples into the big model for model training, the converged big model can better understand the semantics and intentions in the user's natural language commands, and has stronger adaptability to ambiguous, complex commands and different expressions, thereby significantly improving the accuracy of understanding complex customer control commands and improving the accuracy of home customer control. By compressing the converged big model, the reasoning speed of the big model is effectively improved.

[0096] In this embodiment, through large-scale data training and deep learning algorithms, it is possible to better understand the semantics and intentions in the user's natural language commands, and has stronger adaptability to ambiguous, complex commands and different expressions, thereby significantly improving the accuracy of understanding customer control commands and enhancing user experience.

[0097] The intent understanding model is deployed on the terminal device, and most user command data and control operations are processed locally on the terminal, avoiding the transmission of large amounts of sensitive data to the cloud, effectively reducing the risk of data privacy leakage, and meeting users' increasing demand for data security.

[0098] End-side processing reduces delays caused by network transmission and cloud queuing, enables fast command parsing and device control response, and enables home (hotel) devices to respond to user operations instantly. This is especially advantageous in scenarios that require high response timeliness (such as fast switching of lights, emergency electrical appliance operations, etc.).

[0099] The local data recording and self-learning mechanism of the terminal device enables the model to be personalized and optimized according to the usage habits of specific users and changes in the home (hotel) environment, providing customized smart customer control services for different users, further enhancing user satisfaction and the practicality of the smart customer control system.

[0100] Embodiment 2

[0101] See also Figure 2 , is a schematic diagram of the structure of a home guest control system 100 based on a large model provided in a second embodiment of the present invention, including:

[0102] The model training module 10 is used to obtain home customer control samples and input the home customer control samples into the large model for model training until the large model converges.

[0103] Optionally, the model training module 10 is further used to: obtain the parameter weights and weight distribution of the large model after convergence, and determine the quantization parameters according to the parameter weights and the weight distribution;

[0104] The floating-point number parameters in the converged large model are quantized and converted according to the quantization parameters to obtain the pre-trained large model.

[0105] Furthermore, the model training module 10 is also used to: obtain neuron evaluation parameters, and obtain the parameter values ​​to be evaluated of the neurons in the large model after quantization conversion according to the neuron evaluation parameters;

[0106] Sorting the neurons according to the parameter values ​​to be evaluated to obtain an evaluation sorting list, and determining neurons to be pruned among the neurons according to the evaluation sorting list;

[0107] The neurons to be pruned are deleted to obtain the pre-trained large model.

[0108] Furthermore, the model training module 10 is also used to: segment the home customer control sample according to the large model to obtain a segmentation sample, and encode the segmentation template to obtain word embedding information;

[0109] Performing vector conversion on the word embedding information to obtain an embedding vector, and performing position encoding on the embedding vector to obtain a position vector;

[0110] Processing the position vector using a multi-head self-attention mechanism to obtain a self-attention feature, and encoding the self-attention feature to obtain a home guest control feature;

[0111] Decoding the home customer control features to obtain customer control prediction parameters and prediction target devices, and determining model loss according to the customer control prediction parameters and the prediction target devices;

[0112] Parameters of the large model are updated according to the model loss until the large model converges.

[0113] Preferably, the model training module 10 is further used to: obtain standard customer control parameters and standard equipment corresponding to the home customer control sample, and calculate the parameter similarity between the standard customer control parameters and the customer control prediction parameters;

[0114] Determining a first loss according to the parameter similarity, and calculating the pronunciation similarity between the standard device and the predicted target device;

[0115] A second loss is determined according to the pronunciation similarity, and a weighted operation is performed on the first loss and the second loss to obtain the model loss.

[0116] The model compression module 11 is used to compress the converged large model to obtain the pre-trained large model and acquire the user's home control instructions.

[0117] The customer control analysis module 12 is used to input the user's home customer control command into the pre-trained large model for customer control analysis, obtain target home equipment and equipment customer control parameters, and generate customer control adjustment instructions according to the equipment customer control parameters.

[0118] The parameter adjustment module 13 is used to control the target home appliance to adjust parameters according to the customer control adjustment instruction.

[0119] Optionally, the parameter adjustment module 13 is further used to: obtain the home environment of the user corresponding to the user home guest control instruction;

[0120] If there is one target home appliance in the home environment, the customer control adjustment instruction is sent to the target home appliance to control the target home appliance to adjust parameters;

[0121] If the number of the target home devices in the home environment is greater than 1, obtaining the location information of the user and the target home devices to obtain the user coordinates and the device coordinates;

[0122] The distance between the device coordinates and the user coordinates is calculated to obtain the interval distance, and the customer control adjustment instruction is sent to the target home device corresponding to the minimum interval distance to control the target home device corresponding to the minimum interval distance to adjust parameters.

[0123] Furthermore, the parameter adjustment module 13 is also used to: collect images of the user to obtain a facial image of the user, and obtain an eye image in the facial image of the user;

[0124] Performing gaze point recognition on the eye image to obtain a user gaze point, and determining a user gaze area according to the user gaze point;

[0125] If there is one of the target home appliances in the user's gaze area, the customer control adjustment instruction is sent to the target home appliance to control the target home appliance to adjust parameters;

[0126] If the number of the target home devices in the user's gaze area is greater than 1, calculating the gaze distance between the device coordinates and the user's gaze point;

[0127] Sending the customer control adjustment instruction to the target home device corresponding to the minimum gaze distance to control the target home device corresponding to the minimum gaze distance to adjust parameters;

[0128] If the target home appliance does not exist in the user's gaze area, the customer control adjustment instruction is sent to the target home appliance corresponding to the minimum interval distance to control the target home appliance corresponding to the minimum interval distance to adjust parameters.

[0129] In this embodiment, by inputting home customer control samples into the big model for model training, the converged big model can better understand the semantics and intentions in the user's natural language commands, and has stronger adaptability to ambiguous, complex commands and different expressions, thereby significantly improving the accuracy of understanding complex customer control commands and improving the accuracy of home customer control. By compressing the converged big model, the reasoning speed of the big model is effectively improved.

[0130] Embodiment 3

[0131] Figure 3 2 is a block diagram of a terminal device 2 provided in the third embodiment of the present application. Figure 3 As shown, the terminal device 2 of this embodiment includes: a processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the processor 20, such as a program of a home customer control method based on a large model. When the processor 20 executes the computer program 22, the steps in each embodiment of the home customer control method based on a large model are implemented.

[0132] Exemplarily, the computer program 22 may be divided into one or more modules, which are stored in the memory 21 and executed by the processor 20 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 22 in the terminal device 2. The terminal device may include, but is not limited to, a processor 20 and a memory 21.

[0133] The processor 20 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0134] The memory 21 may be an internal storage unit of the terminal device 2, such as a hard disk or memory of the terminal device 2. The memory 21 may also be an external storage device of the terminal device 2, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 2. Further, the memory 21 may also include both an internal storage unit and an external storage device of the terminal device 2. The memory 21 is used to store the computer program and other programs and data required by the terminal device. The memory 21 may also be used to temporarily store data that has been output or is to be output.

[0135] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional unit.

[0136] If the integrated module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Among them, the computer-readable storage medium can be non-volatile or volatile. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in computer-readable storage media can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable storage media do not include electrical carrier signals and telecommunication signals.

[0137] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A home customer control method based on a large model, characterized in that: The method comprises: Acquire home customer control samples, and input the home customer control samples into the large model for model training until the large model converges; Performing model compression on the converged large model to obtain the pre-trained large model, and acquiring the user's home control instructions; Inputting the user's home customer control command into the pre-trained large model for customer control analysis, obtaining target home equipment and equipment customer control parameters, and generating customer control adjustment instructions according to the equipment customer control parameters; The target home appliance is controlled to adjust parameters according to the customer control adjustment instruction.

2. The home customer control method based on a large model as claimed in claim 1, characterized in that: Controlling the target home appliance to adjust parameters according to the customer control adjustment instruction includes: Acquire the home environment of the user corresponding to the user home guest control command; If there is one target home appliance in the home environment, the customer control adjustment instruction is sent to the target home appliance to control the target home appliance to adjust parameters; If the number of the target home devices in the home environment is greater than 1, obtaining the location information of the user and the target home devices to obtain the user coordinates and the device coordinates; The distance between the device coordinates and the user coordinates is calculated to obtain the interval distance, and the customer control adjustment instruction is sent to the target home device corresponding to the minimum interval distance to control the target home device corresponding to the minimum interval distance to adjust parameters.

3. The home customer control method based on a large model as claimed in claim 2, characterized in that: If the number of the target home devices in the home environment is greater than 1, the method further includes: Capturing an image of the user to obtain a facial image of the user, and acquiring an eye image in the facial image of the user; Performing gaze point recognition on the eye image to obtain a user gaze point, and determining a user gaze area according to the user gaze point; If there is one of the target home appliances in the user's gaze area, the customer control adjustment instruction is sent to the target home appliance to control the target home appliance to adjust parameters; If the number of the target home devices in the user's gaze area is greater than 1, calculating the gaze distance between the device coordinates and the user's gaze point; Sending the customer control adjustment instruction to the target home device corresponding to the minimum gaze distance to control the target home device corresponding to the minimum gaze distance to adjust parameters; If the target home appliance does not exist in the user's gaze area, the customer control adjustment instruction is sent to the target home appliance corresponding to the minimum interval distance to control the target home appliance corresponding to the minimum interval distance to adjust parameters.

4. The home customer control method based on a large model as claimed in claim 1, characterized in that: The converged large model is compressed to obtain the pre-trained large model, including: Obtaining parameter weights and weight distribution of the large model after convergence, and determining quantization parameters according to the parameter weights and the weight distribution; The floating-point number parameters in the converged large model are quantized and converted according to the quantization parameters to obtain the pre-trained large model.

5. The home customer control method based on a large model as claimed in claim 4, characterized in that: After the floating point number parameters in the converged large model are quantized according to the quantization parameters, the method further includes: Obtaining neuron evaluation parameters, and obtaining the parameter values ​​to be evaluated of the neurons in the large model after quantization conversion according to the neuron evaluation parameters; Sorting the neurons according to the parameter values ​​to be evaluated to obtain an evaluation sorting list, and determining neurons to be pruned among the neurons according to the evaluation sorting list; The neurons to be pruned are deleted to obtain the pre-trained large model.

6. The method for home customer control based on a large model as claimed in claim 1, characterized in that: Inputting the home customer control samples into the large model for model training until the large model converges, including: Segmenting the home customer control sample according to the large model to obtain segmentation samples, and encoding the segmentation template to obtain word embedding information; Performing vector conversion on the word embedding information to obtain an embedding vector, and performing position encoding on the embedding vector to obtain a position vector; Processing the position vector using a multi-head self-attention mechanism to obtain a self-attention feature, and encoding the self-attention feature to obtain a home guest control feature; Decoding the home customer control features to obtain customer control prediction parameters and prediction target devices, and determining model loss according to the customer control prediction parameters and the prediction target devices; Parameters of the large model are updated according to the model loss until the large model converges.

7. The method for home customer control based on a large model as claimed in claim 6, characterized in that: Determining a model loss according to the customer-controlled prediction parameter and the prediction target device includes: Obtaining standard customer control parameters and standard equipment corresponding to the home customer control sample, and calculating parameter similarity between the standard customer control parameters and the customer control prediction parameters; Determining a first loss according to the parameter similarity, and calculating the pronunciation similarity between the standard device and the predicted target device; A second loss is determined according to the pronunciation similarity, and a weighted operation is performed on the first loss and the second loss to obtain the model loss.

8. A home guest control system based on a large model, characterized in that: The system comprises: A model training module, used for obtaining home customer control samples and inputting the home customer control samples into a large model for model training until the large model converges; A model compression module, used to compress the converged large model to obtain the pre-trained large model and obtain the user's home control instructions; A customer control analysis module, used to input the user's home customer control command into the pre-trained large model for customer control analysis, obtain target home equipment and equipment customer control parameters, and generate a customer control adjustment command according to the equipment customer control parameters; The parameter adjustment module is used to control the target home appliance to adjust the parameters according to the customer control adjustment instruction.

9. The home guest control system based on a large model as claimed in claim 8, characterized in that: The parameter adjustment module is also used for: Acquire the home environment of the user corresponding to the user home guest control command; If there is one target home appliance in the home environment, the customer control adjustment instruction is sent to the target home appliance to control the target home appliance to adjust parameters; If the number of the target home devices in the home environment is greater than 1, obtaining the location information of the user and the target home devices to obtain the user coordinates and the device coordinates; The distance between the device coordinates and the user coordinates is calculated to obtain the interval distance, and the customer control adjustment instruction is sent to the target home device corresponding to the minimum interval distance to control the target home device corresponding to the minimum interval distance to adjust parameters.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images