Multi-home equipment interaction method for intelligently assisting image-text generation and storage medium

Abstract

The invention discloses a multi-home equipment interaction method for intelligently assisting image-text generation and a storage medium. According to the method, the cellular automaton model is utilized to predict interaction information possibly output by a user in the next time step. The cellular automaton model can predict possible interaction of the user based on predefined rules and conditions according to the current state and historical interaction of the user; through the prediction of the cellular automaton model, the system can better understand the historical interaction and habits of the user, thereby generating the interaction content which better accords with the preference and personalized demand of the user. The personalized interaction can enhance the emotional connection between the user and the home equipment and improve the user satisfaction. By providing more individuation and continuity, the user is more likely to participate in the control and management of the home equipment, so that the positive participation degree and the use intention of the user on the equipment are improved. The cellular automaton model can generate certain creative content according to rules and conditions, which can bring some surprises and interests to users.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to a multi-home device interaction method and storage medium for intelligent auxiliary image and text generation. Background Art

[0002] The interactive communication between household appliances is called home device interaction. In recent years, thanks to the rapid development of the Internet of Things technology, home device interaction has entered a more intelligent and convenient era. The Internet of Things technology allows various home appliances (such as smart lights, smart speakers, smart door locks, etc.) to be connected to each other through the Internet, thereby realizing remote monitoring, control and data transmission, and thus enabling deeper intelligent interaction between these appliances.

[0003] Voice recognition technology allows people to communicate with home devices by issuing voice commands. We call this technology voice assistants (such as Siri, Alexa, Google Assistant, etc.). These voice assistants can receive user commands and then perform related actions. To achieve this goal, they need to use natural language processing technology, which is a technology specifically used to analyze and understand user input text to help electrical appliances generate natural language responses or commands, thereby achieving a more intelligent working state. In addition, if we install various sensors (such as temperature sensors, light sensors, etc.) on home devices, these sensors can detect changes in the environment and interact accordingly based on changes in the environment.

[0004] Traditional intelligent assisted image and text generation technology is based on predefined rules and conditions. The generator can generate corresponding image and text information based on the user's input and context. Use pre-designed templates to fill the user's input information into the appropriate position to generate image and text content. Generate appropriate text responses based on the input keywords and patterns, which is often used in scenarios such as automatic SMS replies. Identify keywords in the input text and then generate relevant image and text content based on these keywords.

[0005] However, traditional intelligent assisted image and text generation technology has some defects:

[0006] (1) Lack of creativity and personalization: Traditional technologies are usually based on predefined rules, templates or patterns, which makes it difficult to generate creative and personalized graphic content, limiting the user experience.

[0007] (2) Limited flexibility: Traditional technologies have limitations in adapting to diverse user needs and situations and cannot flexibly generate complex and varied graphic information.

[0008] (3) Insufficient context understanding: Traditional technologies often have difficulty in deeply understanding the context of user input, resulting in the generated graphic and text content lacking accuracy and coherence.

[0009] To this end, a multi-home device interaction method and storage medium for intelligent auxiliary image and text generation are proposed. Summary of the invention

[0010] In view of this, the embodiments of the present invention hope to provide a multi-home device interaction method and storage medium for intelligent auxiliary image and text generation to solve or alleviate the technical problems existing in the prior art, namely, insufficient creativity and personalization, limited flexibility and insufficient context understanding, and at least provide a beneficial option for this;

[0011] The technical solution of the embodiment of the present invention is achieved as follows:

[0012] First aspect

[0013] A method for multi-home device interaction with intelligent auxiliary image and text generation: aims to achieve a more intelligent, personalized and coherent home device interaction experience, which includes:

[0014] S1. Prediction: Interactive prediction using cellular automata model:

[0015] At this stage, the present invention uses the cellular automaton model to predict the interactive information that the user may output in the next time step. The cellular automaton model can predict the user's possible interaction based on the current state and the user's historical interaction, based on predefined rules and conditions. The specific steps are as follows:

[0016] Cell state and input: Each cell represents the state of a device, and historical user interactions are encoded as input.

[0017] Rules and Cell Transition Functions: Define a set of rules that represent the transition rules of the device state. Use cell transition functions to calculate the next state.

[0018] S2. Update: Generate interactive text using LSTM:

[0019] At this stage, the present invention uses LSTM to generate text information related to the predicted interaction to capture contextual information and then output the corresponding text. The specific steps are as follows:

[0020] Input and hidden state: Use the interactions predicted by the cellular automaton as input and combine them with the previously generated text to form the output. Use hidden states to capture contextual information.

[0021] Output: Use the output layer to calculate the output through weights and biases, as well as the Softmax function, as the generated interactive text;

[0022] S3. Integration: Fusion of Cellular Automata Prediction and LSTM Generation:

[0023] At this stage, the present invention fuses the prediction results of the cellular automaton model with the text generated by LSTM to generate the final interrogative interactive text. The specific steps are as follows:

[0024] Fusion weight calculation: Calculate the fusion weight based on the predicted confidence and the generated confidence to balance the influence of the two;

[0025] Fusion result generation: Generate fusion result according to fusion weights in a linear weighted manner as the final inquiry-style interactive text;

[0026] Through this method, the present invention can combine the prediction of cellular automata and the generation of LSTM to generate more coherent and personalized interactive texts for home appliances, while fully considering contextual information and user needs, thus improving the user experience. Compared with traditional technologies, this method can better cope with complex situations and natural language expressions, and provide higher quality image and text generation.

[0027] Second aspect

[0028] A storage medium stores program instructions for executing the multi-home device interaction method for intelligent auxiliary image and text generation as described above.

[0029] This storage medium can be a computer hard disk, a solid state drive, a memory, a cloud storage, etc. The storage medium stores program instructions for a multi-home device interaction method for intelligent auxiliary image and text generation, which describe the implementation details of the entire method, including the various stages of prediction, update, and integration.

[0030] Specifically, the program instructions in the storage medium may include the following:

[0031] (1) Cellular automaton model implementation instructions: This part of the instructions covers the construction of the cellular automaton model, the definition of the state transition function, the setting of rules, etc. It describes how to predict the possible user interaction information in the next time step based on the current state and the user's historical interaction.

[0032] (2) LSTM model implementation instructions: This part of the instructions describes the architecture of the LSTM model, the processing of input and hidden states, the definition of the output layer, etc. It explains how to use LSTM to generate text information related to the predicted interaction.

[0033] (3) Fusion and integration instructions: These instructions cover how to calculate fusion weights, perform linear weighted fusion, and generate the final interactive text. It describes how to fuse the prediction results of the cellular automaton model with the output of the LSTM model to generate optimized graphic content.

[0034] (4) Data processing and preprocessing instructions: In practical applications, data processing and preprocessing are very important steps. These instructions describe how to prepare training data, encode input, perform feature engineering, etc.

[0035] (5) User interface and interaction instructions: If user interface and interaction are involved, the instructions in the storage medium may also include relevant content such as how to receive user input and display generated interactive text.

[0036] (6) These program instructions in the storage medium constitute the implementation basis of the entire multi-home device interaction method for intelligent assisted image and text generation. By executing these instructions, the system can predict user interactions and generate interactive texts in actual applications, and integrate the outputs of the cellular automaton model and the LSTM model to provide a more intelligent, personalized and smooth home device interaction experience.

[0037] In summary, compared with the prior art, the beneficial effects of the multi-home device interaction method and storage medium for intelligent auxiliary image and text generation provided by the present invention are as follows:

[0038] 1. Personalized interaction: Through the prediction of the cellular automaton model, the system can better understand the user's historical interactions and habits, thereby generating interactive content that is more in line with user preferences and personalized needs. This personalized interaction can enhance the user's emotional connection with home devices and improve user satisfaction. By providing a more personalized and coherent interactive experience, users are more likely to participate in the control and management of home devices, thereby increasing the user's active participation and willingness to use the devices. The cellular automaton model can generate certain creative content based on rules and conditions, which can bring some surprises and fun to users. Compared with traditional methods, this method is more likely to produce novel and creative interactive content.

[0039] 2. Coherence and Fluency: Through the application of the LSTM model, the system can capture contextual information and generate more coherent interactive text. This helps ensure that the interactive content is more semantically coherent and fluent, improving the communication effect between users and devices. The LSTM model has advantages in generating text and can better simulate the characteristics of human natural language expression. This means that the generated interactive text is more natural and easier for users to understand and accept, thereby reducing communication barriers.

[0040] 3. Context awareness: The fusion process in the integration phase fully considers the prediction of the cellular automaton model and the generation of the LSTM model, so that the system can better understand and integrate the current context when generating interactive text, improving the pertinence and practicality of the interactive content. This method can flexibly adapt to different home devices and user needs, and achieve a wider range of applications by adjusting rules and model parameters. This enables the system to provide consistent and effective interactive support in different home environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 It is a logical schematic diagram of the present invention;

[0043] Figure 2 This is a schematic diagram of a procedure of Embodiment 5 of the present invention;

[0044] Figure 3 This is a schematic diagram of a procedure of Embodiment 5 of the present invention;

[0045] Figure 4 This is a schematic diagram of a procedure of Embodiment 5 of the present invention; DETAILED DESCRIPTION

[0046] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below;

[0047] See also Figure 1 , this specific implementation will provide relevant technical solutions:

[0048] A multi-home device interaction method for intelligent auxiliary image and text generation includes the following steps:

[0049] S1. Prediction: Interactive prediction using cellular automata model:

[0050] At this stage, this specific implementation uses a cellular automaton model to predict the interactive information that the user may output in the next time step. This is calculated by applying cellular automaton rules based on the current state and the user's historical interactive information. Specifically:

[0051] S1.1, Cell state and input:

[0052] Each device corresponds to a cell, which is used to represent the current state of the device.

[0053] User historical interactions are encoded into input vectors to influence predictions.

[0054] S1.2, rules and cell transformation functions:

[0055] This specific implementation defines a set of rules R for describing the transition behavior of device states.

[0056] The cell transition function f calculates the expected state at the next time step based on the rules and the current state and input.

[0057] Logical principle example: Assume that this embodiment has a smart lamp, which can be on or off. This embodiment wants to predict the possible operation of the user in the next time step (for example, turning on the light). This embodiment can define a rule that if the current state is off and there is an instruction to turn on the light in the user history, the next state is predicted to be on.

[0058] S2. Update: Using LSTM to generate interactive text

[0059] At this stage, this embodiment uses an LSTM model to generate text information related to the predicted interaction to capture contextual information and then outputs the corresponding text:

[0060] S2.1, input and hidden state:

[0061] Use the interactions predicted by the cellular automaton model as input, combined with previously generated text.

[0062] The input is converted into a continuous vector through word embedding and then input into the LSTM model.

[0063] S2.2, LSTM model:

[0064] The LSTM model updates the hidden state to capture the semantic information of the previous text.

[0065] The hidden state is updated at each time step, reflecting the evolution of contextual information.

[0066] S2.3, Output:

[0067] Using the output layer, the hidden states of the LSTM are mapped to a probability distribution over the vocabulary.

[0068] The Softmax function is used to convert the output into probabilities.

[0069] Logical principle example: Assume that this specific implementation wants to generate a sentence to answer the user's question about the status of the lamp. In the input stage, this specific implementation combines the word embedding of the user's question with the word embedding of the previously generated content and inputs it into the LSTM. The LSTM will generate the corresponding answer text based on the context information and the semantics of the user's question.

[0070] S3. Integration: Fusion of Cellular Automata Prediction and LSTM Generation

[0071] At this stage, this specific implementation combines the prediction results of the cellular automaton model with the output of the LSTM model to generate the final inquiry-style interactive text:

[0072] S3.1. Fusion weight calculation:

[0073] Using the predicted confidence and the generated confidence, the fusion weight Wt is calculated.

[0074] This weight represents the relative weight of using the cellular automaton predictions and the LSTM generation in the ensemble.

[0075] S3.2, fusion result generation:

[0076] Through linear weighting, the cellular automaton prediction results and the text generated by LSTM are integrated into the final inquiry-style interactive text.

[0077] Logical principle example: Assume that the cellular automaton of this embodiment predicts that the user may want to turn on the light. The text generated by LSTM is an encouraging sentence, reminding the user that they can adjust the brightness of the light as needed. By fusing weights, this embodiment can determine the final generated interactive text based on the importance of the cellular automaton's prediction and the encouraging content generated by LSTM to provide a more inspiring response.

[0078] In summary, the entire process includes cellular automaton prediction, LSTM generation and fusion to produce more personalized, coherent and creative graphic content in the interaction of multiple home devices.

[0079] Further, the generated interactive text:

[0080] After obtaining the interactive text data message, the method of generating it into graphics and text can be:

[0081] (1) Use text-to-image libraries: Use some existing text-to-image libraries. These libraries provide simple API interfaces. By inputting the text into the API, you can output the image and convert the input text into a specified image format (PNG or JPEG). Preferred text-to-image libraries include ImageMagick, Pillow (Python), and GraphicsMagick.

[0082] (2) Command line tools: You can also use command line tools to convert text to images. For example, on Linux or Mac operating systems, use the command line tools pandoc and ImageMagick to directly convert text files to interactive images.

[0083] The technical features of the above-described specific embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described specific embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0084] Embodiment 1

[0085] According to the above specific implementation methods and embodiments, this embodiment further provides the following technical solutions:

[0086] Assume that the cellular automaton model predicts the next possible interaction based on the current state and the user's historical interaction. The cellular automaton model:

[0087] State Cell: Ct represents the state at time step t.

[0088] Input: Xt represents the input at time step t, including user history and current context; Ut represents the user interaction information at time step t;

[0089] Rules: R represents a rule set of a cellular automaton, which represents the transition rules of the device state; for example, a rule may specify that if the user mentions "turn on the light", the state changes to "on".

[0090] Next Cell: represents the state of the cell at time step t+1, which is calculated based on the current state, input and rules. The core logic of the cellular automaton model is to predict the next state given the current state, input and rules. This can be expressed by the following formula:

[0091] C t+1 =f(CT,Xt,R)

[0092] f is a function of the state transition rule, which defines how to calculate the next state based on the current state, input and rules. Specifically, f can be a logic function, a mapping function, a rule set, etc. The specific form depends on the designed cellular automation model.

[0093] CellState: Each cell represents the state of a device, which can be the brightness, color, etc. of a light. Si_t represents the state of cell i at time step t.

[0094] Cell transition function: Maps the current state, input, and rule to the next state:

[0095] Si t+1 =Transition(Si t , t, Ut, R)

[0096] Wherein, Transition is a defined cellular transition function, which calculates the next state based on the current state, user interaction information and rules, and the next state is equal to the desired state. The Transition function can be implemented based on a rule base, a machine learning model or other methods. For example, a rule base can be used to match user interactions and perform corresponding state transitions. Alternatively, a machine learning model can be used to predict state transitions, such as using a classifier to predict the next state based on user interactions.

[0097] Preferably, the logic function f: It determines the next state based on the current state and input. Assume that this embodiment has a smart door lock device, the state Ct represents the lock state of the door (locked is 1, unlocked is 0), the user history and current context are encoded as input Xt, and the user's interactive information at time step t is Ut. This embodiment can define the logic function f as follows:

[0098] f(Ct,Xt,R)=

[0099] If Ut contains an unlock instruction, the next state is unlock (0);

[0100] If Ut contains a lock instruction, the next state is locked (1);

[0101] Otherwise, the state remains unchanged.

[0102] Preferably, mapping function f: It maps the value of the next state according to the current state and input. Assume that there is a smart curtain device in this embodiment, the state Ct represents the degree of opening and closing of the curtain (0 is closed, 1 is fully open), the user history and current context are encoded as input Xt, and the user's interactive information at time step t is Ut. In this embodiment, the mapping function f can be defined as follows:

[0103] f(Ct,Xt,R)=

[0104] If Ut contains the command to open the curtains, the next state is fully open (1);

[0105] If Ut contains the instruction to close the curtains, the next state is Close (0);

[0106] Otherwise, the state will increase the opening and closing degree, making the curtains slightly open.

[0107] Preferably, the state transition function f is a rule set that calculates the next state based on a set of rules. Assume that in this embodiment there is an intelligent audio device, the state Ct represents the volume level (0-10), the user history and current context are encoded as input Xt, and the user's interaction information at time step t is Ut. In this embodiment, the rule set R can be defined as follows:

[0108] If Ut contains the "increase volume" instruction, the next state is the current state plus 2, but not more than 10; if Ut contains the "decrease volume" instruction, the next state is the current state minus 2, but not less than 0;

[0109] Otherwise, the status remains unchanged.

[0110] Preferably, the Transition function is implemented based on a rule base, and the training of the rule base performs the following steps:

[0111] P1. Rule base definition: First, define a rule base, which contains different rules. Each rule is associated with a state transition, such as state transition according to user instructions.

[0112] P2. User interaction matching: At each time step, the Transition function matches the user interaction information with the rules in the rule library. If the user's interaction information meets the conditions of a rule, then the rule will be triggered.

[0113] P3, State Transition: The triggered rule will specify the state transition method. According to the matched rule, the Transition function will calculate the state change defined in the rule to determine the next state.

[0114] P4. Update state: Finally, the Transition function applies the calculated next state to the cellular automaton model, updates the state of the device, and thus realizes the state transition.

[0115] For example, the Transition function:

[0116] Rule base example:

[0117] If the user mentions "turn on the light", the state becomes "on".

[0118] If the user mentions "turn off the light", the state becomes "off".

[0119]

[0120] The rule base shown in the above example is that the Transition function traverses each rule in the rule base, checks whether the user interaction information meets the conditions of the rule, and if so, executes the corresponding state transition operation to finally obtain the next state. Through this rule base-based implementation method, the Transition function can flexibly perform state transitions according to different user interaction situations, thereby realizing the state prediction and transition functions in the cellular automaton model.

[0121] Embodiment 2

[0122] According to the above specific implementation methods and embodiments, this embodiment further provides the following technical solutions:

[0123] Use an LSTM-based text generation model to generate inquiry-style interactive text, including:

[0124] Input: It represents the input at time step t, including the interactions predicted by the cellular automaton and the previously generated text;

[0125] Hidden state: Ht represents the hidden state at time step t, which is used to capture contextual information.

[0126] Output: Ot represents the output at time step t, as the generated interactive text.

[0127] In S2, input: It represents the input at time step t, including the interaction predicted by the cellular automaton and the previously generated text:

[0128] It=[E(Ut);E(Gt)]

[0129] Among them, E represents the word embedding function, Ut represents the interaction predicted by the cellular automaton, and Gt represents the text generated before time step t; at time step t, the hidden state is used to capture context information. It is obtained by updating the previous hidden state and input (including the interaction predicted by the cellular automaton and the previously generated text) through the LSTM model. At time step t, the output is obtained by linear transformation and Softmax operation on the hidden state, which represents the probability distribution of the generated interactive text.

[0130] Hidden state: Ht represents the hidden state at time step t, which is used to capture contextual information:

[0131] Ht =LSTM(It, Ht-1)

[0132] Among them, LSTM represents the update process of the LSTM model, and Ht-1 represents the hidden state at time step t-1;

[0133] Output: Ot represents the output at time step t, as the generated interactive text:

[0134] Ot=Softmax(WoHt+bo)Ot=Softmax(WoHt+bo)

[0135] Among them, Wo and bo are the weight and bias of the output layer, and Softmax represents the Softmax function, which converts the result of linear transformation into probability distribution.

[0136] Through this process, we can use the LSTM model to generate the interactive text for the next time step based on the previously generated content and the interactions predicted by the cellular automaton. The entire process includes the representation of the input, the update of the hidden state, and the calculation of the output to achieve the generation of LSTM-based interrogative interactive text.

[0137] Specifically, when using the Softmax function: a set of real values ​​is converted into a probability distribution. Given a real vector z = [z_1, z_2, ..., z_n], the Softmax function converts each element zi into a probability value p_i, so that the sum of all probability values ​​is equal to 1. The formula of the Softmax function is as follows:

[0138]

[0139] Where e is a natural constant (approximately equal to 2.71828). The Softmax function exponentially modifies each element in the input vector and then divides the exponential value by the sum of the exponentials of all elements. This ensures that each probability value is between 0 and 1 and that the sum of all probability values ​​is 1, thus forming a probability distribution. Based on the characteristics of the exponential function, its main goal is to normalize the input real values ​​so that they can be interpreted as probabilities:

[0140] P1. Amplify differences: The Softmax function exponentially scales each element of the input vector, which amplifies larger values ​​while compressing smaller values, thereby increasing the differences between them.

[0141] P2, Normalized probability: The exponential sum in the denominator will ensure that the sum of the output probability values ​​is 1. This means that the Softmax function can convert the original real values ​​into relative probability distributions so that they can be used to represent the probabilities of different categories.

[0142] P3, Category selection: The Softmax function makes the category corresponding to the largest probability value in the output the most likely category because it has a stronger response to larger input values.

[0143] P4. The logic of the Softmax function makes it have good properties in classification tasks. It can effectively map the original input into a probability distribution, thereby helping the model to make predictions and decisions. In the LSTM-based text generation model, the Softmax function is used to convert the linear transformation results output by the LSTM model into the probability distribution of the generated interactive text. In this way, we can select the text with the highest probability as the final generated result.

[0144] Embodiment 3

[0145] According to the above specific implementation methods and embodiments, this embodiment further provides the following technical solutions:

[0146] Use word embedding to map the input discrete symbols, interactions and texts into continuous vector representations, then use the LSTM model to update the hidden state, and finally get the generated interactive text through the output layer.

[0147] Specifically, word embedding is used for discrete symbol mapping:

[0148] P1. Input representation: Our model receives interactions predicted by cellular automata and previously generated text as input, which are usually discrete symbols (such as words). In order to process them in the model, we first need to map these discrete symbols to continuous vector representations.

[0149] P2. Word embedding technology: We use word embedding technology, such as Word2Vec, GloVe or BERT, to map discrete symbols into a continuous vector space. In this way, each discrete symbol is represented as a real number vector that semantically preserves the relationship between symbols. This enables the model to better capture the semantic information of the text.

[0150] P3, LSTM model updates the hidden state:

[0151] Input combination: After word embedding, we get a continuous vector representation of the interaction predicted by the cellular automaton and the previously generated text. These vectors will be used as input to the LSTM model.

[0152] LSTM model: LSTM (Long Short-Term Memory) is a recurrent neural network that is particularly suitable for processing sequence data. It has a hidden state that can capture contextual information in the sequence.

[0153] Hidden state update: At each time step, the LSTM model receives the input of the current time step and the hidden state of the previous time step, and then calculates a new hidden state. This hidden state will contain contextual information from the previous time step of the input sequence, which helps the model understand the context of the text.

[0154] P4, the output layer generates interactive text:

[0155] LSTM model output: At each time step, the hidden state of the LSTM model will be used as output to capture the context information of the current time step.

[0156] Linear transformation: We pass the output of the LSTM model through a linear transformation (weighted sum) to the output layer. This linear transformation maps the model output to a representation that is more suitable for generating interactive text.

[0157] Softmax probability distribution: After the linear transformation, we will use the Softmax function to convert the model output into a probability distribution. Each possible interactive text will be associated with a probability value, indicating the possibility that the text is generated.

[0158] Through this process, we can map discrete interactions and text inputs into continuous vector representations, capture contextual information and update states through the LSTM model, and finally obtain the probability distribution of the generated interactive text through the output layer. This allows us to select the interactive text with the highest probability as the final generated result based on the model's prediction.

[0159] Embodiment 4

[0160] According to the above specific implementation methods and embodiments, this embodiment further provides the following technical solutions:

[0161] In S3 of this embodiment: Linear weighted fusion is used: In this stage, we will fuse the prediction results of the cellular automaton model and the output of the LSTM model in a linear weighted manner to generate the final inquiry-style interactive text. Here we involve two key aspects: fusion weight and fusion result.

[0162] Specifically, the goal of the fusion is to obtain more accurate, interesting and appropriate home device interaction text by combining the outputs of two different models. The cellular automaton model predicts that the interaction information may be more based on rules and historical data, while the LSTM model can better capture the context and generate natural language.

[0163] Fusion weight: At time step t, we use the fusion weight Wt to ​​balance the interaction information predicted by the cellular automaton and the text information generated by the LSTM. The fusion weight is calculated by the following formula:

[0164] Wt=α·Confidence(Ct+1)+(1-α)·Confidence(O)t)

[0165] Among them, α is a parameter that controls the weight balance, and Confidence represents the confidence assessment of the prediction or generation. The larger α is, the more inclined to use the interactive information predicted by the cellular automaton, and the smaller α is, the more inclined to use the text information generated by the LSTM.

[0166] Fusion results: Use the fusion weights to fuse the prediction results of the cellular automaton model and the output of the LSTM model together in a linear weighted manner:

[0167] Ft=Wt·Ot+(1-Wt)·Ct+1

[0168] The fusion result Ft is the final query-style interactive text generated at time step t. It combines the information of the two models and selectively fuses the text predicted by the cellular automation and generated by the LSTM according to the weight balance;

[0169] Ot is the interactive text output generated by LSTM, and Ct+1 is the next state predicted by the cellular automaton. The fusion result Ft is the final interrogative interactive text generated at time step t. By adjusting the fusion weights, we can balance the contributions between the cellular automaton model and the LSTM model to a certain extent, thereby generating more comprehensive and adaptive interactive texts. Through this fusion process, we can effectively integrate the information from the cellular automaton and LSTM models to generate more accurate, interesting and appropriate interactive texts for home appliances. The logic of this process helps to improve the intelligence of the system and the user experience in practical applications.

[0170] Furthermore, the logical principle of the fusion process is to combine the advantages of different models and use the fusion weight to control the weight distribution of the two, so as to achieve a better balance when generating interactive text. This enables the model to flexibly and selectively fuse the outputs of different models according to task requirements and the quality of generated results, thereby generating more creative, accurate and personalized interactive texts for home devices. This fusion method has the potential to improve system performance and user experience.

[0171] Further, about α: Initially, you can set the initial value of α based on experience or intuition. For example, if you think that cellular automaton predictions are more reliable, you can set a larger initial value; if you think that LSTM generation is more accurate, you can set a smaller initial value. Then, adjust according to the actual effect. During training and validation, you can use cross-validation to determine the best α value. Divide the training data into a training set and a validation set, and select the α value that performs best on the validation set by trying different α values ​​and evaluating the quality of the generated results on the validation set. You can consider using adaptive methods to dynamically adjust the value of α. For example, automatically adjust the size of α based on factors such as the progress of model training and the confidence of generated text. This may require some extra work, but it can adapt to the needs of the model at different stages. Use hyperparameter optimization techniques, such as Bayesian optimization or grid search, to search for the optimal α value. These methods can automatically search for the best parameter configuration within a certain range to achieve better performance. If you have multiple cellular automaton models and LSTM models, you can consider using model integration methods. In this case, you can set different α values ​​for each model separately, and then merge their outputs by weighted averaging or voting. If user feedback is available, you can adjust the α value based on user preferences and satisfaction. For example, users may prefer the generated interactive text or trust the predictions of the cellular automaton more.

[0172] Embodiment 5

[0173] According to the above specific implementation methods and embodiments, this embodiment further provides the following technical solutions:

[0174] This embodiment further provides a storage medium, which stores a multi-home device interaction method for realizing the intelligent auxiliary image and text generation as described in the above embodiments 1 to 4. Figures 2 to 4 , which is presented in the form of C++ pseudocode, the principle is as follows:

[0175] S1: Cellular Automata Prediction:

[0176] This section demonstrates how to use the cellular automaton model to predict the state of the device at the next time step. Cellular automaton is a mathematical model of discrete time and space that is used to describe the evolution of states. Here, we predict the next state based on the current state, user input, and rules by calling the CellAutomatonPredict function. This embodies the concept of state prediction through rules and historical data.

[0177] S2: LSTM model update:

[0178] This section shows how to use an LSTM-based model to update hidden states and generate text. LSTM is a recurrent neural network that processes sequence data. In the code, the LSTMUpdate function shows how to update the hidden state based on the input and the previous hidden state. This process captures contextual information, allowing for better understanding of the context and generating appropriate text.

[0179] S3: Linear weighted fusion:

[0180] This section shows how to fuse the state predicted by the cellular automaton and the text generated by the LSTM in a linear weighted manner to generate the final interactive text. The LinearWeightedFusion function calculates the fusion result based on the predicted state, LSTM output, and fusion weight. The fusion weight is controlled by the parameter α, which can be adjusted according to model performance and requirements to balance the contribution of the two types of information.

[0181] This code shows how to combine the cellular automaton model and the LSTM-based text generation model to generate text suitable for home device interaction. Each stage represents a key step in the entire process, covering aspects such as prediction, context capture, and information fusion to achieve a smarter, interesting, and personalized home device interaction experience. This example pseudocode can be used as a starting point for actual implementation and can be customized and extended according to specific needs.

Claims

1. A multi-home device interaction method for intelligent auxiliary image and text generation, characterized by: The steps include: S1. Prediction: Using the cellular automaton model, based on the current state and the user's historical interactions, predict the interaction information that the user may output in the next time step, and output the expected state based on the current state, input and rule calculation; S2, update: using LSTM, input the expected state in S1, including the interaction predicted by the cellular automaton and the previously generated text, and output text after capturing the context information; S3. Integration: Integrate the prediction results of the cellular automaton model with the output of the LSTM text generation model and fuse them in a linear weighted form to generate the final interrogative interactive text.

2. The method for multi-home device interaction according to claim 1, characterized in that: In S1: Assume that the cellular automaton model predicts the next possible interaction based on the current state and the user's historical interaction. The cellular automaton model: State Cell: Ct represents the state at time step t; Input: Xt represents the input at time step t, including user history and current context; Rule: R represents a rule set of a cellular automaton, and the rule set represents a transition rule of a device state; Next Cell: Indicates the state of the cell at time step t+1, which is calculated based on the current state, input and rules: C t+1 =f(CT,Xt,R) f is the function of the state transition rule, which defines how to calculate the next state based on the current state, input and rules.

3. The method for multi-home device interaction according to claim 2, characterized in that: In said S2: Cell State: Each cell represents the state of a device, Si_t represents the state of cell i at time step t; Cell transition function: Maps the current state, input, and rule to the next state: If t+1 =Transition(If t ,t,Ut,R) Ut represents the user interaction information at time step t; Among them, Transition is a defined cell transition function, which calculates the next state according to the current state, user interaction information and rules, and the next state is equal to the expected state.

4. The method for multi-home device interaction according to claim 2, characterized in that: In said S2: a LSTM-based text generation model is used to generate inquiry-type interactive text, including: Input: It represents the input at time step t, including the interactions predicted by the cellular automaton and the previously generated text; Hidden state: Ht represents the hidden state at time step t, which is used to capture contextual information; Output: Ot represents the output at time step t, as the generated interactive text.

5. The method for multi-home device interaction according to claim 4, characterized in that: In S2, input: It represents the input at time step t, including the interaction predicted by the cellular automaton and the previously generated text: It=[E(Ut);E(Gt)] Where E represents the word embedding function, Ut represents the interaction predicted by the cellular automaton, and Gt represents the text generated before time step t; Hidden state: Ht represents the hidden state at time step t, which is used to capture contextual information: H t =LSTM(It,Ht-1) Among them, LSTM represents the update process of the LSTM model, and Ht-1 represents the hidden state at time step t-1; Output: Ot represents the output at time step t, as the generated interactive text: Ot=Softmax(WoHt+bo)Ot=Softmax(WoHt+bo) Among them, Wo and bo are the weight and bias of the output layer, and Softmax represents the Softmax function.

6. The method for multi-home device interaction according to claim 5, characterized in that: Use word embedding to map the input discrete symbols, interactions and texts into continuous vector representations, then use the LSTM model to update the hidden state, and finally get the generated interactive text through the output layer.

7. The method for multi-home device interaction according to claim 4, characterized in that: In the S3: linear weighted fusion is used: Fusion weight: Wt represents the fusion weight at time step t: Wt=α·Confidence(Ct+1)+(1-α)·Confidence(Ot) where α is a parameter that controls the weight balance, and Confidence represents the confidence assessment of the prediction or generation; the larger the α, the more inclined to use the interactive information predicted by the cellular automaton, and the smaller the α, the more inclined to use the text information generated by the LSTM; Fusion results: Ft=Wt·Ot+(1-Wt)·Ct+1 Where Ct+1 is the next state predicted by the cellular automaton; the fusion result Ft is the final interrogative interactive text generated at time step t.

8. A storage medium, characterized in that: The storage medium stores program instructions for executing the multi-home device interaction method as described in any one of claims 1 to 7.

Images