GPT model-based intelligent device interaction method, apparatus, and system

Abstract

To provide a GPT(R) model-based intelligent device interaction method, apparatus, and system.SOLUTION: A method includes causing an intelligent device side to obtain user interaction information by masking initial interaction information, transmit it to a proxy layer together with user preference information, process the user interaction information through a query proxy, and transmit the information to a GPT model. The method further includes causing the model to perform intent recognition and task subdivision based on the processed interaction information, and return task processing information of the processed interaction information, and also causes a response proxy to accept corresponding information, obtain interaction response information based on the user preference information and the task processing information, and transmit it to the intelligent device side. The method further includes causing the intelligent device side to modify the interaction response information based on user privacy data, and respond based on the modified information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to the technical field of human-computer interaction, and in particular to an interaction method, apparatus and system for intelligent devices based on the GPT model. [Background technology]

[0002] In modern society, intelligent devices have become deeply ingrained in people's daily lives, and the human-computer interaction function of devices such as smart speakers and in-vehicle devices is particularly important. However, there are still many challenges and difficulties in the current interaction systems of intelligent devices. First, traditional interaction methods cannot meet people's needs for more natural and intelligent interaction methods. Second, due to the hardware constraints of the devices, these devices cannot express high-quality digital humans (i.e., virtual humans). And most importantly, large-scale language models based on the Generative Pre-Trained Transformer (GPT) model deployed in the cloud can provide personalized services by digital humans, but at the same time, they also bring about problems in data security and privacy. Therefore, in the process of uploading and analyzing user data in real time, how to provide more accurate interaction effects on the premise of ensuring data security and protecting user privacy is currently an important issue to be solved. Summary of the Invention [Problem to be solved by the invention]

[0003] The present invention provides an intelligent device interaction method, apparatus and system based on the GPT model to solve the shortcomings in the prior art, such as the contradiction between the interaction effect and the system complexity of the intelligent device, and the problem of user privacy leakage when providing personalized services. [Means for solving the problem]

[0004] The present invention provides an intelligent device interaction method based on GPT model, which includes:

[0005] That is, the user interaction information and the user preference information transmitted from the intelligent device are received. The user interaction information is information obtained after the intelligent device has performed a masking process on the initial interaction information input by the user.

[0006] After processing the user interaction information to obtain post-processing interaction information, the post-processing interaction information is sent to a GPT model, so that the GPT model returns task processing information of the post-processing interaction information.

[0007] After receiving the task processing information returned from the GPT model, the interaction response information is obtained by fusing the task processing information with the user preference information, and the interaction response information is sent to the intelligent device, so that the intelligent device modifies the interaction response information based on the user privacy data related to the user interaction information, and responds to the user based on the modified information.

[0008] According to the intelligent device interaction method based on the GPT model provided in the present invention, when obtaining interaction response information by fusing based on the above-mentioned user preference information and the task processing information, specifically, the following is included:

[0009] That is, by performing matching based on each noun in the task processing information and each preference label in the user preference information, preference labels matching each noun in the task processing information are obtained.

[0010] The task processing information is then processed by the intelligent device, and the task processing information is processed by the intelligent device. The task processing information is processed by the intelligent device. The task processing information is processed by the intelligent device. The task processing information is processed by the intelligent device. The task processing information is processed by the intelligent device. The task processing information is processed by the intelligent device. The task processing information is processed by the intelligent device. The task processing information is processed by the intelligent device. The task processing information is processed by the intelligent device. The task processing information is processed by the intelligent device. The task processing information is processed by the intelligent device. After the intelligent device receives the interaction response information, the intelligent device corrects the interaction response information based on user privacy data related to the user interaction information to obtain correction processing information, and the intelligent device executes the correction processing information to realize a response to the user.

[0011] According to the intelligent device interaction method based on the GPT model provided in the present invention, when sending the above-mentioned post-processing interaction information to the GPT model, it specifically includes:

[0012] That is, the processed interaction information is vectorized to obtain a processed interaction vector.

[0013] A search is performed in a knowledge base of each field based on the post-processing interaction vector, thereby identifying field knowledge and its field knowledge vector that match the post-processing interaction vector in the knowledge base of each field.

[0014] The processed interaction vector is combined with the domain knowledge vector of the matched domain knowledge to obtain a fused query vector, and the fused query vector is sent to a GPT model.

[0015] According to the intelligent device interaction method based on the GPT model provided in the present invention, when sending the above-mentioned fused query vector to the GPT model, it specifically includes:

[0016] That is, by sending the fused query vector to multiple GPT models, task processing information of the post-processing interaction information returned from the multiple GPT models is obtained.

[0017] The post-processing interaction information returned from the multiple GPT models is fused or selected based on task processing information to obtain optimized processing information, which is fused with the user preference information to obtain the interaction response information.

[0018] According to the intelligent device interaction method based on the GPT model provided in the present invention, the proxy layer is constructed according to the following steps:

[0019] That is, an initial model is constructed based on a pre-trained text generation model.

[0020] Sample input information is input to the initial model to obtain test output information output from the initial model.

[0021] The proxy layer is obtained by fine-tuning the initial model based on an evaluation of the test output information or based on differences between the test output information and sample output information.

[0022] The present invention further provides an intelligent device interaction method based on GPT model, the method includes:

[0023] That is, initial interaction information input by the user is accepted, and the initial interaction information is subjected to a masking process to obtain user interaction information.

[0024] The user interaction information and the user preference information of the user are sent to a proxy layer, whereby the proxy layer processes the user interaction information to obtain post-processing interaction information, and then sends the post-processing interaction information to a GPT model, so that the GPT model returns task processing information of the post-processing interaction information.

[0025] The proxy layer receives the interaction response information sent by the proxy layer, modifies the interaction response information based on user privacy data related to the user interaction information, and responds to the user based on the modified information. The interaction response information is obtained by fusing the user preference information and the task processing information after the proxy layer receives the task processing information of the processed interaction information returned from the GPT model.

[0026] According to the intelligent device interaction method based on the GPT model provided in the present invention, the interaction response information is modified according to the user privacy data related to the above-mentioned user interaction information, and when responding to the user based on the modified information, the following specifically includes:

[0027] That is, based on the user privacy data related to the user interaction information, the interaction response information is modified to obtain correction process information, and the correction process information is executed to realize a response to the user.

[0028] The interaction response information is obtained by the proxy layer obtaining preference labels that match each noun in the task processing information by matching each noun in the task processing information with each preference label in the user preference information, and then replacing the corresponding noun in the task processing information based on the label value of the preference label matched to each noun in the task processing information.

[0029] According to the intelligent device interaction method based on the GPT model provided in the present invention, the method further includes:

[0030] That is, initial interaction information and corresponding revised information of multiple users are collected, and the initial interaction information corresponding to the revised information that does not meet the user's expectations is recorded as a directional optimization sample.

[0031] Intent recognition is performed on the directional optimization sample to obtain a user's intent corresponding to the directional optimization sample, and a type label is set for relevant text in the directional optimization sample that is related to the user's intent based on the user's intent.

[0032] The related text is divided into words to obtain keywords of the related text, and a search is performed within a knowledge base of each field based on the keywords of the related text and the type label of the related text to obtain field knowledge matching the related text.

[0033] A relevance relationship is established between the related text and the domain knowledge matched with the related text, so that when a search is performed in the knowledge base of each domain based on the post-processing interaction vector of the post-processing interaction information corresponding to the corresponding initial interaction information after the same user inputs the same initial interaction information as the directional optimization sample again, the domain knowledge matched with the related text is identified as the domain knowledge matching the corresponding post-processing interaction vector, or a response is made to the initial interaction information based on the domain knowledge matched with the related text.

[0034] According to the intelligent device interaction method based on the GPT model provided in the present invention, the method further includes:

[0035] That is, feedback information input by the user after the response is obtained, and an image of the user inputting the feedback information is collected.

[0036] Emotion recognition is performed on the feedback information to obtain an emotion type of the feedback information.

[0037] A facial expression type when the user inputs the feedback information is acquired by performing facial expression recognition on an image when the user inputs the feedback information.

[0038] If the feedback information contains a negative keyword, if the emotion type of the feedback information is a negative emotion, or if the facial expression type when the user inputs the feedback information is a negative facial expression, it is determined that an interaction barrier exists at that point.

[0039] If an interaction barrier exists at that time, each piece of question and answer history information corresponding to the user is obtained, and question and answer history information matching the initial interaction information is searched for from each piece of question and answer history information, and a response is made based on the question and answer history information matching the initial interaction information.

[0040] According to the intelligent device interaction method based on the GPT model provided in the present invention, the method further includes:

[0041] That is, the type of network condition at that time is confirmed, and when the type of network condition is a network condition deterioration or a network disconnection state, the user interaction information and the user preference information of the user are sent to a local GPT model deployed locally, thereby obtaining local response information returned from the local GPT model, and responding to the user based on the local response information.

[0042] According to the intelligent device interaction method based on the GPT model provided in the present invention, the local GPT model is obtained by performing model quantization compression on the sample GPT model.

[0043] The present invention further provides an intelligent device interaction method based on the GPT model, the method includes:

[0044] That is, the intelligent device side receives initial interaction information input by the user and acquires user interaction information by performing a masking process on the initial interaction information.

[0045] The intelligent device side sends the user interaction information and the user preference information of the user to a proxy layer.

[0046] The proxy layer receives user interaction information and user preference information sent from the intelligent device side, processes the user interaction information to obtain post-processing interaction information, and then sends the post-processing interaction information to the GPT model, which then returns task processing information of the post-processing interaction information.

[0047] After receiving the task processing information of the post-processing interaction information returned from the GPT model, the proxy layer obtains interaction response information by fusing it based on the user preference information and the task processing information, and transmits the interaction response information to the intelligent device side.

[0048] The intelligent device side receives the interaction response information sent by the proxy layer, modifies the interaction response information based on the user privacy data related to the user interaction information, and responds to the user based on the modified information.

[0049] The present invention further provides an intelligent device interaction apparatus based on the GPT model, the apparatus including:

[0050] Information receiving unit: receives user interaction information and user preference information transmitted from the intelligent device side. The user interaction information is information obtained after the intelligent device side performs masking processing on the initial interaction information input by the user.

[0051] Inquiry Proxy: After processing the user interaction information to obtain post-processing interaction information, it sends the post-processing interaction information to the GPT model, so that the GPT model returns task processing information of the post-processing interaction information.

[0052] Response Proxy: After receiving the task processing information returned from the GPT model, it obtains interaction response information by fusing the user preference information and the task processing information, and transmits the interaction response information to the intelligent device side, so that the intelligent device side modifies the interaction response information based on user privacy data related to the user interaction information, and responds to the user based on the modified information.

[0053] The present invention further provides an intelligent device interaction apparatus based on the GPT model, the apparatus including:

[0054] An input processing unit: receives initial interaction information input by a user, and obtains user interaction information by performing a masking process on the initial interaction information.

[0055] An information forwarding unit: sends the user interaction information and the user preference information of the user to a proxy layer, so that the proxy layer processes the user interaction information to obtain post-processing interaction information, and then sends the post-processing interaction information to a GPT model, so that the GPT model returns task processing information of the post-processing interaction information.

[0056] An information response unit: receives the interaction response information sent by the proxy layer, modifies the interaction response information according to the user privacy data related to the user interaction information, and responds to the user based on the modified information. The interaction response information is obtained by fusing the user preference information and the task processing information after the proxy layer receives the task processing information of the processed interaction information returned from the GPT model.

[0057] The present invention further provides an intelligent device interaction system based on the GPT model, which includes a proxy layer and an intelligent device side.

[0058] In the proxy layer, an interaction apparatus of an intelligent device based on the GPT model corresponding to any of the above-mentioned proxy layers is deployed.

[0059] An intelligent device interaction device based on the GPT model corresponding to any of the above-mentioned intelligent device sides is deployed on the intelligent device side. Effect of the Invention

[0060] In the method, apparatus and system for intelligent device interaction based on the GPT model provided by the present invention, the initial interaction information input by the user is masked by the intelligent device side to obtain user interaction information, which is then sent to the proxy layer together with user preference information. Then, the query proxy in the proxy layer processes the user interaction information to obtain post-processing interaction information, and then sends the post-processing interaction information to the GPT model. As a result, the GPT model performs intent recognition and task subdivision based on the post-processing interaction information, and returns task processing information of the post-processing interaction information. After receiving the task processing information of the post-processing interaction information returned from the GPT model, the response proxy obtains interaction response information of the user interaction information by fusing it based on the user preference information and the task processing information, and sends the interaction response information to the intelligent device side. Next, the intelligent device side modifies the interaction response information based on the user privacy data related to the user interaction information, and responds to the user based on the modified information. In this way, a solution is provided to realize high-quality personalized intelligent interaction while protecting user privacy, which improves the accuracy of the response while ensuring that user privacy data remains only on the intelligent device, thus protecting the safety of user privacy.

[0061] In order to more clearly describe the technical solutions of the present invention or the prior art, the following briefly describes the drawings that need to be used in the description of the embodiments or the prior art, it goes without saying that the drawings described below are only some of the embodiments of the present invention, and those skilled in the art can obtain other drawings from these drawings without requiring creative labor. [Brief description of the drawings]

[0062] [Figure 1]FIG. 1 is a schematic flowchart 1 of an intelligent device interaction method based on the GPT model provided in the present invention. [Diagram 2] FIG. 2 is a schematic flowchart 2 of the intelligent device interaction method based on the GPT model provided in the present invention. [Diagram 3] FIG. 3 is a schematic flowchart 3 of the intelligent device interaction method based on the GPT model provided in the present invention. [Figure 4] FIG. 4 is a schematic structural diagram 1 of an intelligent device interaction device based on the GPT model provided in the present invention. [Diagram 5] FIG. 5 is a schematic structural diagram 2 of an intelligent device interaction device based on the GPT model provided in the present invention. [Figure 6] FIG. 6 is a schematic structural diagram of an intelligent device interaction system based on the GPT model provided in the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0063] In order to make the objectives, technical solutions and advantages of the present invention clearer, the following will clearly and concisely describe the technical solutions of the present invention in combination with the drawings in the present invention. It goes without saying that the embodiments described herein are only some of the embodiments of the present invention, and not all of the embodiments. All other embodiments that can be obtained by those skilled in the art based on the embodiments of the present invention without requiring creative labor, all belong to the scope of protection of the present invention.

[0064] FIG. 1 is a schematic flow chart 1 of an intelligent device interaction method based on the GPT model provided by the present invention. The method is applied to a proxy layer deployed in the cloud, which is interposed between the intelligent device side used for interaction and the GPT model deployed in the cloud. The intelligent device side includes, but is not limited to, smart speakers, in-vehicle devices, etc. As shown in FIG. 1, the method includes:

[0065] Step 110: receiving user interaction information and user preference information sent from the intelligent device side, where the user interaction information is information obtained after the intelligent device side performs masking processing on the initial interaction information input by the user.

[0066] Step 120: After processing the user interaction information to obtain post-processing interaction information, the post-processing interaction information is sent to a GPT model, so that the GPT model returns task processing information of the post-processing interaction information.

[0067] Step 130: after receiving the task processing information returned from the GPT model, obtain interaction response information by fusing based on the user preference information and the task processing information, and send the interaction response information to the intelligent device side, so that the intelligent device side modifies the interaction response information based on the user privacy data related to the user interaction information, and responds to the user based on the modified information.

[0068] Specifically, after the user inputs the initial interaction information through a human-computer interface (e.g., a voice input interface or a text input interface) provided by the intelligent device side, in order to prevent the user's privacy from being leaked, the intelligent device side performs a masking process on the initial interaction information and replaces the data of the initial interaction information related to the user's privacy (e.g., name, contact information, specific address, etc.) with a predetermined symbol (e.g., "*"), so that the user interaction information corresponding to the initial interaction information can be obtained. Then, the intelligent device side obtains the user preference information of the user. The user preference information is obtained by performing a user profile analysis on various interaction behaviors of the user on the intelligent device side (which may be local interaction behaviors on the intelligent device side, or may be interaction behaviors synchronized in a mobile application installed on the intelligent device side and linked with the user's mobile phone). The intelligent device side transmits the user preference information and the user interaction information together to a proxy layer of the cloud. The proxy layer of the cloud includes at least a query proxy and a response proxy.

[0069] After receiving the user interaction information and user preference information transmitted from the intelligent device side, the inquiry proxy can process the user interaction information to obtain post-processing interaction information. The above processing pattern can be predefined in the inquiry proxy, and the user interaction information and user preference information can be organized according to the processing pattern to obtain the post-processing interaction information. In this way, after transmitting the post-processing interaction information to the GPT model in the cloud, the GPT model can perform intent recognition and task subdivision based on the post-processing interaction information, rather than feeding back the user interaction information as it is. The processing pattern can be obtained by testing based on the specific type of the GPT model. For example, by manually setting different prompts and then testing them one by one, a prompt that can achieve the corresponding interaction needs (i.e., obtain the processing method of the task) can be obtained, but this is not specifically limited in the embodiment of the present invention. For example, when the user interaction information is a command control type, the GPT model may perform intent recognition and task subdivision based on the post-processing interaction information, and then return task processing information capable of instructing the execution steps of the corresponding command. For example, when the command represented by the user interaction information is a take-out lunch order, the GPT model may return task processing information instructing processing steps such as selecting a take-out platform, selecting food to order, and determining contact information and address, rather than executing the command of take-out order as is. Also, when the user interaction information is an information suggestion type, the GPT model may perform intent recognition and task subdivision based on the post-processing interaction information, and then return task processing information capable of instructing the acquisition steps of the corresponding information. For example, when the waiting-for-suggestion information represented by the user interaction information is a restaurant near the user's home, the GPT model may return task processing information instructing processing steps such as selecting a dining area and selecting a restaurant style based on the user's home address, rather than directly returning the selected restaurant.As is clear from the above, the task processing information returned by the GPT model dictates how to process user interaction information.

[0070] After receiving the task processing information of the processed interaction information returned by the GPT model, the response proxy obtains interaction response information that is more in line with the user's needs and preferences through fusion based on the user preference information and the task processing information, so as to improve the user's interaction experience, and sends the interaction response information to the intelligent device side. The intelligent device side modifies the above interaction response information based on the user privacy data related to the user interaction information, and responds to the user based on the modified information. This improves the accuracy of the response while ensuring that the user privacy data is only kept on the intelligent device side, thereby protecting the safety of the user's privacy.

[0071] In some embodiments, when the user interaction information is a command control system and the response proxy acquires the interaction response information of the user interaction information by fusing based on the user preference information and the task processing information, matching may be performed based on each noun in the task processing information and each preference label in the user preference information to acquire a preference label matching each noun in the task processing information. Here, if the similarity between any noun and all preference labels is lower than a predetermined threshold, the noun does not have a matching preference label. Also, if the similarity between any noun and any preference label is not only higher than the similarity between any other preference labels but also higher than a predetermined threshold, the noun is matched to the preference label. Then, based on the label value of the preference label matched to each noun in the task processing information, the corresponding noun in the task processing information is replaced to acquire the interaction response information of the user interaction information. After receiving the interaction response information, the intelligent device side can supplement and complete the corresponding part in the interaction response information based on the user privacy data related to the user interaction information (e.g., address information, contact information, etc.), obtain correction processing information, and call a local execution interpreter to execute the correction processing information, thereby realizing a response to the user. Note that the local GPT on the intelligent device side can be used to analyze the interaction response information and the user privacy data to identify the part in the interaction response information to be supplemented, and the user privacy data can be used to supplement it.

[0072] In some other embodiments, when the user interaction information is an information suggestion system and the response proxy acquires the interaction response information of the user interaction information by fusing based on the user preference information and the task processing information, a preference label matching each noun in the task processing information may be acquired by matching each noun in the task processing information with each preference label in the user preference information. Then, the complete task processing information is acquired by replacing the corresponding noun in the task processing information based on the label value of the preference label matched to each noun in the task processing information. Then, the complete task processing information is acquired by directly executing the step indicated by the complete task processing information. After receiving the interaction response information, the intelligent device side selects the interaction response information based on the user privacy data related to the user interaction information (for example, selects restaurants within a narrower area / range based on the user's home address), acquires the selected response information, and returns the selected response information to the user, thereby realizing a response to the user.

[0073] Based on any of the above embodiments, when the query proxy sends the post-processing interaction information to the GPT model, the post-processing interaction information may be vectorized to obtain a post-processing interaction vector, and then a search may be performed in the knowledge base of each field based on the post-processing interaction vector. The knowledge base of each field stores the field knowledge of each field and the field knowledge vector of the field knowledge. By vector search, the field knowledge and the field knowledge vector that match the post-processing interaction vector in the knowledge base of each field can be identified. Then, the post-processing interaction vector is combined with the field knowledge vector of the matched field knowledge to obtain a fusion query vector, and the fusion query vector is sent to the GPT model. The post-processing interaction vector of the post-processing interaction information and the field knowledge vector of the matched field knowledge are combined and fused and then sent to the GPT model, which helps the GPT model understand the true meaning of the post-processing interaction information, and improves the accuracy of the interaction response.

[0074] Here, by transmitting the fusion query vector to multiple GPT models, task processing information of post-processing interaction information returned from the multiple GPT models is obtained, and optimal task processing information may be selected from among them based on the task processing information of the post-processing interaction information returned from the multiple GPT models, or the task processing information returned from the multiple GPT models may be fused to obtain optimization processing information. The optimization processing information is used to obtain interaction response information of user interaction information by fusing with user preference information. That is, when obtaining interaction response information by fusing based on user preference information and task processing information, the response proxy may select from the task processing information returned from the multiple GPT models, or may obtain the interaction response information by fusing with the user preference information.

[0075] Based on any of the above embodiments, in order to improve the data processing capability of the query proxies and response proxies of the proxy layer, reinforcement learning may be performed on the proxy layer (query proxies and response proxies) based on the following steps.

[0076] First, in order to utilize the text understanding and text generation functions of a conventional text generation model (e.g., GPT model), an initial model corresponding to the query proxy and the response proxy of the proxy layer is respectively constructed based on a pre-trained text generation model. It should be noted that the same or different text generation models may be used for the query proxy and the response proxy, and the embodiment of the present invention does not specifically limit this. Next, the collected sample input information is input into the initial model to obtain the test output information output from the initial model. It should be noted that the sample input information corresponding to the query proxy and the response proxy is different. For the query proxy, the corresponding sample input information can be user interaction information and processing patterns. For the response proxy, the corresponding sample input information can be user preference information and task processing information. Based on the test output information output from the initial model, it is evaluated, and then the model loss is calculated based on the evaluation of the test output information. In this way, the model parameters of the initial model are reverse-adjusted based on the model loss to realize fine-tuning of the initial model, thereby obtaining a trained query proxy or response proxy. In addition, after setting accurate sample output information for the sample input information, the difference between the sample output information and the test output information is calculated, and the model loss is calculated based on this difference. Then, the model parameters of the initial model are back-tuned based on the model loss to realize fine-tuning of the initial model, thereby obtaining a trained query proxy or answer proxy.

[0077] In the method provided in the embodiment of the present invention, the initial interaction information input by the user is masked by the intelligent device side to obtain user interaction information, and the information is sent to the proxy layer together with the user preference information. Then, the query proxy in the proxy layer processes the user interaction information to obtain post-processing interaction information, and then the post-processing interaction information is sent to the GPT model. As a result, the GPT model performs intent recognition and task subdivision based on the post-processing interaction information, and returns task processing information of the post-processing interaction information. After receiving the task processing information of the post-processing interaction information returned from the GPT model, the response proxy obtains interaction response information of the user interaction information by fusing based on the user preference information and the task processing information, and sends the interaction response information to the intelligent device side. Next, the intelligent device side modifies the interaction response information based on the user privacy data related to the user interaction information, and responds to the user based on the modified information. As a result, the accuracy of the response is improved while ensuring that the user privacy data remains only on the intelligent device side, thereby protecting the safety of the user's privacy.

[0078] 2 is a schematic flowchart 2 of an intelligent device interaction method based on the GPT model provided in the present invention, which is applied to the intelligent device side. As shown in FIG. 2, the method includes:

[0079] Step 210: Accept initial interaction information input by a user, and obtain user interaction information by performing a masking process on the initial interaction information.

[0080] Step 220: Send the user interaction information and the user preference information of the user to a proxy layer, so that the proxy layer processes the user interaction information to obtain post-processing interaction information, and then sends the post-processing interaction information to a GPT model, so that the GPT model returns task processing information of the post-processing interaction information.

[0081] Step 230: Accept the interaction response information sent by the proxy layer, modify the interaction response information according to the user privacy data related to the user interaction information, and respond to the user based on the modified information. The interaction response information is obtained by fusing the user preference information and the task processing information after the proxy layer accepts the task processing information of the processed interaction information returned from the GPT model.

[0082] In the method provided in the embodiment of the present invention, the initial interaction information input by the user is masked by the intelligent device side to obtain user interaction information, and the information is sent to the proxy layer together with the user preference information. Then, the query proxy in the proxy layer processes the user interaction information to obtain post-processing interaction information, and then the post-processing interaction information is sent to the GPT model. As a result, the GPT model performs intent recognition and task subdivision based on the post-processing interaction information, and returns task processing information of the post-processing interaction information. After receiving the task processing information of the post-processing interaction information returned from the GPT model, the response proxy obtains interaction response information of the user interaction information by fusing based on the user preference information and the task processing information, and sends the interaction response information to the intelligent device side. Next, the intelligent device side modifies the interaction response information based on the user privacy data related to the user interaction information, and responds to the user based on the modified information. As a result, the accuracy of the response is improved while ensuring that the user privacy data remains only on the intelligent device side, thereby protecting the safety of the user's privacy.

[0083] Based on any of the above embodiments, when the user interaction information is a command control system, the interaction response information is modified based on user privacy data related to the above user interaction information, and when responding to the user based on the modified information, the following specifically includes:

[0084] That is, based on the user privacy data related to the user interaction information, the interaction response information is modified to obtain correction process information, and the correction process information is executed to realize a response to the user.

[0085] The interaction response information is obtained by the proxy layer obtaining preference labels that match each noun in the task processing information by matching each noun in the task processing information with each preference label in the user preference information, and then replacing the corresponding noun in the task processing information based on the label value of the preference label matched to each noun in the task processing information.

[0086] Based on any of the above embodiments, when the user interaction information is information suggestion information, the interaction response information is modified based on the user privacy data related to the above user interaction information, and a response to the user based on the modified information specifically includes the following:

[0087] That is, the interaction response information is selected based on the user privacy data related to the user interaction information, selected response information is obtained, and the selected response information is returned to the user, thereby realizing a response to the user.

[0088] The interaction response information is obtained by the proxy layer obtaining preference labels that match each noun in the task processing information by matching each noun in the task processing information with each preference label in the user preference information, obtaining complete task processing information by replacing the corresponding noun in the task processing information based on the label value of the preference label matched to each noun in the task processing information, and then executing the complete task processing information.

[0089] It should be mentioned that the interaction method presented in the above embodiment on the intelligent device side is similar to that presented in the corresponding embodiment on the proxy layer, so it will not be described in further detail here.

[0090] According to any of the above embodiments, if a user inputs a voice but the voice is misrecognized, for example, the Kongo River is misrecognized as Kongo Mountain, it may be difficult to provide an accurate response. In particular, when the proxy proxy in the proxy layer searches in the knowledge base of each field based on the post-processing interaction vector of the post-processing interaction information, it is difficult to search for truly related field knowledge, so it is difficult to guarantee the accuracy of the response. In order to further improve the accuracy of the interaction response, the intelligent device side may collect multiple users' initial interaction information and corresponding corrected information (i.e., the result of the intelligent device side correcting the interaction response information according to the corresponding initial interaction information based on the user privacy data related to the user interaction information), and record the initial interaction information corresponding to the corrected information that does not meet the user's expectations as a directional optimization sample. Here, based on the corresponding user's reaction after the intelligent device side responds based on the corrected information (e.g., the user's facial expression or the feedback information input by the user), it may be determined whether the corrected information meets the user's expectations.

[0091] Then, intent recognition is performed for each directional optimization sample to obtain the user's intent corresponding to each directional optimization sample. Then, for each directional optimization sample, a type label is set for related text in the directional optimization sample related to the user's intent based on the user's intent corresponding to the directional optimization sample. For example, if the user's intent is to watch a movie, the type label of the related text can be set to "movie". Then, the related text is divided into words to obtain keywords of the related text, and a search is performed in the knowledge base of each field based on the keywords of the related text and the type label of the related text to obtain field knowledge matching the related text. Here, the field knowledge that has the highest relevance to the keywords in the related text and whose type matches the type label of the related text can be selected as the field knowledge matching the related text. Next, a relationship of relevance is built between the related text and the field knowledge matched to the related text.

[0092] After the relevance relationship is established, when the same user again inputs the same initial interaction information as the directional optimization sample, the query proxy in the proxy layer searches in the knowledge base of each domain based on the post-processing interaction vector of the post-processing interaction information corresponding to the corresponding initial interaction information. Since the above-mentioned relevance relationship is established between the related text in the directional optimization sample and the domain knowledge matched to the related text, the domain knowledge can be immediately detected and the domain knowledge matched to the related text can be identified as the domain knowledge matching the corresponding post-processing interaction vector. This allows the GPT model to more accurately understand the user's intent and the true meaning of each part of the text in the post-processing interaction information (even if the text is input incorrectly), thereby improving the accuracy of the response. In addition, after the same user again inputs the same initial interaction information as the directional optimization sample, the intelligent device may respond directly based on the domain knowledge matched to the relevant text described above, thereby realizing quick understanding of the user's intent and quick response while avoiding excessive reliance on an external GPT model.

[0093] Based on any of the above embodiments, in order to further improve the degree of protection of user privacy, when the intelligent device side transmits the user interaction information and the user preference information to the inquiry proxy, the user preference information may be obtained after the true user preferences are obfuscated. In this case, the true user profile label related to the user interaction information in the true user preferences may be identified based on the user interaction information, and the related true user profile label may be superconceptualized (for example, mala hotpot may be superconceptualized as spicy food), thereby obtaining user preference information including the obfuscated profile label corresponding to the related true user profile label. After receiving the interaction response information transmitted from the response proxy, the intelligent device side may first modify the interaction response information based on the true user preferences, and then secondarily modify the interaction response information after the first modification using the user privacy data related to the user interaction information, thereby obtaining modification processing information.

[0094] When the user interaction information is information suggestion information, the interaction response information may be first corrected by a local interference removal model on the intelligent device side. The interference removal model includes four network layers. The first layer is a true preference extraction layer that can be obtained by constructing based on LSTM (Long Short-Term Memory) or its modified result, and is used to extract features of the hidden layer of the related true user profile label based on the interaction behavior corresponding to the related true user profile label. The second layer is an obfuscated preference extraction layer having a structure similar to that of the true preference extraction layer, and is used to extract features of the hidden layer of the obfuscated profile label based on the interaction behavior according to the obfuscated profile label corresponding to the related true user profile label. The third layer is a response vector conversion layer that can be obtained by constructing based on a fully connected network, and is used to convert the type distribution of each response item (e.g., description information of each restaurant) in the interaction response information into a response vector. Note that the type distribution may be obtained by a statistical method by classifying the description information in each response item using a classifier in advance and acquiring the type of each response item. The fourth layer is a correction layer, which is a fully connected layer, and is used to convert the hidden layer features of the related true user profile labels, the hidden layer features of the obscured profile labels, and the combination result of the answer vectors into a new corrected type distribution, and use the corrected type distribution to select response items corresponding to types with high occupancy rates and combine them as the interaction response information after the first correction.

[0095] In the case where the user interaction information is of a command control system, by matching each noun in the interaction response information with the above-mentioned related true user profile label in the true user preferences, a noun in the interaction response information that matches with the above-mentioned related true user profile label is obtained, and by replacing the matched noun based on the above-mentioned related true user profile label, it is possible to obtain the interaction response information after the first correction.

[0096] Based on any of the above embodiments, the intelligent device side can further improve the interaction experience of the user by appropriately processing the interaction barrier scene. In this case, the intelligent device side may obtain feedback information input by the user after the response, collect an image when the user inputs the feedback information, and perform emotion recognition on the feedback information to obtain the emotion type of the feedback information, or perform facial expression recognition on the image when the user inputs the feedback information to obtain the facial expression type when the user inputs the feedback information. Then, if the feedback information includes a negative keyword, if the emotion type of the feedback information is a negative emotion, or if the facial expression type when the user inputs the feedback information is a negative facial expression, it is determined that an interaction barrier exists at that time. Also, if an interaction barrier exists at that time, each question and answer history information corresponding to the user is obtained, and each question and answer history information is searched and obtained to find question and answer history information matching the initial interaction information from the each question and answer history information, and a response is made based on the question and answer history information matching the initial interaction information.

[0097] Based on any of the above embodiments, in an interaction scene, the user is likely to encounter abnormal situations such as slow line speeds and even network disconnections, for example, when passing through a section outside the network coverage area such as a tunnel or a weak signal. In this case, it is necessary to support the user to use some basic interaction functions even in an offline state. Therefore, a local GPT that can perform inference using local computing power and complete some simple answers may be deployed on the intelligent device side. In this case, the type of network state at that time is confirmed, and when the type of network state is a network state deterioration or a network disconnection state, the user interaction information and the user's user preference information may be sent to the locally deployed local GPT model to obtain local response information returned from the local GPT model, and a response may be made to the user based on the local response information. Here, since the computing power on the intelligent device side is limited, a local GPT model that can be locally deployed may be obtained by performing quantization compression of the model based on a large-scale sample GPT model (e.g., LLaMA-6B). Specifically, the following quantization compression means can realize compression optimization of the model.

[0098] 1. Optimization of the amount of parameters in the model: By compressing floating-point constants from float32 to int8 and using the area mapping method, the amount of parameters in the model can be reduced. In this way, the model can be converted from an fp32 state that requires 24GB of memory to an int8 state that requires only 6GB of memory, resulting in a savings of about four times the amount of calculation.

[0099] 2. Gradient Optimization of Parameters: Next, the parameters of the model may be further optimized, which can be achieved by a degree of gradient optimization of the parameters.

[0100] 3. Use AdamW optimizer: You can use AdamW optimizer. This optimizer requires 6B x 2 bytes = 12GB of memory.

[0101] 4. Use CUDA kernel: You can use CUDA kernel. This tool occupies 1.3GB of memory.

[0102] 5. Mixed-precision optimization: The mixed-precision optimization method can further improve the computation efficiency and reduce the memory occupation.

[0103] 6. High-order matrix transformation based on Lora: Using the Lora high-order matrix transformation technique, a high-order matrix can be transformed into the dot product of two low-order matrices, thereby reducing memory occupation.

[0104] In summary, by using the above optimization technique, it is possible to control the memory required for a part of the LLaMA-6B model with int8 precision to about 25.3 GB. In addition, by quantizing the weights to 4 bits and using a common scale and zero point, it is possible to minimize the loss of precision.

[0105] It should be noted that the above-mentioned quantization method for model compression is just one possible method, and other model compression methods such as model pruning may also be possible methods for constructing a local GPT model, and the embodiments of the present invention are not limited thereto.

[0106] 3 is a schematic flowchart 3 of an intelligent device interaction method based on the GPT model provided in the present invention. As shown in FIG. 3, the method includes:

[0107] Step 310: The intelligent device side receives the initial interaction information input by the user, and obtains user interaction information by performing a masking process on the initial interaction information.

[0108] Step 320: The intelligent device side sends the user interaction information and the user preference information of the user to a proxy layer.

[0109] Step 330: The proxy layer accepts user interaction information and user preference information sent from the intelligent device side, processes the user interaction information to obtain post-processing interaction information, and then sends the post-processing interaction information to the GPT model, so that the GPT model returns task processing information of the post-processing interaction information.

[0110] Step 340: After the proxy layer receives the task processing information of the post-processing interaction information returned from the GPT model, it obtains interaction response information by fusing it based on the user preference information and the task processing information, and sends the interaction response information to the intelligent device side.

[0111] Step 350: The intelligent device side receives the interaction response information sent by the proxy layer, modifies the interaction response information based on the user privacy data related to the user interaction information, and responds to the user based on the modified information.

[0112] It should be noted that the interaction method presented in this embodiment is similar to the interaction method presented in the embodiment corresponding to the proxy layer and the interaction method presented in the embodiment corresponding to the intelligent device side, and therefore will not be described in further detail here.

[0113] In the method provided in the embodiment of the present invention, the initial interaction information input by the user is masked by the intelligent device side to obtain user interaction information, and the information is sent to the proxy layer together with the user preference information. Then, the query proxy in the proxy layer processes the user interaction information to obtain post-processing interaction information, and then the post-processing interaction information is sent to the GPT model. As a result, the GPT model performs intent recognition and task subdivision based on the post-processing interaction information, and returns task processing information of the post-processing interaction information. After receiving the task processing information of the post-processing interaction information returned from the GPT model, the response proxy obtains interaction response information of the user interaction information by fusing based on the user preference information and the task processing information, and sends the interaction response information to the intelligent device side. Next, the intelligent device side modifies the interaction response information based on the user privacy data related to the user interaction information, and responds to the user based on the modified information. As a result, the accuracy of the response is improved while ensuring that the user privacy data remains only on the intelligent device side, thereby protecting the safety of the user's privacy.

[0114] Next, the interaction device of the intelligent device based on the GPT model provided by the present invention is described. Note that the interaction device of the intelligent device based on the GPT model described below and the interaction method of the intelligent device based on the GPT model described above can be referred to each other.

[0115] According to any of the above embodiments, FIG. 4 is a schematic structural diagram 1 of an intelligent device interaction device based on the GPT model provided in the present invention. The device is deployed in the proxy layer. As shown in FIG. 4, the device includes:

[0116] Information receiving unit 410: receives user interaction information and user preference information sent from the intelligent device side. The user interaction information is information obtained after the intelligent device side performs masking processing on the initial interaction information input by the user.

[0117] Inquiry Proxy 420: After processing the user interaction information to obtain post-processing interaction information, it sends the post-processing interaction information to the GPT model, so that the GPT model returns the task processing information of the post-processing interaction information.

[0118] Response Proxy 430: After receiving the task processing information of the post-processing interaction information returned from the GPT model, it obtains interaction response information by fusing based on the user preference information and the task processing information, and transmits the interaction response information to the intelligent device side, so that the intelligent device side modifies the interaction response information based on the user privacy data related to the user interaction information, and responds to the user based on the modified information.

[0119] Based on any of the above embodiments, when the user interaction information is a command control system, obtaining interaction response information by fusing based on the above user preference information and the task processing information specifically includes the following:

[0120] That is, by performing matching based on each noun in the task processing information and each preference label in the user preference information, preference labels matching each noun in the task processing information are obtained.

[0121] The task processing information is updated based on the label value of the preference label that is matched to each noun in the task processing information, thereby obtaining interaction response information. After receiving the interaction response information, the intelligent device side obtains correction processing information by correcting the interaction response information based on user privacy data related to the user interaction information, and executes the correction processing information to realize a response to the user.

[0122] Based on any of the above embodiments, when the user interaction information is an information suggestion type, obtaining interaction response information of the user interaction information by replacing corresponding nouns in the task processing information based on the label values ​​of preference labels matched to each noun in the task processing information specifically includes the following:

[0123] That is, by performing matching based on each noun in the task processing information and each preference label in the user preference information, preference labels matching each noun in the task processing information are obtained.

[0124] Based on the label value of the preference label matched to each noun in the task process information, the corresponding noun in the task process information is replaced to obtain the complete task process information.

[0125] The complete task processing information is executed to obtain interaction response information of the user interaction information. After receiving the interaction response information, the intelligent device side selects the interaction response information based on user privacy data related to the user interaction information, obtains selected response information, and returns the selected response information to the user to realize a response to the user.

[0126] According to any of the above embodiments, when the above-mentioned post-processing interaction information is sent to the GPT model, the following is specifically included:

[0127] That is, the processed interaction information is vectorized to obtain a processed interaction vector.

[0128] A search is performed in a knowledge base of each field based on the post-processing interaction vector, thereby identifying field knowledge and its field knowledge vector that match the post-processing interaction vector in the knowledge base of each field.

[0129] The processed interaction vector is combined with the domain knowledge vector of the matched domain knowledge to obtain a fused query vector, and the fused query vector is sent to a GPT model.

[0130] According to any of the above embodiments, when sending the above fused query vector to the GPT model, the following is specifically included:

[0131] That is, by sending the fused query vector to multiple GPT models, task processing information of the post-processing interaction information returned from the multiple GPT models is obtained.

[0132] The post-processing interaction information returned from the multiple GPT models is fused or selected based on task processing information to obtain optimized processing information, which is fused with the user preference information to obtain the interaction response information.

[0133] According to any of the above embodiments, the proxy layer is constructed according to the following steps.

[0134] That is, an initial model is constructed based on a pre-trained text generation model.

[0135] Sample input information is input to the initial model to obtain test output information output from the initial model.

[0136] The proxy layer is obtained by fine-tuning the initial model based on an evaluation of the test output information or based on differences between the test output information and sample output information.

[0137] According to any of the above embodiments, FIG. 5 is a schematic structural diagram 2 of an intelligent device interaction device based on the GPT model provided in the present invention. The device is deployed on the intelligent device side. As shown in FIG. 5, the device includes:

[0138] Input processing unit 510: Accepts initial interaction information input by a user, and performs masking processing on the initial interaction information to obtain user interaction information.

[0139] An information forwarding unit 520: sends the user interaction information and the user preference information of the user to a proxy layer, so that the proxy layer processes the user interaction information to obtain post-processing interaction information, and then sends the post-processing interaction information to a GPT model, so that the GPT model returns task processing information of the post-processing interaction information.

[0140] Information response unit 530: receives the interaction response information sent by the proxy layer, modifies the interaction response information according to the user privacy data related to the user interaction information, and responds to the user according to the modified information. The interaction response information is obtained by fusing the user preference information and the task processing information after the proxy layer receives the task processing information of the processed interaction information returned from the GPT model.

[0141] Based on any of the above embodiments, when the user interaction information is a command control system, the interaction response information is modified based on user privacy data related to the above user interaction information, and when responding to the user based on the modified information, the following specifically includes:

[0142] That is, based on the user privacy data related to the user interaction information, the interaction response information is modified to obtain correction process information, and the correction process information is executed to realize a response to the user.

[0143] The interaction response information is obtained by the proxy layer obtaining preference labels that match each noun in the task processing information by matching each noun in the task processing information with each preference label in the user preference information, and then replacing the corresponding noun in the task processing information based on the label value of the preference label matched to each noun in the task processing information.

[0144] Based on any of the above embodiments, when the user interaction information is information suggestion information, the interaction response information is modified based on the user privacy data related to the above user interaction information, and a response to the user based on the modified information specifically includes the following:

[0145] That is, the interaction response information is selected based on the user privacy data related to the user interaction information, selected response information is obtained, and the selected response information is returned to the user, thereby realizing a response to the user.

[0146] The interaction response information is obtained by the proxy layer obtaining preference labels that match each noun in the task processing information by matching each noun in the task processing information with each preference label in the user preference information, obtaining complete task processing information by replacing the corresponding noun in the task processing information based on the label value of the preference label matched to each noun in the task processing information, and then executing the complete task processing information.

[0147] Based on any of the above embodiments, the apparatus further comprises an optimization unit, which is used for:

[0148] That is, initial interaction information and corresponding revised information of multiple users are collected, and the initial interaction information corresponding to the revised information that does not meet the user's expectations is recorded as a directional optimization sample.

[0149] Intent recognition is performed on the directional optimization sample to obtain a user's intent corresponding to the directional optimization sample, and a type label is set for relevant text in the directional optimization sample that is related to the user's intent based on the user's intent.

[0150] The related text is divided into words to obtain keywords of the related text, and a search is performed within a knowledge base of each field based on the keywords of the related text and the type label of the related text to obtain field knowledge matching the related text.

[0151] A relevance relationship is established between the related text and the domain knowledge matched with the related text, so that when a search is performed in the knowledge base of each domain based on the post-processing interaction vector of the post-processing interaction information corresponding to the corresponding initial interaction information after the same user inputs the same initial interaction information as the directional optimization sample again, the domain knowledge matched with the related text is identified as the domain knowledge matching the corresponding post-processing interaction vector, or a response is made to the initial interaction information based on the domain knowledge matched with the related text.

[0152] According to any of the above embodiments, the apparatus further comprises an interaction barrier processing unit, which is used for:

[0153] That is, feedback information input by the user after the response is obtained, and an image of the user inputting the feedback information is collected.

[0154] Emotion recognition is performed on the feedback information to obtain an emotion type of the feedback information.

[0155] A facial expression type when the user inputs the feedback information is acquired by performing facial expression recognition on an image when the user inputs the feedback information.

[0156] If the feedback information contains a negative keyword, if the emotion type of the feedback information is a negative emotion, or if the facial expression type when the user inputs the feedback information is a negative facial expression, it is determined that an interaction barrier exists at that point.

[0157] If an interaction barrier exists at that time, each piece of question and answer history information corresponding to the user is obtained, and question and answer history information matching the initial interaction information is searched for from each piece of question and answer history information, and a response is made based on the question and answer history information matching the initial interaction information.

[0158] According to any of the above embodiments, the apparatus further comprises a local processing unit for:

[0159] That is, the type of network condition at that time is confirmed, and when the type of network condition is a network condition deterioration or a network disconnection state, the user interaction information and the user preference information of the user are sent to a local GPT model deployed locally, thereby obtaining local response information returned from the local GPT model, and responding to the user based on the local response information.

[0160] Based on any of the above embodiments, the local GPT model is obtained by performing model quantization compression on the sample GPT model.

[0161] According to any of the above embodiments, a schematic structural diagram of an intelligent device interaction system based on the GPT model provided in the present invention is shown in Figure 6. As shown in Figure 6, the system includes a proxy layer 610 and an intelligent device side 620.

[0162] The proxy layer 610 deploys an intelligent device interaction device based on the GPT model provided in the corresponding embodiment above. The intelligent device side 620 deploys an intelligent device interaction device based on the GPT model provided in the corresponding embodiment above.

[0163] The above-described embodiment of the device is merely schematic, and the above units described as separate components may or may not be physically separated. Also, the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed among multiple network units. The objective of the solution of the embodiment can be realized by selecting some or all of the modules according to actual needs. Also, those skilled in the art can understand and implement it without any creative work.

[0164] From the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be realized by combining software with a required general-purpose hardware platform, or of course by hardware. Based on this understanding, the above technical solutions can be embodied essentially or in the form of a software product that contributes to the prior art. The computer software product can be stored in a computer-readable storage medium such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes some instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to execute the method described in each embodiment or any part of the embodiment.

[0165] Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention, and are not limiting. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art may modify the technical solutions described in the above embodiments or make equivalent replacements for some of the technical features thereof, and such modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent device interaction method based on the GPT model, the method being applied to a proxy layer, receiving user interaction information and user preference information transmitted from an intelligent device side, the user interaction information being information obtained after the intelligent device side has subjected initial interaction information input by a user to a masking process; After processing the user interaction information to obtain post-processing interaction information, the post-processing interaction information is sent to the GPT model, so that the GPT model returns task processing information of the post-processing interaction information; After receiving the task processing information returned from the GPT model, obtain interaction response information by fusing based on the user preference information and the task processing information, and transmit the interaction response information to the intelligent device side, so that the intelligent device side modifies the interaction response information based on user privacy data related to the user interaction information, and responds to the user based on the modified information. The method according to claim 1, further comprising:

2. When acquiring interaction response information by fusing based on the above-mentioned user preference information and the task processing information, specifically, performing matching based on each noun in the task processing information and each preference label in the user preference information, thereby obtaining preference labels matching each noun in the task processing information; The interaction response information is acquired by replacing the corresponding noun in the task processing information based on the label value of the preference label matched to each noun in the task processing information, and the intelligent device side, after receiving the interaction response information, acquires correction processing information by correcting the interaction response information based on user privacy data related to the user interaction information, and executes the correction processing information to realize a response to the user.

2. The intelligent device interaction method according to claim 1, comprising:

3. Specifically, when transmitting the processed interaction information to the GPT model, vectorizing the processed interaction information to obtain a processed interaction vector; searching a knowledge base for each field based on the post-processing interaction vector to identify a field knowledge and its field knowledge vector that match the post-processing interaction vector in the knowledge base for each field; Combining the processed interaction vector with the domain knowledge vector of the matched domain knowledge to obtain a fusion query vector, and sending the fusion query vector to the GPT model; 3. The method for intelligent device interaction based on the GPT model according to claim 1 or 2, comprising:

4. Specifically, when the fused query vector is sent to the GPT model, Transmitting the fused query vector to a plurality of the GPT models to obtain task processing information of the post-processing interaction information returned from the plurality of the GPT models; Acquire optimization processing information by fusing or selecting the post-processing interaction information returned from a plurality of the GPT models based on the task processing information, and the optimization processing information is used to acquire the interaction response information by fusing with the user preference information.

4. The intelligent device interaction method according to claim 3, comprising:

5. The proxy layer comprises: We build an initial model based on a pre-trained text generation model, inputting sample input information into the initial model to obtain test output information output from the initial model; obtaining the proxy layer by fine-tuning the initial model based on an evaluation of the test output information or based on a difference between the test output information and sample output information; 3. The method for intelligent device interaction based on the GPT model according to claim 1 or 2, characterized in that it is constructed based on the steps of:

6. An intelligent device interaction method based on the GPT model, which is applied to an intelligent device side, receiving initial interaction information input by a user and performing a masking process on the initial interaction information to obtain user interaction information; Send the user interaction information and the user preference information of the user to a proxy layer, so that the proxy layer processes the user interaction information to obtain post-processing interaction information, and then sends the post-processing interaction information to the GPT model, so that the GPT model returns task processing information of the post-processing interaction information; receiving interaction response information sent by the proxy layer, modifying the interaction response information based on user privacy data related to the user interaction information, and responding to the user based on the modified information, the interaction response information being obtained by fusing the user preference information and the task processing information after the proxy layer receives the task processing information of the post-processing interaction information returned from the GPT model; The method according to claim 1, further comprising:

7. When the interaction response information is modified based on the user privacy data related to the user interaction information and a response is made to the user based on the modified information, specifically, and correcting the interaction response information based on user privacy data related to the user interaction information to obtain correction processing information, and executing the correction processing information to realize a response to the user; The interaction response information is obtained by the proxy layer obtaining preference labels that match each noun in the task processing information by performing matching based on each noun in the task processing information and each preference label in the user preference information, and then replacing the corresponding noun in the task processing information based on the label value of the preference label matched to each noun in the task processing information.

8. The method further comprises: Collect initial interaction information and corresponding modified information of a plurality of users, and record the initial interaction information corresponding to the modified information that does not meet the user's expectations as a directional optimization sample; performing intent recognition on the directional optimization sample to obtain a user intent corresponding to the directional optimization sample, and setting a type label for associated text in the directional optimization sample that is associated with the user intent based on the user intent; Segmenting the related text into words, acquiring keywords of the related text, and searching a knowledge base of each field based on the keywords of the related text and the type label of the related text to acquire field knowledge matching the related text; A relevance relationship is established between the related text and the domain knowledge matched to the related text, so that when a search is performed in the knowledge base of each domain based on the post-processing interaction vector of the post-processing interaction information corresponding to the corresponding initial interaction information after the same user again inputs the same initial interaction information as the directional optimization sample, the domain knowledge matched to the related text is identified as the domain knowledge matching the corresponding post-processing interaction vector, or a response is made to the initial interaction information based on the domain knowledge matched to the related text.

8. The method for intelligent device interaction based on the GPT model according to claim 6 or 7, comprising:

9. The method further comprises: obtaining feedback information input by the user after the response and collecting an image of the user inputting the feedback information; performing emotion recognition on the feedback information to obtain an emotion type of the feedback information; acquiring a facial expression type when the user inputs the feedback information by performing facial expression recognition on an image when the user inputs the feedback information; If the feedback information includes a negative keyword, if the emotion type of the feedback information is a negative emotion, or if the facial expression type when the user inputs the feedback information is a negative facial expression, it is determined that an interaction barrier exists at that time; If an interaction barrier exists at that time, acquire question and answer history information corresponding to the user, search for question and answer history information matching the initial interaction information from the question and answer history information, and respond based on the question and answer history information matching the initial interaction information.

8. The method for intelligent device interaction based on the GPT model according to claim 6 or 7, comprising:

10. The method further comprises: The intelligent device interaction method based on the GPT model according to claim 6 or 7, characterized in that: checking the type of network condition at that time; and when the type of network condition is a network condition deterioration or a network disconnection state, sending the user interaction information and the user preference information of the user to a local GPT model deployed locally, thereby obtaining local response information returned from the local GPT model, and responding to the user based on the local response information.

11. The method for intelligent device interaction based on a GPT model according to claim 10, characterized in that the local GPT model is obtained by performing model quantization compression on a sample GPT model.

12. An intelligent device interaction method based on a GPT model, comprising: the intelligent device side receives initial interaction information input by a user and acquires user interaction information by performing a masking process on the initial interaction information; The intelligent device side sends the user interaction information and the user preference information of the user to a proxy layer; The proxy layer accepts the user interaction information and the user preference information sent from the intelligent device side, processes the user interaction information to obtain post-processing interaction information, and then sends the post-processing interaction information to the GPT model, so that the GPT model returns task processing information of the post-processing interaction information; The proxy layer receives the task processing information of the post-processing interaction information returned from the GPT model, and then obtains interaction response information by fusing the user preference information and the task processing information, and transmits the interaction response information to the intelligent device side; the intelligent device side receives the interaction response information transmitted by the proxy layer, modifies the interaction response information based on user privacy data related to the user interaction information, and responds to the user based on the modified information; The method according to claim 1, further comprising:

13. An intelligent device interaction apparatus based on the GPT model, deployed in a proxy layer, an information receiving unit for receiving user interaction information and user preference information transmitted from an intelligent device, the user interaction information being information obtained after the intelligent device has subjected initial interaction information input by a user to a masking process; a query proxy that processes the user interaction information to obtain post-processing interaction information, and then sends the post-processing interaction information to the GPT model, so that the GPT model returns task processing information of the post-processing interaction information; and a response proxy that receives the task processing information returned from the GPT model, and then obtains interaction response information by fusing the task processing information with the user preference information and the user preference information, and transmits the interaction response information to the intelligent device side, whereby the intelligent device side modifies the interaction response information based on user privacy data related to the user interaction information, and responds to the user based on the modified information.

14. An intelligent device interaction device based on the GPT model, which is deployed on the intelligent device side, an input processing unit for receiving initial interaction information input by a user and performing a masking process on the initial interaction information to obtain user interaction information; an information forwarding unit, which sends the user interaction information and the user preference information of the user to a proxy layer, so that the proxy layer processes the user interaction information to obtain post-processing interaction information, and then sends the post-processing interaction information to a GPT model, so that the GPT model returns task processing information of the post-processing interaction information; An apparatus characterized in that it includes an information response unit that receives interaction response information sent by the proxy layer, modifies the interaction response information based on user privacy data related to the user interaction information, and responds to the user based on the modified information, wherein the interaction response information is obtained by fusing the user preference information and the task processing information after the proxy layer receives the task processing information of the processed interaction information returned from the GPT model.

15. An intelligent device interaction system based on a GPT model, comprising: Including a proxy layer and an intelligent device side, The proxy layer is deployed with an interaction device for the intelligent device based on the GPT model as claimed in claim 13; The system according to claim 14, wherein an interaction device for an intelligent device based on the GPT model is deployed on the intelligent device side.

Images