Personalized voice question and answer method and device based on large model and refrigeration equipment

Abstract

The invention discloses a personalized voice question and answer method and device based on a large model and refrigeration equipment, and belongs to the technical field of refrigeration equipment. The personalized voice question-answering method based on the large model comprises the following steps: receiving first input of a user; the first input is used for inputting a user task instruction; in response to a first input, determining a target sub-scene based on the user task instruction; acquiring short-time basic information corresponding to the target sub-scene at the current acquisition moment, and acquiring identity information of the user and long-time basic information corresponding to the user task instruction; and inputting at least one of the long-term basic information and the short-term basic information and the user task instruction into the large model, and obtaining a question and answer result output by the large model. According to the personalized voice question-answering method based on the large model, the accuracy of the question-answering result is improved, the processing mode is simple, the maintenance difficulty is reduced, the method is suitable for different task scenes, the generalization ability is high, and the user experience is improved.

Description

Technical Field

[0001] The present application belongs to the technical field of refrigeration equipment, and in particular, relates to a personalized voice question-and-answer method, device and refrigeration equipment based on a large model. Background Art

[0002] With the development of science and technology, smart devices have more room for development; however, the personalization level of the intelligent question-answering module of smart devices is still relatively low. In related technologies, user personalization issues are mainly handled by building complex databases and performing complex natural language parsing. However, the above methods are difficult to build and maintain, and have poor generalization capabilities, which affects the user experience. Summary of the invention

[0003] This application aims to solve at least one of the technical problems existing in the related art. To this end, this application proposes a personalized voice question-and-answer method, device and refrigeration equipment based on a large model, which improves the accuracy of the question-and-answer results, and has a simple processing method, does not require the construction of a complex database and complex natural language processing, reduces the difficulty of maintenance, is suitable for different task scenarios, has strong generalization ability, and improves the user experience.

[0004] In a first aspect, the present application provides a personalized voice question-answering method based on a large model, which is applied to refrigeration equipment, and the method includes: Receive a first input from a user; the first input is used to input a user task instruction; In response to the first input, determining a target sub-scenario based on the user task instruction in response to the first input; Acquire short-term basic information corresponding to the target sub-scene at the current acquisition time, and acquire the identity information of the user and the long-term basic information corresponding to the user task instruction; the short-term basic information is used to characterize at least one of the short-term biometric information of the user, the interaction between the user and the refrigeration device within the target time period, the storage information in the refrigeration device, and the environment information of the refrigeration device; Input the long-term basic information and at least one of the short-term basic information and the user task instruction into the large model to obtain the question and answer result output by the large model.

[0005] According to the large model-based personalized voice question-and-answer method of the present application, by receiving user task instructions, based on the user's actual needs, long-term basic information and short-term basic information related to the needs and matching the user's identity information are obtained, and the obtained information is used as prompt words for the large model to process the user's task instructions, providing more detailed and comprehensive information for processing user tasks, and personalized processing is performed in combination with the user's actual characteristics to obtain question-and-answer results corresponding to the user instructions, thereby improving the accuracy of the question-and-answer results. The processing method is simple, and there is no need to build a complex database and perform complex natural language processing, thereby reducing the difficulty of maintenance. In addition, the obtained long-term basic information and short-term basic information can be applied to different task scenarios, have strong generalization capabilities, and improve user experience.

[0006] According to the large model-based personalized voice question-answering method of the present application, the step of obtaining the identity information of the user and the long-term basic information corresponding to the user task instruction and the short-term basic information corresponding to the current acquisition moment in the target sub-scenario includes: Obtaining the user's identity information and the long-term basic information corresponding to the user's task instruction from the long memory database; Under the target sub-scenario, semantically classify the user task instruction to obtain a semantic classification result corresponding to the user task instruction; When it is determined based on the semantic classification result that the user task instruction does not need semantic reasoning, acquiring the short-term basic information corresponding to the current collection moment from the short memory database; In the case where it is determined based on the semantic classification result that the user task instruction needs to be semantically inferred, semantically inference is performed on the user task instruction to obtain a semantic inference result; Based on the semantic reasoning result, acquiring short-term interaction data corresponding to the semantic reasoning result from interaction data between the user and the refrigeration device; Based on the semantic reasoning result and the short-term interaction data, the short-term basic information corresponding to the current collection moment is obtained.

[0007] According to the large model-based personalized voice question-answering method of the present application, before receiving the first input from the user, the method further includes: Acquire sample interaction data between the user and the refrigeration device within a first short period at the current collection moment, and environmental information of the refrigeration device within the first short period; The short memory database is updated based on the sample interaction data between the user and the refrigeration device in the first short time period, the environmental information of the refrigeration device in the first short time period, and the sample interaction data between the user and the refrigeration device in the previous short time period corresponding to the first short time period.

[0008] According to the large model-based personalized voice question-answering method of the present application, the updating of the short memory database based on the sample interaction data between the user and the refrigeration device in the first short time period, the environmental information of the refrigeration device in the first short time period, and the sample interaction data between the user and the refrigeration device in the previous short time period corresponding to the first short time period includes: Performing semantic classification on the sample interaction data between the user and the refrigeration device in the first short period of time to obtain a sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration device; processing the sample interaction data between the user and the refrigeration device within the first short period based on the sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration device and the target time; A short-term information summary is performed based on the processed interaction data between the user and the refrigeration device in the first short-time period, the environmental information of the refrigeration device in the first short-time period, and the interaction data between the user and the refrigeration device in the previous short-time period to update the short memory database.

[0009] According to the large model-based personalized voice question-answering method of the present application, the sample interaction data between the user and the refrigeration device in the first short period is processed based on the sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration device and the target time, including: When it is determined based on the sample semantic classification result that no semantic reasoning is required for the sample interaction data between the user and the refrigeration device, retaining the sample interaction data between the user and the refrigeration device within the first short period of time; In the case where it is determined based on the sample semantic classification result that semantic reasoning is required for the sample interaction data between the user and the refrigeration device, semantic reasoning is performed on the sample interaction data between the user and the refrigeration device within a target time period to obtain a sample reasoning result; the target time period is a time period determined based on the target moment and the first short-time period; Based on the sample reasoning result, acquiring sample short-term interaction data corresponding to the sample reasoning result from the sample interaction data between the user and the refrigeration device within the target time period; Based on the sample reasoning result and the sample short-time interaction data, the sample interaction data between the user and the refrigeration device in the first short-time period is updated.

[0010] According to the large model-based personalized voice question-answering method of the present application, before receiving the first input from the user, the method further includes: A long memory database is constructed based on the long-term biometric information of the user and the basic information of the user.

[0011] According to the large model-based personalized voice question-answering method of the present application, the long memory database is constructed based on the long-term biometric information of the user and the basic information of the user, including: Obtain at least one user corpus; Constructing an association relationship between at least one of the long-term biometric information of the user and the basic information of the user and the user corpus; The association relationship between the user's long-term biometric information, the user's basic information, and at least one of the long-term biometric information and the basic information and the user corpus is stored in the long memory database.

[0012] In a second aspect, the present application provides a personalized voice question-answering device based on a large model, which is applied to refrigeration equipment, and the device includes: A first processing module, configured to receive a first input from a user; the first input is used to input a user task instruction; A second processing module, configured to determine a target sub-scenario in response to the first input and based on the user task instruction; A third processing module is used to obtain short-term basic information corresponding to the target sub-scene at the current acquisition time, and to obtain the identity information of the user and the long-term basic information corresponding to the user task instruction; the short-term basic information is used to characterize at least one of the short-term biometric information of the user, the interaction between the user and the refrigeration device within the target time period, the storage information in the refrigeration device, and the environmental information of the refrigeration device; The fourth processing module is used to input the long-term basic information and at least one of the short-term basic information and the user task instruction into the large model to obtain the question and answer result output by the large model.

[0013] According to the large model-based personalized voice question-and-answer device of the present application, by receiving user task instructions, based on the actual needs of the user, long-term basic information and short-term basic information related to the needs and matching the identity information of the user are obtained, and the obtained information is used as prompt words for the large model to process the user task instructions, providing more detailed and comprehensive information for processing user tasks, and personalized processing is performed in combination with the actual characteristics of the user to obtain question-and-answer results corresponding to the user instructions, thereby improving the accuracy of the question-and-answer results, and the processing method is simple, without the need to build a complex database and perform complex natural language processing, thereby reducing the difficulty of maintenance. In addition, the obtained long-term basic information and short-term basic information can be applied to different task scenarios, with strong generalization capabilities, thereby improving the user experience.

[0014] In a third aspect, the present application provides a refrigeration device, comprising: A personalized voice question-answering device based on a large model as described in the second aspect.

[0015] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the large model-based personalized voice question-answering method as described in the first aspect above.

[0016] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the large model-based personalized voice question-answering method as described in the first aspect above.

[0017] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects: By receiving user task instructions, based on the user's actual needs, the long-term basic information and short-term basic information related to the needs and matching the user's identity information are obtained, and the obtained information is used as prompt words for the large model to process the user's task instructions, providing more detailed and comprehensive information for processing user tasks. Combined with the user's actual characteristics, personalized processing is performed to obtain question and answer results corresponding to the user instructions, thereby improving the accuracy of the question and answer results. The processing method is simple, and there is no need to build a complex database and perform complex natural language processing, which reduces the difficulty of maintenance. In addition, the obtained long-term basic information and short-term basic information can be applied to different task scenarios, with strong generalization ability, thereby improving the user experience.

[0018] Furthermore, by obtaining long-term basic information from a pre-built long-memory database, and based on the actual situation of whether the user task instruction requires semantic reasoning, different methods are selected to obtain short-term basic information. When the user task instruction requires reasoning, the short-term basic information is determined through the semantic reasoning results and interaction data. When the user task instruction does not require reasoning, the short-term basic information is obtained from the short-memory database. Different short-term basic information acquisition methods are set for different user task instructions to improve the accuracy of the acquired short-term basic information, and only information related to the task instruction is obtained. This can handle the personalized needs of users and reduce the complexity of task processing.

[0019] Furthermore, by storing the user's long-term biometric information and basic information, the background information of different users is effectively stored, enriching the data in the long memory database, providing comprehensive background information for subsequent voice questions and answers, and improving the user experience.

[0020] Furthermore, by acquiring multiple user corpora and establishing an association relationship between the user's long-term biometric information and one or more of the user's basic information and the user corpora, on the basis of storing the user's long-term basic information, the association relationship between each category of data in each long-term basic information and the user corpus is further stored, thereby improving the accuracy of acquiring long-term basic information based on user task instructions.

[0021] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which: Figure 1 This is one of the flow charts of the personalized voice question-answering method based on a large model provided in an embodiment of the present application; Figure 2 This is the second flow chart of the personalized voice question-answering method based on a large model provided in an embodiment of the present application; Figure 3 This is the third flow chart of the personalized voice question-answering method based on a large model provided in the embodiment of the present application; Figure 4 This is the fourth flow chart of the personalized voice question-answering method based on a large model provided in an embodiment of the present application; Figure 5 This is the fifth flow chart of the personalized voice question-answering method based on a large model provided in the embodiment of the present application; Figure 6This is one of the principle schematic diagrams of the large-model-based personalized voice question-answering method provided in the embodiment of the present application; Figure 7 This is the sixth flow chart of the personalized voice question-answering method based on a large model provided in an embodiment of the present application; Figure 8 This is the second principle schematic diagram of the large model-based personalized voice question-answering method provided in the embodiment of the present application; Fig. 9 This is the third principle schematic diagram of the large model-based personalized voice question-answering method provided in the embodiment of the present application; Fig.10 This is the fourth principle schematic diagram of the large model-based personalized voice question-answering method provided in the embodiment of the present application; Fig.11 This is the seventh flow chart of the personalized voice question-answering method based on a large model provided in the embodiment of the present application; Fig.12 This is the eighth flow chart of the personalized voice question-answering method based on a large model provided in an embodiment of the present application; Fig.13 This is the ninth flow chart of the personalized voice question-answering method based on a large model provided in an embodiment of the present application; Fig.14 This is the tenth flowchart of the large model-based personalized voice question-answering method provided in the embodiment of the present application; Fig.15 This is the eleventh flow chart of the personalized voice question-answering method based on a large model provided in an embodiment of the present application; Fig.16 It is a structural diagram of a personalized voice question-and-answer device based on a large model provided in an embodiment of the present application; Fig.17 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0024] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0025] In combination with the accompanying drawings, the personalized voice question and answer method based on a large model, the personalized voice question and answer device based on a large model, the refrigeration equipment and the readable storage medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0026] Among them, the personalized voice question and answer method based on the large model can be applied to the terminal, and can be specifically executed by the hardware or software in the terminal.

[0027] The terminal includes but is not limited to portable communication devices such as mobile phones or tablet computers. It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer.

[0028] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.

[0029] The large model-based personalized voice question-and-answer method provided in the embodiment of the present application can be executed by a refrigeration device, such as a single-chip microcomputer; in some embodiments, it can also be executed by a server connected to the refrigeration device, or jointly executed by a server and a single-chip microcomputer, etc.

[0030] The embodiment of the present application provides a personalized voice question and answer method based on a big model. The execution subject of the personalized voice question and answer method based on a big model can be a refrigeration device, or an electronic device that is communicatively connected to the refrigeration device, or a functional module or functional entity in the electronic device that can implement the personalized voice question and answer method based on a big model. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The personalized voice question and answer method based on a big model provided in the embodiment of the present application is described below using an electronic device as an example of the execution subject.

[0031] like Figure 1As shown, the large model-based personalized voice question-answering method includes: step 110, step 120 and step 130.

[0032] The personalized voice question-answering method based on large models can be applied to refrigeration equipment.

[0033] The refrigeration equipment can be understood as refrigeration storage equipment in a broad sense, including but not limited to refrigerators, freezers, display cabinets, beverage cabinets, wine cabinets, fresh-keeping cabinets, refrigerated vending machines and other refrigeration storage equipment. The refrigeration equipment has various structural forms and a wide range of applications.

[0034] Step 110: receiving a first input from a user; In this step, the first input is used to input a user task instruction.

[0035] User task instructions are instructions sent by users to refrigeration equipment.

[0036] The first input may be in at least one of the following ways: First, the first input may be a touch operation, including but not limited to a click operation, a slide operation, and a press operation.

[0037] In this implementation, receiving the first input from the user may be receiving a touch operation of the user in the display area of ​​the terminal display screen.

[0038] In order to reduce the user error operation rate, the effective area of ​​the first input can be limited to a specific area, such as the upper middle area of ​​the question-and-answer interface; or when the question-and-answer interface is displayed, the target control is displayed on the current interface, and the first input can be achieved by touching the target control; or the first input can be set to a continuous multiple tapping operation on the display area within a target time interval.

[0039] Secondly, the first input may be a physical key input.

[0040] In this embodiment, a physical button corresponding to the question and answer is provided on the terminal body, and receiving the user's first input can be receiving the operation of the user pressing the corresponding physical button; the first input can also be a combined operation of pressing multiple physical buttons at the same time.

[0041] Third, the first input may be voice input.

[0042] In this implementation, when the terminal receives a voice such as "roast duck", it can trigger the broadcasting of the corresponding cooking method of roast duck.

[0043] Of course, in other embodiments, the first input may also be in other forms, including but not limited to character input, etc., which can be determined according to actual needs and is not limited in this embodiment of the present application.

[0044] It is understandable that in different application scenarios, user task instructions may be different.

[0045] For example, when the user needs to cook a dish, the user task instruction may be a recipe for dish X.

[0046] For another example, when the user needs to know the food stored in the refrigeration equipment, the user task instruction may be "What other fruits are there in the refrigerator?".

[0047] In some embodiments, the user may input user task instructions through a display device disposed on the refrigeration equipment.

[0048] In other embodiments, the user may also input user task instructions through a voice recognition device.

[0049] The specific input method of the user task instruction can be determined based on the actual situation and is not limited in this application.

[0050] Step 120: In response to the first input, determine a target sub-scenario based on a user task instruction; In this step, the target sub-scenario is a specific task scenario corresponding to the user's task instruction.

[0051] For example, the target sub-scenario may be to query the recipe of a specific dish.

[0052] For another example, the target sub-scenario may be to query the food stored in a certain area of ​​the refrigeration equipment.

[0053] In the actual execution process, natural language recognition can be performed based on user task instructions to determine the target sub-scenario.

[0054] Of course, in the actual implementation process, the target sub-scenario can be determined in any feasible way, and this application does not limit it.

[0055] like Figure 2 As shown, in some embodiments, determining the target sub-scene based on the user task instruction may also include: Determine the target scenario based on user task instructions; Combine the user's task instructions and target scenes to obtain task scene information; The mission scenario information is input into the target large model, and the target sub-scenario output by the target large model is obtained.

[0056] In this embodiment, the target scenario is a broad category of task scenarios corresponding to the user's task instructions.

[0057] Target scenarios may include: food management, health management, knowledge Q&A, and recipe recommendations.

[0058] The target sub-scene can be any sub-scene in the target scene.

[0059] The task scenario information is the information required to determine the target sub-scenario corresponding to the user's task instruction.

[0060] The target big model can be a generative big model.

[0061] Generative large models are deep learning models with the number of parameters on the order of billions or more.

[0062] Continue to refer Figure 2 ,In the actual execution process, after obtaining the user task ,instructions, the user task instructions can be sent to the scenario ,distribution module, which includes various types of scenarios, such as ,recipe recommendation scenarios, user health management scenarios, and user ,daily chat scenarios.

[0063] The scene distribution module can be a classification network model such as a neural network and machine learning.

[0064] The specific model of the scene distribution module can be determined based on actual conditions and is not limited in this application.

[0065] The scene distribution module can output the target scene based on the user task instructions.

[0066] Continue to refer Figure 2 ,The scene distribution module can convert the format of the content corresponding to the user task instructions.

[0067] After receiving the user task instruction input by the user and determining the target scene, the user task instruction can be spliced ​​with the prompt project template to obtain the field "content: user task instruction, field: target scene", that is, the task scene information.

[0068] In the actual execution process, after obtaining the task scenario information, the task scenario information can be input into the target large model to obtain the target sub-scenario output by the target large model.

[0069] According to the large model-based personalized voice question-and-answer method provided in the embodiment of the present application, a relatively broad large-category task scenario corresponding to the user task instruction is first determined through the user task instruction, and based on the large-category task scenario and the user task instruction, the specific task scenario under the large-category task scenario is further determined. Through layer-by-layer analysis, the difficulty of determining the target sub-scenario based on the user task instruction is reduced, and the determination efficiency and accuracy are improved.

[0070] During the actual execution process, after receiving the user task instruction, in response to the user task instruction, natural language recognition is performed on the user task instruction, keywords in the user task instruction are extracted, the target sub-scene corresponding to the user task instruction is determined, and the user's identity information and the long-term basic information corresponding to the user task instruction and the short-term basic information corresponding to the target sub-scene at the current acquisition moment are obtained.

[0071] Step 130: Acquire short-term basic information corresponding to the target sub-scene at the current acquisition moment, and acquire the user's identity information and long-term basic information corresponding to the user's task instruction; In this embodiment, the identity information is information that can distinguish different user identities.

[0072] The identity information can be obtained by the user actively providing it to the refrigeration device, or by the refrigeration device performing facial recognition on the user or by recognizing the voice of the user's input of a user task instruction.

[0073] For example, in the actual execution process, after the user inputs the user task instruction, a page may pop up on the display device of the refrigeration equipment for the user to input identity information, such as user name or user nickname.

[0074] For another example, during the actual execution process, after the user inputs a user task instruction, the refrigeration equipment can perform facial recognition on the user through an image sensor to obtain the user's identity information.

[0075] The long-term basic information is information that does not change over a long period of time for users interacting with the refrigeration equipment.

[0076] Long-term basic information can be user portrait information.

[0077] Long-term basic information includes, but is not limited to: user name, user family members, user preferences, user long-term biometric information, user city, user gender, and events experienced by the user.

[0078] The short-term basic information is information that may change in a short period of time for a user interacting with the refrigeration device and is related to an application scenario corresponding to the user's task instruction.

[0079] It should be noted that the long-term basic information and short-term basic information required for different target sub-scenarios may be different.

[0080] The short-term basic information may be temporary information.

[0081] The short-term basic information is used to represent at least one of the user's short-term biometric information, the interaction between the user and the refrigeration device during the target time period, the storage information in the refrigeration device, and the environment information of the refrigeration device. The short-term biometric information is the user's physical health status on that day.

[0082] For example, short-term biometric information may indicate that the user has a fever today and the body temperature is 38 degrees Celsius.

[0083] Short-term biometric information can also include user-defined feature information, such as user dietary preferences and health management goals.

[0084] For example, the user's dietary preference may be that the user wants to eat a light diet today.

[0085] For example, the health management goal can be to ensure the current intake of protein and vitamins.

[0086] The target time period is a preset time period.

[0087] The specific duration of the target time period can be based on user customization or determined based on actual conditions. For example, the target time period can be 1 day or half a day, etc., which is not limited in this application.

[0088] The interaction situation between the user and the refrigeration device within the target time period is the eating situation of the user within the target time period determined based on the interaction data between the user and the refrigeration device within the target time period.

[0089] For example, when the user takes out apples, chocolate, green vegetables, and pork today, it is determined that the user has eaten apples, chocolate, green vegetables, and pork today.

[0090] For another example, when the user puts in potatoes, onions, chicken legs and beef brisket today, it is determined that the user tends to eat potatoes, onions, chicken legs and beef brisket today.

[0091] The storage information in the refrigeration device is information related to the storage object stored in the refrigeration device.

[0092] The storage information may include: the name of the storage object and the storage duration of the storage object.

[0093] The environmental information is the environmental information of the location where the refrigeration equipment is located.

[0094] Environmental information includes, but is not limited to, season information, weather information, and temperature information.

[0095] It should be noted that the short-term basic information may change in accordance with different application scenarios.

[0096] For example, when a user wants to make a recipe recommendation, the user's physical condition today (ie, short-term biometric information) and the ingredients stored in the refrigeration device (ie, storage information) can be obtained.

[0097] For another example, when a user needs to manage food, for example, the user queries how many new apples are put in today, the user's interaction with the refrigeration equipment today can be obtained. Short-term basic information In the same application scenario, the short-term basic information of the same user may also change.

[0098] Continuing with the example of a user wanting to make a recipe recommendation, the short-term biometric information obtained from the user today may indicate that he or she has a cold, and the characteristics of the user's cold need to be considered when making recipe recommendations. After a week, the user's short-term biometric information may change to indicate that he or she is asymptomatic and in good health. At this time, when recommending recipes, compared with previously recommended recipes, we do not need to consider whether the user has a cold.

[0099] During the actual execution process, the user's long-term basic information and short-term basic information may be obtained based on the actual situation of the user's task instruction and the user's identity information.

[0100] In the actual implementation process, the user's long-term basic information and short-term basic information can be obtained in any feasible way.

[0101] For example, the specific content of the long-term basic information and the specific content of the short-term basic information to be acquired are displayed on a display device of the refrigeration equipment.

[0102] In other embodiments, after receiving the user task instruction, the refrigeration device may actively inquire the user based on the specific content of the user task instruction, and ask the user to answer corresponding questions to obtain the long-term basic information and the short-term basic information.

[0103] For another example, a temperature sensor may be provided on the outside of the refrigeration equipment to collect temperature information of the environment in which the refrigeration equipment is located; the refrigeration equipment may also be connected to a network to retrieve information about the environment in which the refrigeration equipment is located.

[0104] Step 130: input at least one of the long-term basic information and the short-term basic information and the user task instruction into the large model to obtain the question and answer result output by the large model.

[0105] In this step, the question and answer result is the question and answer result corresponding to the user's task instruction.

[0106] During the actual execution process, the user's task instructions and the specific circumstances of the acquired long-term basic information and short-term basic information can be input into the big model, and the big model analyzes and processes the obtained data to output the question and answer results corresponding to the user's task instructions.

[0107] It is understandable that the acquired long-term basic information or short-term basic information may be empty. In the process of processing the user task instruction, the user task instruction can be processed based on the actual situation of the acquired information to output the question and answer results.

[0108] Taking the user task instruction of "Recommend what is suitable to eat today?" as an example, at this time, the user's long-term basic information, that is, the user's preferences, the user's long-term biometric information, and the user's gender, etc. can be obtained based on the user's task instruction, and suitable dishes can be recommended to the user based on the user's long-term biometric information, user preferences, and user gender.

[0109] For example, if the user's long-term biometric information shows that he or she has diabetes, foods high in sugar should be avoided during the recommendation process.

[0110] In some embodiments, food may be recommended to the user further based on the user's short-term biometric information.

[0111] For example, if the user has a sore throat today, we should try to recommend light food to the user and reduce the recommendation of spicy food.

[0112] In some embodiments, food may be recommended to the user based on the environment information of the refrigeration device.

[0113] For example, when environmental information shows that the weather today is hot and the temperature is high, green bean soup, ice jelly and other heat-relief foods can be recommended to users.

[0114] In the actual execution process, after receiving the user task instruction, the target sub-scenario is determined by performing natural language analysis on the user task instruction. Based on the user's identity information and the user task instruction, the long-term basic information and short-term basic information to be obtained are limited. The short-term basic information to be obtained is further limited through the target sub-scenario, and the short-term basic information related to the target sub-scenario at the current collection time is obtained to obtain the newer short-term basic information that may change and is related to the execution of the user task instruction, so as to ensure the timeliness of the obtained short-term basic information. Based on the understanding of the overall characteristics of the user through the long-term basic information, the user's preferences at the current collection time are further understood based on the newer short-term basic information, so as to personalize the processing of the user task instruction, so that the question and answer results output by the large model based on the long-term basic information and the short-term basic information can better meet the user's usage needs at the current collection time, improve the accuracy of the output questions, and improve the user experience.

[0115] According to the large model-based personalized voice question-and-answer method provided in the embodiment of the present application, by receiving user task instructions, based on the actual needs of the user, long-term basic information and short-term basic information related to the needs and matching the identity information of the user are obtained, and the obtained information is used as prompt words for the large model to process the user task instructions, providing more detailed and comprehensive information for processing user tasks, and personalized processing is performed in combination with the actual characteristics of the user to obtain question-and-answer results corresponding to the user instructions, thereby improving the accuracy of the question-and-answer results, and the processing method is simple, without the need to build a complex database and perform complex natural language processing, thereby reducing the difficulty of maintenance. In addition, the obtained long-term basic information and short-term basic information can be applied to different task scenarios, have strong generalization capabilities, and enhance user experience.

[0116] In some embodiments, obtaining the user's identity information and the long-term basic information corresponding to the user's task instruction and the short-term basic information corresponding to the target sub-scene at the current acquisition time may include: Obtain the user's identity information and the long-term basic information corresponding to the user's task instructions from the long memory database; Under the target sub-scenario, semantic classification is performed on the user task instructions to obtain the semantic classification results corresponding to the user task instructions; When it is determined based on the semantic classification result that the user task instruction does not need semantic reasoning, the corresponding short-term basic information at the current collection time is obtained from the short memory database; When it is determined based on the semantic classification result that semantic reasoning is required for the user task instruction, semantic reasoning is performed on the user task instruction to obtain a semantic reasoning result; Based on the semantic reasoning result, short-term interaction data corresponding to the semantic reasoning result is obtained from the interaction data between the user and the refrigeration device; Based on the semantic reasoning results and short-term interaction data, the corresponding short-term basic information at the current collection moment is obtained.

[0117] In this embodiment, the long memory database is a database related to long-term basic information that is constructed based on continuous iteration of identity information of different categories of users.

[0118] The long memory database is used to store data related to user profile information.

[0119] Different categories of identity information include, but are not limited to: user name, user family members, user preferences, user long-term biometric information, user city, user gender, and events experienced by the user.

[0120] It is understandable that the long memory database can store user portrait information of multiple users.

[0121] The short memory database is a database related to short-term basic information that is constructed based on the continuous iteration of the interaction data between the user and the refrigeration equipment.

[0122] The short memory database is used to store temporary information other than user profiles.

[0123] The long memory database and the short memory database may be dynamically updated databases.

[0124] In the actual execution process, after receiving the user task instruction, the user's identity information can be determined first, and based on the identity information and the user task instruction, the target is retrieved from the long memory database to obtain the long-term basic information required for processing the user task instruction.

[0125] Taking the user task instruction of "what should I eat to be good for my health" as an example, the long-term basic information of the user issuing the instruction can be determined from the long-term basic information of multiple users based on the identity information of the user issuing the instruction, and based on the user's long-term biometric information required for "what should I eat to be good for my health" as an example, the required long-term basic information can be determined, such as the user is overweight, has three highs, and has poor gastrointestinal health.

[0126] The semantic classification result is the classification result obtained by determining whether the user task instruction requires semantic reasoning in the target sub-scenario.

[0127] The semantic classification results include: semantic reasoning is required or not required.

[0128] In the actual execution process, the user task instructions can be input into the classification model to obtain the semantic classification results.

[0129] Taking the user task instruction "What did I take from the refrigerator at 10 o'clock today" as an example, "What did I take from the refrigerator at 10 o'clock today" is a relatively intuitive instruction. The refrigeration equipment can directly determine the question and answer result based on the time recorded in the log and the items taken out by the user. At this time, the semantic classification result corresponding to "What did I take from the refrigerator at 10 o'clock today" can be obtained without the need for semantic reasoning.

[0130] When user task instructions do not require semantic reasoning, the corresponding short-term basic information at the current collection moment can be directly obtained from the pre-established short memory database.

[0131] Taking the user task instruction "What fruit did I eat today" as an example, in the actual execution process, the refrigeration equipment cannot accurately understand what the user ate today, but it can perform semantic reasoning on "What fruit did I eat today" and infer "What fruit did the user take out of the refrigerator today" as the user task instruction corresponding to "What fruit did I eat today". At this time, the semantic classification result corresponding to "What fruit did I eat today" can be required for semantic reasoning.

[0132] The short-term interaction data is interaction data related to the user's task instructions extracted from the interaction data between the user and the refrigeration device.

[0133] Taking the items taken out by the user today including apples, bananas, potatoes and eggplants as an example, the short-term interaction data is the apples and bananas determined from apples, bananas, potatoes and eggplants based on "what fruits the user took out from the refrigerator today".

[0134] In the actual execution process, the semantic reasoning results and the short-term interaction data can be spliced ​​together to obtain the corresponding short-term basic information at the current collection time.

[0135] like Figure 3 As shown, in the actual execution process, after the target sub-scene is determined, the target sub-scene can be input into a processing module corresponding to the target sub-scene to process the user task instruction.

[0136] It is understandable that, when the user is a new user, that is, before the current collection time, the user who inputs the user task instruction has not generated any interaction data with the refrigeration device, the retrieved long-term basic information may be empty. In this case, the search result may output None.

[0137] like Figure 4 As shown, when the user task instruction requires reasoning, the user task instruction can be input into the reasoning module to obtain the semantic reasoning result corresponding to the user instruction.

[0138] Continue to refer Figure 4 , based on the user task instructions, the interaction data related to the semantic reasoning results are obtained from the log storing the interaction data between the user and the refrigeration equipment, and the interaction data is preprocessed, and the interaction data is input into the natural language processing parsing model to obtain the parsing results, and a reasoning prompt engineering template is designed, and the parsing results are input into the reasoning prompt engineering template. In the future, the question and answer results can be obtained based on the filled reasoning prompt engineering template.

[0139] Based on the content of the reasoning prompt engineering template, we can extract apples and bananas from the interaction data between the user and the refrigeration equipment, such as the actual situation of the ingredients taken out, that is, apples, bananas, potatoes and eggplants, and get "Please list the {fruits} taken out by the user: apples and bananas", and obtain the short-term basic information required to process the user's task instructions.

[0140] It should be noted that, in this application scenario, the long-term basic information queried based on the user task instruction may be empty.

[0141] Continue to refer Figure 4 In the actual execution process, based on the results of semantic reasoning, a prompt engineering template for rewriting user task instructions can be designed, key information in the user task instructions can be extracted, and the extracted key information can be filled into the designed prompt engineering template. The prompt engineering template filled with key information can be filled into the prompt engineering template of the input large model. The prompt engineering template filled with key information is used as the user task instruction, and the long-term basic information and short-term basic information can be obtained based on the user task instruction.

[0142] Taking the user task instruction "What fruits did I eat today" as an example, a template related to the semantic reasoning result can be designed. Some of the storage objects taken out of the refrigeration equipment today may be the question and answer results of "What fruits did I eat today".

[0143] All storage objects taken out by the user from the refrigeration device today may be stored in the interaction log of the refrigeration device in the form of: the user takes out food: apple, banana, potato and eggplant.

[0144] Based on the storage form of storage objects in the interaction log of the refrigeration equipment, a prompt engineering template for users to retrieve storage objects can be designed, such as please list the {slot} retrieved by the user.

[0145] In the actual execution process, the key information in the user's task instruction, that is, the fruit, can be extracted, and the fruit can be filled into the template to obtain "Please list the {fruits} taken out by the user".

[0146] "Please list the {fruits} taken out by the user" can be used as the content that needs to be filled in the slot corresponding to the user task instruction in the prompt engineering template input to the large model.

[0147] Based on “Please list the {fruits} taken out by the user”, obtain the corresponding short-term basic information at the current collection moment.

[0148] Continue to refer Figure 3 ,When the user task instructions do not require semantic reasoning, the long-term basic information and ,short-term basic information can be directly obtained from the long-memory database and the short-memory database.

[0149] The acquired long-term basic information and short-term basic information are filled into the prompt engineering template, and the filled prompt engineering template is input into the big model to obtain the question and answer results output by the big model.

[0150] Continue to refer Figure 3 In some embodiments, a prompt engineering template may be set corresponding to the target sub-scene, and the prompt engineering template may include user task instructions. In addition, the acquired long-term basic information and short-term basic information may be stored in corresponding positions in the prompt engineering template.

[0151] For example, long-term basic information is filled into the user portrait information position in the prompt engineering template, short-term basic information is filled into the temporary information position, and the prompt engineering template filled with prompt information is input into the target large model to obtain the question and answer results output by the target large model.

[0152] In some embodiments, the prompt project template may also include specific task requirements and contextual information.

[0153] In this embodiment, the specific task information is background information that needs to be provided to the large model in the target sub-scene.

[0154] The specific task information may include requirements such as role positioning of the target large model and length limitation of the output content.

[0155] For example, the specific task information may be that the length of the output content does not exceed 100 words.

[0156] In some embodiments, the long-term basic information related to the user's task instructions can be retrieved from the long memory database through a retrieval enhancement generation method.

[0157] In this embodiment, a Retrieval-Augmented Generation (RAG) method provides the large language model with information retrieved from a database, and uses the retrieved information as a basis for the large model to generate an answer.

[0158] Retrieval-augmented generation can include two stages: retrieving context-relevant information and using the retrieved knowledge to guide the generation process.

[0159] The RAG method uses the information found by the retrieval algorithm as context to help the big model answer user task instructions; both the query and the retrieved context are injected into the prompt engineering template sent to the target big model.

[0160] Taking the user task instruction of "Using the ingredients in my refrigerator, recommend me what to eat for dinner" as an example, the user task instruction is input into the scenario distribution module, the target scenario is determined to be recipe recommendation, the user task instruction is distributed to the recipe recommendation scenario, the actual needs of the user task instruction are determined, and based on the actual needs, long-term basic information is obtained from the long memory database, and short-term basic information is obtained from the short memory database.

[0161] The specific process of obtaining long-term basic information from the long memory database is: The user task instruction "Use the ingredients in my refrigerator to recommend me what to eat for dinner" is input into the retrieval enhancement generation module to retrieve long-term basic information from the long memory database.

[0162] At this point, the retrieved knowledge fragments may include: {document: "Recommend dinner for me"; metadata: "User health status: the user has chronic hypertension and high uric acid"}; {document: "Recommend a recipe for me"; metadata: "User location: District B, City A"}; {document: "Recommended dinner"; metadata: "Basic user information: Gender: Male; Age: 65 years old"}.

[0163] The retrieved long-term basic information can be filled in the position corresponding to the long-term basic information of the prompt project template.

[0164] Obtaining short-term basic information from the short memory database may include: {Users added new ingredients today. Please use fresh ingredients first when recommending recipes. Ingredient information: beef brisket}; {The user ate carrots today. Please consider the nutritional content of the above ingredients and do not recommend foods with the same nutritional value repeatedly}; {The user has the following ingredients in the refrigerator: potatoes, onions, chicken legs, beef brisket, eggs, cabbage, beans, green vegetables and pork. Please strictly follow the ingredients in the refrigerator for recommendations}; {It is autumn now, please choose seasonal recipe recommendations according to the season}.

[0165] The obtained short-term basic information can be filled in the position corresponding to the short-term basic information of the prompt project template.

[0166] After the long-term basic information and the short-term basic information are filled into the prompt project template, the requirements for processing the user task instructions and the context information and other contents may be further added to obtain the filled prompt project template.

[0167] After filling in all the information in the prompt project template, the prompt information obtained may be: “Please recommend ingredients to users based on their basic conditions in their long-term basic information and their daily conditions in their short-term basic information.

[0168] Requirements: Refer to as much background information as possible, and it is strictly forbidden to create without background information.

[0169] Long-term basic information: User basic information: Gender: Male; Age: 65 years old; User location: Xicheng District, Beijing; User health status: The user has chronic hypertension and high uric acid; Short basic information: The user added new ingredients today. Please use fresh ingredients first when recommending recipes. Ingredient information: beef brisket; The user ate carrots today. Please consider the nutritional content of the above ingredients and do not recommend foods with the same nutritional value repeatedly. The user has the following ingredients in the refrigerator: potatoes, onions, chicken legs, beef brisket, eggs, cabbage, beans, green vegetables and pork. Please strictly follow the ingredients in the refrigerator for recommendations; Now is autumn, please choose seasonal recipe recommendations according to the season.

[0170] User task instruction: Recommend me what to eat for dinner using the ingredients in my refrigerator.

[0171] The above prompt information is input into the big model. Based on the prompt information, the big model processes the user task instructions and outputs the question and answer results.

[0172] According to the large model-based personalized voice question-and-answer method provided in the embodiment of the present application, long-term basic information is obtained from a pre-constructed long-memory database, and different methods are selected to obtain short-term basic information based on the actual situation of whether the user task instruction requires semantic reasoning. When the user task instruction requires reasoning, the short-term basic information is determined through the semantic reasoning results and interaction data. When the user task instruction does not require reasoning, the short-term basic information is obtained from the short-memory database. Different short-term basic information acquisition methods are set for different user task instructions to improve the accuracy of the acquired short-term basic information, and only information related to the task instruction is obtained. This can handle the user's personalized needs and reduce the complexity of task processing.

[0173] In some embodiments, before step 110, the method may further include: Acquire sample interaction data between the user and the refrigeration device in a first short period at the current collection moment, and environmental information of the refrigeration device in the first short period; The short memory database is updated based on the sample interaction data between the user and the refrigeration device in the first short time period, the environmental information of the refrigeration device in the first short time period, and the sample interaction data between the user and the refrigeration device in the previous short time period corresponding to the first short time period.

[0174] In this embodiment, the first short period is a period corresponding to the current acquisition moment.

[0175] The specific duration of the first short time period can be based on user customization or determined based on actual conditions. For example, the first short time period can be 1 hour or 2 hours, etc., which is not limited in this application.

[0176] The previous short time period is the previous short time period closest to the first short time period.

[0177] For example, when the first short time period is a time period corresponding to 9 am to 10 am, the previous short time period is a time period corresponding to 8 am to 9 am.

[0178] Continue to refer Figure 5 Taking the first short-time period of 1 hour as an example, in the actual execution process, the sample interaction data between the user and the refrigeration equipment in the first short-time period of the current collection time can be obtained by reading the user's interaction log in that hour.

[0179] In some embodiments, in the process of reading the interaction log of the hour, the user's interaction log may be integrated into a more reasonable format, such as "interaction object: user A, interaction content: Y".

[0180] In some embodiments, after obtaining the user's sample interaction data in the first short period and the sample interaction data in the previous short period, data cleaning may be performed on the sample interaction data to delete abnormal conversation content in the interaction log.

[0181] In this embodiment, the abnormal conversation content may be sample interaction data such as speech misrecognition and false wake-up.

[0182] Abnormal conversation content may also include noisy interactions and repeated interactions.

[0183] During the actual execution process, a set of sample interaction data can be deleted, but sample interaction data cannot be added or the content of the sample interaction data cannot be rewritten.

[0184] During the actual execution process, after obtaining the sample interaction data of the first short period, the sample interaction data of the first short period can be fused with the sample interaction data of the previous short period to obtain new sample interaction data of the first short period, and the short memory database is updated based on the new sample interaction data of the first short period and the environmental information of the refrigeration equipment in the first short period.

[0185] According to the large model-based personalized voice question-and-answer method provided in the embodiment of the present application, by acquiring the sample interaction data between the refrigeration equipment and the first short-time period of the current collection moment, the latest sample interaction data can be acquired, so as to obtain new sample interaction data of the first short-time period based on the latest sample interaction data and the sample interaction data of the previous short-time period, and update the short memory database based on the new sample interaction data of the first short-time period and the environmental information of the refrigeration equipment in the first short-time period, without storing the sample interaction data of each period, reducing content redundancy, so as to construct a concise short memory database with a small amount of data.

[0186] In some embodiments, based on the sample interaction data between the user and the refrigeration device in the first short period, the environment information of the refrigeration device in the first short period, and the sample interaction data between the user and the refrigeration device in the previous short period corresponding to the first short period, updating the short memory database includes: Performing semantic classification on the sample interaction data between the user and the refrigeration device in the first short period of time to obtain a sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration device; Based on the sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration device and the target time, processing the sample interaction data between the user and the refrigeration device within the first short period; Based on the processed interaction data between the user and the refrigeration device in the first short period, the environmental information of the refrigeration device in the first short period, and the interaction data between the user and the refrigeration device in the previous short period, a short-term information summary is performed to update the short memory database.

[0187] In this embodiment, the sample interaction data between the user and the refrigeration device in the first short period of time may be text generated by the refrigeration device based on the user's interaction actions, or may be voice interaction information between the user and the refrigeration device.

[0188] For example, when a user directly takes out two apples from the refrigeration device, the interaction log of the refrigeration device may record that at D time E minute on C day, user X takes out two apples.

[0189] For another example, the user's voice interaction message may be: Bring me two apples.

[0190] The sample semantic classification result is a classification result obtained by determining whether semantic reasoning is required for the sample interaction data.

[0191] In the actual implementation process, the sample interaction data between the user and the refrigeration equipment can be input into the classification model to obtain the sample semantic classification results.

[0192] The target time is a preset time.

[0193] The specific value of the target time can be determined based on user customization or based on actual conditions. For example, the target time can be 0 o'clock or 5 o'clock in the morning, and this application does not limit it.

[0194] During the actual execution process, after determining the sample semantic classification results of the user's task instructions, the sample interaction data between the user and the refrigeration equipment in the first short time period can be processed differently based on the different situations of the sample semantic classification results, through the sample interaction data and the target time, so as to obtain the latest sample interaction data between the user and the refrigeration equipment in the first short time period, which characterizes the interaction situation between the user and the refrigeration equipment in the first short time period.

[0195] The processed interaction data between the user and the refrigeration device in the first short period and the interaction data between the user and the refrigeration device in the previous short period are merged, and the short memory database is updated based on the merged data and the environmental information of the refrigeration device in the first short period.

[0196] According to the large model-based personalized voice question-and-answer method provided in the embodiment of the present application, by processing the sample interaction data between the user and the refrigeration device in the first short time period, the environmental information of the refrigeration device in the first short time period, and the sample interaction data between the user and the refrigeration device in the previous short time period, the information in the short memory database is jointly updated, thereby dynamically updating the short memory database to ensure the real-time and accuracy of the data in the short memory database.

[0197] In some embodiments, based on the sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration device and the target time, processing the sample interaction data between the user and the refrigeration device within the first short period includes: When it is determined based on the sample semantic classification result that no semantic reasoning is required for the sample interaction data between the user and the refrigeration device, retaining the sample interaction data between the user and the refrigeration device within the first short period; In the case where it is determined based on the sample semantic classification result that semantic reasoning is required for the sample interaction data between the user and the refrigeration device, semantic reasoning is performed on the sample interaction data between the user and the refrigeration device within the target time period to obtain a sample reasoning result; Based on the sample reasoning result, sample short-term interaction data corresponding to the sample reasoning result is obtained from the sample interaction data between the user and the refrigeration equipment within the target time period; Based on the sample reasoning result and the sample short-time interaction data, the sample interaction data between the user and the refrigeration device in the first short-time period is updated.

[0198] In this embodiment, the target time period is a time period determined based on the target time and the first short time period.

[0199] Taking the target time as 0 o'clock and the first short-term period as the period corresponding to 3 pm to 4 pm as an example, the target time period can be a time period determined based on 0 o'clock to 4 o'clock, and the target time period can also be a time period determined based on the time when the sample interaction data that needs to be inferred within the first short-term period ends from 0 o'clock.

[0200] The sample reasoning result is the reasoning result that represents the user's usage intention obtained by performing natural language reasoning on the user's task instructions.

[0201] For example, when the user's interaction data is taking out two carrots, the sample inference result is removal.

[0202] For another example, when the user's interaction data is to put in beef, the sample inference result is to add.

[0203] When the sample reasoning result is removal, it is determined that the user's usage intention is to eat the removed storage object, that is, the user eats two radishes within the first short period of time.

[0204] When the sample reasoning result is addition, it is determined that the user's usage intention is to tend to eat the newly added storage object, that is, the user tends to eat beef in the future.

[0205] During the actual execution process, when it is determined based on the sample semantic classification results that the sample interaction data between the user and the refrigeration device does not require semantic reasoning, the form of the sample interaction data between the user and the refrigeration device within the first short time period can be kept unchanged, and the short memory database can be updated based on the sample interaction data between the user and the refrigeration device within the first short time period.

[0206] When it is determined based on the sample semantic classification results that semantic reasoning is required for the sample interaction data between the user and the refrigeration device, the question and answer data in the sample interaction data between the user and the refrigeration device in the first short time period corresponding to the target moment to the current collection moment is analyzed and reasoned to update the sample interaction data between the user and the refrigeration device in the first short time period.

[0207] In the actual execution process, different prompt engineering templates can be designed for different sample inference results of question and answer data.

[0208] For example, when the inference result is removal, the prompt engineering template can be: {The user ate: [slot] today, please consider the nutritional content of the above ingredients and do not repeatedly recommend foods containing the same nutrients}.

[0209] For another example, when the inference result is addition, the prompt engineering template can be: {The user added new ingredients today, please use fresh ingredients first when recommending recipes, ingredient information: [slot]}.

[0210] In the actual execution process, the interaction data between the user and the refrigeration equipment in the first short period can be updated based on the user usage intention represented by the inference result, such as filling the storage objects involved in the interaction data into the prompt project template corresponding to the inference result.

[0211] It is understandable that after the user interacts with the refrigeration device, the storage information in the refrigeration device will also change. At this time, the storage information in the refrigeration device at the current collection time can be updated based on the interaction data of each stage.

[0212] Likewise, a prompt engineering template may be designed for storage information in the refrigeration equipment and environmental information of the refrigeration equipment.

[0213] The project template for prompting storage information in the refrigeration equipment can be: {The user's refrigerator has the following ingredients: [slot], please make recommendations strictly based on the ingredients already in the refrigerator}.

[0214] The project template for prompting the environmental information of the refrigeration equipment may be: {The current season is: [slot], please select seasonal recipe recommendations according to the season}.

[0215] According to the large model-based personalized voice question-and-answer method provided in the embodiment of the present application, by determining whether the sample interaction data between the user and the refrigeration device requires semantic reasoning, different methods are used to process the sample interaction data between the user and the refrigeration device. When reasoning is required, the sample interaction data is first reasoned to obtain a sample reasoning result. Based on the sample reasoning result and the sample interaction data, the sample interaction data between the user and the refrigeration device within a first short period is updated. When reasoning is not required, the sample interaction data between the user and the refrigeration device within the first short period is maintained. The method of processing the sample interaction data between the user and the refrigeration device within the first short period can be flexibly selected to improve the availability of the processed sample interaction data.

[0216] In some embodiments, semantic reasoning of sample interaction data between the user and the refrigeration device within the target time period to obtain a sample reasoning result may also include: Segment the sample interaction data between each user and the refrigeration equipment within the target period to obtain the segmentation results corresponding to each sample interaction data; Extract the operation object and operation content in each sample interaction data from the word segmentation results; Based on the operation object and the operation content, a sample reasoning result of the user in the first short time period is obtained.

[0217] In this embodiment, the word segmentation result is the result obtained by dividing the sample interaction data into individual characters or individual words.

[0218] The operation object is the object involved in the process of interaction between the user and the refrigeration equipment.

[0219] The operation content refers to the operation content of the user on the operation object.

[0220] For example, the operation content may be taking out the operation object, or putting in the operation object.

[0221] In the actual execution process, you can use jieba to segment the sample interaction data, and design and fill business logic slots based on the segmentation results.

[0222] Taking the sample interaction data of taking two apples as an example, after analyzing taking two apples, the word segmentation result obtained can be: ["take", "two", "apple"].

[0223] According to the word segmentation results, the designed business logic slots can be: [{"take":None},{"two":None},{"apple":None}].

[0224] Combined with business logic to fill in Figure 6 Business logic slots shown.

[0225] Based on the filled business logic slots, a prompt engineering template related to the usage intent can be designed.

[0226] The prompt project template related to the usage intention can be: Template (before filling): User has consumed fruit: {slot}.

[0227] Template (after filling): User has consumed fruit: apple.

[0228] According to the large model-based personalized voice question and answer method provided in the embodiment of the present application, by performing word segmentation on the sample interaction data, key information in the sample interaction data is effectively extracted, thereby determining the sample inference result of the sample interaction data, converting the sample interaction data into a more intuitive form, and improving the processing efficiency of the voice question and answer.

[0229] In some embodiments, updating the short memory database based on the updated sample interaction data between the user and the refrigeration device in the first short period, the environment information of the refrigeration device in the first short period, and the sample interaction data between the user and the refrigeration device in the previous short period may further include: Based on the sample interaction data between the user and the refrigeration device in the previous short period, and the environment information of the refrigeration device in the previous short period, obtaining the first short-term summary information; Obtaining second short-term summary information based on the updated sample interaction data between the user and the refrigeration device in the first short-term period and the environmental information of the refrigeration device in the first short-term period; Replace the environment information of the refrigeration equipment in the previous short period in the first short-term summary information with the environment information of the refrigeration equipment in the first short period in the second short-term summary information; The sample interaction data between the user and the refrigeration device in the previous short period in the first short summary information and the updated sample interaction data between the user and the refrigeration device in the first short period in the second short summary information are concatenated to update the short memory database.

[0230] In this embodiment, the first short-term summary information is information obtained by summarizing sample interaction data between the user and the refrigeration device in the previous short-term period and environmental information of the refrigeration device.

[0231] The second short-term summary is information obtained by summarizing the sample interaction data between the user and the refrigeration device within the first short period of time, and the environmental information of the refrigeration device.

[0232] Continue to refer Figure 5 In the actual execution process, the sample interaction data between the user and the refrigeration equipment in the first short period and the environmental information of the refrigeration equipment in the first short period can be input into the target big model to obtain the summary information corresponding to the first short period. The target big model can analyze the sample interaction data in the first short period and organize it according to the needs to obtain the second short-time summary information.

[0233] In some embodiments, the target big model may organize the interaction data of the first short period into a key-value format.

[0234] It can be understood that the second short-term summary information does not involve the user portrait in the long memory database, so as to reduce the conflict between the short memory database and the long memory database.

[0235] It can be understood that the sample interaction data between the user and the refrigeration device in the previous short period and the environmental information of the refrigeration device in the previous short period can be processed in the same way as the interaction data between the user and the refrigeration device in the first short period to obtain the first short-term summary information. To avoid repetition, they are not described here.

[0236] Continue to refer Figure 5 In some embodiments, after obtaining the first short summary information and the second short summary information, the first short summary information and the second short summary information may be spliced, and the spliced ​​data may be input into the target large model to generate new short summary information.

[0237] The target large model may replace the environmental information of the refrigeration equipment in the previous short period in the first short period summary information with the environmental information of the refrigeration equipment in the first short period in the second short period summary information.

[0238] For the newly added interaction information, the updated interaction data between the user and the refrigeration device in the first short period in the second short summary information can be directly spliced ​​to the interaction data between the user and the refrigeration device in the previous short period in the first short summary information to obtain new short summary information to update the short memory database.

[0239] It should be noted that, in the process of sorting and analyzing the first short summary information and the second short summary information, the target large model cannot summarize the summary or output new information not included in the first short summary information and the second short summary information.

[0240] In the actual execution process, after the first short summary information and the second short summary information are input into the target large model, new short summary information output by the target large model can be obtained, so as to update the short memory database based on the new short summary information.

[0241] In the actual execution process, the new short-term summary information does not need to be vectorized and stored in the vector database, but can be directly spliced ​​with the prompt project in symbolic form.

[0242] like Figure 7 As shown, in the actual execution process, after receiving the user task instruction input by the user, the target sub-scenario corresponding to the user task instruction can be obtained through the scenario distribution module. Based on the target sub-scenario, the user task instruction is processed by the reasoning construction and non-reasoning retrieval modules to obtain the filled prompt word template, and the filled prompt word template is input into the large model to obtain the question and answer result.

[0243] The inference construction and non-inference retrieval modules can be based on Figure 4 The process shown processes user task instructions.

[0244] In addition, the inference model 1 can be used to perform Figure 4 The reasoning module of the left branch process in the figure, the reasoning module 2 can be used for execution Figure 4 The reasoning module of the right branch in .

[0245] Of course, the inference model 1 can be used to perform Figure 4 The reasoning module of the right branch process in the figure, reasoning module 2 can be used to execute Figure 4 The reasoning module of the left branch.

[0246] According to the large model-based personalized voice question-and-answer method provided in an embodiment of the present application, the first short-term summary information is obtained through the sample interaction data between the user and the refrigeration device in the previous short period and the environmental information of the refrigeration device in the previous short period, and the second short-term summary information is obtained through the sample interaction data between the user and the refrigeration device in the first short period and the environmental information of the refrigeration device in the first short period. The interaction data of each period is effectively analyzed and sorted, the short-term summary information corresponding to each period is obtained, and the information is updated based on the obtained short-term summary information. The short-term summary information of the two periods is condensed and summarized, thereby reducing content redundancy, waste of computing resources and delay problems caused by the high update frequency.

[0247] In some embodiments, before step 110, the method may further include: A long memory database is constructed based on the user's long-term biometric information and basic information.

[0248] In this embodiment, the long-term biometric information is the user's health information that remains unchanged over a long period of time.

[0249] For example, long-term biometric information may include the user's condition of suffering from a chronic disease.

[0250] The user's basic information may include: gender, age, region, preferences, family members, and user experience.

[0251] In the actual implementation process, the user's long-term biometric information and the user's basic information can be collected by providing a question and answer page to the user.

[0252] Of course, in the actual implementation process, the user's long-term biometric information and the user's basic information can also be obtained through any other feasible methods.

[0253] It is understandable that the long-term biometric information and basic information of different users may be different. Different storage areas can be established for different users, and the long-term biometric information and basic information of multiple users can be stored in a long memory database to build a long memory database.

[0254] According to the large model-based personalized voice question and answer method provided in the embodiment of the present application, by storing the user's long-term biometric information and the user's basic information, the background information of different users is effectively stored, the data in the long memory database is enriched, and comprehensive background information is provided for subsequent voice questions and answers, thereby improving the user experience.

[0255] In some embodiments, building a long memory database based on the user's long-term biometric information and the user's basic information may also include: Obtain at least one user corpus; Constructing an association relationship between the user's long-term biometric information and at least one of the user's basic information and the user corpus; The long-term biometric information, the basic information, and the association between at least one of the long-term biometric information and the basic information and the user corpus are stored in a long memory database.

[0256] In this embodiment, the user corpus is sentences that may be involved in the process of voice interaction between the user and the refrigeration device.

[0257] For example, user data can be: recommend a recipe to me, recommend a dish to me, and what I should eat to be healthy, etc.

[0258] The association relationship is whether one or more of the long-term biometric information and the user's basic information is needed in the process of processing the user corpus.

[0259] For example, the association relationship may be: when the user corpus is "what should I eat to be healthy", the user's physical health information, that is, long-term biometric information, is needed. At this time, an association relationship between "what should I eat to be healthy" and long-term biometric information can be constructed.

[0260] In the actual implementation process, the long-term biometric information, the basic information, and the association between one or more of the long-term biometric information and the basic information and the user corpus can be stored in the long memory database to construct a long memory database.

[0261] It should be noted that if the key-value pairs are directly vectorized and stored, in actual retrieval, if the retrieval is directly based on the user's task instructions, the required long-term basic information may not be retrieved.

[0262] The reason is that the long-term basic information and the user task instructions may be uncorrelated in terms of semantic similarity.

[0263] For example, a user's long-term biometric information can be used as background information for recommending recipes to the user. However, in actual vector distance calculation, the user's long-term biometric information may be far away from the user's recommended recipe information.

[0264] By constructing the association between long-term biometric information and one or more of the basic information and the user's task instructions, the problem of irrelevant semantic similarity in the retrieval process can be alleviated.

[0265] In the actual implementation process, Figure 8 The key-value information shown is converted into Fig. 9 Slice data shown.

[0266] Among them, document is a category in the long-term basic information, which is used to be stored in the vector library, and metadatas is the detailed information of the long-term basic information, which does not participate in vector embedding and storage.

[0267] In the actual retrieval process, the user task instructions can be embedded into vectors and similarity calculated with the embedded documents in the vector library.

[0268] In the actual implementation process, Fig.10 As shown, multiple user corpora can be collected to establish a Mapping corpus.

[0269] It can be understood that the Mapping corpus is a pre-established table used to build a long memory database.

[0270] like Fig.11 As shown, different user corpora can construct different data.

[0271] After determining the Mapping corpus, a complete slice file can be constructed by combining the user's long-term biometric information and basic information.

[0272] like Fig.12 As shown in Figure 6 Construct a new slice based on the metadatas information in the slice data shown.

[0273] It is understandable that the Mapping corpus can be updated dynamically, that is, it can be continuously updated and increased according to business needs.

[0274] In the actual execution process, the Mapping corpus can be iterated through the large model.

[0275] like Fig.13As shown, a prompt engineering template used for iteration can be designed in advance, and the prompt engineering template can be input into the large model to obtain the output result of the large model.

[0276] The output result of the large model can be a category of long-term basic information, such as the user's long-term biometric information and the user's basic information.

[0277] The Mapping corpus corresponding to the category of the long-term basic information shown in the output result is taken out, and the user corpus is filled into the Mapping corpus, the slice data is reconstructed, and the reconstructed slice data is stored in the vector library.

[0278] Among them, the prompt engineering template used for iteration needs to include the categories of long-term basic information included in the long memory database and user corpus.

[0279] According to the large model-based personalized voice question-and-answer method provided in the embodiment of the present application, by acquiring multiple user corpora and establishing an association relationship between the user's long-term biometric information and one or more of the user's basic information and the user corpora, on the basis of storing the user's long-term basic information, the association relationship between each category of data in each long-term basic information and the user corpus is further stored, thereby improving the accuracy of acquiring long-term basic information based on user task instructions.

[0280] like Fig.14 As shown, in some embodiments, before step 110, the method may further include: Acquire interaction data between the user and the refrigeration device within a first long period at the current collection time; A long memory database is constructed based on the interaction data between the user and the refrigeration device in a first long period at a current collection moment and the interaction data between the user and the refrigeration device in at least one first long period before the first long period at the current collection moment.

[0281] In this embodiment, the first long-term period is a preset period for processing the interaction data to obtain the long-term basic information of the user.

[0282] The specific value of the first long period can be based on user customization or determined based on actual execution conditions; for example, the first long period can be one day or half a day, which is not limited in this application.

[0283] It is understandable that the first long period may include multiple groups of interaction data. Taking the first long period as one day as an example, at nine o'clock in the morning, user A may interact with the refrigeration equipment to generate a group of interaction data, and at 12 o'clock in the afternoon, user B may interact with the refrigeration equipment to generate a group of interaction data.

[0284] Interaction data refers to the data generated by operations such as asking and answering questions between users and refrigeration equipment.

[0285] The interaction data can be obtained by reading the interaction log of the refrigeration equipment.

[0286] In some embodiments, the interaction data may be organized into the same format, such as “interaction object: A, interaction content: X”.

[0287] The at least one previous first long time period is one or more first long time periods before the first long time period corresponding to the current acquisition moment.

[0288] For example, if the first long period corresponding to the current collection time is Wednesday, then at least one previous first long period may be Tuesday and Monday.

[0289] It can be understood that if the first long time period corresponding to the current acquisition moment is different, at least one previous first long time period corresponding thereto will also be different.

[0290] Continue to refer Fig.14 In some embodiments, after obtaining the interaction data between the user and the refrigeration device, abnormal conversation content in the interaction data may be deleted.

[0291] In this embodiment, the abnormal conversation content may be interaction data such as misrecognition of speech and false wake-up.

[0292] Abnormal conversation content may also include noisy interactions and repeated interactions.

[0293] During the actual execution process, a certain set of interaction data can be deleted, but interaction data cannot be added or the content of the interaction data cannot be rewritten.

[0294] In some embodiments, a long memory database may be constructed using a strategy of "keeping peak values ​​unchanged and updating valley values".

[0295] In this embodiment, the interaction data can be processed during a period when the user does not interact with the refrigeration device to build a long memory database. For example, the refrigeration device processes the interaction data at night or during a lunch break to build a long memory database.

[0296] During the time when the user interacts with the refrigeration device, the refrigeration device may store the interaction data.

[0297] It can be understood that the objects interacting with the refrigeration device may be different in different first long time periods.

[0298] According to the large model-based personalized voice question-and-answer method provided in the embodiment of the present application, the user's long-term basic information is jointly determined by acquiring the interaction data of multiple first long-time periods between the user and the refrigeration equipment before the first long-time period corresponding to the current collection moment and the interaction data of the first long-time period corresponding to the current collection moment, and by iterating the data of different first long-time periods, the user's long-term basic information is continuously enriched, thereby improving the accuracy of the determined long-term basic information and improving the data quality of the constructed long memory database.

[0299] In some embodiments, constructing a long memory database based on the interaction data between the user and the refrigeration device in the first long period at the current collection time and the interaction data between the user and the refrigeration device in at least one first long period before the first long period at the current collection time may include: Processing the interaction data between the user and the refrigeration device in at least one previous first long-term period to obtain at least one first long-term summary information; Processing the interaction data between the user and the refrigeration device in the first long-term period at the current collection moment to obtain second long-term summary information; A long memory database is constructed based on the first long-term summary information and the second long-term summary information.

[0300] In this embodiment, the first long-term summary information is information obtained by analyzing and sorting the interaction data between the user and the refrigeration device in the first long-term period before the first long-term period corresponding to the current collection moment.

[0301] The second long-term summary information is information obtained by analyzing and sorting the interaction data between the user and the refrigeration device in the first long-term period corresponding to the current collection moment.

[0302] In some embodiments, the first long summary information and the second long summary information may be output in a target format, such as a json format, to focus on the key and specific value of the interaction data.

[0303] In some embodiments, after obtaining the interaction data between the user and the refrigeration device in at least one previous first long period and the interaction data between the user and the refrigeration device in the first long period corresponding to the current collection moment, multiple groups of interaction data in each first long period can be spliced ​​together to obtain spliced ​​interaction data.

[0304] In some embodiments, the interaction data may be spliced ​​sequentially based on the time when it was generated.

[0305] In other embodiments, the splicing may also be performed sequentially based on the interaction objects corresponding to the interaction data.

[0306] In the actual execution process, the interaction data can be spliced ​​with the iterative prompt project, or directly spliced ​​into the prompt words, or uploaded synchronously after forming a document to obtain the target interaction data.

[0307] In the actual execution process, after the spliced ​​interaction data is obtained, the spliced ​​interaction data can be processed to construct a long memory database.

[0308] Continue to refer Fig.14 In some embodiments, after obtaining the interaction data, the interaction data can be input into the target macro model to obtain summary information.

[0309] In the actual execution process, after obtaining the interaction data between the user and the refrigeration device in each first long period, the interaction data between the user and the refrigeration device in each first long period can be analyzed by the target large model.

[0310] During the analysis process, the target big model can focus on information such as user name, user family members, user preferences, user long-term biometric information, user city, user gender, and events experienced by the user in the interaction data to obtain multiple first long-term summary information and second long-term summary information.

[0311] The user's name, user's family members, user's preferences, user's long-term biometric information, user's city, user's gender, and events experienced by the user in the interaction data can be the default information that the target large model needs to pay attention to.

[0312] The information that the target large model needs to pay attention to by default can be based on user customization or determined based on actual execution conditions, and this application does not limit it.

[0313] Of course, the target large model can also focus on other data information, and the specific circumstances of the other data information are not limited in this application.

[0314] When the target large model pays attention to other information, a new field such as "mechine" can be added to mark the title and content of the new information of interest.

[0315] In the actual implementation process, after obtaining the plurality of first long-term summary information and the second long-term summary information, the plurality of first long-term summary information and the second long-term summary information may be stored in a long memory database for subsequent use.

[0316] In the actual execution process, the plurality of first long-term summary information and second long-term summary information may be further processed, such as sorted and merged, to construct a long memory database.

[0317] According to the large model-based personalized voice question-and-answer method provided in the embodiment of the present application, by processing the interaction data of the first long-term period of the current collection moment and the interaction data between the user and the refrigeration equipment in the previous multiple first long-term periods, the long-term summary information corresponding to each first long-term period is obtained respectively, and based on the new interaction data and the historical interaction data, the long-term basic information of the user is jointly determined, thereby improving the accuracy and completeness of constructing the long memory database and improving the data quality of the long memory database.

[0318] In some embodiments, constructing a long memory database based on each of the first long-term summary information and the second long-term summary information may include: splicing the first long-term summary information based on the time sequence to obtain the third long-term summary information corresponding to at least one previous first long-term period; A long memory database is constructed based on the second long-term summary information and the third long-term summary information.

[0319] In this embodiment, the third long-term summary information is summary information obtained by concatenating a plurality of first long-term summary information.

[0320] In some embodiments, each piece of first long-term summary information may be concatenated based on a time sequence.

[0321] In some other embodiments, each first long-term summary information may be further scored based on at least one of a time sequence and an interactive object.

[0322] In the actual execution process, after obtaining multiple first long-term summary information, the multiple first long-term summary information can be spliced ​​to obtain summary information corresponding to multiple first long-term periods before the first long-term period at the current acquisition moment (i.e., the third long-term summary information), thereby constructing a long memory database based on the old summary information (i.e., the third long-term summary information) and the new summary information (i.e., the second long-term summary information).

[0323] According to the large model-based personalized voice question and answer method provided in the embodiment of the present application, through the first long-term summary information and the second long-term summary information, the user's long-term basic information can be jointly determined based on the historical interaction data and the interaction data of the period in which the current collection moment is located, thereby improving the accuracy of the long-term basic information, providing data support for subsequent large model-based personalized voice question and answer, providing personalized services to users, and improving the user experience.

[0324] like Fig.15 As shown, in some embodiments, constructing a long memory database based on the second long-term summary information and the third long-term summary information may include: Acquire a first feature category included in the third long-term summary information and a second feature category included in the second long-term summary information; Based on the data corresponding to the second feature category, the data corresponding to the first feature category which is the same as the second feature category in the first feature category is updated, and the data corresponding to the feature category which is different from the first feature category in the second feature category is updated to the third long-term summary information; and the fourth long-term summary information is obtained; The fourth long-term summary information is stored in the long memory database.

[0325] In this embodiment, the first feature category is the feature representing the long-term basic information of the user included in the third long-term summary information.

[0326] The second feature category is the feature representing the long-term basic information of the user included in the second long-term summary information.

[0327] It can be understood that the number of first feature categories will continue to increase with the increase in the number of iterations.

[0328] The fourth long-term summary information is summary information obtained by updating the third long-term summary information.

[0329] In the actual execution process, after obtaining the third long summary information and the second long summary information, the information corresponding to the feature categories in the third long summary information and the second long summary information may be respectively extracted and compared to see whether they need to be updated.

[0330] It can be understood that the second long-term summary information is summary information of the most recent interaction data, and the old long-term summary information (ie, the third long-term summary information) can be updated based on new information.

[0331] For example, taking the case where the second feature category of the second long summary information includes the user's age, and the first feature category of the third long summary information also includes the user's age, if the second long summary information can obtain that the user's age has increased by 1 year based on the interaction data between the user and the refrigeration device, the data corresponding to the user's age included in the first feature category of the third long summary information can be updated to a value after the current stored age is increased by 1.

[0332] For another example, when the second feature category of the second long summary information includes user allergens, but the first feature category of the third long summary information does not include user allergens, the data corresponding to the user allergens can be updated to the third long summary information to obtain fourth long summary information.

[0333] In actual execution, after obtaining a plurality of first long summary information and second long summary information, the plurality of first long summary information and second long summary information may be input into the target large model to obtain fourth long summary information output by the target large model.

[0334] During the actual execution process, the target large model splices multiple first long-term summary information to obtain the third long-term summary information. The target large model analyzes the new and old summary information (i.e., the second long-term summary information and the third long-term summary information), updates the data corresponding to the same feature category, and directly splices the newly added information after the old summary information (i.e., the third long-term summary information). After sorting and analyzing the new and old summary information, the target large model outputs the fourth long-term summary information.

[0335] It is understandable that in order to ensure the accuracy of the summary information, the target large model does not need to summarize the summary or output additional new content.

[0336] According to the large model-based personalized voice question-answering method provided in the embodiment of the present application, the new second long summary information is analyzed and compared with the third long summary information, the data of the feature category also stored in the third long summary information is updated by the new second long summary information, and the feature category included in the second long summary information but not included in the third long summary information is supplemented to the third long summary information, so as to effectively update the third long summary information, continuously enrich the data in the fourth long summary information obtained based on the third long summary information, and improve the data quality of the fourth long summary information.

[0337] Continue to refer Fig.15 In some embodiments, storing the fourth long-term summary information in the long memory database may include: performing at least one of slicing processing and vector embedding processing on the fourth long-term summary information to obtain target long-term summary information; Store the target long-term summary information into the long memory database.

[0338] In this embodiment, the target long-term summary information is summary information obtained by processing the fourth long-term summary information.

[0339] The slicing process is to extract the required information from the target long-term summary information.

[0340] Vector embedding is the process of converting the target long-term summary information into vector information.

[0341] In the actual execution process, after obtaining the fourth long-term summary information, the long-term summary information may be sliced ​​and vector embedded to obtain summary information represented by a vector, and the summary information represented by the vector is stored in a long memory database.

[0342] The long memory database may be a vector database.

[0343] Vector Database is a database system used to store and query vector data.

[0344] In the actual execution process, after obtaining the fourth long-term summary information, the fourth long-term summary information may be sliced ​​and vector embedded, and the processed data may be stored in the long memory database. At this time, the long memory database is a vector database.

[0345] According to the large model-based personalized voice question-answering method provided in the embodiment of the present application, the fourth long-term summary information is sliced ​​to improve the flexibility of data selection, and the fourth long-term summary information is vector embedded to reduce the complexity of the long memory database, thereby improving the storage efficiency and retrieval efficiency of the long memory database.

[0346] The large-model-based personalized voice question-answering method provided in the embodiment of the present application can be executed by a large-model-based personalized voice question-answering device. In the embodiment of the present application, the large-model-based personalized voice question-answering device performs the large-model-based personalized voice question-answering method as an example to illustrate the large-model-based personalized voice question-answering device provided in the embodiment of the present application.

[0347] The embodiment of the present application also provides a personalized voice question and answer device based on a large model.

[0348] like Fig.16 As shown, the large model-based personalized voice question-answering device includes: a first processing module 1610 , a second processing module 1620 and a third processing module 1630 .

[0349] The first processing module 1610 is used to receive a first input from a user; the first input is used to input a user task instruction; A second processing module 1620 is used to determine a target sub-scene in response to the first input and based on the user task instruction; The third processing module 1630 is used to obtain the short-term basic information corresponding to the target sub-scene at the current acquisition time, and obtain the user's identity information and the long-term basic information corresponding to the user's task instruction; the short-term basic information is used to represent at least one of the user's short-term biometric information, the interaction between the user and the refrigeration device in the target time period, the storage information in the refrigeration device, and the environment information of the refrigeration device; The fourth processing module 1640 is used to input at least one of the long-term basic information and the short-term basic information and the user task instruction into the large model to obtain the question and answer result output by the large model.

[0350] According to the large-model-based personalized voice question-and-answer device provided in the embodiment of the present application, by receiving user task instructions, long-term basic information and short-term basic information related to the needs and matching the user's identity information are obtained based on the user's actual needs. The obtained information is used as prompt words for the large-model to process the user's task instructions, providing more detailed and comprehensive information for processing user tasks. Combined with the user's actual characteristics, personalized processing is performed to obtain question-and-answer results corresponding to the user instructions, thereby improving the accuracy of the question-and-answer results. The processing method is simple, without the need to build a complex database and perform complex natural language processing, thereby reducing the difficulty of maintenance. In addition, the obtained long-term basic information and short-term basic information can be applied to different task scenarios, have strong generalization capabilities, and enhance user experience.

[0351] In some embodiments, the third processing module 1630 may also be used to: Obtain the user's identity information and the long-term basic information corresponding to the user's task instructions from the long memory database; When it is determined based on the semantic classification result that the user task instruction does not need semantic reasoning, the corresponding short-term basic information at the current collection time is obtained from the short memory database; When it is determined based on the semantic classification result that semantic reasoning is required for the user task instruction, semantic reasoning is performed on the user task instruction to obtain a semantic reasoning result; Based on the semantic reasoning result, short-term interaction data corresponding to the semantic reasoning result is obtained from the interaction data between the user and the refrigeration device; Based on the semantic reasoning results and short-term interaction data, the corresponding short-term basic information at the current collection moment is obtained.

[0352] In some embodiments, the apparatus may further include a fourth processing module, configured to: Acquire sample interaction data between the user and the refrigeration device in a first short period at the current collection moment, and environmental information of the refrigeration device in the first short period; The short memory database is updated based on the sample interaction data between the user and the refrigeration device in the first short time period, the environmental information of the refrigeration device in the first short time period, and the sample interaction data between the user and the refrigeration device in the previous short time period corresponding to the first short time period.

[0353] In some embodiments, the apparatus may further include a fifth processing module, configured to: Performing semantic classification on the sample interaction data between the user and the refrigeration device in the first short period of time to obtain a sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration device; Based on the sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration device and the target time, processing the sample interaction data between the user and the refrigeration device within the first short period; Based on the processed interaction data between the user and the refrigeration device in the first short period, the environmental information of the refrigeration device in the first short period, and the interaction data between the user and the refrigeration device in the previous short period, a short-term information summary is performed to update the short memory database.

[0354] In some embodiments, the apparatus may further include a sixth processing module, configured to: When it is determined based on the sample semantic classification result that no semantic reasoning is required for the sample interaction data between the user and the refrigeration device, retaining the sample interaction data between the user and the refrigeration device within the first short period; In the case where it is determined based on the semantic classification result that semantic reasoning is required for the sample interaction data between the user and the refrigeration device, semantic reasoning is performed on the sample interaction data between the user and the refrigeration device within a target time period to obtain a sample reasoning result; the target time period is a time period between the target moment and the first short time period; Based on the sample reasoning result, sample short-term interaction data corresponding to the sample reasoning result is obtained from the sample interaction data between the user and the refrigeration equipment within the target time period; Based on the sample reasoning result and the sample short-time interaction data, the sample interaction data between the user and the refrigeration device in the first short-time period is updated.

[0355] In some embodiments, the apparatus may further include an eighth processing module, configured to: A long memory database is constructed based on the user's long-term biometric information and basic information.

[0356] In some embodiments, the apparatus may further include a ninth processing module, configured to: Obtain at least one user corpus; Constructing an association relationship between at least one of the long-term biometric information and the basic information and the user corpus; The long-term biometric information, the basic information, and the association between at least one of the long-term biometric information and the basic information and the user corpus are stored in a long memory database.

[0357] The personalized voice question-and-answer device based on the large model in the embodiment of the present application can be a refrigeration device, or an electronic device connected to the refrigeration device for communication, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than the terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a car electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.

[0358] The large model-based personalized voice question-answering device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0359] The personalized voice question-answering device based on a large model provided in the embodiment of the present application can achieve Figures 1 to 15 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0360] The embodiment of the present application also provides a refrigeration device.

[0361] In some embodiments, the refrigeration equipment includes: a personalized voice question-and-answer device based on a large model as described in any of the above embodiments.

[0362] The refrigeration equipment can be understood as refrigeration storage equipment in a broad sense, including but not limited to refrigerators, freezers, display cabinets, beverage cabinets, wine cabinets, fresh-keeping cabinets, refrigerated vending machines and other refrigeration storage equipment. The refrigeration equipment has various structural forms and a wide range of applications.

[0363] In some embodiments, the refrigeration device further includes: a speaker.

[0364] In this embodiment, the speaker is used to output the question and answer results.

[0365] The speaker is electrically connected to the personalized voice question-answering device based on the large model.

[0366] According to the refrigeration device provided in the embodiment of the present application, the question and answer results can be directly output through the speaker, which is simple and convenient.

[0367] In some embodiments, the refrigeration equipment further includes: a display device.

[0368] In this embodiment, the display device can be used to receive the user's first input or display information such as question and answer results.

[0369] The display device is electrically connected to the personalized voice question-answering device based on the large model.

[0370] According to the refrigeration equipment provided in the embodiment of the present application, the user's task instructions and information that needs to be displayed through images in the question and answer results are received through a display device, which is applicable to a variety of scenarios and improves the user experience.

[0371] During the actual execution process, the user may input a user task instruction through the display device, and the refrigeration equipment may output the question and answer result through the speaker and the display device after processing the user task instruction.

[0372] According to the refrigeration equipment provided in the embodiment of the present application, by receiving user task instructions, based on the actual needs of the user, long-term basic information and short-term basic information related to the needs and matching the identity information of the user are obtained, and the obtained information is used as a prompt word for the large model to process the user task instructions, so as to provide more detailed and comprehensive information for processing user tasks, and personalized processing is performed in combination with the actual characteristics of the user to obtain question and answer results corresponding to the user instructions, thereby improving the accuracy of the question and answer results, and the processing method is simple, without the need to build a complex database and perform complex natural language processing, thereby reducing the difficulty of maintenance. In addition, the obtained long-term basic information and short-term basic information can be applied to different task scenarios, with strong generalization ability, thereby improving the user experience.

[0373] In some embodiments, Fig.17 As shown, an embodiment of the present application also provides an electronic device 1700, including a processor 1701, a memory 1702, and a computer program stored in the memory 1702 and executable on the processor 1701. When the program is executed by the processor 1701, each process of the embodiment of the above-mentioned large-model-based personalized voice question-and-answer method is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0374] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0375] The embodiment of the present application also provides a personalized voice question-answering system based on a large model.

[0376] In some embodiments, the large model-based personalized voice question-answering system includes: a scenario distribution module, a specific scenario module, a long memory database iteration module, and a short memory database iteration module.

[0377] In this embodiment, the scene distribution module is used to determine a specific scene (ie, a target sub-scene) based on a user task instruction.

[0378] The specific scenario module is used to output the question and answer results corresponding to the user task instructions based on the specific scenario and the user task instructions.

[0379] The long memory database iteration module is used to form a user portrait based on the interaction data between the user and the refrigeration equipment to build a long memory database.

[0380] The short memory database iteration module is used to form temporary information to construct a short memory database based on the interaction data between the user and the refrigeration equipment.

[0381] Both the long memory database iteration module and the short memory database iteration module may include an interactive data cleaning module.

[0382] The interaction data cleaning module is used to clean the interaction data and delete unnecessary interaction data.

[0383] According to the large model-based personalized voice question and answer system provided in the embodiment of the present application, by receiving user task instructions, long-term basic information and short-term basic information related to the needs and matching the user's identity information are obtained based on the user's actual needs. The obtained information is used as prompt words for the large model to process the user's task instructions, providing more detailed and comprehensive information for processing user tasks. Combined with the user's actual characteristics, personalized processing is performed to obtain question and answer results corresponding to the user instructions, thereby improving the accuracy of the question and answer results. The processing method is simple, and there is no need to build a complex database and perform complex natural language processing, which reduces the difficulty of maintenance. In addition, the obtained long-term basic information and short-term basic information can be applied to different task scenarios, with strong generalization ability, thereby improving the user experience.

[0384] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned embodiment of the personalized voice question-and-answer method based on a large model are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0385] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0386] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned large model-based personalized voice question-answering method.

[0387] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0388] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned embodiment of the personalized voice question and answer method based on a large model, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0389] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0390] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0391] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0392] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0393] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0394] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A personalized voice question-answering method based on a large model, characterized in that: Applied to refrigeration equipment, the method comprises: Receive a first input from a user; the first input is used to input a user task instruction; In response to the first input, determining a target sub-scenario based on the user task instruction; Acquire short-term basic information corresponding to the target sub-scene at the current acquisition time, and acquire the identity information of the user and the long-term basic information corresponding to the user task instruction; the short-term basic information is used to characterize at least one of the short-term biometric information of the user, the interaction between the user and the refrigeration device within the target time period, the storage information in the refrigeration device, and the environment information of the refrigeration device; Input the long-term basic information and at least one of the short-term basic information and the user task instruction into the large model to obtain the question and answer result output by the large model.

2. The large model-based personalized voice question-answering method according to claim 1, characterized in that: The acquiring of the short-term basic information corresponding to the target sub-scene at the current acquisition moment, and the acquiring of the identity information of the user and the long-term basic information corresponding to the user task instruction, include: Obtaining the user's identity information and the long-term basic information corresponding to the user's task instruction from the long memory database; Under the target sub-scenario, semantically classify the user task instruction to obtain a semantic classification result corresponding to the user task instruction; When it is determined based on the semantic classification result that the user task instruction does not need semantic reasoning, acquiring the short-term basic information corresponding to the current collection moment from the short memory database; In the case where it is determined based on the semantic classification result that the user task instruction needs to be semantically inferred, semantically inference is performed on the user task instruction to obtain a semantic inference result; Based on the semantic reasoning result, acquiring short-term interaction data corresponding to the semantic reasoning result from interaction data between the user and the refrigeration device; Based on the semantic reasoning result and the short-term interaction data, the short-term basic information corresponding to the current collection moment is obtained.

3. The large model-based personalized voice question-answering method according to claim 1, characterized in that: Before receiving the first input from the user, the method further includes: Acquire sample interaction data between the user and the refrigeration device within a first short period at the current collection moment, and environmental information of the refrigeration device within the first short period; The short memory database is updated based on the sample interaction data between the user and the refrigeration device in the first short time period, the environmental information of the refrigeration device in the first short time period, and the sample interaction data between the user and the refrigeration device in the previous short time period corresponding to the first short time period.

4. The large model-based personalized voice question-answering method according to claim 3, characterized in that: The updating of the short memory database based on the sample interaction data between the user and the refrigeration device in the first short time period, the environment information of the refrigeration device in the first short time period, and the sample interaction data between the user and the refrigeration device in a previous short time period corresponding to the first short time period includes: Performing semantic classification on the sample interaction data between the user and the refrigeration device in the first short period of time to obtain a sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration device; processing the sample interaction data between the user and the refrigeration device within the first short period based on the sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration device and the target time; A short-term information summary is performed based on the processed interaction data between the user and the refrigeration device in the first short-time period, the environmental information of the refrigeration device in the first short-time period, and the interaction data between the user and the refrigeration device in the previous short-time period to update the short memory database.

5. The large model-based personalized voice question-answering method according to claim 4, characterized in that: The processing of the sample interaction data between the user and the refrigeration device within the first short period based on the sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration device and the target time includes: When it is determined based on the sample semantic classification result that no semantic reasoning is required for the sample interaction data between the user and the refrigeration device, retaining the sample interaction data between the user and the refrigeration device within the first short period of time; In the case where it is determined based on the sample semantic classification result that semantic reasoning is required for the sample interaction data between the user and the refrigeration device, semantic reasoning is performed on the sample interaction data between the user and the refrigeration device within a target time period to obtain a sample reasoning result; the target time period is a time period determined based on the target moment and the first short-time period; Based on the sample reasoning result, acquiring sample short-term interaction data corresponding to the sample reasoning result from the sample interaction data between the user and the refrigeration device within the target time period; Based on the sample reasoning result and the sample short-time interaction data, the sample interaction data between the user and the refrigeration device in the first short-time period is updated.

6. The large model-based personalized voice question-answering method according to any one of claims 1 to 5, characterized in that: Before receiving the first input from the user, the method further includes: A long memory database is constructed based on the long-term biometric information of the user and the basic information of the user.

7. The large model-based personalized voice question-answering method according to claim 6, characterized in that: The step of constructing a long memory database based on the long-term biometric information of the user and the basic information of the user includes: Obtain at least one user corpus; Constructing an association relationship between at least one of the long-term biometric information of the user and the basic information of the user and the user corpus; The association relationship between the user's long-term biometric information, the user's basic information, and at least one of the long-term biometric information and the basic information and the user corpus is stored in the long memory database.

8. A personalized voice question-answering device based on a large model, characterized in that: Applied to refrigeration equipment, the device comprises: A first processing module, configured to receive a first input from a user; the first input is used to input a user task instruction; A second processing module, configured to determine a target sub-scenario in response to the first input and based on the user task instruction; A third processing module is used to obtain short-term basic information corresponding to the target sub-scene at the current acquisition time, and to obtain the identity information of the user and the long-term basic information corresponding to the user task instruction; the short-term basic information is used to characterize at least one of the short-term biometric information of the user, the interaction between the user and the refrigeration device within the target time period, the storage information in the refrigeration device, and the environmental information of the refrigeration device; The fourth processing module is used to input the long-term basic information and at least one of the short-term basic information and the user task instruction into the large model to obtain the question and answer result output by the large model.

9. A refrigeration device, characterized in that: include: A personalized voice question-answering device based on a large model as described in claim 8.

10. The refrigeration device according to claim 9, characterized in that: The refrigeration equipment further comprises: A speaker is electrically connected to the large model-based personalized voice question-answering device.

11. The refrigeration device according to claim 9, characterized in that: The refrigeration equipment further comprises: A display device is electrically connected to the large model-based personalized voice question-and-answer device.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the large model-based personalized voice question-answering method as described in any one of claims 1 to 7 is implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the large model-based personalized voice question-answering method as described in any one of claims 1 to 7 is implemented.

Images