Personalized Voice Q&A Method, Device and Refrigeration Equipment Based on Large Language Model

Abstract

The present application discloses a personalized voice question-and-answer method, device, and refrigeration equipment based on a large model, which belongs to the technical field of refrigeration equipment. The personalized voice question-and-answer method based on a large model includes: receiving a first input from a user; the first input is used to input a user task instruction; in response to the first input, determining a target sub-scene based on the user task instruction; obtaining short-term basic information corresponding to the target sub-scene at the current acquisition moment, and obtaining the user's identity information and long-term basic information corresponding to the user task instruction; inputting at least one of the long-term basic information and the short-term basic information and the user task instruction into the large model to obtain the question-and-answer result output by the large model. The personalized voice question-and-answer method based on a large model of the present application improves the accuracy of the question-and-answer result, and has a simple processing method, reduces the difficulty of maintenance, is applicable to different task scenarios, has strong generalization ability, and improves the user experience.

Description

Technical Field

[0001] The present application belongs to the technical field of refrigeration equipment, and in particular relates to a large-model-based personalized voice question-answering method, device, and refrigeration equipment. Background Art

[0002] Advances in science and technology have brought more room for development for smart devices; however, the level of personalization in intelligent question-and-answer modules remains relatively low. Related technologies primarily address user personalization by building complex databases and performing sophisticated natural language parsing. However, these approaches are difficult to build and maintain, and their generalization capabilities are poor, impacting the user experience. Summary of the Invention

[0003] This application aims to solve at least one of the technical problems existing in the related art. To this end, this application proposes a large-scale model-based personalized voice question-answering method, device, and refrigeration equipment to improve the accuracy of question-answering results. The method is simple to process, does not require the construction of a complex database and complex natural language processing, reduces maintenance difficulty, is applicable to different task scenarios, has strong generalization capabilities, and enhances the user experience.

[0004] In a first aspect, the present application provides a large-model-based personalized voice question-answering method, which is applied to refrigeration equipment. The method includes:

[0005] Receive a first input from a user; the first input is used to input a user task instruction;

[0006] In response to the first input, determining a target sub-scenario based on the user task instruction;

[0007] Acquiring short-term basic information corresponding to the target sub-scene at the current collection time, and acquiring the identity information of the user and long-term basic information corresponding to the user's task instruction; the short-term basic information is used to represent at least one of the user's short-term biometric information, the user's interaction with the refrigeration device during the target time period, the storage information in the refrigeration device, and the environment information of the refrigeration device;

[0008] At least one of the long-term basic information and the short-term basic information and the user task instruction are input into the large model to obtain the question and answer result output by the large model.

[0009] According to the large model-based personalized voice question-and-answer method of the present application, by receiving user task instructions, long-term basic information and short-term basic information related to the needs and matching the user's identity information are obtained based on the user's actual needs. The obtained information is used as prompt words for the large model to process the user's task instructions, providing more detailed and comprehensive information for processing user tasks. Combined with the user's actual characteristics, personalized processing is performed to obtain question-and-answer results corresponding to the user instructions, thereby improving the accuracy of the question-and-answer results. The processing method is simple, and there is no need to build a complex database and perform complex natural language processing, which reduces the difficulty of maintenance. In addition, the obtained long-term basic information and short-term basic information can be applied to different task scenarios, have strong generalization capabilities, and enhance user experience.

[0010] According to the large model-based personalized voice question-answering method of the present application, the step of obtaining the user's identity information and the long-term basic information corresponding to the user's task instruction, as well as the short-term basic information corresponding to the target sub-scenario at the current collection moment, includes:

[0011] Obtaining the user's identity information and the long-term basic information corresponding to the user's task instruction from a long memory database;

[0012] Under the target sub-scenario, semantically classify the user task instruction to obtain a semantic classification result corresponding to the user task instruction;

[0013] When it is determined based on the semantic classification result that the user task instruction does not require semantic reasoning, acquiring the short-term basic information corresponding to the current collection moment from the short memory database;

[0014] In a case where it is determined based on the semantic classification result that the user task instruction requires semantic reasoning, performing semantic reasoning on the user task instruction to obtain a semantic reasoning result;

[0015] Based on the semantic reasoning result, obtaining short-term interaction data corresponding to the semantic reasoning result from interaction data between the user and the refrigeration device;

[0016] Based on the semantic reasoning result and the short-term interaction data, the short-term basic information corresponding to the current collection moment is obtained.

[0017] According to the large model-based personalized voice question-answering method of the present application, before receiving the first input from the user, the method further includes:

[0018] Acquire sample interaction data between the user and the refrigeration device within a first short period at the current collection moment, and environmental information of the refrigeration device within the first short period;

[0019] The short memory database is updated based on the sample interaction data between the user and the refrigeration device in the first short time period, the environmental information of the refrigeration device in the first short time period, and the sample interaction data between the user and the refrigeration device in the previous short time period corresponding to the first short time period.

[0020] According to the large model-based personalized voice question-answering method of the present application, updating the short memory database based on the sample interaction data between the user and the refrigeration appliance in the first short time period, the environmental information of the refrigeration appliance in the first short time period, and the sample interaction data between the user and the refrigeration appliance in the previous short time period corresponding to the first short time period includes:

[0021] performing semantic classification on the sample interaction data between the user and the refrigeration appliance within the first short period to obtain a sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration appliance;

[0022] processing the sample interaction data between the user and the refrigeration appliance within the first short period based on a sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration appliance and a target time;

[0023] A short-term information summary is performed based on the processed interaction data between the user and the refrigeration device in the first short-time period, the environmental information of the refrigeration device in the first short-time period, and the interaction data between the user and the refrigeration device in the previous short-time period to update the short memory database.

[0024] According to the large model-based personalized voice question-answering method of the present application, processing the sample interaction data between the user and the refrigeration appliance within the first short period based on the sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration appliance and the target time includes:

[0025] When it is determined based on the sample semantic classification result that no semantic reasoning is required for the sample interaction data between the user and the refrigeration appliance, retaining the sample interaction data between the user and the refrigeration appliance within the first short period;

[0026] In a case where it is determined based on the sample semantic classification result that semantic reasoning is required for the sample interaction data between the user and the refrigeration appliance, semantic reasoning is performed on the sample interaction data between the user and the refrigeration appliance within a target time period to obtain a sample reasoning result; the target time period is a time period determined based on the target moment and the first short-time period;

[0027] Based on the sample reasoning result, obtaining sample short-term interaction data corresponding to the sample reasoning result from the sample interaction data between the user and the refrigeration device within the target time period;

[0028] Based on the sample inference result and the sample short-time interaction data, the sample interaction data between the user and the refrigeration device in the first short-time period is updated.

[0029] According to the large model-based personalized voice question-answering method of the present application, before receiving the first input from the user, the method further includes:

[0030] A long memory database is constructed based on the long-term biometric information of the user and the basic information of the user.

[0031] According to the large model-based personalized voice question-answering method of the present application, the long-memory database is constructed based on the long-term biometric information of the user and the basic information of the user, including:

[0032] Obtain at least one user corpus;

[0033] Establishing an association relationship between the user's long-term biometric information and at least one of the user's basic information and the user corpus;

[0034] The association relationship between the user's long-term biometric information, the user's basic information, at least one of the long-term biometric information and the basic information and the user corpus is stored in the long memory database.

[0035] In a second aspect, the present application provides a personalized voice question-answering device based on a large model, which is applied to refrigeration equipment, and the device includes:

[0036] A first processing module is configured to receive a first input from a user; the first input is used to input a user task instruction;

[0037] a second processing module, configured to determine a target sub-scene in response to the first input and based on the user task instruction;

[0038] a third processing module, configured to obtain short-term basic information corresponding to the target sub-scene at a current collection time, and obtain the identity information of the user and long-term basic information corresponding to the user's task instruction; the short-term basic information is used to represent at least one of the user's short-term biometric information, the user's interaction with the refrigeration device during a target time period, storage information in the refrigeration device, and environmental information of the refrigeration device;

[0039] The fourth processing module is used to input at least one of the long-term basic information and the short-term basic information and the user task instruction into the large model to obtain the question and answer results output by the large model.

[0040] According to the large-model-based personalized voice question-and-answer device of the present application, by receiving user task instructions, long-term basic information and short-term basic information related to the needs and matching the user's identity information are obtained based on the user's actual needs. The obtained information is used as prompt words for the large-model to process the user's task instructions, providing more detailed and comprehensive information for processing user tasks. Combined with the user's actual characteristics, personalized processing is performed to obtain question-and-answer results corresponding to the user instructions, thereby improving the accuracy of the question-and-answer results. The processing method is simple, and there is no need to build a complex database and perform complex natural language processing, which reduces the difficulty of maintenance. In addition, the obtained long-term basic information and short-term basic information can be applied to different task scenarios, have strong generalization capabilities, and enhance user experience.

[0041] In a third aspect, the present application provides a refrigeration device, comprising:

[0042] A personalized voice question-answering device based on a large model as described in the second aspect.

[0043] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the large model-based personalized voice question-answering method as described in the first aspect above.

[0044] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the large model-based personalized voice question-answering method as described in the first aspect above.

[0045] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0046] By receiving user task instructions, based on the user's actual needs, the long-term basic information and short-term basic information related to the needs and matching the user's identity information are obtained. The obtained information is used as prompt words for the large model to process the user's task instructions, providing more detailed and comprehensive information for processing user tasks. Combined with the user's actual characteristics, personalized processing is performed to obtain question and answer results corresponding to the user instructions, thereby improving the accuracy of the question and answer results. The processing method is simple, without the need to build a complex database and perform complex natural language processing, reducing maintenance difficulty. In addition, the obtained long-term basic information and short-term basic information can be applied to different task scenarios, with strong generalization ability, thereby improving user experience.

[0047] Furthermore, by obtaining long-term basic information from a pre-built long-memory database, and based on the actual situation of whether the user task instruction requires semantic reasoning, different methods are selected to obtain short-term basic information. When the user task instruction requires reasoning, the short-term basic information is determined through the semantic reasoning results and interaction data. When the user task instruction does not require reasoning, the short-term basic information is obtained from the short-memory database. Different short-term basic information acquisition methods are set for different user task instructions to improve the accuracy of the obtained short-term basic information, and only information related to the task instruction is obtained, which can handle the user's personalized needs and reduce the complexity of task processing.

[0048] Furthermore, by storing the user's long-term biometric information and basic information, the background information of different users can be effectively stored, enriching the data in the long memory database, providing comprehensive background information for subsequent voice questions and answers, and improving the user experience.

[0049] Furthermore, by obtaining a variety of user corpora and establishing an association relationship between the user's long-term biometric information and one or more of the user's basic information and the user corpora, on the basis of storing the user's long-term basic information, the association relationship between each category of data in each long-term basic information and the user corpus is further stored, thereby improving the accuracy of obtaining long-term basic information based on user task instructions.

[0050] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0052] Figure 1 This is one of the flow charts of the large-model-based personalized voice question-answering method provided in the embodiment of the present application;

[0053] Figure 2 This is the second flow chart of the large-model-based personalized voice question-answering method provided in the embodiment of the present application;

[0054] Figure 3 This is the third flow chart of the large-model-based personalized voice question-answering method provided in the embodiment of the present application;

[0055] Figure 4 This is the fourth flow chart of the large model-based personalized voice question-answering method provided in the embodiment of the present application;

[0056] Figure 5This is the fifth flow chart of the large model-based personalized voice question-answering method provided in the embodiment of the present application;

[0057] Figure 6 This is one of the principle diagrams of the large-model-based personalized voice question-answering method provided in the embodiment of the present application;

[0058] Figure 7 This is the sixth flow chart of the large model-based personalized voice question-answering method provided in the embodiment of the present application;

[0059] Figure 8 This is the second schematic diagram of the principle of the large model-based personalized voice question-answering method provided in the embodiment of the present application;

[0060] Figure 9 This is the third principle diagram of the large-model-based personalized voice question-answering method provided in the embodiment of the present application;

[0061] Figure 10 This is the fourth principle diagram of the large-model-based personalized voice question-answering method provided in an embodiment of the present application;

[0062] Figure 11 This is the seventh flow chart of the large-model-based personalized voice question-answering method provided in the embodiment of the present application;

[0063] Figure 12 This is the eighth flow chart of the large-model-based personalized voice question-answering method provided in the embodiment of the present application;

[0064] Figure 13 This is the ninth flow chart of the large-model-based personalized voice question-answering method provided in the embodiment of the present application;

[0065] Figure 14 This is the tenth flow chart of the large model-based personalized voice question-answering method provided in the embodiment of the present application;

[0066] Figure 15 This is the eleventh flow chart of the large-model-based personalized voice question-answering method provided in the embodiment of the present application;

[0067] Figure 16 Schematic diagram of the structure of a personalized voice question-answering device based on a large model provided in an embodiment of the present application;

[0068] Figure 17 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0070] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0071] Below, in combination with the accompanying drawings, the personalized voice question and answer method based on a large model, the personalized voice question and answer device based on a large model, the refrigeration equipment and the readable storage medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0072] Among them, the personalized voice question-answering method based on the large model can be applied to the terminal, and can be specifically executed by the hardware or software in the terminal.

[0073] The terminal includes but is not limited to portable communication devices such as mobile phones or tablet computers. It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer.

[0074] In the following embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse, and a joystick.

[0075] The large-model-based personalized voice question-and-answer method provided in the embodiments of the present application can be executed by a refrigeration device, such as a single-chip microcomputer; in some embodiments, it can also be executed by a server connected to the refrigeration device, or jointly by a server and a single-chip microcomputer, etc.

[0076] The embodiment of the present application provides a personalized voice question-answering method based on a large model. The execution subject of the personalized voice question-answering method based on a large model can be a refrigeration device, or an electronic device that is communicatively connected to the refrigeration device, or a functional module or functional entity in the electronic device that can implement the personalized voice question-answering method based on a large model. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The personalized voice question-answering method based on a large model provided in the embodiment of the present application is explained below using an electronic device as an example of the execution subject.

[0077] like Figure 1 As shown, the large model-based personalized voice question-answering method includes: step 110, step 120 and step 130.

[0078] The personalized voice question-answering method based on large models can be applied to refrigeration equipment.

[0079] The refrigeration equipment can be understood as refrigeration storage equipment in a broad sense, including but not limited to refrigerators, freezers, display cabinets, beverage cabinets, wine cabinets, fresh-keeping cabinets and refrigerated vending machines and other refrigeration storage equipment. The refrigeration equipment has various structural forms and a wide range of applications.

[0080] Step 110: Receive a first input from a user;

[0081] In this step, the first input is used to input a user task instruction.

[0082] User task instructions are instructions issued by users to refrigeration equipment.

[0083] The first input may be in at least one of the following ways:

[0084] First, the first input may be a touch operation, including but not limited to a click operation, a slide operation, and a press operation.

[0085] In this embodiment, receiving the first input from the user may be receiving a touch operation of the user on the display area of the terminal display screen.

[0086] In order to reduce the user error operation rate, the effective area of the first input can be limited to a specific area, such as the upper middle area of the question-and-answer interface; or when the question-and-answer interface is displayed, the target control is displayed on the current interface, and the first input can be achieved by touching the target control; or the first input can be set to a continuous multiple tapping operation on the display area within a target time interval.

[0087] Secondly, the first input may be a physical key input.

[0088] In this embodiment, the terminal body is provided with a physical button corresponding to the question and answer, and receiving the user's first input can be receiving the operation of the user pressing the corresponding physical button; the first input can also be a combined operation of pressing multiple physical buttons at the same time.

[0089] Third, the first input may be voice input.

[0090] In this embodiment, when the terminal receives a voice such as "roast duck", it can trigger the broadcast of the corresponding cooking method of roast duck.

[0091] Of course, in other embodiments, the first input may also be in other forms, including but not limited to character input, etc., which can be determined according to actual needs and is not limited in this embodiment of the present application.

[0092] It is understandable that in different application scenarios, user task instructions may be different.

[0093] For example, when the user needs to cook a dish, the user task instruction may be a recipe for dish X.

[0094] For another example, when a user needs to know the food stored in a refrigeration device, the user task instruction may be "What other fruits are there in the refrigerator?"

[0095] In some embodiments, the user may input user task instructions through a display device provided on the refrigeration equipment.

[0096] In other embodiments, the user may also input user task instructions through a voice recognition device.

[0097] The specific input method of the user task instruction can be determined based on actual conditions and is not limited in this application.

[0098] Step 120: In response to the first input, determine a target sub-scene based on the user task instruction;

[0099] In this step, the target sub-scenario is the specific task scenario corresponding to the user task instruction.

[0100] For example, the target sub-scenario may be to query the recipe of a specific dish.

[0101] For another example, the target sub-scenario may be to query the food stored in a certain area of the refrigeration equipment.

[0102] During the actual execution process, natural language recognition can be performed based on user task instructions to determine the target sub-scenario.

[0103] Of course, in the actual implementation process, the target sub-scene can be determined in any feasible way, and this application does not limit it.

[0104] like Figure 2 As shown, in some embodiments, determining the target sub-scene based on the user task instruction may also include:

[0105] Determine the target scenario based on user task instructions;

[0106] Combine user task instructions and target scenarios to obtain task scenario information;

[0107] Input the mission scenario information into the target large model and obtain the target sub-scenario output by the target large model.

[0108] In this embodiment, the target scenario is a broad category of task scenarios corresponding to the user's task instructions.

[0109] Target scenarios may include: food management, health management, knowledge Q&A, and recipe recommendations.

[0110] The target sub-scene can be any sub-scene in the target scene.

[0111] The task scenario information is the information required to determine the target sub-scenario corresponding to the user task instruction.

[0112] The target big model can be a generative big model.

[0113] Generative large models are deep learning models with billions of parameters or more.

[0114] Continue to refer Figure 2 ,In the actual execution process, after obtaining the user task ,instructions, the user task instructions can be sent to the scenario ,distribution module, which includes various types of scenarios, such as ,recipe recommendation scenarios, user health management scenarios, and user ,daily chat scenarios.

[0115] The scene distribution module can be a classification network model such as a neural network and machine learning.

[0116] The specific model of the scene distribution module can be determined based on actual conditions and is not limited in this application.

[0117] The scene distribution module can output the target scene based on the user task instructions.

[0118] Continue to refer Figure 2 ,The scene distribution module can convert the format of the content corresponding to the ,user task instructions.

[0119] After receiving the user task instruction input by the user and determining the target scenario, the user task instruction can be spliced with the prompt project template to obtain the field "content: user task instruction, field: target scenario", that is, the task scenario information.

[0120] During the actual execution process, after obtaining the task scenario information, the task scenario information can be input into the target large model to obtain the target sub-scenario output by the target large model.

[0121] According to the large-model-based personalized voice question-and-answer method provided in the embodiment of the present application, a relatively broad large-category task scenario corresponding to the user task instruction is first determined through the user task instruction. Based on the large-category task scenario and the user task instruction, the specific task scenario under the large-category task scenario is further determined. Through layer-by-layer analysis, the difficulty of determining the target sub-scenario based on the user task instruction is reduced, and the determination efficiency and accuracy are improved.

[0122] During the actual execution process, after receiving the user task instruction, the user task instruction is responded to by performing natural language recognition on the user task instruction, extracting keywords in the user task instruction, determining the target sub-scene corresponding to the user task instruction, obtaining the user's identity information and the long-term basic information corresponding to the user task instruction and the short-term basic information corresponding to the target sub-scene at the current acquisition moment.

[0123] Step 130: Acquire the short-term basic information corresponding to the target sub-scene at the current acquisition moment, and acquire the user's identity information and the long-term basic information corresponding to the user's task instruction;

[0124] In this embodiment, the identity information is information that can distinguish different user identities.

[0125] The identity information can be obtained by the user actively providing it to the refrigeration device, or by the refrigeration device performing facial recognition on the user or voice recognition of the user's input of a user task instruction.

[0126] For example, during the actual execution process, after the user inputs the user task instruction, a page may pop up on the display device of the refrigeration equipment for the user to input identity information, such as user name or user nickname.

[0127] For another example, during the actual execution process, after the user inputs a user task instruction, the refrigeration equipment can perform facial recognition on the user through an image sensor to obtain the user's identity information.

[0128] The long-term basic information is information about users interacting with the refrigeration equipment that remains unchanged for a long time.

[0129] Long-term basic information can be user portrait information.

[0130] Long-term basic information includes but is not limited to: user name, user family members, user preferences, user long-term biometric information, user city, user gender, and events experienced by the user.

[0131] The short-term basic information is information that may change in a short period of time when a user interacts with the refrigeration device and is related to an application scenario corresponding to the user's task instruction.

[0132] It should be noted that the long-term basic information and short-term basic information required for different target sub-scenarios may be different.

[0133] The short-term basic information may be temporary information.

[0134] The short-term basic information represents at least one of the user's short-term biometric information, the user's interaction with the refrigeration device during the target time period, the storage information within the refrigeration device, and the refrigeration device's environmental information. The short-term biometric information represents the user's physical health status for the day.

[0135] For example, the short-term biometric information may indicate that the user has a fever today and the body temperature is 38 degrees Celsius.

[0136] Short-term biometric information can also include user-defined feature information, such as user dietary preferences and health management goals.

[0137] For example, the user's dietary preference may be that the user wants to eat a light diet today.

[0138] For example, the health management goal can be to ensure the current intake of protein and vitamins.

[0139] The target time period is a preset time period.

[0140] The specific duration of the target time period can be user-defined or determined based on actual conditions. For example, the target time period can be 1 day or half a day, etc., which is not limited in this application.

[0141] The interaction situation between the user and the refrigeration device during the target time period is the eating situation of the user during the target time period determined based on the interaction data between the user and the refrigeration device during the target time period.

[0142] For example, when the user takes out apples, chocolate, green vegetables, and pork today, it is determined that the user has eaten apples, chocolate, green vegetables, and pork today.

[0143] For another example, when the user puts potatoes, onions, chicken legs, and beef brisket in the food today, it is determined that the user tends to eat potatoes, onions, chicken legs, and beef brisket today.

[0144] The storage information in the refrigeration device is information related to the storage objects stored in the refrigeration device.

[0145] The storage information may include: the name of the storage object and the storage duration of the storage object.

[0146] The environmental information is the environmental information of the location where the refrigeration equipment is located.

[0147] Environmental information includes but is not limited to: season information, weather information, and temperature information.

[0148] It should be noted that the short-term basic information can change accordingly with different application scenarios.

[0149] For example, when a user wants to make a recipe recommendation, the user's physical condition today (i.e., short-term biometric information) and the ingredients stored in the refrigeration equipment (i.e., storage information) can be obtained.

[0150] For example, if a user needs to manage food, such as checking how many new apples were added today, the user's interaction with the refrigeration equipment today can be obtained. Short-term basic information In the same application scenario, the short-term basic information of the same user may also change.

[0151] Continuing with the example of a user wanting to recommend a recipe, the short-term biometric information obtained for the user today may indicate a cold, and the user's cold characteristics need to be considered when recommending a recipe. A week later, the user's short-term biometric information may change to asymptomatic and healthy. At this time, when recommending a recipe, compared with the previous recipe recommendation, whether the user has a cold can be ignored.

[0152] During the actual execution process, the user's long-term basic information and short-term basic information can be obtained based on the actual situation of the user's task instruction and the user's identity information.

[0153] In the actual implementation process, the user's long-term basic information and short-term basic information can be obtained in any feasible way.

[0154] For example, the specific content of the long-term basic information and the specific content of the short-term basic information to be acquired are displayed on a display device of the refrigeration equipment.

[0155] In other embodiments, after receiving the user task instruction, the refrigeration device may proactively query the user based on the specific content of the user task instruction, and ask the user to answer corresponding questions to obtain long-term basic information and short-term basic information.

[0156] For another example, a temperature sensor may be provided on the outside of the refrigeration equipment to collect temperature information of the environment in which the refrigeration equipment is located; the refrigeration equipment may also be connected to a network to retrieve information about the environment in which the refrigeration equipment is located.

[0157] Step 130: Input at least one of the long-term basic information and the short-term basic information and the user task instruction into the large model to obtain the question-answering result output by the large model.

[0158] In this step, the question-and-answer result is the question-and-answer result corresponding to the user's task instruction.

[0159] During the actual execution process, the user task instructions and the specific circumstances of the acquired long-term basic information and short-term basic information can be input into the big model. The big model analyzes and processes the obtained data and outputs the question-and-answer results corresponding to the user task instructions.

[0160] It is understandable that the acquired long-term basic information or short-term basic information may be empty. In the process of processing user task instructions, the user task instructions can be processed based on the actual situation of the acquired information to output the question and answer results.

[0161] Taking the user task instruction of "Recommend what to eat today?" as an example, at this time, based on the user task instruction, the user's long-term basic information, namely the user's preferences, the user's long-term biometric information, and the user's gender, etc., can be obtained. Based on the user's long-term biometric information, user preferences, and user gender, suitable dishes can be recommended to the user.

[0162] For example, if the user's long-term biometric information shows that he or she has diabetes, foods high in sugar should be avoided during the recommendation process.

[0163] In some embodiments, food may be recommended to the user based on the user's short-term biometric information.

[0164] For example, if the user has a sore throat today, we should try to recommend light food to the user and reduce the recommendation of spicy and irritating food.

[0165] In some embodiments, food may be recommended to the user based on the environmental information of the refrigeration device.

[0166] For example, when environmental information shows that the weather today is hot and the temperature is high, summer-cooling foods such as mung bean soup and ice jelly can be recommended to the user.

[0167] During the actual execution process, after receiving the user task instruction, the target sub-scenario is determined by performing natural language analysis on the user task instruction. Based on the user's identity information and user task instruction, the long-term basic information and short-term basic information to be obtained are limited. The short-term basic information to be obtained is further limited through the target sub-scenario, and the short-term basic information related to the target sub-scenario at the current collection moment is obtained to obtain the newer short-term basic information that may change and is related to the execution of the user task instruction, thereby ensuring the timeliness of the obtained short-term basic information. Based on the understanding of the overall characteristics of the user through the long-term basic information, the user's preferences at the current collection moment are further understood based on the newer short-term basic information, thereby personalizing the processing of the user task instruction, so that the question and answer results output by the large model based on the long-term basic information and the short-term basic information can be more in line with the user's usage needs at the current collection moment, thereby improving the accuracy of the output questions and improving the user experience.

[0168] According to the large-model-based personalized voice question-and-answer method provided in the embodiment of the present application, by receiving user task instructions, long-term basic information and short-term basic information related to the needs and matching the user's identity information are obtained based on the user's actual needs. The obtained information is used as prompt words for the large-model to process the user's task instructions, providing more detailed and comprehensive information for processing user tasks. Combined with the user's actual characteristics, personalized processing is performed to obtain question-and-answer results corresponding to the user instructions, thereby improving the accuracy of the question-and-answer results. The processing method is simple, and there is no need to build a complex database and perform complex natural language processing, thereby reducing the difficulty of maintenance. In addition, the obtained long-term basic information and short-term basic information can be applied to different task scenarios, have strong generalization capabilities, and enhance the user experience.

[0169] In some embodiments, obtaining the user's identity information and the long-term basic information corresponding to the user's task instruction, as well as the short-term basic information corresponding to the target sub-scene at the current acquisition moment, may include:

[0170] Obtain the user's identity information and the long-term basic information corresponding to the user's task instructions from the long memory database;

[0171] In the target sub-scenario, semantic classification is performed on the user task instructions to obtain the semantic classification results corresponding to the user task instructions;

[0172] When the user task instruction is determined based on the semantic classification result without semantic reasoning, the corresponding short-term basic information at the current collection moment is obtained from the short memory database;

[0173] When it is determined based on the semantic classification result that the user task instruction requires semantic reasoning, semantic reasoning is performed on the user task instruction to obtain a semantic reasoning result;

[0174] Based on the semantic reasoning results, short-term interaction data corresponding to the semantic reasoning results are obtained from the interaction data between the user and the refrigeration equipment;

[0175] Based on the semantic reasoning results and short-term interaction data, the corresponding short-term basic information at the current collection moment is obtained.

[0176] In this embodiment, the long memory database is a database related to long-term basic information that is constructed through continuous iteration based on different categories of identity information of users.

[0177] The long memory database is used to store data related to user profile information.

[0178] Different categories of identity information include, but are not limited to: user name, user family members, user preferences, user long-term biometric information, user city, user gender, and events experienced by the user.

[0179] It is understandable that the long memory database can store user portrait information of multiple users.

[0180] The short memory database is a database related to short-term basic information that is built based on the continuous iteration of interaction data between users and refrigeration equipment.

[0181] The short memory database is used to store temporary information other than user profiles.

[0182] The long memory database and the short memory database may be dynamically updated databases.

[0183] During the actual execution process, after receiving the user task instruction, the user's identity information can be determined first, and based on the identity information and the user task instruction, the target is retrieved from the long memory database to obtain the long-term basic information required to process the user task instruction.

[0184] Taking the user task instruction "What should I eat to be good for my health" as an example, the long-term basic information of the user issuing the instruction can be determined from the long-term basic information of multiple users based on the identity information of the user issuing the instruction, and the required long-term basic information can be determined based on the user's long-term biometric information required for "What should I eat to be good for my health" as an example, such as the user is overweight, has three highs, and has poor gastrointestinal health.

[0185] The semantic classification result is the classification result obtained by determining whether the user task instruction requires semantic reasoning in the target sub-scenario.

[0186] The semantic classification results include: semantic reasoning is required or not required.

[0187] During the actual execution process, user task instructions can be input into the classification model to obtain semantic classification results.

[0188] Taking the user task instruction "What did I take out of the refrigerator at 10 o'clock today" as an example, "What did I take out of the refrigerator at 10 o'clock today" is a relatively intuitive instruction. The refrigeration equipment can directly determine the question and answer result based on the time recorded in the log and the items taken out by the user. At this time, the semantic classification result corresponding to "What did I take out of the refrigerator at 10 o'clock today" can be obtained without the need for semantic reasoning.

[0189] When user task instructions do not require semantic reasoning, the corresponding short-term basic information at the current collection moment can be directly obtained from the pre-established short-term memory database.

[0190] Taking the user task instruction "What fruit did I eat today" as an example, during the actual execution process, the refrigeration equipment cannot accurately understand what the user ate today, but it can perform semantic reasoning on "What fruit did I eat today" and infer "What fruit did the user take out of the refrigerator today" as the user task instruction corresponding to "What fruit did I eat today". At this time, the semantic classification result corresponding to "What fruit did I eat today" can be required for semantic reasoning.

[0191] The short-term interaction data is interaction data related to the user's task instructions extracted from the interaction data between the user and the refrigeration equipment.

[0192] Taking the items taken out by the user today as an example, including apples, bananas, potatoes and eggplants, the short-term interaction data is the apples and bananas determined from apples, bananas, potatoes and eggplants based on "what fruits the user took out of the refrigerator today".

[0193] During the actual execution process, the semantic reasoning results and short-term interaction data can be spliced together to obtain the corresponding short-term basic information at the current collection moment.

[0194] like Figure 3 As shown, in the actual execution process, after the target sub-scene is determined, the target sub-scene can be input into the processing module corresponding to the target sub-scene to process the user task instruction.

[0195] It is understandable that if the user is a new user, that is, the user who inputs the user task instruction has not generated any interaction data with the refrigeration equipment before the current collection time, the retrieved long-term basic information may be empty. In this case, the search result may output None.

[0196] like Figure 4 As shown, when the user task instruction requires reasoning, the user task instruction can be input into the reasoning module to obtain the semantic reasoning result corresponding to the user instruction.

[0197] Continue to refer Figure 4 Based on the user task instructions, the interaction data related to the semantic reasoning results are obtained from the log storing the interaction data between the user and the refrigeration equipment, and the interaction data is preprocessed. The interaction data is input into the natural language processing parsing model to obtain the parsing results, and a reasoning prompt engineering template is designed. The parsing results are input into the reasoning prompt engineering template. Subsequently, the question and answer results can be obtained based on the filled-in reasoning prompt engineering template.

[0198] Based on the content of the inference prompt engineering template, we can extract apples and bananas from the user's interaction data with the refrigeration equipment, such as the actual situation of the ingredients taken out, that is, taking out apples, bananas, potatoes, and eggplants, and get "Please list the {fruits} taken out by the user: apples and bananas", and obtain the short-term basic information required to process the user's task instructions.

[0199] It should be noted that, in this application scenario, the long-term basic information queried based on the user task instruction may be empty.

[0200] Continue to refer Figure 4 In the actual execution process, based on the results of semantic reasoning, a prompt engineering template for rewriting user task instructions can be designed, key information in the user task instructions can be extracted, and the extracted key information can be filled into the designed prompt engineering template. The prompt engineering template filled with key information can be filled into the prompt engineering template of the input large model. The prompt engineering template filled with key information is used as the user task instruction, and based on the user task instruction, long-term basic information and short-term basic information can be obtained.

[0201] Taking the user task instruction "What fruits did I eat today" as an example, a template related to the semantic reasoning results can be designed. Some of the storage objects taken out of the refrigeration equipment today may be the question and answer results of "What fruits did I eat today".

[0202] All the storage objects that the user takes out from the refrigeration device today may be stored in the interaction log of the refrigeration device in the form of: User takes out ingredients: apple, banana, potato, and eggplant.

[0203] Based on the storage format of storage objects in the interaction log of the refrigeration equipment, a prompt engineering template for users to retrieve storage objects can be designed, for example, please list the {slot} retrieved by the user.

[0204] During the actual execution process, the key information in the user's task instruction, that is, the fruit, can be extracted, and the fruit can be filled into the template to obtain "Please list the {fruits} taken out by the user".

[0205] "Please list the {fruits} taken out by the user" can be used as the content that needs to be filled in the slot corresponding to the user task instruction in the prompt engineering template input to the large model.

[0206] Based on the query “Please list the {fruits} taken out by the user”, the corresponding short-term basic information at the current collection moment is obtained.

[0207] Continue to refer Figure 3 ,When user task instructions do not require semantic reasoning, long-term ,basic information and short-term basic information can be ,obtained directly from the long memory database and the short-term ,basic information.

[0208] The acquired long-term basic information and short-term basic information are filled into the prompt engineering template, and the filled prompt engineering template is input into the large model to obtain the question-answering results output by the large model.

[0209] Continue to refer Figure 3 In some embodiments, a prompt project template may be set corresponding to the target sub-scene. The prompt project template may include user task instructions. In addition, the acquired long-term basic information and short-term basic information may be stored in corresponding locations in the prompt project template.

[0210] For example, long-term basic information is filled into the user portrait information position in the prompt engineering template, short-term basic information is filled into the temporary information position, and the prompt engineering template filled with prompt information is input into the target large model to obtain the question and answer results output by the target large model.

[0211] In some embodiments, the prompt project template may also include specific task requirements and contextual information.

[0212] In this embodiment, the specific task information is background information that needs to be provided to the large model in the target sub-scene.

[0213] The specific task information may include requirements such as the role positioning of the target large model and the length limit of the output content.

[0214] For example, the specific task information may be that the length of the output content does not exceed 100 words.

[0215] In some embodiments, the long-term basic information related to the user's task instructions can be retrieved from the long-term memory database through a retrieval enhancement generation method.

[0216] In this embodiment, a Retrieval-Augmented Generation (RAG) method provides the large language model with information retrieved from a database, and uses the retrieved information as a basis for the large model to generate answers.

[0217] Retrieval-augmented generation can include two stages: retrieving context-relevant information and using the retrieved knowledge to guide the generation process.

[0218] The RAG method uses the information found by the retrieval algorithm as context to help the large model answer user task instructions; both the query and the retrieved context are injected into the prompt engineering template sent to the target large model.

[0219] Taking the user task instruction "Use the ingredients in my refrigerator to recommend me what to eat for dinner" as an example, the user task instruction is input into the scenario distribution module, the target scenario is determined to be recipe recommendation, the user task instruction is distributed to the recipe recommendation scenario, the actual needs of the user task instruction are determined, and based on the actual needs, long-term basic information is obtained from the long memory database, and short-term basic information is obtained from the short memory database.

[0220] The specific process of obtaining long-term basic information from the long memory database is:

[0221] The user task instruction "Use the ingredients in my refrigerator to recommend me what to eat for dinner" is input into the retrieval enhancement generation module to retrieve long-term basic information from the long memory database.

[0222] At this point, the retrieved knowledge fragments may include: {document: "Recommend dinner for me"; metadata: "User health status: The user has chronic hypertension and high uric acid"};

[0223] {document: "Recommend a recipe for me"; metadata: "User location: District B, City A"};

[0224] {document: "Recommended dinner"; metadata: "User basic information: Gender: Male; Age: 65 years old"}.

[0225] The retrieved long-term basic information can be filled in the position corresponding to the long-term basic information of the prompt project template.

[0226] Obtaining short-term basic information from the short-term memory database may include:

[0227] {Users added new ingredients today. Please use fresh ingredients first when recommending recipes. Ingredient information: beef brisket};

[0228] {The user ate radish today. Please consider the nutritional content of the above ingredients and do not recommend foods with the same nutritional value repeatedly};

[0229] The user has the following ingredients in their refrigerator: potatoes, onions, chicken legs, beef brisket, eggs, cabbage, beans, green vegetables, and pork. Please strictly follow the ingredients already in the refrigerator when making recommendations.

[0230] {It is autumn now, please choose seasonal recipe recommendations according to the season}.

[0231] The obtained short-term basic information can be filled in the corresponding position of the prompt project template short-term basic information.

[0232] After the long-term basic information and the short-term basic information are filled into the prompt project template, requirements for processing user task instructions and context information may be further added to obtain a filled prompt project template.

[0233] After filling in all the information in the prompt project template, the prompt information you get may be:

[0234] “Please recommend ingredients to users based on their basic conditions in their long-term basic information and their daily conditions in their short-term basic information.

[0235] Requirements: Refer to as much background information as possible, and it is strictly forbidden to create without background information.

[0236] Long-term basic information:

[0237] User basic information: Gender: Male; Age: 65 years old;

[0238] User location: Xicheng District, Beijing;

[0239] User health status: The user has chronic hypertension and high uric acid;

[0240] Short basic information:

[0241] The user added new ingredients today. When recommending recipes, please use fresh ingredients first. Ingredient information: beef brisket;

[0242] The user ate radish today. Please consider the nutritional content of the above ingredients and do not recommend foods with the same nutritional value repeatedly.

[0243] The user has the following ingredients in their refrigerator: potatoes, onions, chicken legs, beef brisket, eggs, cabbage, beans, green vegetables, and pork. Please strictly follow the ingredients already in the refrigerator for recommendations.

[0244] Now it is autumn, please choose seasonal recipe recommendations according to the season.

[0245] User task instruction: Recommend me what to eat for dinner using the ingredients in my refrigerator.

[0246] The above prompt information is input into the big model. Based on the prompt information, the big model processes the user task instructions and outputs the question and answer results.

[0247] According to the large model-based personalized voice question-answering method provided in the embodiment of the present application, long-term basic information is obtained from a pre-built long-memory database, and different methods are selected to obtain short-term basic information based on the actual situation of whether the user task instruction requires semantic reasoning. When the user task instruction requires reasoning, the short-term basic information is determined through the semantic reasoning results and interaction data. When the user task instruction does not require reasoning, the short-term basic information is obtained from the short-memory database. Different short-term basic information acquisition methods are set for different user task instructions to improve the accuracy of the acquired short-term basic information, and only information related to the task instruction is obtained. This can handle the user's personalized needs and reduce the complexity of task processing.

[0248] In some embodiments, before step 110, the method may further include:

[0249] Acquire sample interaction data between the user and the refrigeration device within a first short period at the current collection moment, as well as environmental information of the refrigeration device within the first short period;

[0250] The short memory database is updated based on the sample interaction data between the user and the refrigeration device in the first short time period, the environmental information of the refrigeration device in the first short time period, and the sample interaction data between the user and the refrigeration device in the previous short time period corresponding to the first short time period.

[0251] In this embodiment, the first short period is the period corresponding to the current acquisition moment.

[0252] The specific duration of the first short time period can be user-defined or determined based on actual conditions. For example, the first short time period can be 1 hour or 2 hours, etc., which is not limited in this application.

[0253] The previous short time period is the previous short time period closest to the first short time period.

[0254] For example, when the first short time period is a time period corresponding to 9:00 am to 10:00 am, the previous short time period is a time period corresponding to 8:00 am to 9:00 am.

[0255] Continue to refer Figure 5 Taking the first short-time period of 1 hour as an example, in the actual execution process, the sample interaction data between the user and the refrigeration equipment in the first short-time period at the current collection moment can be obtained by reading the user's interaction log in that hour.

[0256] In some embodiments, during the process of reading the interaction logs for that hour, the user's interaction logs may be integrated into a more reasonable format, such as "interaction object: user A, interaction content: Y".

[0257] In some embodiments, after obtaining the user's sample interaction data in the first short period and the sample interaction data in the previous short period, data cleaning can be performed on the sample interaction data to delete abnormal conversation content in the interaction log.

[0258] In this embodiment, the abnormal conversation content may be sample interaction data such as speech misrecognition and false wake-up.

[0259] Abnormal conversation content may also include noisy interactions and repeated interactions.

[0260] During the actual execution process, a set of sample interaction data can be deleted, but sample interaction data cannot be added or the content of the sample interaction data cannot be rewritten.

[0261] During the actual execution process, after obtaining the sample interaction data of the first short period, the sample interaction data of the first short period can be fused with the sample interaction data of the previous short period to obtain new sample interaction data of the first short period, and the short memory database is updated based on the new sample interaction data of the first short period and the environmental information of the refrigeration equipment in the first short period.

[0262] According to the large model-based personalized voice question-and-answer method provided in the embodiment of the present application, by obtaining the sample interaction data between the user and the refrigeration equipment within the first short period of the current collection moment, the latest sample interaction data can be obtained, and based on the latest sample interaction data and the sample interaction data of the previous short period, new sample interaction data of the first short period is obtained. Based on the new sample interaction data of the first short period and the environmental information of the refrigeration equipment in the first short period, the short memory database is updated. There is no need to store the sample interaction data of each period, which reduces content redundancy and constructs a concise and small-data-volume short memory database.

[0263] In some embodiments, updating the short memory database based on sample interaction data between the user and the refrigeration device in a first short period, information about the environment in which the refrigeration device is located in the first short period, and sample interaction data between the user and the refrigeration device in a previous short period corresponding to the first short period includes:

[0264] Performing semantic classification on the sample interaction data between the user and the refrigeration device in the first short period to obtain a sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration device;

[0265] Processing the sample interaction data between the user and the refrigeration device within a first short period based on the sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration device and the target time;

[0266] A short-term information summary is performed based on the processed interaction data between the user and the refrigeration device in the first short-term period, the environmental information of the refrigeration device in the first short-term period, and the interaction data between the user and the refrigeration device in the previous short-term period to update the short memory database.

[0267] In this embodiment, the sample interaction data between the user and the refrigeration device in the first short period may be text generated by the refrigeration device based on the user's interaction actions, or may be voice interaction information between the user and the refrigeration device.

[0268] For example, when a user directly takes out two apples from the refrigeration device, the interaction log of the refrigeration device may record that at D time E minute on C day, user X took out two apples.

[0269] For another example, the user's voice interaction message may be: Bring me two apples.

[0270] The sample semantic classification result is the classification result obtained by determining whether semantic reasoning is required for the sample interaction data.

[0271] During the actual implementation process, the sample interaction data between the user and the refrigeration equipment can be input into the classification model to obtain the sample semantic classification results.

[0272] The target time is a preset time.

[0273] The specific value of the target time can be user-defined or determined based on actual conditions. For example, the target time can be 0:00 or 5:00 in the morning, which is not limited in this application.

[0274] During the actual execution process, after determining the sample semantic classification results of the user's task instructions, the sample interaction data between the user and the refrigeration equipment in the first short time period can be processed differently based on the different situations of the sample semantic classification results, through the sample interaction data and the target time, so as to obtain the latest sample interaction data between the user and the refrigeration equipment in the first short time period, which represents the interaction situation between the user and the refrigeration equipment in the first short time period.

[0275] The processed interaction data between the user and the refrigeration device in the first short period and the interaction data between the user and the refrigeration device in the previous short period are fused, and the short memory database is updated based on the fused data and the environmental information of the refrigeration device in the first short period.

[0276] According to the large model-based personalized voice question-and-answer method provided in the embodiment of the present application, by processing the sample interaction data between the user and the refrigeration device in the first short time period, the environmental information of the refrigeration device in the first short time period, and the sample interaction data between the user and the refrigeration device in the previous short time period, the information in the short memory database is jointly updated, thereby dynamically updating the short memory database to ensure the real-time and accuracy of the data in the short memory database.

[0277] In some embodiments, processing the sample interaction data between the user and the refrigeration appliance within the first short period based on the sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration appliance and the target time includes:

[0278] When it is determined based on the sample semantic classification result that no semantic reasoning is required for the sample interaction data between the user and the refrigeration appliance, retaining the sample interaction data between the user and the refrigeration appliance within the first short period;

[0279] When it is determined based on the sample semantic classification result that semantic reasoning is required for the sample interaction data between the user and the refrigeration device, semantic reasoning is performed on the sample interaction data between the user and the refrigeration device during the target period to obtain a sample reasoning result;

[0280] Based on the sample inference results, obtain the sample short-term interaction data corresponding to the sample inference results from the sample interaction data between the user and the refrigeration equipment during the target period;

[0281] Based on the sample reasoning result and the sample short-time interaction data, the sample interaction data between the user and the refrigeration device in the first short-time period is updated.

[0282] In this embodiment, the target time period is a time period determined based on the target time and the first short time period.

[0283] Taking the target time as 0 o'clock and the first short-time period as the period corresponding to 3 pm to 4 pm as an example, the target time period can be a time period determined based on 0 o'clock to 4 o'clock, and the target time period can also be a time period determined based on the time when the sample interaction data that needs to be inferred within 0 o'clock to the first short-time period ends.

[0284] The sample reasoning result is the reasoning result that represents the user's usage intention obtained by performing natural language reasoning on the user's task instructions.

[0285] For example, when the user's interaction data is taking out two carrots, the sample inference result is removal.

[0286] For example, when the user's interaction data is to put in beef, the sample inference result is to add.

[0287] When the sample reasoning result is removal, it is determined that the user's usage intention is to eat the removed storage object, that is, the user eats two radishes within the first short time period.

[0288] When the sample reasoning result is addition, it is determined that the user's usage intention is to tend to eat the newly added storage object, that is, the user tends to eat beef in the future.

[0289] During the actual execution process, when it is determined based on the sample semantic classification results that the sample interaction data between the user and the refrigeration equipment does not require semantic reasoning, the form of the sample interaction data between the user and the refrigeration equipment within the first short time period can be kept unchanged, and the short memory database can be updated based on the sample interaction data between the user and the refrigeration equipment within the first short time period.

[0290] When it is determined based on the sample semantic classification results that semantic reasoning is required for the sample interaction data between the user and the refrigeration device, the question and answer data in the sample interaction data between the user and the refrigeration device in the first short time period corresponding to the target suitability to the current acquisition moment is analyzed and reasoned to update the sample interaction data between the user and the refrigeration device in the first short time period.

[0291] In the actual implementation process, different prompt engineering templates can be designed for different sample inference results of question and answer data.

[0292] For example, when the inference result is removal, the prompt engineering template can be: {The user ate: [slot] today. Please consider the nutritional composition of the above ingredients and do not repeatedly recommend foods containing the same nutrients}.

[0293] For example, when the inference result is addition, the prompt project template can be: {The user added new ingredients today, please use fresh ingredients first when recommending recipes, ingredient information: [slot]}.

[0294] During the actual execution process, the interaction data between the user and the refrigeration equipment in the first short period can be updated based on the user usage intention represented by the inference result. For example, the storage objects involved in the interaction data are filled in the prompt project template corresponding to the inference result.

[0295] It is understandable that after the user interacts with the refrigeration device, the stored information in the refrigeration device will also change. At this time, the stored information in the refrigeration device at the current collection moment can be updated based on the interaction data of each stage.

[0296] Similarly, a prompt engineering template may be designed for the storage information in the refrigeration equipment and the environmental information of the refrigeration equipment.

[0297] The project template for prompting storage information in the refrigeration equipment can be: {The user's refrigerator has the following ingredients: [slot]. Please strictly follow the ingredients already in the refrigerator for recommendations.}

[0298] A project template for prompting environmental information of the refrigeration equipment may be: {The current season is: [slot], please select seasonal recipe recommendations based on the season}.

[0299] According to the large model-based personalized voice question-answering method provided in the embodiment of the present application, by determining whether the sample interaction data between the user and the refrigeration device requires semantic reasoning, different methods are used to process the sample interaction data between the user and the refrigeration device. If reasoning is required, the sample interaction data is first reasoned to obtain a sample reasoning result. Based on the sample reasoning result and the sample interaction data, the sample interaction data between the user and the refrigeration device in the first short time period is updated. If reasoning is not required, the sample interaction data between the user and the refrigeration device in the first short time period is maintained. The method of processing the sample interaction data between the user and the refrigeration device in the first short time period can be flexibly selected to improve the availability of the processed sample interaction data.

[0300] In some embodiments, performing semantic reasoning on sample interaction data between the user and the refrigeration appliance within the target time period to obtain a sample reasoning result may further include:

[0301] Perform word segmentation on the sample interaction data between each user and the refrigeration equipment during the target period to obtain the word segmentation results corresponding to each sample interaction data;

[0302] Extract the operation object and operation content in each sample interaction data from the word segmentation results;

[0303] Based on the operation object and operation content, a sample inference result of the user in the first short time period is obtained.

[0304] In this embodiment, the word segmentation result is the result obtained by dividing the sample interaction data into individual characters or individual words.

[0305] The operation object is the object involved in the interaction between the user and the refrigeration equipment.

[0306] The operation content refers to the operation content of the user on the operation object.

[0307] For example, the operation content may be taking out the operation object or putting in the operation object.

[0308] In the actual execution process, you can use Jieba to segment the sample interaction data, and design and fill the business logic slots based on the segmentation results.

[0309] Taking the sample interaction data of "take two apples" as an example, the word segmentation result of analyzing "take two apples" can be: ["take", "two", "apple"].

[0310] According to the word segmentation results, the designed business logic slot can be: [{"take":None},{"two":None},{"apple":None}].

[0311] Combined with business logic to fill in Figure 6 Business logic slots shown.

[0312] Based on the filled business logic slots, a prompt engineering template related to usage intent can be designed.

[0313] The prompt project template related to the usage intention can be:

[0314] Template (before filling): User has consumed fruit: {slot}.

[0315] Template (after filling): User has consumed fruit: apple.

[0316] According to the large-model-based personalized voice question-answering method provided in the embodiment of the present application, by segmenting the sample interaction data, key information in the sample interaction data is effectively extracted, thereby determining the sample inference results of the sample interaction data, converting the sample interaction data into a more intuitive form, and improving the processing efficiency of voice question-answering.

[0317] In some embodiments, updating the short memory database based on the updated sample interaction data between the user and the refrigeration appliance during the first short period, the environmental information of the refrigeration appliance during the first short period, and the sample interaction data between the user and the refrigeration appliance during the previous short period may further include:

[0318] Obtaining first short-term summary information based on sample interaction data between the user and the refrigeration device in a previous short period and environmental information of the refrigeration device in the previous short period;

[0319] Obtaining second short-term summary information based on the updated sample interaction data between the user and the refrigeration device during the first short-term period and the environmental information of the refrigeration device during the first short-term period;

[0320] Replacing the environmental information of the refrigeration equipment in the previous short period in the first short summary information with the environmental information of the refrigeration equipment in the first short period in the second short summary information;

[0321] The sample interaction data between the user and the refrigeration device in the previous short period in the first short summary information and the updated sample interaction data between the user and the refrigeration device in the first short period in the second short summary information are concatenated to update the short memory database.

[0322] In this embodiment, the first short-term summary information is information obtained by summarizing sample interaction data between the user and the refrigeration device in a previous short period and environmental information of the refrigeration device.

[0323] The second short-term summary is information obtained by summarizing the sample interaction data between the user and the refrigeration device within the first short period of time, as well as the environmental information of the refrigeration device.

[0324] Continue to refer Figure 5 In the actual execution process, the sample interaction data between the user and the refrigeration equipment in the first short period and the environmental information of the refrigeration equipment in the first short period can be input into the target large model to obtain the summary information corresponding to the first short period. The target large model can analyze the sample interaction data in the first short period and organize it according to the needs to obtain the second short-term summary information.

[0325] In some embodiments, the target big model may organize the interaction data of the first short period into a key-value format.

[0326] It is understandable that the second short-term summary information does not involve the user portrait in the long memory database, so as to reduce the conflict between the short memory database and the long memory database.

[0327] It can be understood that the sample interaction data between the user and the refrigeration equipment in the previous short period and the environmental information of the refrigeration equipment in the previous short period can be processed in the same way as the interaction data between the user and the refrigeration equipment in the first short period to obtain the first short-term summary information. To avoid repetition, they are not described here.

[0328] Continue to refer Figure 5 In some embodiments, after obtaining the first short summary information and the second short summary information, the first short summary information and the second short summary information may be spliced together, and the spliced data may be input into the target large model to generate new short summary information.

[0329] The target large model may replace the environmental information of the refrigeration equipment in the previous short period in the first short period summary information with the environmental information of the refrigeration equipment in the first short period in the second short period summary information.

[0330] For the newly added interaction information, the updated interaction data between the user and the refrigeration device in the first short period in the second short summary information can be directly spliced to the interaction data between the user and the refrigeration device in the previous short period in the first short summary information to obtain new short summary information to update the short memory database.

[0331] It should be noted that, in the process of collating and analyzing the first short summary information and the second short summary information, the target large model cannot summarize the summary or output new information not included in the first short summary information and the second short summary information.

[0332] In actual execution, after the first short summary information and the second short summary information are input into the target large model, new short summary information output by the target large model can be obtained, and the short memory database is updated based on the new short summary information.

[0333] In the actual execution process, the new short-term summary information does not need to be vectorized and stored in the vector database, but can be directly spliced with the prompt project in symbolic form.

[0334] like Figure 7 As shown, in the actual execution process, after receiving the user task instruction input by the user, the target sub-scenario corresponding to the user task instruction can be obtained through the scenario distribution module. Based on the target sub-scenario, the user task instruction is processed through the reasoning construction and non-reasoning retrieval modules to obtain the filled prompt word template, and the filled prompt word template is input into the large model to obtain the question and answer result.

[0335] The inference construction and non-inference retrieval modules can be based on Figure 4 The process shown processes user task instructions.

[0336] In addition, the inference model 1 can be used to perform Figure 4 The reasoning module of the left branch process, reasoning module 2 can be used to execute Figure 4 The reasoning module of the right branch.

[0337] Of course, the inference model 1 can be used to perform Figure 4 The reasoning module of the right branch process, reasoning module 2 can be used to execute Figure 4 The reasoning module of the left branch.

[0338] According to the large model-based personalized voice question-and-answer method provided in an embodiment of the present application, first short-term summary information is obtained through sample interaction data between the user and the refrigeration device in the previous short period and environmental information of the refrigeration device in the previous short period. Second short-term summary information is obtained through sample interaction data between the user and the refrigeration device in the first short period and environmental information of the refrigeration device in the first short period. The interaction data of each period is effectively analyzed and organized to obtain the short-term summary information corresponding to each period. The short-term summary information obtained is updated, and the short-term summary information of the two periods is simplified and summarized, thereby reducing content redundancy, waste of computing resources, and delay problems caused by the high update frequency.

[0339] In some embodiments, before step 110, the method may further include:

[0340] A long memory database is constructed based on the user's long-term biometric information and basic information.

[0341] In this embodiment, the long-term biometric information is the user's health information that remains unchanged over a long period of time.

[0342] For example, the long-term biometric information may be information about the user's chronic disease.

[0343] The user's basic information may include: gender, age, region, preferences, family members, and user experience.

[0344] During the actual implementation process, the user's long-term biometric information and the user's basic information can be collected by providing a question-and-answer page to the user.

[0345] Of course, in the actual implementation process, the user's long-term biometric information and basic information can also be obtained through any other feasible methods.

[0346] It is understandable that the long-term biometric information and basic information of different users may be different. Different storage areas can be established for different users, and the long-term biometric information and basic information of multiple users can be stored in a long memory database to build a long memory database.

[0347] According to the large model-based personalized voice question-and-answer method provided in the embodiment of the present application, by storing the user's long-term biometric information and the user's basic information, the background information of different users is effectively stored, the data in the long memory database is enriched, and comprehensive background information is provided for subsequent voice questions and answers, thereby improving the user experience.

[0348] In some embodiments, building a long memory database based on the user's long-term biometric information and the user's basic information may also include:

[0349] Obtain at least one user corpus;

[0350] Constructing an association relationship between the user's long-term biometric information and at least one of the user's basic information and the user corpus;

[0351] The long-term biometric information, the basic information, and the association between at least one of the long-term biometric information and the basic information and the user corpus are stored in a long memory database.

[0352] In this embodiment, the user corpus is sentences that may be involved in the process of voice interaction between the user and the refrigeration device.

[0353] For example, user corpus can be: recommend a recipe to me, recommend a dish to me, and what I should eat to be healthy, etc.

[0354] The association relationship is whether one or more of the long-term biometric information and the user's basic information is needed in the process of processing the user corpus.

[0355] For example, the association relationship may be: when the user corpus is "What should I eat to be healthy?", the user's physical health information, that is, long-term biometric information, is required. At this time, an association relationship between "What should I eat to be healthy?" and long-term biometric information can be constructed.

[0356] During the actual execution process, the long-term biometric information, basic information, and the association between one or more of the long-term biometric information and basic information and the user corpus can be stored in the long memory database to construct a long memory database.

[0357] It should be noted that if the key-value pairs are directly vectorized and stored, in actual retrieval, if the retrieval is directly based on the user's task instructions, the required long-term basic information may not be retrieved.

[0358] The reason is that long-term basic information and user task instructions may be unrelated in semantic similarity.

[0359] For example, a user's long-term biometric information can be used as background information for recommending recipes to the user. However, in actual vector distance calculation, the user's long-term biometric information may be far away from the user's recommended recipe information.

[0360] By constructing the association between long-term biometric information and one or more of the basic information and the user's task instructions, the problem of irrelevant semantic similarity in the retrieval process can be alleviated.

[0361] In the actual implementation process, Figure 8The key-value information shown is converted into Figure 9 Slice data shown.

[0362] Among them, document is a category in the long-term basic information and is used to be stored in the vector library. Metadatas is the detailed information of the long-term basic information and does not participate in vector embedding and storage.

[0363] In the actual retrieval process, the user task instructions can be embedded into vectors and similarity calculated with the embedded documents in the vector library.

[0364] In the actual implementation process, Figure 10 As shown, multiple user corpora can be collected to create a Mapping corpus.

[0365] It can be understood that the Mapping corpus is a pre-established table used to build a long memory database.

[0366] like Figure 11 As shown, different user corpora can construct different data.

[0367] After determining the Mapping corpus, a complete slice file can be constructed by combining the user's long-term biometric information and basic information.

[0368] like Figure 12 As shown, each user corpus in the Mapping corpus can be compared with Figure 6 Construct a new slice using the metadatas information in the slice data shown.

[0369] It is understandable that the Mapping corpus can be updated dynamically, that is, it can be continuously updated and added according to business needs.

[0370] In the actual execution process, the Mapping corpus can be iterated through the large model.

[0371] like Figure 13 As shown, a prompt engineering template used for iteration can be designed in advance, and the prompt engineering template is input into the large model to obtain the output result of the large model.

[0372] The output result of the large model can be a category of long-term basic information, such as the user's long-term biometric information and the user's basic information.

[0373] The Mapping corpus corresponding to the category of the long-term basic information shown in the output result is taken out, and the user corpus is filled into the Mapping corpus, the slice data is reconstructed, and the reconstructed slice data is stored in the vector library.

[0374] Among them, the prompt engineering template used for iteration needs to include the categories of long-term basic information included in the long memory database and user corpus.

[0375] According to the large-model-based personalized voice question-and-answer method provided in the embodiment of the present application, by obtaining a variety of user corpora and establishing an association relationship between the user's long-term biometric information and one or more of the user's basic information and the user corpora, on the basis of storing the user's long-term basic information, the association relationship between each category of data in each long-term basic information and the user corpus is further stored, thereby improving the accuracy of obtaining long-term basic information based on user task instructions.

[0376] like Figure 14 As shown, in some embodiments, before step 110, the method may further include:

[0377] Acquire interaction data between the user and the refrigeration device within a first long period at the current collection moment;

[0378] A long memory database is constructed based on interaction data between the user and the refrigeration device within a first long period at a current collection moment and interaction data between the user and the refrigeration device within at least one first long period before the first long period at the current collection moment.

[0379] In this embodiment, the first long-term period is a preset period for processing the interaction data to obtain the long-term basic information of the user.

[0380] The specific value of the first long period can be user-defined or determined based on actual execution conditions; for example, the first long period can be one day or half a day, which is not limited in this application.

[0381] It is understandable that the first long time period may include multiple sets of interaction data. Taking the first long time period as one day as an example, at 9 a.m., user A may interact with the refrigeration equipment, generating a set of interaction data, and at 12 noon, user B may interact with the refrigeration equipment, generating a set of interaction data.

[0382] Interaction data refers to data generated by operations such as questions and answers between users and refrigeration equipment.

[0383] Interaction data can be obtained by reading the interaction log of the refrigeration equipment.

[0384] In some embodiments, the interaction data may be organized into the same format, such as "interaction object: A, interaction content: X".

[0385] The at least one previous first long time period is one or more first long time periods before the first long time period corresponding to the current acquisition moment.

[0386] Taking Wednesday as an example, the first long period corresponding to the current collection moment may be Tuesday and Monday.

[0387] It is understandable that if the first long time period corresponding to the current acquisition moment is different, the at least one previous first long time period corresponding thereto will also be different.

[0388] Continue to refer Figure 14 In some embodiments, after obtaining the interaction data between the user and the refrigeration device, abnormal conversation content in the interaction data may be deleted.

[0389] In this embodiment, the abnormal conversation content may be interaction data such as speech misrecognition and false wake-up.

[0390] Abnormal conversation content may also include noisy interactions and repeated interactions.

[0391] During the actual execution process, a set of interaction data can be deleted, but interaction data cannot be added or the content of the interaction data cannot be rewritten.

[0392] In some embodiments, a long memory database may be constructed using a strategy of "keeping peak values unchanged and updating valley values."

[0393] In this embodiment, the interaction data may be processed during a period when the user does not interact with the refrigeration device to build a long memory database. For example, the refrigeration device processes the interaction data at night or during lunch breaks to build a long memory database.

[0394] During the interaction between the user and the refrigeration appliance, the refrigeration appliance may store the interaction data.

[0395] It can be understood that the objects interacting with the refrigeration device may be different in different first long time periods.

[0396] According to the large model-based personalized voice question-and-answer method provided in the embodiment of the present application, the user's long-term basic information is jointly determined by obtaining the interaction data between the user and the refrigeration equipment for multiple first long-term periods before the first long-term period corresponding to the current collection moment and the interaction data for the first long-term period corresponding to the current collection moment. By iterating the data of different first long-term periods, the user's long-term basic information is continuously enriched, thereby improving the accuracy of the determined long-term basic information and improving the data quality of the constructed long-memory database.

[0397] In some embodiments, constructing a long memory database based on interaction data between a user and a refrigeration appliance within a first long period at a current collection time and interaction data between the user and the refrigeration appliance within at least one first long period before the first long period at the current collection time may include:

[0398] Processing interaction data between the user and the refrigeration device in at least one previous first long-term period to obtain at least one first long-term summary information;

[0399] Processing the interaction data between the user and the refrigeration device in the first long-term period at the current collection moment to obtain second long-term summary information;

[0400] A long memory database is constructed based on the first long-term summary information and the second long-term summary information.

[0401] In this embodiment, the first long-term summary information is information obtained by analyzing and arranging interaction data between the user and the refrigeration device within a first long-term period before the first long-term period corresponding to the current collection moment.

[0402] The second long-term summary information is information obtained by analyzing and arranging the interaction data between the user and the refrigeration equipment in the first long-term period corresponding to the current collection moment.

[0403] In some embodiments, the first long summary information and the second long summary information may be output in a target format, such as a JSON format, to focus on the key and specific value of the interaction data.

[0404] In some embodiments, after obtaining the interaction data between the user and the refrigeration device in at least one previous first long period and the interaction data between the user and the refrigeration device in the first long period corresponding to the current collection moment, multiple groups of interaction data in each first long period can be spliced together to obtain spliced interaction data.

[0405] In some embodiments, the interaction data may be spliced sequentially based on the time when it was generated.

[0406] In other embodiments, the splicing may be performed sequentially based on the interaction objects corresponding to the interaction data.

[0407] During the actual execution process, the interaction data can be spliced with the iterative prompt project, or directly spliced into the prompt words, or uploaded synchronously after forming a document to obtain the target interaction data.

[0408] In an actual execution process, after the spliced interaction data is obtained, the spliced interaction data may be processed to construct a long memory database.

[0409] Continue to refer Figure 14 In some embodiments, after obtaining the interaction data, the interaction data can be input into the target macro model to obtain summary information.

[0410] In the actual implementation process, after obtaining the interaction data between the user and the refrigeration device in each first long period, the interaction data between the user and the refrigeration device in each first long period can be analyzed by using the target large model.

[0411] During the analysis process, the target big model can focus on information such as user name, user family members, user preferences, user long-term biometric information, user city, user gender, and events experienced by the user in the interaction data to obtain multiple first long-term summary information and second long-term summary information.

[0412] The user name, user family members, user preferences, user long-term biometric information, user city, user gender, and events experienced by the user in the interaction data can be the default information that the target large model needs to pay attention to.

[0413] The information that the target large model needs to pay attention to by default can be based on user customization or determined based on actual execution conditions, and this application does not limit it.

[0414] Of course, the target large model can also focus on other data information, and this application does not limit the specific circumstances of other data information.

[0415] When the target large model focuses on other information, a new field such as "mechine" can be added to mark the title and content of the new information.

[0416] In actual execution, after obtaining the plurality of first long-term summary information and the second long-term summary information, the plurality of first long-term summary information and the second long-term summary information may be stored in a long memory database for subsequent use.

[0417] During actual execution, the plurality of first long-term summary information and second long-term summary information may be further processed, such as sorted and integrated, to construct a long memory database.

[0418] According to the large model-based personalized voice question-and-answer method provided in the embodiment of the present application, by processing the interaction data of the first long-term period of the current collection moment and the interaction data between the user and the refrigeration equipment in the previous multiple first long-term periods, the long-term summary information corresponding to each first long-term period is obtained respectively. Based on the new interaction data and the historical interaction data, the long-term basic information of the user is jointly determined, thereby improving the accuracy and completeness of constructing the long-memory database and improving the data quality of the long-memory database.

[0419] In some embodiments, constructing a long memory database based on the first long-term summary information and the second long-term summary information may include:

[0420] splicing the first long-term summary information based on the time sequence to obtain third long-term summary information corresponding to at least one previous first long-term period;

[0421] A long memory database is constructed based on the second long-term summary information and the third long-term summary information.

[0422] In this embodiment, the third long summary information is summary information obtained by concatenating multiple first long summary information.

[0423] In some embodiments, each piece of first long-term summary information may be concatenated based on a time sequence.

[0424] In other embodiments, the first long-term summary information may be further scored based on at least one of a time sequence and an interactive object.

[0425] During actual execution, after obtaining multiple first long-term summary information, the multiple first long-term summary information can be spliced to obtain summary information corresponding to multiple first long-term periods before the first long-term period at the current acquisition moment (i.e., third long-term summary information), thereby constructing a long memory database based on the old summary information (i.e., third long-term summary information) and the new summary information (i.e., second long-term summary information).

[0426] According to the large model-based personalized voice question and answer method provided in the embodiment of the present application, through the first long-term summary information and the second long-term summary information, the user's long-term basic information can be jointly determined based on historical interaction data and interaction data of the period in which the current collection moment is located, thereby improving the accuracy of the long-term basic information, providing data support for subsequent large model-based personalized voice question and answer, providing personalized services to users, and improving the user experience.

[0427] like Figure 15 As shown, in some embodiments, constructing a long memory database based on the second long-term summary information and the third long-term summary information may include:

[0428] Acquire a first feature category included in the third long-term summary information and a second feature category included in the second long-term summary information;

[0429] Based on the data corresponding to the second feature category, the data corresponding to the first feature category that is the same as the second feature category in the first feature category is updated, and the data corresponding to the feature category that is different from the first feature category in the second feature category is updated to the third long-term summary information; thereby obtaining the fourth long-term summary information.

[0430] The fourth long-term summary information is stored in the long memory database.

[0431] In this embodiment, the first feature category is the feature representing the user's long-term basic information included in the third long-term summary information.

[0432] The second feature category is the feature representing the long-term basic information of the user included in the second long-term summary information.

[0433] It can be understood that the number of first feature categories will continue to increase as the number of iterations increases.

[0434] The fourth long-term summary information is summary information obtained by updating the third long-term summary information.

[0435] In actual execution, after obtaining the third long summary information and the second long summary information, information corresponding to feature categories in the third long summary information and the second long summary information may be extracted and compared to determine whether they need to be updated.

[0436] It is understandable that the second long-term summary information is summary information of the most recent interaction data, and the old long-term summary information (ie, the third long-term summary information) may be updated based on new information.

[0437] For example, if the second feature category of the second long summary information includes the user's age, and the first feature category of the third long summary information also includes the user's age, and the second long summary information can obtain that the user's age has increased by 1 year based on the interaction data between the user and the refrigeration device, the data corresponding to the user's age included in the first feature category of the third long summary information can be updated to a value obtained by adding 1 to the currently stored age.

[0438] For another example, if the second feature category of the second long summary information includes user allergens, but the first feature category of the third long summary information does not include user allergens, the data corresponding to the user allergens can be updated to the third long summary information to obtain fourth long summary information.

[0439] In actual execution, after obtaining the plurality of first long summary information and second long summary information, the plurality of first long summary information and second long summary information may be input into the target large model to obtain fourth long summary information output by the target large model.

[0440] During the actual execution process, the target large model splices multiple first long-term summary information to obtain the third long-term summary information. The target large model analyzes the new and old summary information (i.e., the second long-term summary information and the third long-term summary information), updates the data corresponding to the same feature category, and directly splices the newly added information after the old summary information (i.e., the third long-term summary information). After sorting and analyzing the new and old summary information, the target large model outputs the fourth long-term summary information.

[0441] It is understandable that in order to ensure the accuracy of the summary information, the target large model does not need to summarize the summary or output additional new content.

[0442] According to the large-model-based personalized voice question-and-answer method provided in an embodiment of the present application, the new second long summary information is analyzed and compared with the third long summary information, and the data of the feature categories also stored in the third long summary information is updated using the new second long summary information. The feature categories included in the second long summary information but not included in the third long summary information are supplemented to the third long summary information, thereby effectively updating the third long summary information, continuously enriching the data in the fourth long summary information obtained based on the third long summary information, and improving the data quality of the fourth long summary information.

[0443] Continue to refer Figure 15 In some embodiments, storing the fourth long-term summary information in the long memory database may include:

[0444] performing at least one of slicing processing and vector embedding processing on the fourth long-term summary information to obtain target long-term summary information;

[0445] Store the target long-term summary information into the long memory database.

[0446] In this embodiment, the target long-term summary information is summary information obtained by processing the fourth long-term summary information.

[0447] Slicing is done to extract the required information from the target long-term summary information.

[0448] Vector embedding is the process of converting target long-term summary information into vector information.

[0449] In the actual execution process, after obtaining the fourth long-term summary information, the long-term summary information can be sliced and vector embedded to obtain summary information represented by a vector, and the summary information represented by the vector is stored in the long memory database.

[0450] The long memory database may be a vector database.

[0451] Vector Database is a database system used to store and query vector data.

[0452] In the actual execution process, after obtaining the fourth long-term summary information, the fourth long-term summary information can be sliced and vector embedded, and the processed data is stored in the long memory database. At this time, the long memory database is a vector database.

[0453] According to the large model-based personalized voice question-answering method provided in the embodiment of the present application, the fourth long-term summary information is sliced to improve the flexibility of data selection, and the fourth long-term summary information is vector embedded to reduce the complexity of the long memory database, thereby improving the storage efficiency and retrieval efficiency of the long memory database.

[0454] The large-model-based personalized voice question-answering method provided in the embodiments of this application can be executed by a large-model-based personalized voice question-answering device. In the embodiments of this application, the large-model-based personalized voice question-answering device executing the large-model-based personalized voice question-answering method is used as an example to illustrate the large-model-based personalized voice question-answering device provided in the embodiments of this application.

[0455] The embodiment of the present application also provides a personalized voice question-answering device based on a large model.

[0456] like Figure 16 As shown, the large model-based personalized voice question-answering device includes: a first processing module 1610 , a second processing module 1620 and a third processing module 1630 .

[0457] The first processing module 1610 is configured to receive a first input from a user; the first input is used to input a user task instruction;

[0458] A second processing module 1620 is configured to determine a target sub-scene in response to the first input and based on the user task instruction;

[0459] The third processing module 1630 is configured to obtain short-term basic information corresponding to the target sub-scene at the current acquisition time, as well as user identity information and long-term basic information corresponding to the user's task instruction; the short-term basic information is used to represent at least one of the user's short-term biometric information, the user's interaction with the refrigeration device during the target time period, storage information in the refrigeration device, and environmental information of the refrigeration device;

[0460] The fourth processing module 1640 is used to input at least one of the long-term basic information and the short-term basic information and the user task instruction into the large model to obtain the question and answer results output by the large model.

[0461] According to the large-model-based personalized voice question-and-answer device provided in the embodiment of the present application, by receiving user task instructions, long-term basic information and short-term basic information related to the needs and matching the user's identity information are obtained based on the user's actual needs. The obtained information is used as prompt words for the large-model to process the user's task instructions, providing more detailed and comprehensive information for processing user tasks. Combined with the user's actual characteristics, personalized processing is performed to obtain question-and-answer results corresponding to the user instructions, thereby improving the accuracy of the question-and-answer results. The processing method is simple, and there is no need to build a complex database and perform complex natural language processing, which reduces the difficulty of maintenance. In addition, the obtained long-term basic information and short-term basic information can be applied to different task scenarios, have strong generalization capabilities, and enhance user experience.

[0462] In some embodiments, the third processing module 1630 may also be configured to:

[0463] Obtain the user's identity information and the long-term basic information corresponding to the user's task instructions from the long memory database;

[0464] When the user task instruction is determined based on the semantic classification result without semantic reasoning, the corresponding short-term basic information at the current collection moment is obtained from the short memory database;

[0465] When it is determined based on the semantic classification result that the user task instruction requires semantic reasoning, semantic reasoning is performed on the user task instruction to obtain a semantic reasoning result;

[0466] Based on the semantic reasoning results, short-term interaction data corresponding to the semantic reasoning results are obtained from the interaction data between the user and the refrigeration equipment;

[0467] Based on the semantic reasoning results and short-term interaction data, the corresponding short-term basic information at the current collection moment is obtained.

[0468] In some embodiments, the apparatus may further include a fourth processing module configured to:

[0469] Acquire sample interaction data between the user and the refrigeration device within a first short period at the current collection moment, as well as environmental information of the refrigeration device within the first short period;

[0470] The short memory database is updated based on the sample interaction data between the user and the refrigeration device in the first short time period, the environmental information of the refrigeration device in the first short time period, and the sample interaction data between the user and the refrigeration device in the previous short time period corresponding to the first short time period.

[0471] In some embodiments, the apparatus may further include a fifth processing module configured to:

[0472] Performing semantic classification on the sample interaction data between the user and the refrigeration device in the first short period to obtain a sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration device;

[0473] Processing the sample interaction data between the user and the refrigeration device within a first short period based on the sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration device and the target time;

[0474] A short-term information summary is performed based on the processed interaction data between the user and the refrigeration device in the first short-term period, the environmental information of the refrigeration device in the first short-term period, and the interaction data between the user and the refrigeration device in the previous short-term period to update the short memory database.

[0475] In some embodiments, the apparatus may further include a sixth processing module configured to:

[0476] When it is determined based on the sample semantic classification result that no semantic reasoning is required for the sample interaction data between the user and the refrigeration appliance, retaining the sample interaction data between the user and the refrigeration appliance within the first short period;

[0477] When it is determined based on the semantic classification result that semantic reasoning is required for the sample interaction data between the user and the refrigeration device, semantic reasoning is performed on the sample interaction data between the user and the refrigeration device within a target time period to obtain a sample reasoning result; the target time period is a time period between the target moment and the first short time period;

[0478] Based on the sample inference results, obtain the sample short-term interaction data corresponding to the sample inference results from the sample interaction data between the user and the refrigeration equipment during the target period;

[0479] Based on the sample reasoning result and the sample short-time interaction data, the sample interaction data between the user and the refrigeration device in the first short-time period is updated.

[0480] In some embodiments, the apparatus may further include an eighth processing module configured to:

[0481] A long memory database is constructed based on the user's long-term biometric information and basic information.

[0482] In some embodiments, the apparatus may further include a ninth processing module configured to:

[0483] Obtain at least one user corpus;

[0484] Establishing an association relationship between at least one of the long-term biometric information and basic information and the user corpus;

[0485] The long-term biometric information, the basic information, and the association between at least one of the long-term biometric information and the basic information and the user corpus are stored in a long memory database.

[0486] The large-scale model-based personalized voice question-and-answer device in the embodiments of the present application can be a refrigeration device, or an electronic device that is communicatively connected to the refrigeration device, or a component of the electronic device, such as an integrated circuit or chip. The electronic device can be a terminal or other device other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. It can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc., and the embodiments of the present application are not specifically limited.

[0487] The large model-based personalized voice question-answering device in the embodiments of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.

[0488] The personalized voice question-answering device based on the large model provided in the embodiment of the present application can achieve Figures 1 to 15 To avoid repetition, the various processes implemented in the method embodiment are not described here.

[0489] An embodiment of the present application also provides a refrigeration device.

[0490] In some embodiments, the refrigeration equipment includes: a personalized voice question-and-answer device based on a large model as described in any of the above embodiments.

[0491] The refrigeration equipment can be understood as refrigeration storage equipment in a broad sense, including but not limited to refrigerators, freezers, display cabinets, beverage cabinets, wine cabinets, fresh-keeping cabinets and refrigerated vending machines and other refrigeration storage equipment. The refrigeration equipment has various structural forms and a wide range of applications.

[0492] In some embodiments, the refrigeration device further includes: a speaker.

[0493] In this embodiment, the speaker is used to output the question and answer results.

[0494] The loudspeaker is electrically connected to the personalized voice question-answering device based on the large model.

[0495] According to the refrigeration device provided in the embodiment of the present application, the question and answer results can be directly output through the speaker, which is simple and convenient.

[0496] In some embodiments, the refrigeration equipment further includes: a display device.

[0497] In this embodiment, the display device can be used to receive the user's first input or display information such as question and answer results.

[0498] The display device is electrically connected to the personalized voice question-answering device based on the large model.

[0499] According to the refrigeration equipment provided in the embodiment of the present application, the user's task instructions and information that needs to be displayed through images in the question and answer results are received through a display device, which is applicable to various scenarios and improves the user experience.

[0500] During the actual execution process, the user can input the user task instruction through the display device, and the refrigeration equipment can output the question and answer result through the speaker and the display device after processing the user task instruction.

[0501] According to the refrigeration equipment provided in the embodiment of the present application, by receiving user task instructions, long-term basic information and short-term basic information related to the needs and matching the user's identity information are obtained based on the user's actual needs. The obtained information is used as prompt words for the large model to process the user task instructions, providing more detailed and comprehensive information for processing user tasks. Combined with the user's actual characteristics, personalized processing is performed to obtain question and answer results corresponding to the user instructions, thereby improving the accuracy of the question and answer results. The processing method is simple, and there is no need to build a complex database and perform complex natural language processing, which reduces the difficulty of maintenance. In addition, the obtained long-term basic information and short-term basic information can be applied to different task scenarios, have strong generalization capabilities, and enhance user experience.

[0502] In some embodiments, as Figure 17 As shown, an embodiment of the present application also provides an electronic device 1700, including a processor 1701, a memory 1702, and a computer program stored in the memory 1702 and executable on the processor 1701. When the program is executed by the processor 1701, the various processes of the embodiment of the above-mentioned large-model-based personalized voice question-and-answer method are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0503] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0504] The embodiment of the present application also provides a personalized voice question-answering system based on a large model.

[0505] In some embodiments, the large model-based personalized voice question-answering system includes: a scenario distribution module, a specific scenario module, a long memory database iteration module, and a short memory database iteration module.

[0506] In this embodiment, the scene distribution module is used to determine a specific scene (ie, a target sub-scene) based on a user task instruction.

[0507] The specific scenario module is used to output the question and answer results corresponding to the user task instructions based on the specific scenario and user task instructions.

[0508] The long memory database iteration module is used to form user portraits based on the interaction data between users and refrigeration equipment to build a long memory database.

[0509] The short memory database iteration module is used to form temporary information based on the interaction data between the user and the refrigeration equipment to build a short memory database.

[0510] Both the long memory database iteration module and the short memory database iteration module may include an interactive data cleaning module.

[0511] The interaction data cleaning module is used to clean the interaction data and delete unnecessary interaction data.

[0512] According to the large-model-based personalized voice question-and-answer system provided in the embodiment of the present application, by receiving user task instructions, long-term basic information and short-term basic information related to the needs and matching the user's identity information are obtained based on the user's actual needs. The obtained information is used as prompt words for the large-model to process the user's task instructions, providing more detailed and comprehensive information for processing user tasks. Combined with the user's actual characteristics, personalized processing is performed to obtain question-and-answer results corresponding to the user instructions, thereby improving the accuracy of the question-and-answer results. The processing method is simple, and there is no need to build a complex database and perform complex natural language processing, which reduces the difficulty of maintenance. In addition, the obtained long-term basic information and short-term basic information can be applied to different task scenarios, have strong generalization capabilities, and enhance the user experience.

[0513] An embodiment of the present application also provides a non-transitory computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the various processes of the above-mentioned embodiment of the personalized voice question-and-answer method based on a large model and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0514] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0515] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned large model-based personalized voice question-answering method.

[0516] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0517] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned embodiment of the personalized voice question-and-answer method based on a large model, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0518] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0519] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0520] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of this application.

[0521] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

[0522] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0523] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and intent of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A personalized voice question-answering method based on a large model, characterized in that: Applied to refrigeration equipment, the method comprises: Receive a first input from a user; the first input is used to input a user task instruction; In response to the first input, determining a target sub-scenario based on the user task instruction; Acquiring short-term basic information corresponding to the target sub-scene at the current collection time, and acquiring the identity information of the user and long-term basic information corresponding to the user's task instruction; the short-term basic information is used to represent at least one of the user's short-term biometric information, the user's interaction with the refrigeration device during the target time period, the storage information in the refrigeration device, and the environment information of the refrigeration device; Inputting at least one of the long-term basic information and the short-term basic information and the user task instruction into the large model, and obtaining the question-answering result output by the large model; The acquiring of the short-term basic information corresponding to the target sub-scene at the current acquisition moment, and the acquiring of the user's identity information and the long-term basic information corresponding to the user's task instruction, include: Obtaining the user's identity information and the long-term basic information corresponding to the user's task instruction from a long memory database; Under the target sub-scenario, semantically classify the user task instruction to obtain a semantic classification result corresponding to the user task instruction; When it is determined based on the semantic classification result that the user task instruction does not require semantic reasoning, acquiring the short-term basic information corresponding to the current collection moment from the short memory database; In a case where it is determined based on the semantic classification result that the user task instruction requires semantic reasoning, performing semantic reasoning on the user task instruction to obtain a semantic reasoning result; Based on the semantic reasoning result, obtaining short-term interaction data corresponding to the semantic reasoning result from interaction data between the user and the refrigeration device; Based on the semantic reasoning result and the short-term interaction data, the short-term basic information corresponding to the current collection moment is obtained.

2. The large model-based personalized voice question-answering method according to claim 1, characterized in that: Before receiving the first input from the user, the method further includes: Acquire sample interaction data between the user and the refrigeration device within a first short period at the current collection moment, and environmental information of the refrigeration device within the first short period; The short memory database is updated based on the sample interaction data between the user and the refrigeration device in the first short time period, the environmental information of the refrigeration device in the first short time period, and the sample interaction data between the user and the refrigeration device in the previous short time period corresponding to the first short time period.

3. The large model-based personalized voice question-answering method according to claim 2, characterized in that: The updating of the short memory database based on the sample interaction data between the user and the refrigeration device in the first short time period, the environmental information of the refrigeration device in the first short time period, and the sample interaction data between the user and the refrigeration device in a short time period before the first short time period includes: performing semantic classification on the sample interaction data between the user and the refrigeration appliance within the first short period to obtain a sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration appliance; processing the sample interaction data between the user and the refrigeration appliance within the first short period based on a sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration appliance and a target time; A short-term information summary is performed based on the processed interaction data between the user and the refrigeration device in the first short-time period, the environmental information of the refrigeration device in the first short-time period, and the interaction data between the user and the refrigeration device in the previous short-time period to update the short memory database.

4. The large model-based personalized voice question-answering method according to claim 3, characterized in that: The processing of the sample interaction data between the user and the refrigeration appliance within the first short period based on the sample semantic classification result corresponding to the sample interaction data between the user and the refrigeration appliance and the target time includes: When it is determined based on the sample semantic classification result that no semantic reasoning is required for the sample interaction data between the user and the refrigeration appliance, retaining the sample interaction data between the user and the refrigeration appliance within the first short period; In a case where it is determined based on the sample semantic classification result that semantic reasoning is required for the sample interaction data between the user and the refrigeration appliance, semantic reasoning is performed on the sample interaction data between the user and the refrigeration appliance within a target time period to obtain a sample reasoning result; the target time period is a time period determined based on the target moment and the first short-time period; Based on the sample reasoning result, obtaining sample short-term interaction data corresponding to the sample reasoning result from the sample interaction data between the user and the refrigeration device within the target time period; Based on the sample inference result and the sample short-time interaction data, the sample interaction data between the user and the refrigeration device in the first short-time period is updated.

5. The large model-based personalized voice question-answering method according to any one of claims 1 to 4, characterized in that: Before receiving the first input from the user, the method further includes: Building a long memory database based on the long-term biometric information of the user and the basic information of the user; The long-term biometric information is the user's health information that remains unchanged over a long period of time, and the basic information is one or more of the user's gender, age, location, preferences, family members, and user experience.

6. The large model-based personalized voice question-answering method according to claim 5, characterized in that: The step of constructing a long memory database based on the long-term biometric information of the user and the basic information of the user includes: Obtain at least one user corpus; Establishing an association relationship between the user's long-term biometric information and at least one of the user's basic information and the user corpus; The user's long-term biometric information, the user's basic information and the association relationship are stored in the long memory database.

7. A personalized voice question-answering device based on a large model, characterized in that: The device is used to implement the large model-based personalized voice question-answering method according to any one of claims 1 to 6, and the device is applied to refrigeration equipment, and the device includes: A first processing module is configured to receive a first input from a user; the first input is used to input a user task instruction; a second processing module, configured to determine a target sub-scene in response to the first input and based on the user task instruction; a third processing module, configured to obtain short-term basic information corresponding to the target sub-scene at a current collection time, and obtain the identity information of the user and long-term basic information corresponding to the user's task instruction; the short-term basic information is used to represent at least one of the user's short-term biometric information, the user's interaction with the refrigeration device during a target time period, storage information in the refrigeration device, and environmental information of the refrigeration device; The fourth processing module is used to input at least one of the long-term basic information and the short-term basic information and the user task instruction into the large model to obtain the question and answer results output by the large model.

8. A refrigeration device, characterized in that: include: The large model-based personalized voice question-answering device as described in claim 7.

9. The refrigeration equipment according to claim 8, characterized in that The refrigeration equipment further includes: A speaker is electrically connected to the large model-based personalized voice question-answering device.

10. The refrigeration equipment according to claim 8, characterized in that The refrigeration equipment further includes: A display device is electrically connected to the large model-based personalized voice question-answering device.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the large model-based personalized voice question-answering method according to any one of claims 1 to 6 is implemented.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the large model-based personalized voice question-answering method according to any one of claims 1 to 6 is implemented.

Images