A multi-agent based modular human-machine collaborative decision-making method and device

Abstract

The application discloses a kind of modular man-machine collaborative decision-making method and device based on multi-agent, the method is based on the constraint of the first prompt word constructed by large language model User input information is converted into preliminary context label and demand label, so as to carry out preliminary refinement to the information input by user, then utilize large language model to preliminary context label and demand label carry out semantic reasoning, refinement and completion, so as to obtain more complete context label and demand label, and adjust according to the change of each round user input information, after multiple adjustments, the context label and demand label that can obtain more accurate matching user individualized demand, based on the context label and demand label, more suitable candidate can be matched for user, so as to realize more accurate auxiliary user makes dynamic, reliable individualized recommendation.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of artificial intelligence and large language models, and in particular to a modular human-computer collaborative decision-making method and device based on multiple agents. BACKGROUND

[0002] In the context of rapid information growth and increasingly complex personalized needs, users often face challenges such as information overload, ambiguous preferences, and dynamic changes in scenarios in various online decision-making tasks such as shopping, travel planning, and course selection. For example, on e-commerce platforms, users' initial purchase intentions are usually not clear, and only after continuous browsing, comparison, and information acquisition can the demand gradually become clear and ultimately make a choice. This dynamic evolution from ambiguity to clarity requires the system not only to provide personalized recommendations but also to continuously perceive, understand, and guide users to refine their decision criteria.

[0003] The invention patent application with publication number CN120316250A discloses an intelligent personalized clothing matching method based on a large language model, which includes: obtaining the body information of the user and the scene demand information of the clothing matching, as well as the professional knowledge of clothing matching, and preprocessing them respectively, using the HuggingFace Embedding tool to convert the prompt text into a prompt text embedding vector and the knowledge text into a knowledge text embedding vector; input the prompt text embedding vector into the pre-trained information extraction model to obtain the knowledge text embedding vector, input the knowledge text embedding vector set into the optimized IPEX-LLM large language model, generate specific clothing matching suggestions through a random sampling method, and generate corresponding clothing pictures using the CogView-3 text-to-image model. The invention patent application can solve the technical problem that the existing clothing matching recommendation method based on clothing matchers cannot meet the user's cutting-edge fashion needs as much as possible.

[0004] The invention patent application with publication number CN117891924A discloses a large language model dialogue recommendation method based on reinforcement knowledge selection, belonging to the field of natural language processing. The invention combines the encoder of the T5 model and the RGCN model to improve the relevance and richness of the dialogue content, making the dialogue more natural, smooth, and closely tailored to user needs. The T5 model is used as a strategy model, combined with reinforcement learning, effectively improving the quality and accuracy of dialogue generation, and ensuring the efficiency of knowledge selection. Experiments show that the model of the invention exceeds the current latest benchmark model in both automatic evaluation and manual evaluation. This method effectively solves the above problems through innovative reinforcement learning framework and algorithm, providing users with more intelligent, accurate, and efficient dialogue recommendation services. The proposal of this method not only represents a major advance in dialogue recommendation technology, but also provides a new direction for future research and development of intelligent dialogue agents.

[0005] However, the above two patent applications provide static user portraits, lack continuous perception of dynamic demand, and are difficult to accurately capture subtle preference changes of users in the decision-making process. It can be seen that the existing system often relies on static user portraits and single algorithm model, lacks continuous perception and update of dynamic demand in multiple rounds of interaction, and is difficult to accurately capture subtle preference changes of users in the decision-making process. At the same time, these systems usually lack transparent explanation of recommendation logic, limit the understanding and trust of users on the recommendation results, and lead to low efficiency and satisfaction of overall decision support. SUMMARY

[0006] The modular human-computer collaborative decision-making method based on multiple agents provided by the application can more accurately assist users to make dynamic and reliable personalized recommendations.

[0007] The application provides a modular human-computer collaborative decision-making method based on multiple agents, comprising:

[0008] Based on the current user input information and the historical interaction record, a response information is output through a large language model, when the user input information is a recommendation request, text features and image features are extracted from the natural language description and image information of the current user input information, the preliminary context label and the preliminary demand label of the last round are injected into the dynamic slot of the preset first prompt word template together with the current text features and image features to construct a first prompt word, and the current preliminary context label and preliminary demand label are obtained through the large language model based on the first prompt word;

[0009] Based on the first preliminary context label and the preliminary demand label of the current user, a second and a third prompt word template for generating supplemented context labels and demand labels are constructed, the current preliminary context label and demand label are injected into the second prompt word template and the third prompt word template to obtain a second prompt word and a third prompt word, and the second and third prompt words are input into the large language model to obtain a context label set and a demand label set;

[0010] The context label set and the demand label set are compared with the candidate item label set respectively to obtain a context matching score and a demand matching score, and a comprehensive matching score of the candidate item is obtained based on the context matching score and the demand matching score, and the candidate item with the top N comprehensive matching score is recommended to the user.

[0011] Preferably, a fourth prompt word template is also constructed based on the first primary context label and the first primary demand label of the current user, the fourth prompt word template including a scene judgment question and a recommended candidate item explanation question, the scene judgment question being a question of judging a task scene based on the first generated primary context label and the first primary demand label of the current user and explaining the dimensions that need to be covered in the task scene, and the recommended candidate item explanation question being a question of explaining the recommended reason of each recommended candidate item based on the dimensions that need to be covered.

[0012] The fourth prompt word is obtained by injecting the set of context labels, the set of demand labels, and the candidate item recommendation result into the dynamic slot of the fourth prompt word template, and the explanation of the recommended candidate item is obtained by inputting the fourth prompt word into the large language model, and the explanation of the recommended candidate item and the recommended candidate item are pushed to the user.

[0013] Preferably, the explanation includes the context labels and demand labels that drive the candidate item recommendation result, a comparison of the recommended candidate item with the user's demand difference, and an additional value of the recommended candidate item.

[0014] Preferably, the second prompt word template includes a scene judgment question and a question of supplementing the primary context label.

[0015] The scene judgment question is a question of judging a task scene based on the first generated primary context label and the first primary demand label of the current user, and clearly stating the requirement for context supplement in the task scene.

[0016] The question of supplementing the primary context label is a question of supplementing and format converting the current primary context label based on the requirement for context supplement in the task scene.

[0017] Preferably, the third prompt word template includes a scene judgment question and a question of supplementing the primary demand label.

[0018] The scene judgment question is a question of judging a task scene based on the first generated primary context label and the first primary demand label of the current user, and clearly stating the requirement for demand understanding in the task scene.

[0019] The question of supplementing the primary demand label is a question of supplementing and format converting the current primary demand label based on the requirement for demand understanding in the task scene.

[0020] Preferably, the current preliminary context label and the preliminary demand label are obtained by the large language model based on a first prompt word, wherein the first prompt word is used to enable the large language model to de-duplicate, fuse and classify the current text features and image features with the preliminary context label and the preliminary demand label of the last round to obtain the current preliminary context label and the preliminary demand label, and to format the current preliminary context label and the preliminary demand label.

[0021] Preferably, the text features are extracted from the natural language description of the current user input information, including:

[0022] The text features are extracted from the natural language description by the large language model based on the constraint of the text feature prompt word;

[0023] The text feature prompt word includes the natural language description with injected dynamic slot, and the intention, target object, attribute and preference of the user obtained based on the natural language description of the user input information, and the intention, target object, attribute and preference of the user obtained are formatted.

[0024] Preferably, the image features are extracted from the image information of the current user input information, including:

[0025] The image features are extracted from the image information of the current user input information by the multi-modal perception model based on the constraint of the image feature prompt word;

[0026] The image feature prompt word includes the image information with injected dynamic slot, and the object, attribute and spatial information in the image obtained based on the image information, and the object, attribute and spatial information in the image obtained are formatted.

[0027] Preferably, the response information is output by the large language model based on the current user input information and the historical interaction record, including:

[0028] The response information is output by the large language model based on the constraint of the response prompt word, so as to reply to the current user, and the response information includes an instant answer, a confirmation dialogue and a clarification guide;

[0029] The user input information is divided into question type input, recommendation request and unclassified or casual content by the constraint of the response prompt word, when the user input information is question type input, the large language model directly generates an instant answer based on the historical interaction record to complete the candidate recommendation to the user, when the user input information is a recommendation request or a reply information to the candidate of the last round recommendation, the large language model generates a confirmation dialogue, and when the user input information is unclassified or casual content, the large language model generates a clarification guide.

[0030] The application also provides a multi-agent-based modular man-machine collaborative decision-making device, comprising:

[0031] An information preprocessing agent, which comprises a personalized response module, a text feature understanding module, an image feature understanding module and an information processing module, the personalized response module is used to output response information based on current user input information and historical interaction records through a large language model, the text feature understanding module is used to extract text features from the natural language description of the current user input information when the user input information is a recommendation request, the image feature understanding module is used to extract image features from the image information of the current user input information when the user input information is a recommendation request, and the information processing module is used to inject the preliminary context label and the preliminary demand label of the last round, and the current text features and image features into the dynamic slot in the preset first prompt word template so as to construct a first prompt word, and obtain the current preliminary context label and demand label through the large language model based on the first prompt word;

[0032] A comprehensive reasoning agent, which comprises a personalized prompt word generation module, a context understanding module, a demand understanding module and an information processing module, the personalized prompt word generation module is used to construct second and third prompt word templates for generating complete context labels and demand labels based on the first generated preliminary context label and preliminary demand label of the current user respectively, the context understanding module is used to inject the current preliminary context label into the second prompt word template to obtain a second prompt word, input the second prompt word into the large language model to obtain a completed context label, construct a context label set through multiple completed context labels obtained in multiple rounds, the demand understanding module is used to inject the current demand label into the third prompt word template to obtain a third prompt word, input the third prompt word into the large language model to obtain a completed demand label, and construct a demand label set through multiple completed demand labels obtained in multiple rounds;

[0033] A personalized recommendation agent, which is used to perform cosine similarity comparison between the context label set and the demand label set and a candidate item label set respectively to obtain a context matching score and a demand matching score, obtain a comprehensive matching score of the candidate item based on the context matching score and the demand matching score, and recommend the candidate item with the top N comprehensive matching scores to the user.

[0034] Compared with the prior art, the application has the following beneficial effects:

[0035] The application is based on the constraint of the constructed first prompt word, which converts the user input information into a preliminary context label and a preliminary demand label through a large language model, thereby preliminarily refining the user input information, and then uses the large language model to perform semantic reasoning, refinement and completion on the preliminary context label and the preliminary demand label, so as to obtain a relatively complete context label and demand label. After multiple rounds of adjustment of the changes in the user input information, the context label and demand label that accurately match the user's personalized demand can be obtained, and based on the context label and demand label, a more appropriate candidate can be matched for the user, thereby realizing more accurate assistance for the user to make dynamic and reliable personalized recommendations. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 A flowchart of a modular human-machine collaborative decision-making method based on multiple agents is provided for specific embodiments of the application.

[0037] Figure 2 A schematic diagram of a modular human-machine collaborative decision-making device based on multiple agents is provided for specific embodiments of the application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and technical effect of the application clearer, the application is further described in detail below with reference to the accompanying drawings.

[0039] In order to realize accurate recommendation for user input demand, the application divides the user input information into a preliminary context label and a preliminary demand label through a large language model by constructing a prompt word, and supplements the preliminary context label and the preliminary demand label, thereby more accurately capturing the user's demand and accurately recommending for the user. The specific steps of the modular human-machine collaborative decision-making method based on multiple agents provided by the specific embodiments of the application are as shown in Figure 1 , including:

[0040] S1, responding to the user input information, when the user input information is a recommendation request, converting the user input information into a preliminary context label and a preliminary demand label, the specific steps are:

[0041] Based on the current user input information and the historical interaction record, the response information is output through the large language model, when the user input information is a recommendation request, the text features and image features are extracted from the natural language description and image information of the current user input information, the preliminary context label and the preliminary demand label of the last round, and the current text features and image features are injected into the dynamic slot in the preset first prompt word template to construct the first prompt word, and the current preliminary context label and demand label are obtained through the large language model based on the first prompt word.

[0042] In a specific implementation, the large language model outputs response information based on current user input information and historical interaction records, including:

[0043] The specific embodiments of the present application output response information through a large language model based on the constraints of response prompt words, thereby replying to the current user, and the response information includes instant answers, confirmation dialogues and clarification guides.

[0044] The present application enables the large language model to divide the user input information into question type input, recommendation request and unclassified or casual content through the constraints of the response prompt words. When the user input information is question type input, the large language model directly generates instant answers based on historical interaction records to complete candidate recommendation for the user, thereby ending the process and improving operation efficiency. When the user input information is a recommendation request or a reply to the candidate of the last round of recommendation, the large language model generates confirmation dialogues and provides user input information to the next process, thereby being able to continuously correct the context label set and the demand label set based on the user's reply. When the user input information is unclassified or casual content, the large language model generates clarification guides to guide the user to further ask questions.

[0045] The specific embodiments of the present application do not process the current input in isolation after user intent recognition is completed, but read and call historical interaction records (History), comprehensively analyze the user's past input, label state, recommendation result and explanation information to ensure that the current response is consistent with the entire dialogue link in terms of coherence and context consistency.

[0046] In a specific embodiment, question type input (Question Type): such as information type question proposed by the user (example: “Is this sofa suitable for a small house?”); recommendation type input (Recommendation Request): such as explicit expression of demand or scene description by the user (example: “I want to find a sofa with a simple style”); other input (Other Types): such as casual, unstructured content or temporarily unclassified input.

[0047] In a specific embodiment, the first prompt word is as follows:

[0048] “[System prompt]:

[0049] You are a user input response expert, and your responsibility is to receive user input and determine its intent category (such as information type question, recommendation request or other input). You need to decide whether to directly generate an answer, whether to transfer the request to a downstream module, or whether to issue a clarification guide to the user.

[0050] [Task instruction]:

[0051] Analyze {USER_INPUT} carefully to classify user input into MetaTag, from one of three categories:

[0052] 1) Question (Question-type input): User asks for information about a certain product, service, or concept;

[0053] 2) Recommendation (Recommendation request): User makes a request for recommendations, uploads images, or describes needs;

[0054] 3) Other (Unclassifiable or casual content).

[0055] Perform the following operations according to the classification results:

[0056] Question → Generate clear and concise Answer (e.g., "This sofa is suitable for small homes and easy to maintain.");

[0057] Recommendation → Output ForwardSignal: true and generate a short confirmation phrase (e.g., "I understand, I will recommend suitable furniture according to your description.");

[0058] Other → Output Clarification, guiding the user to provide more details (e.g., "Could you please tell me which type of product you are interested in, or would you like some suggestions in a specific area?").

[0059] [Dynamic slot]:

[0060] Historical interactions: {HISTORY};

[0061] User input: {USER_INPUT}.

[0062] [Output constraints]:

[0063] "Output must be in JSON format and follow the following rules:

[0064] Include MetaTag field (value can only be 'Question', 'Recommendation' or 'Other');

[0065] If MetaTag is Question, return Answer field;

[0066] If MetaTag is Recommendation, return ForwardSignal: true and Response field;

[0067] If the MetaTag is Other, return the Clarification field.

[0068] Example output:

[0069] { "MetaTag": "Question", "Answer": "This sofa is suitable for small households and easy to maintain."};

[0070] { "MetaTag": "Recommendation", "ForwardSignal": true, "Response": "Okay, I will recommend suitable furniture based on your description."};

[0071] { "MetaTag": "Other", "Clarification": "Do you want to find a specific type of product, or do you want to understand the product's maintenance method?"}.

[0072] The personalized response module, in a specific embodiment, extracts text features from the natural language description of the current user input information when the user input information is a recommendation request, including: extracting text features from natural language descriptions based on text feature prompt words through a large language model.

[0073] The text feature prompt words provided by the embodiments of the present application include natural language descriptions injected into dynamic slots, and the user's intent, target object, attribute, and preference extracted from the natural language description of the user input information, and the obtained user's intent, target object, attribute, and preference are format converted, the embodiments of the present application use text feature prompt words to enable the large language model to identify the user's intent, target object, attribute, and preference based on the natural language of the user input information, and convert them into format tags. In a specific embodiment, the format is converted to JSON tag {TEXT_JSON}.

[0074] Specifically, the user's intent (Intent) provided by the embodiments of the present application is what the user wants to accomplish, such as: replace, purchase, cancel, plan, the target object (Object) is the core object related to the task (such as: goods, services, places), and the attribute or preference (Attribute) is the user's feature requirement for the object (such as: style, size, budget, priority).

[0075] In a specific embodiment, the text feature prompt words provided by the present embodiment are as follows:

[0076] "[System prompt]:

[0077] You are a decision-making assistant. Please extract decision-related information based on the text input by the user.

[0078] [Task Instructions]:

[0079] Please extract the core information from the text and output it in the following structured format:

[0080] Intent: The task or goal that the user wants to accomplish (such as buying, comparing, booking, learning);

[0081] Object (Target Object): The core object that users are interested in (such as sofa, courses, travel destination);

[0082] Attribute (or preference): Features or preferences mentioned by the user (such as color, material, style, budget).

[0083] [Dynamic slots]:

[0084] The text entered by the user: {USER_INPUT_TEXT}.

[0085] [Output Constraints]:

[0086] Output only in JSON format; do not add any additional descriptions or comments.

[0087] Output example:

[0088] {

[0089] "Intent": ["Purchase"],

[0090] "Object": ["Sofa"],

[0091] Attribute: ["Modern minimalist style", "Gray", "Space-saving"]

[0092] }

[0093] The specific embodiments of the present invention provide for extracting image features from image information of current user input information, including:

[0094] Based on the constraints of image feature prompts, image features are extracted from the image information of the current user input information through a multimodal perception model. In a specific embodiment, the multimodal perception model is a language-image bootstrapping pre-training model (BLIP).

[0095] The image feature prompt word provided by the embodiment of the present application comprises image information injected into a dynamic slot, and object type, attribute and spatial information in the image are obtained based on the image information, and the obtained object, object attribute and spatial information in the image are format-converted.

[0096] The present application utilizes the image feature prompt word to enable the multi-modal perception model to identify the object, attribute and spatial information of the user based on the natural language mode of the user input information, and convert them into format labels. In an embodiment, the format is converted into a JSON label {IMAGE_JSON}.

[0097] Specifically, the object (Object) provided by the embodiment of the present application is a key object or region related to the task in the image, the attribute (Attributes) is color, shape, material, style, etc., and the spatial information (Spatial Info) is the position or layout of the object in the image.

[0098] In an embodiment, the image feature prompt word provided by the present embodiment is specifically:

[0099] "[System prompt]:

[0100] You are a decision-making assistant. Please extract decision-related information based on the image content input by the user.

[0101] [Task instruction]:

[0102] Please extract the core information in the image and output it in the following structured format:

[0103] Object (object): main objects (such as furniture, equipment, vehicles, etc.) appearing in the image;

[0104] Attribute (attribute): key features of these objects (such as color, material, style);

[0105] SpatialInfo (spatial information): the approximate position or layout of the object (such as against the wall, in the center, stacked).

[0106] [Dynamic slot]:

[0107] User-uploaded picture: {USER_INPUT_IMAGE}.

[0108] [Output constraint]:

[0109] Only output in JSON format, do not add additional explanations or comments.

[0110] Output example:

[0111] {

[0112] "Object": ["Sofa"],

[0113] Attribute: ["Dark Gray", "Modern Minimalist Style", "Leather"],

[0114] "SpatialInfo": ["Place against the wall"]

[0115] }

[0116] In human-computer collaborative decision-making tasks, users often input mixed information, including text and images, all at once. This information can be divided into two main categories: contextual information and requirement information. Contextual information describes the user's background, conditions, or environmental characteristics (such as spatial attributes, resource constraints, and external environment), providing the system with the basis for understanding the decision-making context. Requirement information expresses the user's goals, preferences, or functional requirements, directly pointing to the direction of personalized recommendations or decision generation. In actual interaction, these two types of information are often expressed together. If they are not accurately distinguished, the subsequent reasoning and recommendation logic of the intelligent agent will be unable to accurately understand the user's intent, affecting the final decision-making effect.

[0117] In a specific embodiment of the present invention, the current text features and image features, along with the preliminary context tags and preliminary requirement tags from the previous round, are injected into dynamic slots in a preset first prompt word template to construct the first prompt word. These dynamic slots are {TEXT}, {IMAGE}, {HISTORY_INITIAL_CONTEXT_TAGS}, and {HISTORY_INITIAL_REQUIREMENT_TAGS}. This embodiment utilizes the first prompt word to enable a large language model to deduplicate, fuse, and classify the current text features and image features with the preliminary context tags and preliminary requirement tags from the previous round. Tags related to environment, scene, and layout are categorized as context tags (Initial_ContextTags), while tags related to user goals, requirements, and preferences are categorized as requirement understanding tags (Initial_RequirementTags). Furthermore, when tag conflicts occur, updates are performed based on the latest input.

[0118] In one specific embodiment, the first prompt word provided in this embodiment is:

[0119] [System Prompt]:

[0120] You are an intelligent decision-making assistant whose task is to generate preliminary context and demand tags for this round of updates based on the text and image information input by the user, as well as historical tag content.

[0121] [Task Instructions]:

[0122] 1. Read the latest input of the user corresponding to the label (text features and image features);

[0123] 2. Read the history label library (Initial_ContextTags and Initial_RequirementTags);

[0124] 3. De-duplicate, merge and classify the history label and the current label:

[0125] Initial_ContextTags: Describe the environment, scene, and existing conditions (may include historical scene information and newly identified visual background);

[0126] Initial_RequirementTags: Describe the user's goals, needs, and preferences (mainly based on the user's latest intentions, combined with historical retained labels that have not been denied);

[0127] 4. Output "updated labels" to ensure that the output integrates historical information and reflects the user's current latest input.

[0128] [Dynamic slot]:

[0129] History label: {HISTORY_INITIAL_CONTEXT_TAGS} {HISTORY_INITIAL_REQUIREMENT_TAGS}.

[0130] Current text features: {TEXT}.

[0131] Current image features: {IMAGE}.

[0132] [Output constraints]:

[0133] Strictly output JSON format:

[0134] {

[0135] "Initial_ContextTags": [...],

[0136] "Initial_RequirementTags": [...]

[0137] } Do not output any additional explanations."

[0138] S2, The embodiment of the application supplements the current preliminary context label and preliminary requirement label by a large language model (LLM) to obtain a context label and a requirement label, and the specific steps are as follows: based on the first generated preliminary context label and preliminary requirement label of the current user, second and third prompt word templates for generating supplemented context labels and requirement labels are constructed, in a specific embodiment, the second and third prompt word templates are only generated when the first trigger flow is triggered, and subsequent operations call the second and third prompt word templates, the current preliminary context label and requirement label are injected into the second prompt word template and the third prompt word template to obtain a second prompt word and a third prompt word, and the second and third prompt words are input into the large language model to obtain a context label set and a requirement label set.

[0139] In a specific embodiment, the second prompt word template provided by the embodiment of the application includes a scene judgment question and a question for supplementing the preliminary context label, the scene judgment question is to judge the task scene based on the first generated preliminary context label (Initial_ContextTags) and preliminary requirement label (Initial_RequirementTags) of the current user, and to clearly supplement the requirement of the initial context label in the task scene;

[0140] The question for supplementing the context label is to supplement and format the current preliminary context label based on the requirement of supplementing the initial context label in the task scene, in a specific embodiment, the current preliminary context label is supplemented, refined and structured to obtain a more complete context label.

[0141] Specifically, the embodiment of the application injects the current preliminary context label into the second prompt word template to obtain a second prompt word, and based on the constraint of the second prompt word, the large language model further infers and supplements the preliminary context label Initial_ContextTags to generate a more refined and structured context label set ContextTags, the supplemented context label set not only describes the current environment or context characteristics, but also infers the implicit requirements that the user may derive in the context, so as to constitute the core matching basis of the subsequent recommendation system together with the requirement label set RequirementTags.

[0142] In a specific embodiment, the second prompt word provided by the embodiment is:

[0143] "[System prompt]:

[0144] You are a prompt template design expert. Your task is to design a prompt template for a **context understanding module** that will be invoked in a multi-round decision-making task to guide LLM completion and refinement of context labels.

[0145] [Task Instructions]:

[0146] Please generate the corresponding prompt template according to the following steps:

[0147] 1. Task type identification: Based on Initial_ContextTags and Initial_RequirementTags, think about what kind of decision-making scenario the current task belongs to (such as shopping, education, travel, medical treatment, etc.), and clarify the requirements of the scene for context completion;

[0148] 2. Thinking about planning logic: Think about the functional positioning of the **context understanding agent**, plan how the LLM can complete reasoning, refine semantics and structure based on Initial_ContextTags, and generate complete ContextTags for subsequent personalized recommendations and decision support;

[0149] 3. Prompt template generation:

[0150] Output the complete template according to the following four fields:

[0151] System prompt: Describe the functional positioning of the **context understanding module**;

[0152] Task instructions: Describe the steps of reasoning and completion, including how to use Initial_ContextTags;

[0153] Dynamic slot: List the contents filled dynamically when invoked: {INITIAL_CONTEXT_TAGS};

[0154] Output constraints: Specify that the output format must be JSON, and provide an output example.

[0155] [Dynamic slot]:

[0156] Initial context label: {INITIAL_CONTEXT_TAGS};

[0157] Initial requirement label: {INITIAL_REQUIREMENT_TAGS}.

[0158] [Output constraints]:

[0159] Please output **ContextPromptTemplate** as JSON in the following structure:

[0160] {

[0161] "ContextPromptTemplate": {

[0162] "System Message": "Define the role description of the LLM",

[0163] "Task Instruction": "Detailed description of reasoning completion",

[0164] "Dynamic Slot": {

[0165] "Initial_ContextTags": "{INITIAL_CONTEXT_TAGS}",

[0166] },

[0167] "Output Constraint": "The output must be in JSON and provide format examples, such as {\"ContextTags\":[\"Modern style\", \"Suitable for small households\"]}"

[0168] }

[0169] }.”.

[0170] The third prompt word template provided by the embodiment of the application comprises a scene judgment question and a question of supplementing the preliminary demand label, the scene judgment question is to judge the task scene based on the current preliminary context label and the preliminary demand label, and to clearly define the requirement for understanding the demand in the task scene;

[0171] The question of supplementing the preliminary demand label is to supplement and format the current preliminary demand label based on the requirement for understanding the demand in the task scene.

[0172] The third prompt word is obtained by injecting the current initial requirement tag into the third prompt word template. Based on the constraints of the third prompt word, the initial requirement tag InitialRequirementTags is further semantically refined, reasoned, completed and dynamically evolved by a large language model to generate a structured requirement tag set RequirementTags. The system is driven by a large language model (LLM) and calls the requirement understanding prompt word template provided by the personalized prompt word generation agent to perform deep semantic analysis on Initial_RequirementTags and automatically complete the fuzzy, implicit or potential requirements in user expressions. For example, for vague descriptions such as "wanting to be more design-oriented" or "hoping to be suitable for multiple people", the system will reason to generate more specific attribute dimensions (such as functional configuration, available scenarios, budget range, style elements, etc.) and uniformly encode them into structured requirement tags RequirementTags.

[0173] In a specific embodiment, the third prompt word provided by the embodiment is:

[0174] "[System prompt]:

[0175] You are a prompt word template design expert. Your task is to design a prompt word template for the **Requirement Understanding Module**, which will be called in multiple rounds of decision-making tasks to guide the LLM to complete and refine the requirement tags.

[0176] [Task instructions]:

[0177] Please generate the corresponding prompt word template according to the following steps:

[0178] 1. Task type identification: Based on Initial_ContextTags and Initial_RequirementTags, think about what kind of decision-making scenario the current task belongs to (such as shopping, education, travel, medical treatment, etc.), and clarify the requirements of the scene for requirement understanding;

[0179] 2. Thinking about planning logic: Think about the functional positioning of the **Requirement Understanding Agent**, plan how the LLM will perform deep semantic analysis and reasoning completion based on Initial_RequirementTags, and convert the user's vague or implicit requirements into structured RequirementTags for subsequent personalized recommendation and decision support;

[0180] 3. Prompt word template generation:

[0181] Output the complete template according to the following four fields:

[0182] System prompt: Define the function positioning of the **Requirement Understanding Module**;

[0183] Task instructions: Describe the steps of reasoning and refinement, including how to utilize Initial_RequirementTags;

[0184] Dynamic slots: List the content that is dynamically filled at invocation: {INITIAL_REQUIREMENT_TAGS} ;

[0185] Output constraints: Specify that the output format must be JSON and provide an output example.

[0186] [Dynamic slots]:

[0187] Initial context tags: {INITIAL_CONTEXT_TAGS} ;

[0188] Initial requirement tags: {INITIAL_REQUIREMENT_TAGS}.

[0189] [Output constraints]:

[0190] Please output the **RequirementPromptTemplate** in the following JSON structure strictly:

[0191] {

[0192] "RequirementPromptTemplate": {

[0193] "System prompt": "Define the role description of the LLM",

[0194] "Task instructions": "Detailed instructions for reasoning and completion",

[0195] "Dynamic slots": {

[0196] "Initial_RequirementTags": "{INITIAL_REQUIREMENT_TAGS}",

[0197] },

[0198] "Output constraints": "The output must be in JSON format and provide an example of the format, such as {\"RequirementTags\": [\"Modern style\", \"Design highlights\", \"Multi-person seating\", \"Suitable for parties\", \"Space-saving\"]}"

[0199] }

[0200] }."

[0201] The fourth prompt word template is constructed based on the first generated preliminary context label and preliminary requirement label of the current user, and includes a scene judgment question and a recommended candidate item explanation question. The scene judgment question is to judge the task scene based on the current preliminary context label and preliminary requirement label, and to explain the dimensions that need to be covered in the task scene, including function matching, style correspondence, difference explanation, and accessory value. The recommended candidate item explanation question is to explain the recommended reason of each recommended candidate item based on the dimensions that need to be covered.

[0202] The context label set, requirement label set and candidate item recommendation result are injected into the dynamic slot of the fourth prompt word template to obtain a fourth prompt word. The fourth prompt word is input into a large language model to obtain an explanation of the recommended candidate item. The explanation of the recommended candidate item and the recommended candidate item are pushed to the user.

[0203] Specifically, the explanation provided by the embodiments of the present application includes providing context labels and requirement labels that drive the candidate item recommendation result, comparing and explaining the differences between the recommended candidate item and the user's requirements, and explaining the additional value of the recommended candidate item.

[0204] Based on the recommendation result Recommendation_Results, and in combination with the context label ContextTags and the requirement label RequirementTags, the embodiments of the present application perform label matching analysis and difference explanation on each recommended item to generate structured explanation content Explanations. These explanations not only reveal the matching logic of the recommended item with the user's requirements and scene, but also point out the potential incomplete matching places and additional value, thereby enhancing the transparency and user trust of the recommendation result.

[0205] In a specific implementation, the module is also driven based on a large language model (LLM) and calls a fourth prompt word template (ExplanationPromptTemplate). The system injects ContextTags, RequirementTags and RecommendationResults into the dynamic slot of the fourth prompt word template, and the LLM performs reasoning according to the task instructions in the template to generate the following content for each recommendation result:

[0206] Label matching explanation: analyze which features in the recommended item are highly consistent with ContextTags and RequirementTags (such as “modern style” and “space saving”).

[0207] Discrepancy Analysis: Identify deviations or shortcomings between the recommended item and user tags (e.g., "Size is large, layout needs adjustment").

[0208] Value Supplement: Extract additional advantages of the recommended item (e.g., "Adjustable backrest" "Easy-to-clean fabric").

[0209] Finally, the system will present the recommended results in a conversational interface, accompanied by LLM-generated explanations, enabling users to clearly understand the recommendation logic (why the recommendation, which tags match, what shortcomings or additional values), and make more confident decisions accordingly.

[0210] In a specific embodiment, the fourth prompt word provided by the embodiment is:

[0211] "[System prompt]:

[0212] You are a prompt word template design expert. Your task is to design a prompt word template for the **Explanation Generation Module**, which will be called in multiple rounds of decision-making tasks to guide the LLM to complete and refine the requirement tags.

[0213] [Task instructions]:

[0214] Please generate the corresponding prompt word template according to the following steps:

[0215] 1. Task type identification: Based on Initial_ContextTags and Initial_RequirementTags, think about what kind of decision-making scenario the current task belongs to (such as shopping, education, travel, medical treatment, etc.), and think about the dimensions that the recommendation result explanation needs to cover in this task scenario (such as function matching, style correspondence, discrepancy explanation, additional value).

[0216] 2. Thinking about the planning logic:

[0217] Sort out the function positioning of the **Explanation Generation Agent**, and plan how the LLM uses {CONTEXT_TAGS}, {REQUIREMENT_TAGS}, and {RECOMMENDATION_RESULTS} to generate understandable explanations for each recommended item.

[0218] Including tag matching explanation (which ContextTags / RequirementTags drive the recommendation), discrepancy analysis (which places do not meet user needs), and value explanation (additional advantages).

[0219] 3. Prompt word template generation:

[0220] Output the complete template according to the following four fields:

[0221] System prompt: Explain the function positioning of the **Explanation Generation Module**;

[0222] Task instruction: Describe in detail how LLM performs chain reasoning on recommendation results, including information extraction, label association, difference analysis, and natural language explanation, including how to use ContextTags, RequirementTags, and Recommendation_results;

[0223] Dynamic slot: List the content filled dynamically when called: {CONTEXT_TAGS} {REQUIREMENT_TAGS} {RECOMMENDATION_RESULTS};

[0224] Output constraint: Specify that the output format must be JSON and provide an output example.

[0225] [Dynamic slot]:

[0226] Context tags: {CONTEXT_TAGS};

[0227] Requirement tags: {REQUIREMENT_TAGS};

[0228] Recommendation results: {RECOMMENDATION_RESULTS}.

[0229] [Output constraint]:

[0230] Please output **ExplanationPromptTemplate** in the following JSON structure:

[0231] {

[0232] "ExplanationPromptTemplate": {

[0233] "System prompt": "Define the role of LLM",

[0234] "Task instruction": "Generate detailed instructions for generating recommendation explanations",

[0235] "Dynamic slot": {

[0236] "ContextTags": "{CONTEXT_TAGS}",

[0237] "RequirementTags": "{REQUIREMENT_TAGS}",

[0238] "RecommendationResults": "{RECOMMENDATION_RESULTS}"

[0239] },

[0240] "Output constraints": "Output must be in JSON and provide format examples, such as {\"Explanations\":[{\"item_id\": \"F1234\", \"Explanation\": \"The sofa matches your'modern style' label and highly matches the demand label'save space'.\"}]}"

[0241] }

[0242] }.”.

[0243] The embodiment of the present application is based on the initial context label Initial_ContextTags and the requirement understanding label Initial_RequirementTags, and then uses a large language model to perform scene reasoning to determine the current decision task type, such as shopping, education, travel, and medical treatment. Based on the scene features, customized second, third, and fourth prompt word templates are generated as fixed basic templates for continuous reuse in multiple rounds of interaction. The relevant templates are generated only once and reused in multiple rounds of interaction. The information input by the user is dynamically filled into the slots of the template to ensure that the prompt words remain contextually coherent.

[0244] S3、The embodiment of the present application compares the context label set and the requirement label set with the candidate item label set respectively to obtain a context matching score and a requirement matching score, and obtains a comprehensive matching score of the candidate item based on the context matching score and the requirement matching score. The top N candidate items with the highest comprehensive matching scores are recommended to the user.

[0245] The first i candidate item provided by the embodiment of the present application A set of option label sets are defined in advance, which describe the multi-dimensional attributes of the option (such as functional features, style, scene adaptability, technical parameters, etc.), and are used for matching with the user's context label set and requirement label set. This module uses a label similarity matching algorithm to comprehensively consider the consistency of the candidate item with the user's context and requirement, and the specific formula is as follows: Wherein, represents the comprehensive matching score of the candidate item . and are weight coefficients of context matching and demand matching respectively, which can be flexibly set according to application scenarios, and in a specific embodiment, the weight coefficients are both 0.5.

[0246] The context matching score provided by the embodiment of the application is defined as the cosine similarity between the candidate item tag set and the context tag set . .

[0247] The demand matching score provided by the embodiment of the application is defined as the cosine similarity between the candidate item tag set and the demand tag set . wherein represents that the tag set generates a corresponding semantic vector representation through a pre-trained word vector model (such as Word2Vec, BERT or CLIP, etc.).

[0248] The embodiment of the application sorts all candidate items according to and selects several items with the highest scores as the personalized recommendation results. At the same time, the related information of the recommendation results is input into the explanation generation module to generate personalized explanations, helping the user to understand the basis of the recommendation and enhance the sense of trust.

[0249] On the other hand, the application also provides a modular human-computer collaborative decision-making device based on multi-agent, as shown in Figure 2 , which comprises an information preprocessing agent, a comprehensive reasoning agent and a personalized recommendation agent.

[0250] The information preprocessing agent provided by the embodiment of the application serves as the front-end entrance of the system, is responsible for receiving the multi-modal data (including text, image, etc.) input by the user, and converts it into structured tag information that can be used by the subsequent decision-making process. The information preprocessing agent is composed of four core modules: a personalized response module, a text feature understanding module, an image feature understanding module and an information processing module.

[0251] ​​The personalized response module provided by the embodiment of the present application serves as a first contact of user input, is deployed at a system interaction entrance, and is responsible for performing user intention recognition and preliminary response generation. The personalized response module analyzes user input based on a large language model (LLM), judges the type of the current input, and classifies the input into the following three types: 1, question type input (Question Type): such as a user asking an information type question (example: "Is this sofa suitable for a small house?"); 2, recommendation type input (Recommendation Request): such as a user explicitly expressing a demand or a scene description (example: "I want to find a sofa with a simple style"); and 3, other input (Other Types): such as chatting, unstructured content, or input that cannot be classified temporarily.

[0252] After the intention of the user is recognized, the personalized response module provided by the embodiment of the present application does not process the current input in isolation, but reads and calls a history interaction record (History), comprehensively analyzes the past input, label state, recommendation result, and explanation information of the user, to ensure that the current response is consistent with the entire dialogue link in terms of coherence and context consistency.

[0253] The personalized response of the personalized response module provided by the embodiment of the present application includes:

[0254] Question type input (Question Type): the module directly generates an instant answer (Answer) and embeds information in the historical context (for example: "This sofa is suitable for a small house and is easy to maintain"), to reduce unnecessary downstream module calls.

[0255] Recommendation type input (Recommendation Request): the module outputs ForwardSignal: true, generates a short confirmation phrase (for example: "I understand, I will recommend suitable furniture for you according to your description"), and passes the user input and related historical state to an information preprocessing intelligent agent, so that the intelligent agent generates text features and image features and then enters a subsequent decision-making process.

[0256] Other input (Other Types): the module generates a clarification guide (Clarification) (for example: "Do you want to know which type of product, or do you want to get some suggestions?"), and encourages the user to supplement the demand or provide more explicit expressions.

[0257] The text feature understanding module and the image feature understanding module provided by the embodiment of the present application are used to convert natural language and picture information input by the user into structured features that can be used for subsequent reasoning by a large language model (LLM), to provide prior input for subsequent modules.

[0258] The text feature understanding module provided in this specific embodiment of the invention extracts text features {text} from a user's input natural language description (e.g., "I want to replace the sofa that takes up too much space in the living room"). This module calls a Large Language Model (LLM) to extract key semantic elements, such as intent, target object, attributes, and preferences. It guides the LLM to convert the user's natural language description into JSON tags {text} using a preset dynamic template.

[0259] The intent is what task the user wants to accomplish (e.g., change, purchase, cancel, plan).

[0260] The target object is a core object related to the task (such as: items, services, locations);

[0261] The attribute or preference refers to the user's requirements for the characteristics of an object (such as style, size, budget, priority).

[0262] The image feature understanding module provided in this specific embodiment of the invention extracts image features, including elements such as object type, visual attributes, and spatial information, through a multimodal perception model. For image input, the system calls the multimodal model and constrains its output with task-oriented scene descriptions using prompts, converting them into JSON tags {image}.

[0263] The object type refers to key objects or regions in the image that are relevant to the task.

[0264] The visual attributes include color, shape, material, and style.

[0265] Spatial Info: The position or layout of an object in an image.

[0266] The information processing module provided in this specific embodiment of the invention is used to uniformly integrate the feature information of multimodal inputs and generate preliminary label results that can be used for subsequent decision-making. In human-computer collaborative decision-making tasks, users often input mixed information, including text and images, at once. This information can be divided into two main categories: contextual information and requirement information. Contextual information describes the user's background, conditions, or environmental characteristics (such as spatial attributes, resource constraints, and external environment), providing the system with a basis for understanding the decision-making context; requirement information expresses the user's goals, preferences, or functional requirements, directly pointing to the direction of personalized recommendations or decision generation. In actual interaction, these two types of information are usually expressed in combination. If they are not accurately distinguished, the subsequent reasoning and recommendation logic of the intelligent agent will not be able to accurately understand the user's intent, affecting the final decision-making effect.

[0267] To solve this problem, the module first receives the text features output by the text feature processing module and the image features output by the image feature processing module. At the same time, the system retrieves the stored historical initial context tags and historical initial requirement tags from the interaction history record to form the context information baseline required for the current round of reasoning. Subsequently, the system injects the above information into the dynamic slots {TEXT}, {IMAGE}, and {HISTORY_INITIAL_CONTEXT_TAGS} {HISTORY_INITIAL_REQUIREMENT_TAGS} in the preset prompt word template, guiding the large language model (LLM) to perform deduplication, integration, and semantic merging of historical tags and new tags in the current round during reasoning. The system classifies tags related to the environment, scene, and layout as context tags Initial_ContextTags, and classifies tags related to user goals, requirements, and preferences as requirement understanding tags Initial_RequirementTags, and updates the latest input when there is a conflict between the tags.

[0268] The comprehensive reasoning agent provided by the embodiments of the present application serves as the core logic hub of the system, responsible for generating prompt word templates for subsequent modules based on the initial context tags {initial_context_tags} and initial requirement tags {initial_requirement_tags} output by the information processing module, and further performing tag reasoning and attribute completion to provide high-quality input for subsequent personalized recommendation.

[0269] The comprehensive reasoning agent provided by the embodiments of the present application includes a personalized prompt word generation module, a context understanding module, and a requirement understanding module. After receiving the initial context tags {initial_context_tags} and initial requirement understanding tags {initial_requirement_tags} output by the information preprocessing agent, the personalized prompt word generation module uses a large language model to perform scene reasoning to determine the current decision task type, such as shopping, education, travel, and medical care. Based on the scene features and the positioning of the agent's functions, the personalized prompt word module generates customized prompt word templates for each downstream functional agent (context understanding agent, requirement understanding agent, and explanation generation agent). The completed prompt word templates are passed to the corresponding agents as fixed base templates for continuous reuse in multiple rounds of interaction. Related templates are generated only once and reused in multiple rounds of interaction. Subsequent user input information is dynamically filled into the slots of the templates to ensure that the prompt words remain contextually coherent.

[0270] The context understanding module provided by the embodiments of the present application is used for receiving initial context tags {initial_context_tags} output by an information preprocessing agent, and further performing semantic reasoning and attribute completion on the tags to generate a more refined and structured context attribute tag set {context_tags}. The tags not only describe the current environment or context characteristics, but also infer the implicit needs that the user may derive in the context, so as to, together with requirement tags {requirement_tags} output by a requirement understanding agent, constitute a core matching basis for a subsequent recommendation system. In a specific implementation, the module is driven based on a large language model (LLM), and a preset context understanding prompt word template (provided by a personalized prompt word generation agent) is called to inject {initial_context_tags} into a dynamic slot and perform deep reasoning to complete implicit attribute information. For example, for the description of “need to be suitable for limited space”, the system further completes attribute tags such as “high flexibility, expandable, and small occupied area”.

[0271] The requirement understanding module provided by the embodiments of the present application is used for receiving initial requirement tags {initial_requirement_tags} output by an information preprocessing agent, and further performing semantic refinement, reasoning completion and dynamic evolution on the basis to finally generate a structured requirement attribute tag set {requirement_tags}. In the implementation process, the module is driven based on a large language model (LLM), and a requirement understanding prompt word template provided by a personalized prompt word generation agent is called to perform deep semantic analysis on {initial_requirement_tags} and automatically complete the fuzzy, implicit or potential requirements in the user expression. For example, for the non-specific descriptions such as “want to be more design-oriented” or “hope to be suitable for multiple people”, the system will infer and generate more specific attribute dimensions (such as function configuration, available scenario, budget range, style element, etc.), and uniformly encode them into structured requirement tags {requirement_tags}.

[0272] The personalized recommendation agent provided by the embodiments of the present application is a core decision execution module of the present application, which is used for combining context tags {context_tags} generated by a context understanding agent and requirement tags {requirement_tags} generated by a requirement understanding agent, calling a recommendation algorithm to complete the screening, sorting and recommendation result output of candidate items, and providing an understandable explanation for the recommendation result through an explanation generation mechanism. In the implementation, the agent is composed of a recommendation algorithm module and an explanation generation module.

[0273] The recommendation algorithm module provided by the embodiments of the present application assumes that the first icandidate options A set of predefined option tag sets is defined The tag sets describe the multi-dimensional attributes of the options (such as functional features, styles, scene adaptability, technical parameters, etc.) for matching with the user's context tag set and requirement tag set. The module uses a tag similarity matching algorithm to comprehensively consider the consistency of the candidate options with the user's context and requirements, and the specific formula is as follows: Wherein, represents the comprehensive matching score of the candidate option . and are the weight coefficients of context matching and requirement matching, which can be flexibly set according to the application scenario. In a specific embodiment, the weight coefficients are both 0.5.

[0274] The context matching score provided by the recommendation algorithm module provided by the embodiment of the application is defined as the cosine similarity between the candidate option tag set and the context tag set : . .

[0275] The requirement matching score provided by the recommendation algorithm module provided by the embodiment of the application is defined as the cosine similarity between the candidate option tag set and the requirement tag set : . Wherein represents that the tag set generates a corresponding semantic vector representation through a pre-trained word vector model (such as Word2Vec, BERT, or CLIP, etc.).

[0276] The embodiment of the application utilizes the recommendation algorithm module to sort all candidate options according to , and selects several with the highest scores as the personalized recommendation results. At the same time, the related information of the recommendation results will be input into the explanation generation module to generate personalized explanations, helping the user to understand the basis of the recommendation and enhancing the sense of trust.

[0277] The explanation generation module provided by the embodiment of the application is used to receive the recommendation results { recommendation_results} output by the personalized recommendation agent, and combines the context tags ContextTags and the requirement tags RequirementTags to perform tag matching analysis and difference explanation on each recommendation item, and generates structured explanation content Explanations. These explanations not only reveal the matching logic of the recommendation item with the user's requirements and scene, but also point out the potential incomplete matching places and additional value, enhancing the transparency and user trust of the recommendation results.

[0278] In a specific implementation, this module is also driven based on a large language model (LLM), calling an explanation generation prompt template (ExplanationPromptTemplate) provided by a personalized prompt word generation agent. The system injects ContextTags, RequirementTags, and RecommendationResults into the dynamic slot of the template, and the LLM performs reasoning according to the task instructions in the template to generate the following content for each recommendation result:

[0279] Label matching explanation: Analyze which features in the recommended item are highly consistent with ContextTags and RequirementTags (such as "modern style" and "space-saving").

[0280] Difference analysis: Identify the deviations or deficiencies between the recommended item and the user's tags (such as "large size, need to adjust the layout").

[0281] Value supplement: Extract additional advantages of the recommended item (such as "adjustable backrest" and "easy-to-clean fabric").

[0282] Finally, the system will present the recommendation results on the dialogue interface, along with the explanations generated by the LLM, allowing users to clearly understand the recommendation logic (why the recommendation, which tags match, and what deficiencies or additional values), and make more confident decisions accordingly.

[0283] The specific embodiments of the present application propose a multi-round interaction history storage and calling mechanism to support the continuous updating and dynamic calling of label information, recommendation results, and explanation content in the human-machine collaborative decision-making process. This mechanism ensures that the system can maintain a complete memory of the user's context and demand state in multiple rounds of dialogue, and provide consistency and traceability for subsequent recommendation and explanation generation.

[0284] In implementation, the system uses an interaction record method based on structured JSON data, and generates a complete interaction record (Turn Record) for each round of interaction, which is written into the history database (History) in chronological order. Each interaction record contains the following key fields:

[0285] turn_id: Interaction round identification, used to record the dialogue order;

[0286] user_input: User input content, including text (text) and image (image) and other multi-modal information;

[0287] meta_tag: Input intent classification (e.g., Recommendation, Question, Other), generated by the personalized response agent;

[0288] initial_context_tags / initial_requirement_tags: Initial context tags and initial requirement tags output by the information preprocessing agent;

[0289] context_tags / requirement_tags: More refined tags generated by the combined reasoning agent (context understanding and requirement understanding modules) to support subsequent recommendations and explanations;

[0290] recommendation_results: A list of recommendation results obtained by the personalized recommendation algorithm, including information such as item_id, name, and score;

[0291] recommendation_explanations: Explains the detailed descriptions generated by the module based on the recommendation results, indicating the reasons for tag matching, differences, and added value.

[0292] During multiple rounds of interaction, each user input is fully recorded and added to the interaction history. When the system is called in the next round, it will extract relevant tags and context information from the history and dynamically inject them into the response prompt word template to form an inheritable reasoning context.

[0293] The code for storing interactive history provided in a specific embodiment of the present invention is as follows:

[0294] {

[0295] "turn_id": 1,

[0296] "user_input": {

[0297] text: "I want to buy a sofa",

[0298] "image": "image_7.png"

[0299] },

[0300] "meta_tag": "Recommendation",

[0301] "initial_context_tags": ["Living Room", "Modern Style"],

[0302] "initial_requirement_tags": ["sofa"],

[0303] "context_tags": ["living room", "modern style", "gray tone"],

[0304] "requirement_tags": ["sofa", "save space"],

[0305] "recommendation_results": [

[0306] {

[0307] "item_id": "F1234",

[0308] "name": "Modern Minimalist Double Sofa",

[0309] "score": 0.92,

[0310] },

[0311] {

[0312] "item_id": "F5678",

[0313] "name": "Nordic Style Fabric Sofa",

[0314] "score": 0.87,

[0315] } ]

[0317] "recommendation_explanations": [

[0318] {

[0319] "item_id": "F1234",

[0320] "explanation": "This sofa style matches your 'Modern Minimalist' tag and uses easy-to-clean fabric, meeting the 'Save Space' requirement."

[0321] }

[0322] {

[0323] "item_id": "F5678",

[0324] "explanation": "This sofa is lightweight and easy to move, with a Nordic design that matches your style tag and saves space."

[0325] } ]

[0327] }。”

Claims

1. A modular human-machine collaborative decision-making method based on multi-agent systems, characterized in that, include: Based on the current user input information and historical interaction records, the response information is output through the large language model. When the user input information is a recommendation request, text features and image features are extracted from the natural language description and image information of the current user input information respectively. The preliminary context label and preliminary demand label of the previous round, as well as the current text features and image features, are injected into the dynamic slots in the preset first prompt word template to construct the first prompt word. Based on the first prompt word, the current preliminary context label and preliminary demand label are obtained through the large language model. Based on the initial context labels and initial demand labels of the current user, second and third prompt word templates are constructed to generate supplementary context labels and demand labels. The current initial context labels and demand labels are injected into the second and third prompt word templates to obtain the second and third prompt words. The second and third prompt words are then input into the large language model to obtain the context label set and the demand label set. The context tag set and the demand tag set are compared with the candidate tag set by cosine similarity to obtain the context matching score and the demand matching score, respectively. Based on the context matching score and the demand matching score, the comprehensive matching score of the candidate options is obtained, and the top N candidate options with the highest comprehensive matching scores are recommended to the user.

2. The modular human-machine collaborative decision-making method based on multi-agent systems according to claim 1, characterized in that, Based on the initial contextual tags and initial demand tags of the current user, a fourth prompt word template is constructed. The fourth prompt word template includes a scenario judgment question and a recommendation candidate explanation question. The scenario judgment question is to determine the task scenario based on the initial contextual tags and initial demand tags generated by the current user and the dimensions that need to be covered in the explanation under the task scenario. The recommendation candidate explanation question is to explain the reasons for recommending each recommendation candidate based on the dimensions that need to be covered in the explanation. The context tag set, demand tag set, and candidate recommendation results are injected into the dynamic slot of the fourth prompt word template to obtain the fourth prompt word. The fourth prompt word is then input into the large language model to obtain the explanation of the recommended candidate. The explanation of the recommended candidate and the recommended candidate are then pushed to the user.

3. The modular human-machine collaborative decision-making method based on multi-agent systems according to claim 2, characterized in that, The explanation includes contextual tags and demand tags that provide the driving candidate recommendation results, a comparison of the differences between the recommended candidates and user needs, and an explanation of the additional value of the recommended candidates.

4. The modular human-machine collaborative decision-making method based on multi-agent systems according to claim 1, characterized in that, The second prompt word template includes a scenario judgment question and a question that supplements the initial context labels; The scenario judgment problem involves determining the task scenario based on the initial context labels and initial requirement labels generated by the current user, and clarifying the requirements for supplementing the context under the task scenario. The issue of supplementing the initial context labels involves supplementing and formatting the current initial context labels based on the requirements for context supplementation in the task scenario.

5. The modular human-machine collaborative decision-making method based on multi-agent systems according to claim 1, characterized in that, The third prompt word template includes scenario judgment questions and questions to supplement the initial requirement tags; The scenario judgment problem involves determining the task scenario based on the initial context labels and initial requirement labels generated by the current user, and clarifying the requirements for understanding the requirements under the task scenario. The issue of completing the preliminary requirement tags involves supplementing and formatting the current preliminary requirement tags based on the requirements for understanding the requirements under the task scenario.

6. The modular human-machine collaborative decision-making method based on multi-agent systems according to claim 1, characterized in that, Based on the first prompt word, the large language model obtains the current preliminary context label and preliminary demand label. The first prompt word enables the large language model to deduplicatize, fuse, and classify the current text features and image features with the previous preliminary context label and preliminary demand label to obtain the current preliminary context label and preliminary demand label, and to convert the current preliminary context label and preliminary demand label into a different format.

7. The modular human-machine collaborative decision-making method based on multi-agent systems according to claim 1, characterized in that, Extract text features from the natural language description of the current user input information, including: Text features are extracted from natural speech descriptions based on constraints of text feature prompts using a large language model; The text feature prompts include natural language descriptions injected into dynamic slots and natural language descriptions based on user input information to obtain the user's intent, target object, attributes, and preferences, and the obtained user intent, target object, attributes, and preferences are formatted.

8. The modular human-machine collaborative decision-making method based on multi-agent systems according to claim 1, characterized in that, Image features are extracted from the image information of the current user input, including: Image features based on image feature prompts are used to extract image features from the image information of the current user input information through a multimodal perception model. The image feature prompts include image information injected into dynamic slots and object, attribute, and spatial information obtained from the image information, and the format conversion of the obtained object, object attribute, and spatial information in the image.

9. The modular human-machine collaborative decision-making method based on multi-agent systems according to claim 1, characterized in that, Based on the current user input and historical interaction records, the response information is output through a large language model, including: Based on the constraints of response prompt words, the large language model outputs response information to reply to the current user. The response information includes immediate answers, confirmation phrases, and clarification guidance. The constraints imposed by the response prompts enable the large language model to categorize user input into question-based input, recommendation requests, and unclassifiable or casual content. When the user input is a question-based input, the large language model generates an immediate response based on historical interaction records to recommend candidate options to the user. When the user input is a recommendation request or a response to candidate options recommended in the previous round, the large language model generates a confirmation message. When the user input is unclassifiable or casual content, the large language model generates clarification guidance.

10. A modular human-machine collaborative decision-making device based on multi-agent systems, characterized in that, include: An information preprocessing agent includes a personalized response module, a text feature understanding module, an image feature understanding module, and an information processing module. The personalized response module outputs response information based on the current user input and historical interaction records using a large language model. The text feature understanding module extracts text features from the natural language description of the current user input when the user input is a recommendation request. The image feature understanding module extracts image features from the image information of the current user input when the user input is a recommendation request. The information processing module injects the preliminary context labels and preliminary demand labels from the previous round, as well as the current text features and image features, into a dynamic slot in a preset first prompt word template to construct a first prompt word. Based on the first prompt word, the current preliminary context labels and demand labels are obtained through a large language model. A comprehensive reasoning agent includes a personalized prompt word generation module, a context understanding module, a demand understanding module, and an information processing module. The personalized prompt word generation module is used to construct second and third prompt word templates for generating complete context labels and demand labels based on the initial context labels and initial demand labels generated by the current user. The context understanding module is used to inject the current initial context labels into the second prompt word template to obtain the second prompt word, input the second prompt word into a large language model to obtain the completed context label, and construct a context label set through multiple rounds of obtained completed context labels. The demand understanding module is used to inject the current demand label into the third prompt word template to obtain the third prompt word, input the third prompt word into a large language model to obtain the completed demand label, and construct a demand label set through multiple rounds of obtained completed demand labels. A personalized recommendation agent is used to perform cosine similarity comparison between the context tag set and the demand tag set and the candidate option tag set to obtain a context matching score and a demand matching score, respectively. Based on the context matching score and the demand matching score, a comprehensive matching score of the candidate options is obtained, and the top N candidate options with the highest comprehensive matching scores are recommended to the user.

Images