Large model-based dialogue risk assessment method, system, device and medium

Abstract

The application discloses a large model-based dialogue risk assessment method, system, device and medium, comprising: obtaining dialogue information of a user, performing information extraction on the dialogue information based on a large model to obtain first node information, wherein the first node information comprises a target event; extracting a set of key parameters related to the first node information from a preset sentiment dictionary, and using the set of key parameters to illustrate the target event based on the large model to obtain a candidate answer description; performing risk assessment on the candidate answer description based on the large model to obtain a risk assessment result, and determining a final answer description based on the risk assessment result. The application can accurately capture the real emotions of the user, and then provide dialogue output that meets the risk requirements.

Description

Technical Field

[0001] The present application relates to the field of large models, and in particular to a large model-based dialogue risk assessment method, system, device and medium. Background Art

[0002] In recent years, with the rapid development of artificial intelligence (AI) and natural language processing (NLP), automated dialogue systems based on large language models have become a hot topic in research and application. These systems can simulate human conversations and, through real-time interaction with users, not only provide information and services but also understand and respond to users' emotional states and psychological needs to a certain extent.

[0003] While current large language models can handle complex language structures and conduct extensive conversations, they often struggle to deeply understand user emotions due to a lack of in-depth domain knowledge. These models struggle to accurately capture the true meaning and nuances of user emotions, leading to misunderstandings or inappropriate responses. Furthermore, the sentiment lexicons relied on by most current dialogue systems are often static. This means they operate based on predefined vocabulary and sentiment labels and lack the flexibility to adapt to the diverse and dynamic nature of emotional expression in complex dialogue systems. Consequently, systems often struggle to respond promptly and effectively to rapidly changing conversation content and user emotions.

[0004] Application Contents

[0005] This application provides a large-model-based dialogue risk assessment method, system, device, and medium that can accurately capture the user's true emotions and provide dialogue output that meets risk requirements.

[0006] In a first aspect, the present application provides a large-model-based dialogue risk assessment method, comprising:

[0007] Acquire user conversation information, and extract information from the conversation information based on the large model to obtain first node information, wherein the first node information includes a target event;

[0008] Extracting a key parameter set related to the first node information from a preset sentiment dictionary, and interpreting the target event with the key parameter set based on the large model to obtain candidate answer descriptions;

[0009] A risk assessment is performed on the candidate answer descriptions based on the large model to obtain a risk assessment result, and a final answer description is determined based on the risk assessment result.

[0010] The embodiments of the present application can accurately obtain first-node information such as target events, psychological crisis information, and emotional information by extracting information from the acquired conversation information, thereby facilitating the integration of the first-node information to accurately capture the user's true emotions. By extracting a set of key parameters related to the first-node information from a preset first emotional dictionary, the key parameter set related to the first-node information can be accurately extracted from the emotional dictionary, facilitating the subsequent acquisition of candidate answer descriptions that take into account the user's true emotions. By performing a risk assessment on the candidate answer descriptions, it is possible to comprehensively assess whether the output response of the large model meets the risk requirements, ensure the accuracy of the risk assessment, and thus provide a conversation output that meets the risk requirements. Compared with the prior art, the present application can accurately capture the user's true emotions and thus provide a conversation output that meets the risk requirements.

[0011] Furthermore, before obtaining the user's conversation information, the method further includes:

[0012] Acquiring first psychological data and cleaning the first psychological data to obtain second psychological data, wherein the first psychological data includes professional psychological information and emotional conversation records;

[0013] Node data and edge data are extracted from the second psychological data, and a first sentiment dictionary is constructed based on the node data and the edge data.

[0014] In this way, by introducing professional psychological information and emotional dialogue records, we can effectively deal with users' complex emotional states and psychological crises, and facilitate the subsequent in-depth understanding and effective response to users' emotions.

[0015] Furthermore, before extracting the key parameter set related to the first node information from the preset sentiment dictionary, the method further includes:

[0016] extracting first edge information corresponding to the first node information from the conversation information;

[0017] Inserting the first node information and the first edge information into the first sentiment dictionary to obtain a second sentiment dictionary;

[0018] The similarity values ​​corresponding to each node and each edge in the second sentiment dictionary are respectively calculated, the nodes and edges with high similarity values ​​are respectively merged, and the merged node attributes and edge attributes are respectively updated to obtain a sentiment dictionary.

[0019] In this way, by adaptively updating the sentiment dictionary based on user conversation information, we can ensure that the sentiment dictionary adapts to the rapid changes of complex conversation systems, thereby improving the adaptability of the large language model to new contexts and new words, and facilitating the subsequent in-depth understanding and effective response to user emotions.

[0020] Furthermore, the key parameter set related to the first node information is extracted from a preset emotional dictionary, wherein the first node information also includes the first psychological crisis information and the first emotion information, specifically:

[0021] Extracting second node information related to the first node information from a preset sentiment dictionary;

[0022] Based on the second node information, calculating the path length between the second psychological crisis information and the sentiment dictionary, and extracting the third node information with the shortest path length;

[0023] Based on the second node information, calculating the emotional similarity between the second emotional information and the emotional dictionary, and extracting the fourth node information with the highest emotional similarity;

[0024] Determining a belonging community of the first node information in different sentiment dictionary communities, and extracting a community summary report corresponding to the belonging community, wherein the sentiment dictionary community is a community division result of the sentiment dictionary;

[0025] A sentiment parameter set is obtained based on the second node information, the third node information, the fourth node information and the community summary report.

[0026] In this way, by extracting the key parameter set related to the first node information from the preset first emotional dictionary, the key parameter set related to the first node information can be accurately extracted from the emotional dictionary, which facilitates the subsequent acquisition of candidate answer descriptions that take into account the user's true emotions.

[0027] Furthermore, the determining of the belonging community of the first node information in different sentiment dictionary communities is specifically:

[0028] Calculating a number of modularity values ​​corresponding to the first node information in different sentiment dictionary communities;

[0029] Based on the plurality of modularity values, determining modularity change values ​​of the first node information assigned to different sentiment dictionary communities;

[0030] Based on the modularity change value, a community to which the first node information belongs is determined.

[0031] In this way, by determining the belonging community of the first node information, the subsequent global analysis and output decision of the user's emotions are facilitated, and the subsequent in-depth understanding and effective response to the user's emotions are facilitated.

[0032] Furthermore, the final answer description is determined based on the risk assessment result, specifically:

[0033] Determining whether the risk assessment results meet preset conditions;

[0034] If the risk assessment results meet the preset conditions, the final answer description is determined;

[0035] If the risk assessment result does not meet the preset conditions, the target event is re-interpreted using the key parameter set and the candidate answer description is updated.

[0036] In this way, by performing a risk assessment on the candidate answer descriptions, it is possible to comprehensively evaluate whether the output response of the large model meets the risk requirements, ensure the accuracy of the risk assessment, and then provide a dialogue output that meets the risk requirements.

[0037] Furthermore, after determining the final answer description based on the risk assessment result, the method further includes:

[0038] Extract information from the answer description based on the large model to obtain fifth node information and second edge information;

[0039] The sentiment dictionary is updated based on the fifth node information and the second edge information.

[0040] In this way, by adaptively updating the sentiment dictionary based on the answer description of the large model, it can ensure that the sentiment dictionary adapts to the rapid changes of the complex dialogue system, thereby improving the adaptability of the large language model to new contexts and new words, facilitating the subsequent in-depth understanding and effective response to user emotions.

[0041] In a second aspect, the present application provides a large model-based dialogue risk assessment system, comprising: an acquisition module, an extraction module, and an assessment module;

[0042] The acquisition module is configured to acquire user conversation information, extract information from the conversation information based on the large model, and obtain first node information, wherein the first node information includes a target event;

[0043] The extraction module is configured to extract a key parameter set related to the first node information from a preset sentiment dictionary, interpret the target event with the key parameter set based on the large model, and obtain candidate answer descriptions;

[0044] The evaluation module is used to perform risk evaluation on the candidate answer descriptions based on the large model, obtain a risk evaluation result, and determine a final answer description based on the risk evaluation result.

[0045] The embodiments of the present application can accurately obtain first-node information such as target events, psychological crisis information, and emotional information by extracting information from the acquired conversation information, thereby facilitating the integration of the first-node information to accurately capture the user's true emotions. By extracting a set of key parameters related to the first-node information from a preset first emotional dictionary, the key parameter set related to the first-node information can be accurately extracted from the emotional dictionary, facilitating the subsequent acquisition of candidate answer descriptions that take into account the user's true emotions. By performing a risk assessment on the candidate answer descriptions, it is possible to comprehensively assess whether the output response of the large model meets the risk requirements, ensure the accuracy of the risk assessment, and thus provide a conversation output that meets the risk requirements. Compared with the prior art, the present application can accurately capture the user's true emotions and thus provide a conversation output that meets the risk requirements.

[0046] In a third aspect, the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the large model-based dialogue risk assessment method as described in the present application are implemented.

[0047] In a fourth aspect, the present application provides a readable storage medium storing a program or instruction, which, when executed by a processor, implements the steps of large-model-based dialogue risk assessment as described in the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flowchart of an embodiment of a large model-based conversation risk assessment method provided by this application;

[0049] Figure 2 This is a flowchart of another embodiment of the large model-based conversation risk assessment method provided by this application;

[0050] Figure 3 This is a structural diagram of an embodiment of a large model-based dialogue risk assessment system provided by this application;

[0051] Figure 4 This is a hardware structure diagram of the electronic device provided in this application. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0053] It should be understood that the step numbers used herein are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0054] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0055] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0056] The term "and / or" refers to and includes any and all possible combinations of one or more of the associated listed items.

[0057] In recent years, AI and NLP technologies have developed rapidly, and large-scale language model dialogue systems have become a hot topic in research and application. These automated dialogue systems can not only simulate conversations and provide services, but also respond to users' emotional states and psychological needs. However, large models lack in-depth specialized knowledge and struggle to accurately capture the true meaning of users' emotions, leading to misunderstandings or inappropriate responses. Furthermore, the sentiment lexicons currently relied on by dialogue systems are mostly static and cannot adapt to the diversity and dynamic nature of emotional expression in complex conversations, making it difficult to respond promptly and effectively to rapidly changing conversations and user emotions.

[0058] Next, the nouns involved in this application are analyzed:

[0059] Artificial intelligence (AI) is a new technical discipline that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. A branch of computer science, AI seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thinking. It also encompasses the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0060] Large models, short for Large Language Models (LLMs), refer to machine learning models with large parameters and complex computational structures. These models are typically built using deep neural networks and have billions or even hundreds of billions of parameters. These models are designed to learn complex patterns and features by training on massive amounts of data, resulting in strong generalization capabilities and the ability to make accurate predictions on unseen data.

[0061] Based on this, the embodiments of the present application provide a large-model-based dialogue risk assessment method, system, device and medium, which can accurately capture the user's true emotions and provide dialogue output that meets risk requirements.

[0062] The large-model-based conversation risk assessment method, system, device, and medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the large-model-based conversation risk assessment method in the embodiments of the present application is described.

[0063] The conversation risk assessment method based on a large model provided in the embodiment of the present application relates to the field of large models. The conversation risk assessment method based on a large model provided in the embodiment of the present application can be applied in a terminal, can also be applied in a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, or can be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the conversation risk assessment method based on a large model, etc., but is not limited to the above forms.

[0064] The present application can also be used in numerous general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0065] Example 1

[0066] Please refer to Figure 1 , Figure 1 This is a flowchart of an embodiment of a large model-based conversation risk assessment method provided by the present application, including steps S101 to S103;

[0067] Step S101: Acquire user conversation information, extract information from the conversation information based on a large model, and obtain first node information, wherein the first node information includes a target event;

[0068] In some embodiments, before obtaining the user's conversation information, it also includes: first, obtaining first psychological data, and cleaning the first psychological data to obtain second psychological data, wherein the first psychological data includes professional psychological information and emotional conversation records; second, extracting node data and edge data in the second psychological data, and constructing a first emotional dictionary based on the node data and the edge data.

[0069] Specifically, first, first psychological data is acquired, where the first psychological data includes but is not limited to professional information documents such as information research papers, review articles, psychological counseling guides, and multiple emotional conversation records. The first psychological data is stored in a parseable format, such as a text file, database record, or spreadsheet, using a database system (e.g., MongoDB, MySQL). Second, the first psychological data is cleaned, including detecting and deleting most duplicate content in the first psychological data, applying keyword filtering algorithms and manual screening to eliminate records unrelated to emotional conversations, and converting the first psychological data into a unified format (e.g., TXT or CSV). The cleaned professional psychological information and emotional conversation records are integrated into second psychological data. Third, keywords, phrases, or concepts related to psychological analysis are identified and extracted as node data, as well as relationships between nodes. These relationships are then extracted as edge data. Node data may include target event nodes, psychological crisis nodes, emotion nodes, etc., while edge data may include information such as relationship type, strength, and direction. The node data and edge data are then converted into corresponding data structures and output according to a data structure format. Finally, a first sentiment dictionary is constructed based on the node data and the edge data, wherein an empty graph is created using NetworkX to store the node data and edit data of the sentiment dictionary, and the node data and edit data are added to the graph and stored in a Neo4j database.

[0070] In some embodiments, the target event node may be but not limited to 'Environmental Protection', 'Study Hard', etc., the psychological crisis node may be but not limited to 'Depression Risk', etc., and the emotion node may be but not limited to 'Happiness', etc.

[0071] In some embodiments, for each node, it is necessary to extract the corresponding node name entity_name, node type entity_type and node description entity_description. For 'EnvironmentalProtection (environmental protection)', entity_name is 'EnvironmentalProtection', entity_type is 'target event node', and entity_description is 'a series of activities and concerns related to the protection of the natural environment, sustainable development, etc.'; for 'StudyHard (study hard)', its entity_name is 'StudyHard', entity_type is 'target event node', e The entity_description is 'having a clear purpose of improving knowledge and skills through hard work'; for 'DepressionRisk', the entity_name is 'DepressionRisk', the entity_type is 'Psychological Crisis Node', and the entity_description is 'situations that may potentially lead to depression, such as psychological instability caused by long-term stress, major setbacks, etc.'; for the 'Happiness' node, its entity_name is 'Happiness', the entity_type is 'Emotional Node', and the entity_description is 'a positive emotional state, usually manifested as joy, satisfaction, and optimism'.

[0072] In some embodiments, by defining a task of inputting Prompt, the task can be clearly defined as identifying a specified type of node data and its edge data from the second psychological data. First, the large model is guided to identify the node data and extract the name, type and description of the node data; secondly, based on the extracted node data, the relationship between the node data, such as the relationship description and relationship strength, is further identified and output as edge data; finally, the node data and edge data are output in a preset format.

[0073] It should be noted that the large model is short for Large Language Model (LLM). In some embodiments, the large model can be, but is not limited to, Extract-LLM, ChatGPT series models, ERNIE3.0 Titan model, General Language Model (GLM)-130B model, etc.

[0074] In some embodiments, the database in the large model is provided with basic content in multiple languages, including at least three basic languages: Simplified Chinese, Traditional Chinese and English, and may also include: Indian languages, Iranian languages, other Germanic languages ​​other than English, foreign language types in the Romance languages ​​or Slavic languages, etc.

[0075] In some embodiments, the data structure in the preset format may be any one of a JSON format, a JSON5 format, a STOB format, an ENO format, an AXON format, or a BRE format.

[0076] In some embodiments, the data structure generated in JSON format can be represented as:

[0077] {

[0078] "source":<source entity name>,

[0079] "target":<target entity name>,

[0080] "relationship":<relationship description>,

[0081] "relationship_strength": <relationship strength>,

[0082] }

[0083] In this way, by introducing professional psychological information and emotional dialogue records, we can effectively deal with users' complex emotional states and psychological crises, and facilitate the subsequent in-depth understanding and effective response to users' emotions.

[0084] It should be noted that after constructing the sentiment dictionary, it is necessary to divide the sentiment dictionary into communities to discover sentiment themes and patterns, which facilitates the subsequent detection of changes in the sentiment dictionary. Specifically, the basic sentiment dictionary is divided into communities using the Louvain algorithm, and the designed community summary prompt is used to guide the large model to generate a summary for each community. In this case, all node data and edge data in the sentiment dictionary are used as input to initialize the Louvain algorithm, assigning each node data to a separate community. Through an iterative process, the node data is moved to other communities to improve modularity. The modularity calculation formula is: Where A ij is the weight of the edge between nodes i and j, k i and k j are the degrees of nodes i and j (i.e., the number of edges connected to them), m is the sum of the weights of all edges in the network, this term represents the difference between the actual edge weight and the expected edge weight, δ(c i ,c j) is the Kronecker function, which takes the value 1 when nodes i and j belong to the same community and 0 if they do not. When modularity cannot be increased by further moving individual nodes, the algorithm enters the next stage, merging small communities to form larger ones. At the same time, a specific community summary prompt, t, is used to guide the large language model (LLM) to generate a corresponding community summary report for each generated community. The generated community summary report is then optimized and adjusted to obtain the final community summary report.

[0085] It should be noted that community division is not the focus of this application and will not be expanded here.

[0086] It can be understood that the conversation information includes current conversation information and historical conversation information. Specifically, first, the big model obtains the user's current conversation information in real time through the chat interface, and retrieves the user's historical conversation information from the database to obtain the complete conversation context; secondly, the conversation information is preprocessed to facilitate the big model to extract information; then, the big model will understand and analyze the received preprocessed conversation information to extract the first node information, wherein the first node information includes but is not limited to target events, psychological crisis information and emotional information, and the extracted first node information is organized into a predefined format.

[0087] In some embodiments, target events include, but are not limited to, conversation topics and user intent. Conversation topics are the brief, main topic of the user's current discussion, and user intent is the user's desired goal in the conversation, such as seeking advice, expressing feelings, or seeking support. Psychological crisis information refers to potential serious mental health risks for the user, such as extreme anxiety, depression, and self-harm tendencies. Emotional information is determined based on the user's language and expression, and includes, but is not limited to, happiness, sadness, anger, anxiety, and helplessness.

[0088] In some embodiments, data preprocessing includes but is not limited to data cleaning of conversation information to remove irrelevant characters in the conversation information to ensure the purity of the text; using NLP technology to segment the conversation information to determine the part of speech of each word; converting the text into a numerical vector form suitable for large model processing, etc.

[0089] In some embodiments, the role capabilities and tasks of the system-level Prompt of the large model can be defined first to clarify the tasks and return format of the large model; and the input-level Prompt can be constructed to integrate the user's conversation information, namely the historical conversation information {history} and the current conversation information {user}; finally, the role definition and task description of the system-level Prompt, as well as the historical conversation information and user input of the input-level Prompt, are combined to form the final Extract-Prompt to guide the Extract-LLM to extract information from the conversation information and extract the user's first node information.

[0090] It should be noted that the terms "first," "second," "third," "fourth," or "fifth" do not indicate a specific order; they simply distinguish between terms with the same meaning but from different sources. These terms can be understood as names. The first node is extracted from the user's conversation information. The second node is related to the first node in the sentiment lexicon. The third node is the node with the shortest second psychological crisis path in the sentiment lexicon. The fourth node is the node with the highest emotional similarity to the second emotional information in the sentiment lexicon. The fifth node is extracted from the large model's answer description.

[0091] Step S102: extracting a key parameter set related to the first node information from a preset sentiment dictionary, interpreting the target event with the key parameter set based on the large model, and obtaining candidate answer descriptions;

[0092] In some embodiments, before extracting the key parameter set related to the first node information from the preset emotional dictionary, it also includes: first, identifying and extracting the first edge information corresponding to the first node information from the dialogue information; second, inserting the first node information and the first edge information into the first emotional dictionary to obtain a second emotional dictionary; then, respectively calculating the similarity values ​​corresponding to each node and each edge in the second emotional dictionary, merging the nodes and edges with high similarity values ​​respectively through an attribute matching algorithm, and updating the merged node attributes and edge attributes respectively to obtain an emotional dictionary, wherein the node similarity value is calculated according to the attributes of each node, and the edge similarity value is calculated according to the attributes of each edge. If the similarity value exceeds a preset threshold, the nodes that meet the preset threshold are merged to obtain a new node, and the edges that meet the conditions are also merged to obtain a new edge, and the attributes corresponding to the new node and the new edge are updated to obtain an updated emotional dictionary.

[0093] It should be noted that the similarity value calculation method may be, but is not limited to, cosine similarity or Jaccard similarity coefficient.

[0094] It should be noted that the preset threshold needs to be freely set in advance according to the specific application scenario and data characteristics, and this application does not impose any restrictions.

[0095] It should be noted that the attributes of each node include: name, type, description, etc., and the attributes of each edge include: relationship description, relationship strength, etc.

[0096] In this way, by adaptively updating the sentiment dictionary based on user conversation information, we can ensure that the sentiment dictionary adapts to the rapid changes of complex conversation systems, thereby improving the adaptability of the large language model to new contexts and new words, and facilitating the subsequent in-depth understanding and effective response to user emotions.

[0097] It can be understood that after the emotional dictionary is updated based on the user's conversation information, the key parameter set related to the first node information can be extracted from the emotional dictionary, wherein the first node information also includes the first psychological crisis information and the first emotion information, specifically: first, the second node information related to the first node information is extracted from the preset emotional dictionary, wherein the breadth-first search (BFS) algorithm can be used to find all the second node information related to the first node information and the corresponding edge information, including: initializing the BFS algorithm, taking the first node information as the starting node in the emotional dictionary, starting from the starting node, traversing the adjacent nodes layer by layer, recording all the passed nodes and edges, these nodes and edges constitute a network structure related to the starting node, and storing the traversed second node information and corresponding edge information as the relevant information of this round of conversation, recorded as {relevance}.

[0098] Secondly, based on the second node information, the path length between the second psychological crisis information and the second psychological crisis information in the sentiment dictionary is calculated, and the third node information with the shortest path length is extracted. The second psychological crisis information in the second node information is determined, and the second psychological crisis information is set as the initial node. The Dijkstra algorithm is initialized, and the Dijkstra algorithm is used to calculate the shortest path from the starting node to all other nodes in the sentiment dictionary. The third node information of the top k shortest paths is selected as output, and the path length of the third node information is recorded to guide the LLM analysis of potential risks. The third node information is marked as {risk}.

[0099] Then, based on the second node information, the emotional similarity between the second emotional information and the emotional dictionary is calculated, and the fourth node information with the highest emotional similarity is extracted, wherein the emotional information of the second node information is determined as the reference vector, and each element in the vector represents the node U connected to the user's emotional graph node, and all nodes N in the emotional dictionary are extracted based on the reference vector. i The eigenvector V i, where each element represents the presence or absence of a certain user emotion, and calculates each node N i With the eigenvector V i The emotional similarity value is calculated as follows: According to the emotion similarity value, the top k fourth node information with the highest emotion similarity value is selected as the output. At the same time, the emotion similarity value of the fourth node information is recorded to guide the LLM to analyze the user's related emotions. The fourth node information is marked as {similar}.

[0100] Thirdly, determining the belonging community of the first node information in different emotional dictionary communities, and extracting the community summary report corresponding to the belonging community, wherein the emotional dictionary community is the community division result of the emotional dictionary, including: assigning the first node information to all communities in the emotional dictionary respectively, calculating a plurality of modularity values ​​corresponding to the first node information in different emotional dictionary communities; based on the plurality of modularity values, determining the modularity change value of the first node information assigned to different emotional dictionary communities, wherein the calculation formula for calculating the modularity change value is: Where ΔQ is the modularity change value, is the weighted average of the edges within the community after the first node information is moved, is the weighted average of the edges within the community before the first node information is moved; based on the modularity change value ΔQ, the home community of the first node information is determined, that is, the first node information is assigned to the home community that maximizes the module gain, and the community summary report corresponding to the home community is extracted to guide the LLM analysis of global information. The community summary report is marked as {community_summary}.

[0101] It should be noted that the steps for determining the average weight are: for each community in the sentiment lexicon, calculate the sum of the weights of all edges within the community, then count the number of nodes within the community, and finally divide the sum of the weights of the edges within the community by the square of the number of nodes to obtain the average weight of the edges within the community. For example, if a community has n nodes and the sum of the weights of the edges within the community is W, then the average weight of the edges within the community is W / (n*n).

[0102] It should be noted that modularity is an indicator to measure the quality of community division, which reflects the difference between the density of edges within the community and the density of edges in a random network.

[0103] In this way, by determining the belonging community of the first node information, the subsequent global analysis and output decision of the user's emotions are facilitated, and the subsequent in-depth understanding and effective response to the user's emotions are facilitated.

[0104] Finally, based on the second node information {relevance}, the third node information {risk}, the fourth node information {similar} and the community summary report {community_summary}, a sentiment parameter set is obtained.

[0105] In this way, by extracting the key parameter set related to the first node information from the preset first emotional dictionary, the key parameter set related to the first node information can be accurately extracted from the emotional dictionary, which facilitates the subsequent acquisition of candidate answer descriptions that take into account the user's true emotions.

[0106] It can be understood that when the emotional parameter set is obtained, the emotional parameter set is used together with the dialogue information as the input of the big model, and the target event is interpreted with the key parameter set based on the big model to obtain the candidate answer description; specifically, by first defining the role capabilities and tasks of the system-level Prompt of the big model to clarify the tasks of the big model and the specific response content; and constructing the input-level Prompt to obtain the emotional parameter set and the dialogue information; finally, the system-level Prompt and the input-level Prompt are combined to form the final Dialogue-Prompt, and the Dialogue-Prompt is input into the Dialogue-LLM to interpret the target event with the key parameter set to obtain the candidate answer description.

[0107] In some embodiments, the target event is explained with the key parameter set to obtain candidate answer descriptions, including: obtaining the user's emotional information based on a large model, and matching the corresponding answer style based on the emotional information, explaining the target event with the answer style, and generating an answer description.

[0108] It should be noted that the target answer style is the overall style of the large model answer description. By matching the corresponding answer style according to the user's emotional information, personalized candidate answer descriptions can be provided based on the user's current emotional information, thereby improving the user experience.

[0109] In some embodiments, the response content needs to be evaluated in real time. If the response content does not contain constructive suggestions, the response content needs to be regenerated.

[0110] Step S103: Perform risk assessment on the candidate answer descriptions based on the large model to obtain a risk assessment result, and determine a final answer description based on the risk assessment result.

[0111] It can be understood that when the candidate answer description is obtained, the candidate answer description will be risk assessed. Specifically, the role capabilities and tasks of the system-level Prompt of the large model are first defined to clarify the tasks and evaluation content of the large model, where the role capability is "assessing output risks"; and the input-level Prompt is constructed to obtain the emotional parameter set, the dialogue information and the candidate answer description; finally, the system-level Prompt and the input-level Prompt are combined to form the final Dialogue-Prompt, and the Dialogue-Prompt is input into Evaluate-LLM to perform risk assessment on the candidate answer description to obtain a risk assessment result, where the risk assessment result includes high risk, medium risk and low risk.

[0112] It should be noted that low risk means that the response content has no obvious negative impact on users, and the advice and support provided can help alleviate users' negative emotions; medium risk means that the response content may involve sensitive topics, but overall it provides positive support to users and should be used with caution; high risk means that the response content may aggravate users' negative emotions or involve high-risk behaviors and should be avoided and alternative suggestions should be provided.

[0113] After obtaining the risk assessment results, it is necessary to determine whether the risk assessment results meet the preset conditions; if the risk assessment results meet the preset conditions, the final answer description is determined; if the risk assessment results do not meet the preset conditions, the target event is re-interpreted using the key parameter set and the candidate answer description is updated.

[0114] In some embodiments, if the risk assessment result is low risk, the candidate answer description is directly adopted as the final answer description and returned to the user.

[0115] In some embodiments, if the risk assessment result is medium or high risk, different rewards will be input into Dialogue-Prompt based on the medium or high risk to re-instruct Dialogue-LLM to generate a new response. The "Assessment Reason Field" in the risk assessment level of the data returned by Evaluate-LLM will be marked as {analysis} as input, and the original output of Dialogue-LLM will be marked as {assistant}. The reward for medium risk is: "The dialogue response risk assessment is medium. The response content involves sensitive topics, but overall it is positive and supportive of the user. Please combine Risk analysis: Pay attention to sensitive words and re-modify the generated output. / n## Medium-risk output analysis: / n{information} / n## Dialogue-LLM medium-risk output: / n{assistant}"; The reward for high-risk adoption is: "Dialogue response risk assessment is high!!! The response content may exacerbate the user's negative emotions or involve high-risk behavior. It should be avoided and alternative suggestions should be provided. Please re-generate the output based on the analysis. / n## High-risk output analysis: / n{information} / n## Dialogue-LLM high-risk output: / n{assistant}".

[0116] In this way, by performing a risk assessment on the candidate answer descriptions, it is possible to comprehensively evaluate whether the output response of the large model meets the risk requirements, ensure the accuracy of the risk assessment, and then provide a dialogue output that meets the risk requirements.

[0117] After the final answer description is determined based on the risk assessment result, it also includes: first, extracting information from the answer description based on the large model to obtain the fifth node information and the second edge information; second, updating the sentiment dictionary based on the fifth node information and the second edge information; specifically, using a graph processing tool (such as NetworkX) to create new nodes and edges, extracting the fifth node information and the second edge information from the answer description, and using the Neo4j database to insert the fifth node information and the second edge information into the sentiment dictionary to obtain a third sentiment dictionary, and then respectively calculating the similarity values ​​(such as cosine similarity or Jaccard similarity coefficient) corresponding to each node and each edge in the third sentiment dictionary, merging the nodes and edges with high similarity values ​​through an attribute matching algorithm, and updating the merged node attributes and edge attributes to obtain an updated sentiment dictionary, and re-dividing the updated sentiment dictionary into communities to obtain community division results and corresponding community summary reports for subsequent further use.

[0118] In this way, by adaptively updating the sentiment dictionary based on the answer description of the large model, it can ensure that the sentiment dictionary adapts to the rapid changes of the complex dialogue system, thereby improving the adaptability of the large language model to new contexts and new words, facilitating the subsequent in-depth understanding and effective response to user emotions.

[0119] The embodiments of the present application can accurately obtain first-node information such as target events, psychological crisis information, and emotional information by extracting information from the acquired conversation information, thereby facilitating the integration of the first-node information to accurately capture the user's true emotions. By extracting a set of key parameters related to the first-node information from a preset first emotional dictionary, the key parameter set related to the first-node information can be accurately extracted from the emotional dictionary, facilitating the subsequent acquisition of candidate answer descriptions that take into account the user's true emotions. By performing a risk assessment on the candidate answer descriptions, it is possible to comprehensively assess whether the output response of the large model meets the risk requirements, ensure the accuracy of the risk assessment, and thus provide a conversation output that meets the risk requirements. Compared with the prior art, the present application can accurately capture the user's true emotions and thus provide a conversation output that meets the risk requirements.

[0120] Example 2

[0121] For easier understanding, please refer to Figure 2 , Figure 2 This is a flow chart of another embodiment of the large model-based conversation risk assessment method provided by this application, including:

[0122] Step S201, obtaining dialogue information input by the user;

[0123] Step S202: The large model extracts information from the user's conversation information to obtain first node information;

[0124] Step S203: determining corresponding side information based on the first node information, and updating the sentiment dictionary based on the first node information and the side information;

[0125] Step S204: the large model performs matching in a preset sentiment dictionary based on the first node information to obtain a key parameter set;

[0126] Step S205 , the large model interprets the target event input by the user based on the key parameter set and generates candidate answer descriptions;

[0127] In step S206, the large model performs a risk assessment on the candidate answer descriptions and obtains a risk assessment result. If the risk assessment result does not meet the preset conditions, it is necessary to re-interpret the target event input by the user based on the current candidate answer description and the key parameter set until the generated candidate answer description meets the preset conditions, and the answer description is returned to the user; if the risk assessment result meets the preset conditions, the final answer description is determined and returned to the user.

[0128] Step S207: updating the sentiment dictionary based on the answer description.

[0129] It should be noted that the above steps S201 to S207 have been fully described in the first embodiment, and thus will not be repeated here.

[0130] Example 3

[0131] Please refer to Figure 3 , Figure 3 1 is a schematic structural diagram of an embodiment of a large model-based dialogue risk assessment system provided by the present application, comprising an acquisition module 100, an extraction module 200, and an assessment module 300;

[0132] The acquisition module 100 is used to acquire user conversation information, extract information from the conversation information based on the large model, and obtain first node information, wherein the first node information includes a target event;

[0133] The extraction module 200 is configured to extract a key parameter set related to the first node information from a preset sentiment dictionary, interpret the target event with the key parameter set based on the large model, and obtain candidate answer descriptions;

[0134] The evaluation module 300 is configured to perform risk evaluation on the candidate answer descriptions based on the large model, obtain a risk evaluation result, and determine a final answer description based on the risk evaluation result.

[0135] The information interaction, execution process, and other contents between the modules in the above-mentioned large-model-based dialogue risk assessment system are based on the same concept as the embodiment of the large-model-based dialogue risk assessment method of the first aspect of the present invention, and the technical effects achieved are basically the same. For specific contents, please refer to the description in the first embodiment of the method of the present invention, and will not be repeated here.

[0136] The apparatus embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separate, i.e., they may be located in one location or distributed across multiple network elements. Some or all of these elements may be selected based on actual needs to achieve the objectives of the methods of this embodiment.

[0137] See also Figure 4 , Figure 4 The hardware structure of a terminal device according to another embodiment is shown. The terminal device includes:

[0138] The processor 401 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0139] Memory 402 can be implemented in the form of read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). Memory 402 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in memory 402 and is called by processor 401 to execute the large model-based dialogue risk assessment method of the embodiments of this application.

[0140] Input / output interface 403, used to implement information input and output;

[0141] Communication interface 404, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0142] Bus 405 , which transmits information between various components of the device (e.g., processor 401 , memory 402 , input / output interface 403 , and communication interface 404 );

[0143] The processor 401 , the memory 402 , the input / output interface 403 and the communication interface 404 are connected to each other in communication within the device via a bus 405 .

[0144] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for evaluating dialogue risk based on a large model as described in the first embodiment above is implemented.

[0145] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-monitorable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0146] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application.

[0147] It is particularly pointed out that for those skilled in the art, any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A dialogue risk assessment method based on a large model, characterized in that: include: Acquire user conversation information, and extract information from the conversation information based on the large model to obtain first node information, wherein the first node information includes a target event; Extracting a key parameter set related to the first node information from a preset emotional dictionary, wherein the first node information also includes first psychological crisis information and first emotion information, specifically: extracting second node information related to the first node information from the preset emotional dictionary; calculating, based on the second node information, a path length with the second psychological crisis information in the emotional dictionary, and extracting third node information with the shortest path length; calculating, based on the second node information, an emotional similarity with the second emotional information in the emotional dictionary, and extracting fourth node information with the highest emotional similarity; determining a belonging community of the first node information in different emotional dictionary communities , and extracting a community summary report corresponding to the belonging community, wherein the sentiment dictionary community is a community division result of the sentiment dictionary, including: assigning the first node information to all communities in the sentiment dictionary respectively, calculating a plurality of modularity values ​​corresponding to the first node information in different sentiment dictionary communities; based on the plurality of modularity values, determining a modularity change value of the first node information assigned to different sentiment dictionary communities; based on the second node information, the third node information, the fourth node information and the community summary report, obtaining a key parameter set, interpreting the target event based on the large model with the key parameter set, and obtaining a candidate answer description; A risk assessment is performed on the candidate answer descriptions based on the large model to obtain a risk assessment result, and a final answer description is determined based on the risk assessment result.

2. The large model-based dialogue risk assessment method according to claim 1, characterized in that: Before obtaining the user's conversation information, the method further includes: Acquiring first psychological data and cleaning the first psychological data to obtain second psychological data, wherein the first psychological data includes professional psychological information and emotional conversation records; Node data and edge data are extracted from the second psychological data, and a first sentiment dictionary is constructed based on the node data and the edge data.

3. The large model-based dialogue risk assessment method according to claim 2, characterized in that: Before extracting the key parameter set related to the first node information from the preset sentiment dictionary, the method further includes: extracting first edge information corresponding to the first node information from the conversation information; Inserting the first node information and the first edge information into the first sentiment dictionary to obtain a second sentiment dictionary; The similarity values ​​corresponding to each node and each edge in the second sentiment dictionary are respectively calculated, the nodes and edges with high similarity values ​​are respectively merged, and the merged node attributes and edge attributes are respectively updated to obtain a sentiment dictionary.

4. The large model-based dialogue risk assessment method according to claim 1, characterized in that: The determining of the belonging community of the first node information in different sentiment dictionary communities is specifically: Calculating a number of modularity values ​​corresponding to the first node information in different sentiment dictionary communities; Based on the plurality of modularity values, determining modularity change values ​​of the first node information assigned to different sentiment dictionary communities; Based on the modularity change value, a community to which the first node information belongs is determined.

5. The method for evaluating dialogue risk based on a large model according to claim 1, characterized in that: The final answer description is determined based on the risk assessment result, specifically: Determining whether the risk assessment results meet preset conditions; If the risk assessment results meet the preset conditions, the final answer description is determined; If the risk assessment result does not meet the preset conditions, the target event is re-interpreted using the key parameter set and the candidate answer description is updated.

6. The method for evaluating dialogue risk based on a large model according to claim 1, characterized in that: After determining the final answer description based on the risk assessment result, the method further includes: Extract information from the answer description based on the large model to obtain fifth node information and second edge information; The sentiment dictionary is updated based on the fifth node information and the second edge information.

7. A dialogue risk assessment system based on a large model, characterized in that: include: Acquisition module, extraction module and evaluation module; The acquisition module is configured to acquire user conversation information, extract information from the conversation information based on the large model, and obtain first node information, wherein the first node information includes a target event; The extraction module is configured to extract a key parameter set related to the first node information from a preset emotional dictionary, wherein the first node information also includes first psychological crisis information and first emotion information, specifically: extracting second node information related to the first node information from the preset emotional dictionary; based on the second node information, calculating the path length with the second psychological crisis information in the emotional dictionary, and extracting third node information with the shortest path length; based on the second node information, calculating the emotional similarity with the second emotional information in the emotional dictionary, and extracting fourth node information with the highest emotional similarity; determining the first node information in different emotional dictionary communities , and extracting a community summary report corresponding to the belonging community, wherein the sentiment dictionary community is a community division result of the sentiment dictionary, including: assigning the first node information to all communities in the sentiment dictionary respectively, calculating a plurality of modularity values ​​corresponding to the first node information in different sentiment dictionary communities; determining a modularity change value of the first node information assigned to different sentiment dictionary communities based on the plurality of modularity values; obtaining a key parameter set based on the second node information, the third node information, the fourth node information, and the community summary report, and interpreting the target event with the key parameter set based on the large model to obtain a candidate answer description; The evaluation module is used to perform risk evaluation on the candidate answer descriptions based on the large model, obtain a risk evaluation result, and determine a final answer description based on the risk evaluation result.

8. An electronic device, characterized in that: The system comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the large model-based dialogue risk assessment method according to any one of claims 1 to 6 are implemented.

9. A readable storage medium, characterized in that The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of large-model-based dialogue risk assessment are implemented as described in any one of claims 1 to 6.

Images