Method for generating emotion semantics in cooperation with large model by fusing context and business knowledge

Abstract

This invention discloses a large-scale model-based collaborative generation method for emotion semantics that integrates context and business knowledge, relating to the field of artificial intelligence technology. The method includes: extracting emotion category entries and business terminology entries from a context graph; constructing an emotion layer and a business layer; establishing cross-layer connections and binding unresolved points to generate a two-layer knowledge graph; based on the two-layer knowledge graph, calling a visual model to extract facial cues from image frame data, backfilling emotion layer nodes, performing cross-layer edge weight sorting, filtering business layer nodes, and generating a cross-layer evidence chain; compiling the cross-layer evidence chain into dialogue prompts, concatenating them with a context window, inputting them into a large-scale language model, performing terminology anchor point constraint arrangement, and simultaneously verifying business terminology entries, integrating them into an emotion semantic collaborative output package. This invention enables personalized generation of business responses, enhancing user experience and business adaptability.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for collaborative generation of large-scale model emotion semantics that integrates context and business knowledge. Background Technology

[0002] In recent years, artificial intelligence technology has made breakthroughs in dialogue systems and sentiment analysis. Large-scale models, through multimodal data fusion, have driven the deep development of semantic understanding. Knowledge graphs, as a core technology for structured knowledge representation, are widely used in semantic reasoning and context-aware tasks. In the field of sentiment analysis, multimodal fusion technology is gradually maturing, capable of efficiently integrating text, speech, and image data to achieve multi-dimensional collaboration in sentiment perception. For example, by accurately extracting facial expression features through visual models and combining them with speech prosody analysis for emotion classification, not only is the accuracy of emotion recognition improved, but richer semantic context support is also provided for business scenarios, promoting the practical application of sentiment analysis from a single modality to multimodal collaborative evolution.

[0003] However, existing large-scale models still have room for optimization in key aspects of emotion semantic generation. First, the dynamic collaboration mechanism between emotion analysis and business knowledge is not yet perfect. Large-scale models struggle to associate user emotional cues (such as anxiety and satisfaction) with business scenario identifiers (such as customer service type and product category) in real time, leading to a disconnect between generated content and business specifications. Second, the construction of contextual information lacks an efficient aggregation mapping and compression framework. The processing of multi-source data (text segments, silent audio clips, and image frames) often takes a serial and independent approach, failing to form a unified knowledge graph structure. This results in inefficient cross-layer connections between emotional cues and business definitions, making it difficult to generate logically rigorous evidence chains and limiting the accuracy and practicality of emotion semantic generation. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a large-scale model emotion semantic collaborative generation method that integrates context and business knowledge, solving the problem of consistency between emotion semantics and business objectives.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a large-scale model-based collaborative generation method for emotion semantics that integrates context and business knowledge. The method includes: collecting user text data, speech fragment data, and image frame data, performing sentence segmentation, silence cropping, and face cropping, while simultaneously writing business scenario identifiers and business object identifiers to generate a collection package; based on the collection package, constructing a contextual window and solidifying it as dialogue event nodes, while simultaneously writing deduplicated relational connection records, performing aggregation mapping compression, and generating a context graph; extracting emotion category entries and business phrasing entries from the context graph, constructing an emotion layer and a business layer, performing cross-layer connections and binding unresolved points to generate a two-layer knowledge graph; based on the two-layer knowledge graph, calling a visual model to perform facial expression cue extraction on the image frame data, backfilling emotion layer nodes, performing cross-layer edge weight sorting, filtering business layer nodes, and generating a cross-layer evidence chain; compiling the cross-layer evidence chain into dialogue prompts, splicing them together with the contextual window, inputting them into a large-scale language model, performing phrasing anchor point constraint arrangement, simultaneously verifying business phrasing entries, and integrating them into an emotion semantic collaborative output package.

[0007] As a preferred embodiment of the large-model sentiment semantic collaborative generation method that integrates context and business knowledge as described in this invention, the specific steps for generating the collection package are as follows: Collect user text data, voice segment data, and image frame data, perform consistency checks, and write session number and round index to generate a multimodal record set; Segmenting of user text data in the multimodal record set is performed, and sequential sorting is performed in combination with round index to generate a text segment record set; Based on the text segment record set, silence clipping, segment merging and order preservation are performed on the speech segment data to generate a speech clipping record set; Based on the voice cropping record set, face cropping, size unification, and occlusion removal are performed on the image frame data to obtain the cropping and fusion record set, and business scenario identifiers and business object identifiers are added and integrated into the acquisition package.

[0008] As a preferred embodiment of the large-model emotion semantic collaborative generation method integrating context and business knowledge described in this invention, the specific steps for constructing a contextual window based on the collected data and fixing it as a dialogue event node are as follows: Extract text segment records, speech clipping records, and clipping fusion records from the acquisition package, perform round-window backtracking clipping, and construct a contextual window; The context window is solidified into a dialogue event node, and the language model is input to perform vector extraction and amplitude normalization to obtain the context vector. At the same time, it is bound to the business scenario identifier and the business object identifier to generate a set of dialogue event nodes.

[0009] As a preferred embodiment of the large-scale model emotion semantic collaborative generation method that integrates context and business knowledge as described in this invention, the specific steps for generating the context graph are as follows: The dialogue event node set is checked and verified item by item across rounds, and clues are located, labeled and registered to indicate the relationship type and direction. Unresolved points are also registered, and a relationship connection record set is generated. Based on the relationship connection record set, duplicate nodes are grouped and merged according to the business object identifier, and the aggregation mapping index is fixed. At the same time, deduplicated relationship connection records are written to generate a context graph.

[0010] As a preferred embodiment of the large-scale model emotion semantic collaborative generation method that integrates context and business knowledge as described in this invention, the specific steps for extracting emotion category items and business terminology items from the context graph to construct the emotion layer and business layer are as follows. Based on the context graph, the position of the emotion word fragment is located from the text segment record set, the emotion word fragment competition merging is performed, and the anchor point and emotion category entry are retained. At the same time, it is bound to the round index to generate an emotion entry table. Extract business-related phrase fragments from the context map, perform business-related phrase conflict folding and transcribe them into a business-related item table, combine them with the emotion item table to solidify the hierarchical node list, and generate the emotion layer and business layer.

[0011] As a preferred embodiment of the large-scale model emotion semantic collaborative generation method that integrates context and business knowledge as described in this invention, the specific steps for performing cross-layer connections and binding unresolved points to generate a two-layer knowledge graph are as follows. Perform cross-layer anchor point alignment on the emotion layer and business layer, establish cross-layer connections, and link them with relationship connection records to generate a cross-layer connection table; Perform two-level subgraph backbone pruning on the cross-level connection table, bind unresolved points, intercept caliber drift, and generate a two-level knowledge graph.

[0012] As a preferred embodiment of the large-scale model-based collaborative generation method for emotion semantics that integrates context and business knowledge as described in this invention, the specific steps of extracting facial cues from image frame data based on a two-layer knowledge graph and calling a visual model are as follows. Based on a two-layer knowledge graph, the round index of unresolved points is located and bound, and frame extraction is performed focusing on unresolved point rounds to generate an emoji extraction input set; Input the facial expression extraction input set into the visual model, perform facial expression cue extraction, facial expression cue temporal consensus filtering, and facial expression cue conflict folding to generate a set of facial expression cues.

[0013] As a preferred embodiment of the large-scale model sentiment semantic collaborative generation method that integrates context and business knowledge as described in this invention, the specific steps for generating the cross-layer evidence chain are as follows: The set of facial cues is backfilled into the emotion layer nodes according to the round index, and a consistency check is performed to generate the backfilled emotion layer node set; Perform cross-layer edge weight sorting on the cross-layer edges associated with the backfilled emotion layer node set, and perform weight lifting and cross-layer edge reverse constraint verification on the cross-layer edges that are bound to unresolved points, generating a cross-layer edge sorting list; Based on the cross-layer edge sorting list, business layer nodes are filtered, and business-specific deduplication and cross-layer path evidence assembly are performed to generate a cross-layer evidence chain.

[0014] As a preferred embodiment of the large-scale emotion semantic collaborative generation method integrating context and business knowledge described in this invention, the specific steps of compiling the cross-layer evidence chain into dialogue prompts, concatenating them with the context window, and inputting them into the large-scale language model are as follows. Based on the cross-layer evidence chain, the evidence chain is fragmented and compiled for emotion layer nodes and business scope items, and the round anchor points are extracted and sequentially spliced ​​in combination with the context window to generate a dialogue prompt package. Input the dialogue prompt package into the language model, perform multi-candidate branch generation and orchestration, and output candidate response texts.

[0015] As a preferred embodiment of the large-model emotion semantic collaborative generation method integrating context and business knowledge described in this invention, the specific steps of performing anchor point constraint arrangement, simultaneously verifying business definition entries, and integrating them into an emotion semantic collaborative output package are as follows. Perform a gap scan on the candidate response text to locate the business scope entries, and register the gaps and drift entries to generate a list of gaps and drift entries; Based on the list of gaps and drift items, perform caliber anchor post-arrangement and constraint conflict folding to generate an arrangement response text; The compiled response text is checked bidirectionally and then input into a large language model for anchor point locking and rewriting, and integrated into an emotion semantic collaborative output package.

[0016] The beneficial effects of this invention are as follows: by constructing a two-layer knowledge graph, cross-layer dynamic binding between the emotion layer and the business layer is realized, which plays a role in the precise coordination of business response and real-time emotion, thereby improving service conversion and user satisfaction; by using a large language model to perform caliber anchor point constraint orchestration, personalized generation of business responses is realized, enhancing user experience and business adaptability. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of a large-scale model sentiment semantic collaborative generation method that integrates context and business knowledge.

[0019] Figure 2 The flowchart for generating the acquisition package.

[0020] Figure 3 This is a flowchart for generating a context diagram.

[0021] Figure 4 A flowchart for generating a two-layer knowledge graph. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 This is one embodiment of the present invention, which provides a collaborative generation method for large-scale model emotion semantics that integrates context and business knowledge, including the following steps: S1: Collect user text data, voice segment data and image frame data, and perform sentence segmentation, silence cropping and face cropping. At the same time, write business scenario identifier and business object identifier to generate collection package. S1.1: Collect user text data, voice segment data and image frame data, perform consistency verification, and write session number and round index to generate a multimodal record set; Specifically, when collecting user text data, voice segment data, and image frame data, the collection entries are registered according to the session number and written into the round index. Within a unified time window, user text data, voice segment data, and image frame data are received and written into the session number and round index respectively. A consistency check is performed on the user text data, voice segment data, and image frame data to confirm that the session number and round index are consistent and that the time stamp meets the time window threshold. User text data, voice segment data, and image frame data that fail the consistency check are marked with a gap and a replacement trajectory is registered. User text data, voice segment data, and image frame data that pass the consistency check are merged according to the round index and the session number and round index are fixed to generate a multimodal record set.

[0026] It should be noted that the time window threshold is defined based on the time stamp alignment deviation of the user text data, voice segment data and image frame data corresponding to the session number and round index. It is used to determine the consistency of the acquisition time of user text data, voice segment data and image frame data within the same round index. Example range: 20ms before and after to 100ms before and after.

[0027] S1.2: Perform sentence segmentation on the user text data in the multimodal record set, and sort it sequentially using the round index to generate a text sentence segment record set; Furthermore, when extracting user text data from the multimodal record set, the record containing the user text data is located by the session number and round index, and the user text data is grouped by round index. Sentence segmentation is performed on the user text data under each round index. Sentence segmentation locates the segmentation point by newline character and obtains the text fragments within the segment. Sentence segmentation locates the sentence segmentation point by period, question mark, exclamation mark and semicolon and obtains the text fragments within the sentence. Each text fragment within the sentence is written into the text segment record set and written with the session number, round index and sentence segment number. When a text fragment within the sentence is too short or too long (meaning that the length of the text fragment within the sentence is insufficient to express the complete semantics or the length of the text fragment within the sentence contains multiple independent semantic fragments, resulting in unclear reading breakpoints), adjacent text fragments within the sentence are merged or further segmented at punctuation and the sentence segment number is updated. The text fragments are sorted sequentially by round index and sorted in ascending order by sentence segment number within the same round index to generate the text segment record set.

[0028] S1.3: Based on the text segment record set, perform silence trimming, segment merging and order preservation on the speech segment data to generate a speech trimming record set; Specifically, based on the conversation number and round index in the text segment record set, the speech segment data is located while maintaining the time stamp order. The discrete sampling sequence of the speech segment data is divided into multiple frames according to the frame number. Each frame covers a fixed number of sampling points, and the frame start point is advanced with the sampling point interval between the start positions of two adjacent frames. Within each frame, the amplitude of the sampling points is traversed according to the intra-frame sampling point index to form a squared energy summary value. The squared energy summary value is averaged according to the number of sampling points contained in each frame, and the square root is taken to obtain the frame amplitude. The silence interval is located on the time stamp axis based on the frame amplitude. The silence interval is cut from the speech segment data, and the start and end time stamps of the cut speech segment data are retained. Adjacent speech segment data within the same round index are spliced ​​according to the time stamp order, and when the segment interval reaches the independent segment retention condition, they are retained as independent speech segment data. The speech segment data that has completed silence cutting and segment merging is grouped according to the round index and sorted in ascending order by time stamp to generate a speech cutting record set.

[0029] The formula for calculating the frame amplitude is:

[0030] in, Indicates the frame amplitude. This indicates the number of sampling points contained in each frame. Indicates the index of the intra-frame sampling point. This represents a discrete sampling sequence of speech segment data. Indicates the frame number. This indicates the sampling point interval between the start positions of two adjacent frames.

[0031] It should be noted that the independent segment retention condition refers to the condition in which there is a perceptible pause or semantic boundary between adjacent speech segments within the same round index (meaning that adjacent speech segments are separated in terms of time stamp continuity, duration of silence intervals, or sentence segmentation positions), making it unsuitable to splice adjacent speech segments into the same speech segment.

[0032] Discrete sampling sequence refers to a sequence formed by sampling continuous speech waveforms at equal intervals at a fixed sampling rate during the acquisition of speech segment data, and recording the amplitude of each sampling point in time stamp order.

[0033] S1.4: Based on the voice cropping record set, perform face cropping, size unification, and occlusion removal on the image frame data to obtain the cropping and fusion record set, and add business scenario identifiers and business object identifiers to integrate it into a collection package.

[0034] Specifically, image frame data is located according to the session number and round index in the speech cropping record set. Image frame data is then filtered based on the start and end time markers in the speech cropping record set. The selected image frame data is used to locate the boundary of the face region, and face cropping is performed according to the boundary of the face region to obtain face cropped image frame data. The face cropped image frame data is then scaled and edge-filled to achieve a uniform size (e.g., 224×224 pixels) according to a uniform target size. The occlusion ratio of the face cropped image frame data is determined (by marking occluded pixels within the boundary of the face region and counting the proportion of occluded pixels to the total number of pixels within the boundary of the face region). When the occlusion ratio exceeds the occlusion ratio threshold, the face cropped image frame data is removed. The face cropped image frame data removed due to occlusion is written into the cropping fusion record set, and a business scenario identifier and a business object identifier are attached. The cropping fusion record set, the speech cropping record set, and the text segment record set are then packaged into a collection package.

[0035] It should be noted that the occlusion percentage threshold is defined based on the percentage of occluded pixels within the boundary of the face region in the face cropped image frame data. It is used to determine the triggering criteria for occlusion removal in the face cropped image frame data. Example range: 0.50 to 0.80.

[0036] S2: Based on the acquisition package, construct a context window and solidify it as a dialogue event node. At the same time, write deduplicated relationship connection records, perform aggregation mapping compression, and generate a context graph. S2.1: Extract text segment record set, speech trimming record set and trimming fusion record set from the acquisition package, perform round window backtracking trimming, and construct context window; Specifically, when extracting text segment record sets, speech trimming record sets, and trimming-fusion record sets from the acquisition package, a round index directory is established according to the round index, which binds the round index to the corresponding entry position. The round window backtracking trimming determines the continuous round index interval based on the round index directory. When the length of the continuous round index interval meets the backtracking round number threshold, the text segment record set entries, speech trimming record set entries, and trimming-fusion record set entries within the continuous round index interval are retained. When the length of the continuous round index interval does not meet the backtracking round number threshold, the boundaries of the continuous round index interval are adjusted until the backtracking round number threshold is met. The retained entries are sorted in ascending order by round index, and within the same round index, the text segment record set is sorted in ascending order by segment number, the speech trimming record set is sorted in ascending order by time stamp, and the trimming-fusion record set is sorted in ascending order by time stamp, generating a context window.

[0037] It should be noted that the threshold for the number of backtracking rounds is defined based on the number of consecutive rounds that the context window needs to cover. It is used to determine whether the backtracking range of the round window needs to be expanded to meet the triggering criteria that satisfy the integrity of the dialogue context. Example range: 3 rounds to 8 rounds.

[0038] S2.2: Solidify the context window into a dialogue event node, and input it into the language big model to perform vector extraction and amplitude normalization to obtain the context vector. At the same time, bind it with the business scenario identifier and the business object identifier to generate a set of dialogue event nodes. Furthermore, when solidifying the context window into dialogue event nodes, the text segment record set entries, speech trimming record set entries, and trimming-fusion record set entries within the context window are read according to the session number and round index. A dialogue event node entry is created for each round index, and the dialogue event node entry is written with the session number, round index, and the position index of the three types of entries within the context window. The dialogue event node entries and their corresponding text segment record set entries, speech trimming record set entries, and trimming-fusion record set entries are then concatenated according to the round index order to form the input content for the language large model. The input content is input into the language model and vector extraction is performed to obtain the context vector (referring to the normalized vector representation used to represent the semantic information of the context window, obtained by the language model performing vector extraction on the input content of the language model corresponding to the same round index and normalizing the amplitude). The amplitude normalization is performed on the context vector and the normalized context vector is retained. The normalized context vector is written into the dialogue event node entry, and the business scenario identifier and business object identifier are written into the binding field of the dialogue event node entry. The dialogue event node entries are grouped and encapsulated according to the round index to generate a dialogue event node set.

[0039] It should be noted that the training of a large language model typically consists of two stages: pre-training and alignment training. The large language model is selected from a general pre-trained large language model based on the Transformer architecture. In the pre-training stage, a large-scale text corpus (such as question-and-answer text) is used to perform cleaning, deduplication, and quality screening, as well as word segmentation and serialization. The word sequence is input into the neural network, and the next word prediction or mask prediction target (referring to the label sequence that the large language model wants to fit based on the input word sequence in the pre-training stage, for example, the next word prediction target is the next word number corresponding to the current position, and the mask prediction target is the original word number corresponding to the masked position) is used to obtain gradients using backpropagation and update parameters (such as word vector parameters, feedforward network parameters, and normalization parameters) using an optimizer. Distributed parallelism, learning rate scheduling, and mixed precision control are used to achieve stable convergence. In the alignment training stage, instruction data and human preference data are used for supervised fine-tuning and combined with preference optimization. Safety and compliance filtering are added, and after validation set evaluation and regression testing, a usable large language model is obtained.

[0040] S2.3: Perform cross-round item-by-item comparison and verification of the dialogue event node set, trigger clue location labeling and registration of relationship type and relationship direction, register unresolved points, and generate a relationship connection record set; Specifically, when performing cross-round item-by-item comparison and verification of the dialogue event node set, the dialogue event node set is grouped by session number and arranged in ascending order by round index. Item-by-item comparison and verification is performed on dialogue event nodes of adjacent round indices. This item-by-item comparison and verification includes business scenario identifier consistency verification, business object identifier consistency verification, and round index continuity verification. When the item-by-item comparison and verification triggers clue location annotation, the associated sentence segment sequence number range is located in the text segment record set (referring to the consecutive sentence segment sequence number interval corresponding to the session number and round index locked in the text segment record set by the clue location annotation), and the associated time marker range is located in the voice clipping record set (referring to the start and end time marker interval of the voice segment data corresponding to the session number and round index locked in the voice clipping record set by the clue location annotation). The clipping and fusion record set locates the associated time stamp range (referring to the start and end time stamp interval of the image frame data corresponding to the session number and round index, which is locked in the clipping and fusion record set by the clue location annotation), and solidifies the location result into a clue location annotation; based on the clue location annotation, the relationship type and relationship direction are registered (e.g., the relationship type is "acceptance"), and the session number, round index, business scenario identifier, business object identifier, relationship type, relationship direction, and relationship direction consistency marker are encapsulated into relationship connection record entries. The relationship direction consistency marker is used to register the consistency pass status of the relationship direction in subsequent reverse constraint verification; the business object identifiers in the dialogue event node set that have not triggered the clue location annotation are encapsulated into unresolved point entries and merged with the relationship connection record entries to generate a relationship connection record set.

[0041] S2.4: Based on the relationship connection record set, group and merge duplicate nodes according to the business object identifier, solidify the aggregation mapping index, and write deduplicated relationship connection records to generate a context graph.

[0042] Specifically, the process involves reading relationship connection record entries and unresolved point entries, extracting business object identifiers, round indexes, business scenario identifiers, relationship types, and relationship directions from the relationship connection record entries, and retrieving dialogue event nodes from the dialogue event node set according to the relationship direction to form a candidate list of dialogue event nodes. This candidate list is then grouped by business object identifier to form a business object identifier group list. Within this group list, dialogue event nodes with the same session number, round index, business scenario identifier, and business object identifier are merged, with only one dialogue event node retained. The correspondence between the merged dialogue event node and the retained dialogue event node is solidified into an aggregate mapping index. The dialogue event nodes at both ends of the relationship direction in the relationship connection record entries are updated based on the aggregate mapping index. Duplicate relationship connection record entries are referenced and deleted to obtain deduplicated relationship connection records. Finally, the retained dialogue event nodes, deduplicated relationship connection records, aggregate mapping index, and unresolved point entries are merged and encapsulated to generate a context graph.

[0043] S3: Extract emotion category items and business term items from the context graph, construct the emotion layer and business layer, make cross-layer connections and bind unresolved points to generate a two-layer knowledge graph; S3.1: Based on the context graph, locate the position of the emotion word fragment from the text segment record set, perform competitive merging of emotion word fragments, retain the anchor point and emotion category entry of emotion category, and bind it with the round index to generate an emotion entry table; Specifically, the process involves locating dialogue event nodes from the context graph using round-based indexes and retrieving corresponding text fragments from the text segment record set. It then iterates through the character sequences of these text fragments and locates the position of emotion word fragments within them. These emotion word fragment positions are written to the segment number, start and end character indices, and round-based index, and aggregated into a candidate list of emotion word fragments. The candidate list is grouped by the same round-based index and the same segment number. For grouped entries with overlapping or adjacent emotion word fragment positions, a competitive merging process is performed. This process retains the emotion word fragment positions with more complete coverage and records the merging destination. The merged emotion word fragment positions are then solidified as emotion category entry anchors. These anchors and merged emotion word fragment positions are encapsulated into emotion category entries. Finally, the emotion category entries are bound to the round-based indexes, written to the emotion entry table, and encapsulated, generating the emotion entry table.

[0044] S3.2: Extract business-related phrase fragments from the context diagram, perform business-related phrase conflict folding and transcribe them into a business-related item table, combine them with the emotion item table to solidify the hierarchical node list, and generate the emotion layer and business layer. Furthermore, the dialogue event nodes in the context graph are traversed by the session number and the text segment records within the sentence are retrieved by the round index. The continuous phrase intervals related to the business scenario identifier and the business object identifier are located in the text segment within the sentence and extracted as business-specific phrase segments. The business-specific phrase segments are bound with the round index, sentence segment number, and start and end character index to form a candidate list of business-specific phrase segments. The business-specific phrase conflict folding process groups the candidate list of business-specific phrase fragments according to the business object identifier and round index. For business-specific phrase fragments with mutually exclusive semantics within the same group, one fragment is retained and the conflict folding destination is recorded. The retained result is transcribed into a business-specific entry table and written with the business scenario identifier, business object identifier, and round index. The hierarchical node list is solidified by combining the sentiment entry table and the business-specific entry table. The hierarchical node list solidifies the nodes corresponding to the anchor points of sentiment category entries and the nodes corresponding to business-specific entries, respectively, and retains the round index binding information to generate the sentiment layer and the business layer.

[0045] S3.3: Perform cross-layer anchor point alignment on the emotion layer and business layer, establish cross-layer connections, and attach them to the relationship connection records to generate a cross-layer connection table; Specifically, cross-level alignment candidate groups are formed by grouping business object identifiers and round indexes. Within each cross-level alignment candidate group, cross-level connections are established between the anchor point of the sentiment category entry and the business scope entry with the same business object identifier and round index. The cross-level connection is written with the sentiment category entry anchor point as the starting point, the business scope entry as the ending point, the session number, the business scenario identifier, the business object identifier, and the round index. When attaching relationship connection records, the relationship connection record entries are located based on the relationship connection record entries in the relationship connection record set, according to the business object identifier and round index. The relationship type and relationship direction are written into the attachment field of the cross-level connection, and the results are summarized and encapsulated into a cross-level connection table according to the round index.

[0046] S3.4: Perform two-level subgraph backbone pruning on the cross-level connection table, bind unresolved points, intercept caliber drift, and generate a two-level knowledge graph.

[0047] Specifically, a two-layer subgraph is established based on the session number, round index, business object identifier, sentiment category entry anchor point, and business caliber entry in the cross-layer connection table. Cross-layer connections under the same business object identifier are concatenated in ascending order of round index, and duplicate paths and loop paths of cross-layer connections are deleted, retaining only one continuous path. When the number of cross-layer connections under the same business object identifier exceeds the retention scale corresponding to the main branch retention ratio threshold, the main branch path is retained first according to the continuity of the round index, and non-continuous branches are deleted (for example, the main branch retention ratio threshold is 0.50~0.80). Based on the unresolved point entries in the relationship connection record set, unresolved points are bound to the sentiment category entry anchor point and business caliber entry of the corresponding round index and written into the unresolved point association field. Caliber drift interception performs item-by-item comparison for the business caliber entries corresponding to the same business object identifier in different round indices, and writes caliber drift interception flags for cross-layer connections corresponding to mutually exclusive statements and deletes them from the main branch path. The sentiment layer, business layer, and cross-layer connection table are encapsulated to generate a two-layer knowledge graph.

[0048] It should be noted that the backbone retention percentage threshold is defined based on the percentage of cross-layer connections under the same business object identifier that need to be retained as backbone paths in a two-layer subgraph. It is used to limit the backbone path retention scale and trigger the deletion of non-contiguous branches. Example range: 0.50~0.80.

[0049] The two-layer knowledge graph consists of an emotion layer and a business layer. The emotion layer carries anchor points for emotion category entries and expression clue entries, while the business layer carries business definition entries and establishes connections through a cross-layer connection table. The significance is to align emotion clues and business definitions under the same business object identifier in a unified structure and support cross-layer edge weight sorting and cross-layer evidence chain generation.

[0050] S4: Based on a two-layer knowledge graph, the visual model is called to extract facial cues from image frame data, and the emotion layer nodes are backfilled to perform cross-layer edge weight sorting, filter business layer nodes, and generate a cross-layer evidence chain. S4.1: Based on a two-layer knowledge graph, locate and bind the round index of unresolved points, and perform round-focused frame extraction of unresolved points to generate an expression extraction input set; Specifically, the unresolved point association fields in the two-layer knowledge graph are traversed, and the round index corresponding to the unresolved point is extracted. The round index and business object identifier are grouped to form an unresolved point round index list. Based on the unresolved point round index list, image frame data with the same session number and the same round index are located in the cropping and fusion record set. The image frame data is retained in ascending order according to the time stamp. In the image frame data, the continuous image frame data is extracted according to the focused frame extraction rule and the extraction sequence number is written. The focused frame extraction rule retains representative frames covering the expression changes within the start and end time stamp range of the image frame data and deletes duplicate frames (for example, retaining 1 to 5 frames per second). The extracted image frame data, session number, round index, business scene identifier, business object identifier, time stamp, and extraction sequence number are encapsulated into an expression extraction input set.

[0051] It should be noted that the focused frame extraction rule refers to the frame extraction constraint that selects image frame data that can represent facial expression changes in order of time marker within the start and end time marker range of the image frame data corresponding to the round index of the unresolved point, and deletes duplicate frames.

[0052] S4.2: Input the facial expression extraction input set into the visual model, perform facial expression cue extraction, facial expression cue temporal consensus filtering and facial expression cue conflict folding, and generate a set of facial expression cues; Furthermore, the image frame data in the expression extraction input set are arranged in ascending order by time stamp and frame number order according to the session number and round index. The arranged image frame data, along with the business scenario identifier and business object identifier, are used as input to the visual model. When the visual model outputs the expression clue extraction results, an expression clue entry is generated for each image frame data and written with the session number, round index, time stamp, frame number, and expression clue category information, which are then summarized to form an expression clue candidate list. Expression clue temporal consensus screening compares the expression clue candidate list temporally according to the frame number within the same round index. Expression clue entries that appear repeatedly on multiple consecutive frame numbers are marked as consensus expression clues and retained. Expression clue entries that appear only on a single frame number and are inconsistent with adjacent frame numbers are marked as non-consensus expression clues and deleted. Expression clue conflict folding retains one expression clue entry for expression clue entries pointing to mutually exclusive expressions under the same frame number and registers the conflict folding destination. The consensus expression clues and the conflict-folded expression clue entries are encapsulated and summarized according to the round index to generate an expression clue set.

[0053] It should be noted that visual model training typically includes three parts: data preparation, training, and validation. The visual model is selected from a pre-trained visual model used for facial cue extraction. The visual model adopts a convolutional neural network architecture, collects face images and video frames, performs quality screening and annotation, completes cropping, size unification, and data augmentation, inputs the image frame data into the neural network, and uses facial expression categories or facial cues as training targets. The loss is obtained and the parameters (such as bias parameters) are updated through backpropagation. During the training process, batch iteration and learning rate scheduling are used to control convergence. The accuracy and stability are evaluated on the validation set and abnormal training rounds are rolled back to obtain a visual model that can be used for facial cue extraction.

[0054] S4.3: Backfill the set of facial cues into the emotion layer nodes according to the round index, and perform a consistency check to generate the backfilled emotion layer node set; Specifically, the expression cues are grouped and the corresponding emotion layer nodes for each round index are located. Expression cues under the same round index are written into the expression cues field of the emotion layer node, and the time stamp and the source information of the extracted frame number are retained (referring to the time stamp position of the extracted frame number recorded in the expression cues and the image frame data corresponding to the extracted frame number in the expression extraction input set, which is used to trace the image frame data from which the expression cues originated). Consistency checks are performed on the emotion layer nodes of the same round index to check the consistency between the anchor point of the emotion category entry and the expression cues entry. If the points are consistent, the expression cues entry is retained and a consistency pass mark is registered. If the points are inconsistent, the expression cues entry is registered as a conflict entry and the conflict entry index is retained. The emotion layer nodes that have completed backfilling and consistency checks are summarized and encapsulated by round index to generate a backfilled emotion layer node set.

[0055] S4.4: Perform cross-layer edge weight sorting on the cross-layer edges associated with the backfilled emotion layer node set, and perform weight lifting and cross-layer edge reverse constraint verification on the cross-layer edges that are bound to unresolved points, and generate a cross-layer edge sorting list; Specifically, cross-layer edges are located based on the cross-layer connection table, and the anchor points of sentiment category entries, business caliber entries, and round index binding information at both ends of the cross-layer edge are aggregated with the consistency verification marks in the backfilled sentiment layer node set into the cross-layer edge weight field. Cross-layer edges under the same business object identifier are sorted in descending order of cross-layer edge weight to form a sorting result. When performing weight lifting on cross-layer edges with unresolved points, the cross-layer edges with unresolved points are located based on the unresolved point entries, and a lifting mark is written to the cross-layer edge weight field, so that the cross-layer edges with unresolved points move forward in the sorting result. Cross-layer edge reverse constraint verification checks the relationship direction in the relationship connection record set against the cross-layer edge connection direction one by one in the sorting result. When the relationship direction is inconsistent with the cross-layer edge connection direction, a reverse constraint mark is written and the verification reason is recorded. When the relationship direction is consistent with the cross-layer edge connection direction, the cross-layer edge weight field is retained, and a cross-layer edge sorting list is generated.

[0056] It should be noted that the cross-layer edge weight field includes a base weight value, a lift flag, a lift magnitude value, and a reverse constraint flag. The base weight value is limited to 0 to 1 and is synthesized by a consistency pass flag, a caliber drift interception flag, and a relationship direction consistency flag in a fixed proportion. The consistency pass flag, the caliber drift interception flag, and the relationship direction consistency flag take a pass value of 1 and a fail value of 0, respectively. The cross-layer edge weights are sorted first by the lift flag in descending order, and then by the base weight value in descending order. Cross-layer edges with a reverse constraint flag of 1 are moved to the back in the sorting results. For cross-layer edges with unresolved points, a lift flag and a lift magnitude value are written. The lift magnitude value is 0.05 to 0.20 in the example, and is used for secondary sorting within the same lift flag group.

[0057] S4.5: Based on the cross-layer edge sorting list, filter business layer nodes, and perform business caliber deduplication and cross-layer path evidence assembly to generate a cross-layer evidence chain.

[0058] Specifically, the process involves traversing the cross-layer edges according to their sorting results and grouping them by business object identifier. For each business object identifier, the top-ranked cross-layer edges are retained, and the business layer nodes corresponding to the endpoints of these edges are extracted to form a candidate list of business layer nodes. When performing deduplication and trimming of business statements, the candidate list of business layer nodes is grouped by business object identifier. For business layer nodes with identical business statement entries, one entry is retained and the deduplication direction is recorded. For business layer nodes with mutually exclusive business statement entries, the entry associated with the top-ranked cross-layer edge is retained and the trimming direction is recorded. When performing cross-layer path evidence assembly, the retained business layer nodes are concatenated with the corresponding cross-layer edges and the backfilled sentiment layer node set entries corresponding to the starting points of the cross-layer edges. The concatenation results are encapsulated into a set of cross-layer path evidence entries according to the order of the cross-layer edge sorting list and summarized to generate a cross-layer evidence chain.

[0059] S5: Compile the cross-layer evidence chain into dialogue prompts, and combine them with the context window for splicing. Input the language model, perform caliber anchor point constraint arrangement, check the business caliber items, and integrate them into an emotion semantic collaborative output package.

[0060] S5.1: Based on the cross-layer evidence chain, perform evidence chain fragmentation compilation on emotion layer nodes and business caliber items, and combine the context window to extract round anchor points and sequentially splice them to generate dialogue prompt packages; Specifically, the emotion layer nodes and business clarification entries in the cross-layer evidence chain are grouped by round index, and the evidence chain is fragmented and compiled. The evidence chain fragmentation and compilation organizes the emotion category entry anchor information and expression clue entry consistency check mark, business clarification entry text, relationship type and relationship direction, and unresolved point association information corresponding to the emotion layer node into round fragment content in a fixed order and writes it into the round index. When performing round anchor extraction in conjunction with the context window, the text segment record set entries, voice trimming record set entries and trimming fusion record set entries of the same round index are located in the context window according to the round index and written into the round fragment content. When performing sequential splicing, the round fragment content is spliced ​​incrementally according to the round index, and the order of the text segment record set segment number, the order of the voice trimming record set time mark and the order of the trimming fusion record set time mark are maintained within the same round index to generate a dialogue prompt package.

[0061] S5.2: Input the dialogue prompt package into the language model, perform multi-candidate branch generation and arrangement, and output candidate response text; Furthermore, the dialogue prompts are organized into a language model input sequence according to the conversation number and round index, and formatted and encapsulated. After the language model input sequence is input into the language model, a multi-candidate branch generation and arrangement is triggered. The language model starts multiple generation processes on the same language model input sequence according to the branch number. In each generation process, the business scope entries and unresolved point association information in the dialogue prompts are not deleted. The response content generated by each branch is written with the branch number and round index source information and truncated to the response termination mark to complete the text solidification. The generation results of each branch are summarized and encapsulated into candidate response text according to the branch number order.

[0062] S5.3: Perform a caliber anchor point gap scan on the candidate response text, locate the position of the business caliber item, register the gap and drift item, and generate a list of gap and drift items; Specifically, the business terminology entries in the dialogue prompt package are used as the scanning benchmark to locate the positions of business terminology entries in the candidate response text one by one. The positions of business terminology entries are registered using a start and end character index and bound to the session number, round index, branch number, business scenario identifier, and business object identifier to form a location record. When no business terminology entry position is detected in the candidate response text, the corresponding business terminology entry text is registered as a gap entry and a gap mark is written. When a business terminology entry position is detected in the candidate response text, but there is a wording substitution, affirmation / negation change, or scope expansion / reduction between the business terminology entry text in the location record and the corresponding fragment of the candidate response text, the corresponding business terminology entry text is registered as a drift entry and a drift mark is written, while the business terminology entry position and the corresponding fragment of the candidate response text are retained. The gap entries and drift entries are summarized and grouped and packaged according to the branch number to generate a list of gap and drift entries.

[0063] It should be noted that the start and end character indexing method refers to using the start and end character position indices of the corresponding fragments of the business scope entry text in the candidate response text to mark the position of the business scope entry.

[0064] S5.4: Based on the list of gaps and drift items, perform caliber anchor point post-arrangement and constraint conflict folding to generate arrangement response text; Specifically, when generating orchestrated response text based on the gap and drift item list, candidate response texts are located by session number, round index, and branch number, and corresponding gap and drift items are extracted. For post-positioning of the business scope anchor, the business scope item texts corresponding to the gap items are grouped by business object identifier and inserted into the paragraph positions corresponding to the business object identifiers in the candidate response text. For drift item processing, the corresponding fragments of drift items in the candidate response text are replaced with the business scope item texts registered in the gap and drift item list, while retaining the start and end character index location records for write-back positioning. Constraint conflict folding is performed on the candidate response text; mutually exclusive and duplicate expressions appearing under the same business object identifier are compared item by item, retaining expressions consistent with the business scope item text, while deleting inconsistent expressions and registering the folding destination. The text is then encapsulated and solidified by branch number to generate orchestrated response text.

[0065] S5.5: Perform bidirectional readback verification on the arranged response text, input it into the language big model for anchor point locking and rewriting, and integrate it into an emotion semantic collaborative output package.

[0066] Specifically, the response text is traversed and arranged, and a forward readback check is performed. The forward readback check locates the position of the business scope item text according to the text order and checks the consistency of expression within the same business object identifier paragraph. At the same time, it locates the expression corresponding to the anchor point of the emotion category item and checks the consistency of tone corresponding to the consistency of the expression clue item. Fragments with no detected business scope item text, detected but inconsistent expression, and mutually exclusive expression under the same business object identifier are registered as verification anomaly markers, and the start and end character index location records are retained. The bidirectional readback check continues. The reverse readback verification involves scanning back from the end of the process and checking the round connection expressions corresponding to the relationship type and relationship direction item by item. The verification anomaly markers are reviewed and merged. The arranged reply text, verification anomaly markers, and business scope item texts are input into the language model. The language model performs anchor point locking and rewriting, and replaces the verification anomaly markers with consistent expressions of the business scope item texts. At the same time, it retains the anchor point corresponding expressions of the emotion category items and deletes mutually exclusive expressions. The arranged reply text after anchor point locking and rewriting is encapsulated and integrated into an emotion semantic collaborative output package.

[0067] In summary, this invention achieves cross-layer dynamic binding between the emotion layer and the business layer by constructing a two-layer knowledge graph, which enables precise coordination between business responses and real-time emotions, thereby improving service conversion and user satisfaction; and it enables personalized generation of business responses by using a large language model for caliber anchor point constraint orchestration, thereby enhancing user experience and business adaptability.

[0068] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for generating emotion semantics in a large model by fusing context and business knowledge, characterized in that: include, Collect user text data, voice segment data, and image frame data, and perform sentence segmentation, silence cropping, and face cropping. At the same time, write business scenario identifiers and business object identifiers to generate a collection package. Based on the acquisition package, a contextual window is constructed and solidified as a dialogue event node. At the same time, deduplication relationship connection records are written, aggregation mapping compression is performed, and a contextual graph is generated. Extract emotion category items and business term items from the context graph, construct emotion layer and business layer, make cross-layer connections and bind unresolved points to generate a two-layer knowledge graph; Based on a two-layer knowledge graph, a visual model is invoked to extract facial cues from image frame data, and emotion layer nodes are backfilled to perform cross-layer edge weight sorting, filtering business layer nodes, and generating a cross-layer evidence chain. The cross-layer evidence chain is compiled into dialogue prompts and spliced ​​together with the context window. It is then input into the language model, where it is arranged with anchor point constraints. At the same time, the business-related entries are checked and integrated into an emotion semantic collaborative output package.

2. The method of claim 1, wherein the method further comprises: The specific steps for generating the acquisition package are as follows: Collect user text data, voice segment data, and image frame data, perform consistency checks, and write session number and round index to generate a multimodal record set; Segmenting of user text data in the multimodal record set is performed, and sequential sorting is performed in combination with round index to generate a text segment record set; Based on the text segment record set, silence clipping, segment merging and order preservation are performed on the speech segment data to generate a speech clipping record set; Based on the voice cropping record set, face cropping, size unification, and occlusion removal are performed on the image frame data to obtain the cropping and fusion record set, and business scenario identifiers and business object identifiers are added and integrated into the acquisition package.

3. The large-scale model sentiment semantic collaborative generation method that integrates context and business knowledge as described in claim 2, characterized in that: The specific steps for constructing a contextual window based on the acquired data and embedding it as a dialogue event node are as follows. Extract text segment records, speech clipping records, and clipping fusion records from the acquisition package, perform round-window backtracking clipping, and construct a contextual window; The context window is solidified into a dialogue event node, and the language model is input to perform vector extraction and amplitude normalization to obtain the context vector. At the same time, it is bound to the business scenario identifier and the business object identifier to generate a set of dialogue event nodes.

4. The large-scale model sentiment semantic collaborative generation method that integrates context and business knowledge as described in claim 3, characterized in that: The specific steps for generating the context graph are as follows: The dialogue event node set is checked and verified item by item across rounds, and clues are located, labeled and registered to indicate the relationship type and direction. Unresolved points are also registered, and a relationship connection record set is generated. Based on the relationship connection record set, duplicate nodes are grouped and merged according to the business object identifier, and the aggregation mapping index is fixed. At the same time, deduplicated relationship connection records are written to generate a context graph.

5. The large-scale model sentiment semantic collaborative generation method that integrates context and business knowledge as described in claim 4, characterized in that: The specific steps for extracting emotion category items and business terminology items from the context graph to construct the emotion layer and business layer are as follows. Based on the context graph, the position of the emotion word fragment is located from the text segment record set, the emotion word fragment competition merging is performed, and the anchor point and emotion category entry are retained. At the same time, it is bound to the round index to generate an emotion entry table. Extract business-related phrase fragments from the context map, perform business-related phrase conflict folding and transcribe them into a business-related item table, and solidify the hierarchical node list by combining it with the emotion item table to generate the emotion layer and business layer.

6. The large-scale model sentiment semantic collaborative generation method that integrates context and business knowledge as described in claim 5, characterized in that: The specific steps for performing cross-layer connections and binding unresolved points to generate a two-layer knowledge graph are as follows. Perform cross-layer anchor point alignment on the emotion layer and business layer, establish cross-layer connections, and link them with relationship connection records to generate a cross-layer connection table; Perform two-level subgraph backbone pruning on the cross-level connection table, bind unresolved points, intercept caliber drift, and generate a two-level knowledge graph.

7. The large-scale model sentiment semantic collaborative generation method that integrates context and business knowledge as described in claim 6, characterized in that: The method involves using a two-layer knowledge graph and a visual model to extract facial cues from image frame data. The specific steps are as follows: Based on a two-layer knowledge graph, the round index of unresolved points is located and bound, and frame extraction is performed focusing on unresolved point rounds to generate an emoji extraction input set; Input the facial expression extraction input set into the visual model, perform facial expression cue extraction, facial expression cue temporal consensus filtering, and facial expression cue conflict folding to generate a set of facial expression cues.

8. The large-scale model sentiment semantic collaborative generation method that integrates context and business knowledge as described in claim 7, characterized in that: The specific steps for generating a cross-layer evidence chain are as follows: The set of facial cues is backfilled into the emotion layer nodes according to the round index, and a consistency check is performed to generate the backfilled emotion layer node set; Perform cross-layer edge weight sorting on the cross-layer edges associated with the backfilled emotion layer node set, and perform weight lifting and cross-layer edge reverse constraint verification on the cross-layer edges that are bound to unresolved points, generating a cross-layer edge sorting list; Based on the cross-layer edge sorting list, business layer nodes are filtered, and business-specific deduplication and cross-layer path evidence assembly are performed to generate a cross-layer evidence chain.

9. The large-scale model sentiment semantic collaborative generation method that integrates context and business knowledge as described in claim 8, characterized in that: The process of compiling the cross-layer evidence chain into dialogue prompts, concatenating them with the context window, and inputting them into the language model involves the following steps: Based on the cross-layer evidence chain, the evidence chain is fragmented and compiled for emotion layer nodes and business scope items, and the round anchor points are extracted and sequentially spliced ​​in combination with the context window to generate a dialogue prompt package. Input the dialogue prompt package into the language model, perform multi-candidate branch generation and orchestration, and output candidate response texts.

10. The large-scale model emotion semantic collaborative generation method integrating context and business knowledge as described in claim 9, characterized in that: The process of orchestrating anchor point constraints, verifying business-specific entries, and integrating them into a collaborative emotional semantic output package involves the following steps: Perform a gap scan on the candidate response text to locate the business scope entries, and register the gaps and drift entries to generate a list of gaps and drift entries; Based on the list of gaps and drift items, perform caliber anchor post-arrangement and constraint conflict folding to generate an arrangement response text; The compiled response text is checked bidirectionally and then input into a large language model for anchor point locking and rewriting, and integrated into an emotion semantic collaborative output package.

Images