A method and system for resolving large-scale multi-category customer order data

Abstract

This invention discloses a method and system for parsing large-scale, multi-type customer order data. The method includes the following steps: extracting the remark requirement text from the order data and performing data preprocessing; extracting hidden layer feature vectors and decomposing them into multiple business semantic features; outputting decoupling enhancement features; calculating the similarity between the intent semantic features and a preset intent prototype; outputting an intent activation distribution vector; calculating the matching confidence for each knowledge domain and generating a knowledge feature vector; separating the corresponding semantic subspace features from the decoupling enhancement features; calculating a gating coefficient with the intent activation distribution vector; performing element-wise multiplication of the gating coefficient with the knowledge feature vector to obtain a knowledge representation; concatenating the knowledge representation with the decoupling enhancement features to obtain a fused semantic feature; performing intent recognition and entity extraction based on the fused semantic feature; and outputting an intent probability distribution and a business entity sequence. This invention improves the automation and reliability of parsing specific customer needs.

Description

Technical Field

[0001] This invention relates to the field of text data parsing technology, specifically to a method and system for parsing large-scale, multi-type customer order data. Background Technology

[0002] In modern logistics scheduling, production scheduling, and resource allocation scenarios, customers often need to supplement their requirements through orders or remarks, such as specifying arrival times, designated collaborators, or specific resource usage requirements. These requirements are often expressed in natural language text, characterized by colloquialisms, diverse expressions, and mixed information.

[0003] In existing technologies, the processing of the aforementioned unstructured text mainly relies on manual reading and annotation. While manual processing can meet the needs when the number of orders is small, it is prone to low processing efficiency and reliance on human experience when the order scale is large or the requirements are complex, making it difficult to meet the timeliness and consistency requirements in actual business.

[0004] To improve automation, some existing technologies attempt to identify customer needs using keyword matching or traditional machine learning models. However, since customer notes often contain multiple business intentions and the expression of the same intention varies greatly, the above methods are still prone to problems such as inaccurate identification and insufficient scalability when faced with colloquial, ambiguous, or mixed texts with multiple intentions.

[0005] In addition, existing technologies often lack effective identification verification mechanisms for processing identification results, making it difficult to distinguish and process conflicting identification results. This results in the need for extensive manual intervention in subsequent scheduling decisions, affecting overall scheduling efficiency.

[0006] Moreover, when new demands arise, a great deal of human intervention is often required to identify the new intentions, which consumes a lot of manpower and time.

[0007] Therefore, existing technologies cannot achieve efficient, accurate, and automated demand parsing and structured processing in scenarios involving large-scale, diverse customer orders. They suffer from problems such as low processing efficiency, insufficient data accuracy, and limited automation. How to achieve accurate parsing, structured expression, and stable and reliable automatic processing of unstructured customer demand texts in scheduling scenarios remains an urgent problem to be solved in existing technologies. Summary of the Invention

[0008] To overcome the defects and shortcomings of existing technologies, this invention provides a method and system for parsing large-scale, multi-type customer order data. This addresses the problems of low efficiency in parsing unstructured customer demand text, inaccurate semantic understanding, and lack of consistency and conflict handling mechanisms in existing technologies, thereby improving the automation and reliability of parsing special customer demand in scheduling scenarios.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] This invention provides a method for parsing large-scale, multi-type customer order data, comprising the following steps:

[0011] Extract the notes and requirements text from the order data and perform data preprocessing;

[0012] The preprocessed annotation requirement text is transformed into a sequence of the smallest semantic units, and the hidden layer feature vector is extracted.

[0013] The hidden layer feature vector is decomposed into multiple business semantic features, and the decoupling enhancement features are output. The similarity between the intent semantic features and the preset intent prototype is calculated, and the intent activation distribution vector is output.

[0014] Construct a multi-domain knowledge base, calculate the matching confidence for each knowledge domain, and concatenate them to generate a knowledge feature vector;

[0015] Separate the corresponding semantic subspace features from the decoupled enhancement features, calculate the gating coefficient based on the semantic subspace features and the intent activation distribution vector, perform element-wise multiplication of the gating coefficient and the knowledge feature vector to obtain the knowledge representation, and concatenate the knowledge representation with the decoupled enhancement features to obtain the fused semantic features;

[0016] Intent recognition and entity extraction are performed based on fused semantic features, outputting intent probability distribution and business entity sequence.

[0017] As a preferred technical solution, the preprocessed annotation requirement text is transformed into a sequence of minimal semantic units, and the hidden layer feature vector is extracted, specifically including:

[0018] The annotation requirement text is transformed into a sequence of minimal semantic units using a word segmenter, and the hidden layer feature vector is extracted based on the self-attention mechanism of a multi-layer Transformer encoder.

[0019] As a preferred technical solution, the hidden layer feature vector is decomposed into multiple business semantic features, and the decoupling enhancement features are output, specifically including:

[0020] The hidden layer feature vector is decomposed into multiple business semantic dimensions through multiple parallel linear mapping layers, including action space sub-layer, constraint space sub-layer, time space sub-layer and preference space sub-layer.

[0021] The features output from each linear mapping layer are concatenated, and the concatenated features are residually connected with the hidden layer feature vectors to obtain decoupled enhanced features.

[0022] As a preferred technical solution, the similarity between the semantic features of the intent and the preset intent prototype is calculated, and the intent activation distribution vector is output, specifically including:

[0023] The action space sub-layer outputs intent semantic features, and calculates the similarity between these features and a preset intent prototype. The function generates an intention activation distribution vector after processing.

[0024] As a preferred technical solution, the action space sub-layer calculates the intent classification loss and the prototype comparison loss. The intent classification loss adopts the cross-entropy loss function to distinguish the intent category, and the prototype comparison loss is used to maximize the semantic similarity between the action features and the corresponding intent prototype.

[0025] The constraint space sub-layer is connected to a constraint entity auxiliary classifier, which calculates the constraint auxiliary supervision loss.

[0026] The temporal-space sublayer is connected to a temporal-assisted classifier to calculate the temporal-assisted supervised loss.

[0027] The preference space sub-layer is connected to a preference entity auxiliary classifier, which calculates the preference auxiliary supervision loss.

[0028] As a preferred technical solution, the corresponding semantic subspace features are separated from the decoupling enhancement features, and the gating coefficient is calculated based on the semantic subspace features and the intent activation distribution vector, as follows:

[0029] ;

[0030] in, It is the Sigmoid activation function. Represents the element-wise multiplication operation of vectors. This indicates the intention to activate the distribution vector. Indicates a nonlinear projection layer. Represents semantic subspace features.

[0031] As a preferred technical solution, intent recognition is based on fused semantic features, specifically including:

[0032] The fused semantic features are mapped to obtain the basic classification score. The basic classification score is then weighted and summed with the intent activation distribution vector to obtain the final score. The final score is then processed by the Softmax function to output the final intent probability distribution.

[0033] As a preferred technical solution, entity extraction is performed based on fused semantic features to output a sequence of business entities, specifically including:

[0034] The business entity sequence is extracted by classifying each smallest semantic unit in the fused semantic features through a linear classification layer.

[0035] As a preferred technical solution, it also includes a confidence calculation step for the fused semantic features, which compresses the fused semantic features into a single scalar value through nonlinear mapping to obtain a score value;

[0036] The results of intent recognition and entity extraction are converted into machine-readable structured instructions. The score is verified. If the score is lower than the preset security threshold, the structured instructions are sent to downstream tasks; otherwise, a review request is generated.

[0037] This invention also provides a parsing system for large-scale, multi-category customer order data, used to implement the above-mentioned parsing method for large-scale, multi-category customer order data, including: a data preprocessing module, a feature extraction module, a decoupling module, an intent activation distribution vector output module, a multi-domain knowledge base construction module, a knowledge feature vector generation module, a semantic subspace feature extraction module, a dynamic knowledge gating module, an intent recognition module, and an entity extraction module;

[0038] The data preprocessing module is used to extract the remarks and requirements text from the order data and perform data preprocessing.

[0039] The feature extraction module is used to convert the preprocessed annotation requirement text into a sequence of smallest semantic units and extract the hidden layer feature vector.

[0040] The decoupling module is used to decompose the hidden layer feature vector into multiple business semantic features and output decoupling enhancement features;

[0041] The intent activation distribution vector output module is used to calculate the similarity between the semantic features of the intent and the preset intent prototype, and output the intent activation distribution vector.

[0042] The multi-domain knowledge base construction module is used to construct a multi-domain knowledge base;

[0043] The knowledge feature vector generation module is used to calculate the matching confidence for each knowledge domain and concatenate them to generate a knowledge feature vector.

[0044] The semantic subspace feature extraction module is used to separate the corresponding semantic subspace features from the decoupled enhancement features;

[0045] The dynamic knowledge gating module is used to calculate the gating coefficient based on the semantic subspace features and the intent activation distribution vector. The gating coefficient is multiplied element-wise with the knowledge feature vector to obtain the knowledge representation. The knowledge representation is then concatenated with the decoupled enhancement features to obtain the fused semantic features.

[0046] The intent recognition module is used to recognize intents based on fused semantic features and output an intent probability distribution.

[0047] The entity extraction module is used to extract entities based on fused semantic features and output a sequence of business entities.

[0048] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0049] (1) Compared with the existing technology that directly uses global features for classification, this invention adopts an intent prototype-oriented semantic decoupling mechanism, which decomposes the hidden layer feature vector into multiple business semantic features. Specifically, it is explicitly split into action, constraint, time and preference space sub-layers, and then outputs decoupling enhancement features. The intention prototype vector is used to apply semantic constraints to the action subspace, so that the decoupling features have clear intent orientation. The similarity between the intention semantic features and the preset intention prototype is calculated, and the intention activation distribution vector is output, so that the action features move closer to the corresponding real intention prototype, thereby ensuring the accuracy of decoupling. It effectively solves the semantic entanglement and overfitting problems in small sample scenarios, and significantly improves the generalization ability and intent recognition accuracy under complex instructions.

[0050] (2) Compared with the existing technology of indiscriminately splicing external knowledge, the present invention adopts a dual-factor mechanism that combines intent-driven and content-driven approaches to generate gating coefficients that control specific knowledge base channels. Through the dynamic knowledge gating strategy guided by intent, the feature fusion of the external knowledge base is controlled, and the knowledge representation and decoupled enhancement features are spliced ​​and fused to obtain fused semantic features. This realizes on-demand access to knowledge, effectively suppresses noise interference caused by irrelevant entities, such as same-name matching in non-specific scenarios, and greatly improves the accuracy of knowledge enhancement.

[0051] (3) This invention does not rely on simple literal matching, but transforms the matching results in the multi-domain knowledge base into a numerical confidence vector, obtains knowledge representation based on the gating coefficient, and fuses it with decoupled enhancement features. Thanks to the dynamic knowledge gating strategy, the explicit injection of knowledge features enables correction based on the strong prior signal of the knowledge base when the literal features are ambiguous, which significantly improves the accuracy of entity boundary recognition and classification.

[0052] Specifically, this invention transforms the fuzzy matching degree and alias parsing results in the knowledge base into a numerical confidence vector. Compared with traditional rule-based or exact matching technical solutions, this invention has strong robustness against typos, abbreviations and non-standard expressions in colloquial input, and significantly reduces the cost of data cleaning and secondary verification.

[0053] (4) By calculating the confidence level of the fused semantic features, the present invention outputs the uncertainty score of the prediction result in real time. Compared with the black box blind decision-making of the traditional deep learning model, the present invention provides a safety barrier for automated scheduling. When encountering high-risk or ambiguous instructions, it can actively trigger manual review, effectively avoid erroneous order dispatch, and ensure the safety and reliability of logistics business implementation. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating the method for parsing large-scale, multi-type customer order data according to the present invention.

[0055] Figure 2 This is a schematic diagram illustrating the generation process of the decoupling enhancement feature and the intention activation distribution vector in this invention;

[0056] Figure 3 This is a schematic diagram illustrating the implementation process of generating fused semantic features based on the two-factor mechanism of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0058] Example 1

[0059] like Figure 1 As shown, this embodiment provides a method for parsing large-scale, multi-type customer order data, including the following steps:

[0060] S1: Receive raw order data containing customer requirements, extract unstructured remarks fields as remarks requirement text, perform data preprocessing, and generate preprocessed remarks requirement text.

[0061] In this embodiment, the original order data consists of two parts: one part is structured fields, and the other part is unstructured fields, such as order remarks or additional instructions;

[0062] Specifically, taking the poultry dispatch business scenario as an example, customer orders contain structured information such as poultry breeds and order quantities. However, due to the diversity of actual business scenarios, customers often have many personalized remarks, such as specifying delivery time, specifying farmers, or carpooling delivery. These requirements are usually presented in the form of unstructured remarks text. Therefore, it is not only necessary to process the basic order data, but also to perform semantic recognition and parsing on the unstructured text in the remarks in order to accurately identify the customer's specific needs.

[0063] In this embodiment, the original order data is parsed, and the order remarks field is automatically located and extracted. This unstructured text containing personalized customer requests is defined as the remarks requirement text. Then, the extracted remarks requirement text undergoes data preprocessing for text standardization, specifically including: time standardization, using regular expressions to extract non-standard time descriptions and convert them to a standard format; character encoding standardization, unifying the text encoding format to prevent garbled characters; full-width and half-width character conversion, converting all full-width numbers and letters to half-width forms; and preliminary replacement of common typos, etc.

[0064] S2: Based on deep encoding of the preprocessed notes requirement text using the BERT model, extract hidden layer feature vectors containing global context information;

[0065] In this embodiment, a pre-trained Chinese BERT model (Bidirectional Encoder Representations from Transformers) is used. The input annotation request text is transformed into a token sequence, i.e., a sequence of smallest semantic units, through a word segmenter. The self-attention mechanism of the multi-layer Transformer encoder is used to capture long-distance dependencies in the text, and finally, the hidden layer feature vector containing rich contextual information is output. ;

[0066] S3: Decompose the hidden layer feature vector into multiple business semantic features, output decoupling enhancement features, calculate the similarity between the intent semantic features and the preset intent prototype, and output the intent activation distribution vector.

[0067] like Figure 2 As shown, in this embodiment, the hidden layer feature vector is explicitly split into four semantically orthogonal business subspaces, which are then decomposed into different business semantic dimensions through four parallel linear mapping layers, corresponding to the action space sub-layer, constraint space sub-layer, time space sub-layer and preference space sub-layer, respectively. Semantic constraints are applied to the action subspace using the intention prototype vector, so that the decoupled features have clear intention orientation.

[0068] In this embodiment, the action space sublayer extracts the core features that determine the classification of the main graph from the global semantics, and the mapping formula is as follows:

[0069] ;

[0070] in, and These are the trainable weight matrix and bias vector corresponding to the action space sub-layer, respectively;

[0071] Generated action features Features used to characterize the operational intent contained in user input, as semantic features of intent, such as action verbs like specifying a user or traveling together and their contextual semantics;

[0072] In this embodiment, the constraint space sublayer is used to extract restrictive entity information related to the business subject from the input text. This subspace focuses on identifying such entity features and reducing noise interference to intent recognition. The mapping formula is as follows:

[0073] ;

[0074] in, and These are the trainable weight matrix and bias vector corresponding to the sub-layer of the constraint space, respectively. Represents constraint features;

[0075] In this embodiment, the temporal-space sublayer is used to represent time-related semantic features, such as date, time, and their contextual modifications. Time is a key factor in the scheduling system. This embodiment constructs time independently to avoid time information being obscured by other semantic elements. The mapping formula is as follows:

[0076] ;

[0077] in, and These are the trainable weight matrix and bias vector corresponding to the temporal and spatial sublayers, respectively. Indicates time characteristics;

[0078] In this embodiment, the preference space sublayer is used to represent the user's subjective preference information, such as carpooling partners and priority preferences. Although this type of semantics does not directly define the core action intent, it serves as an auxiliary context to help downstream tasks perform more accurate resource matching. The mapping formula is as follows:

[0079] ;

[0080] in, and These are the trainable weight matrix and bias vector corresponding to the sub-layer of the preference space, respectively. Indicates preference characteristics;

[0081] In this embodiment, in order to integrate the information from each subspace, the features output from the four business subspaces are concatenated and then passed through a nonlinear transformation layer containing LayerNorm and GELU activation functions and the input hidden layer feature vector. Perform residual connections to obtain the final decoupling enhancement features. ;

[0082] In this embodiment, the action features (i.e., semantic features of intent) and intent prototype Similarity calculation was performed. The function generates an intention activation distribution vector, which is represented as follows:

[0083] ;

[0084] The intent prototypes in the intent prototype library are represented as follows: ,in, This represents m intent prototypes, used to characterize predefined intent type semantics. The intent prototypes are high-dimensional semantic prototype vector matrices, where each row corresponds to a specific business intent category, such as specifying a delivery time, carpooling delivery, specifying a farmer, specifying a product type, etc.; the intent activation distribution vector... It serves a dual purpose: firstly, it acts as prior knowledge passed to subsequent intent recognition and dynamic knowledge gating; secondly, it is used during the training phase to calculate the contrastive loss, thus enabling action features to... By aligning with the corresponding true intention prototype, the accuracy of decoupling can be guaranteed;

[0085] In this embodiment, the ability of the four business subspaces to adaptively extract specific semantic features that do not interfere with each other from the unified hidden layer feature vector is determined by the end-to-end multi-task training mechanism. Although the four business subspaces are all linear mapping layers in physical structure, the iterative direction of their weights depends entirely on the specific loss function of their connection.

[0086] Specifically, the action space sublayer is mainly affected by the intention classification loss. Loss compared to prototype The dual constraints include an intent classification loss function using cross-entropy, which allows the sublayer to extract core verb features that can distinguish different business intents, such as specifying users or specifying time. Represented as:

[0087] ;

[0088] in, The total number of intent categories, Labeling as true intent For action feature-based Prediction belongs to the first The probability value of the class intent;

[0089] Prototype contrast loss Used to maximize action features Corresponding Intent Prototype The semantic similarity is specifically represented as:

[0090] ;

[0091] ;

[0092] in, Represents the normalized cosine similarity. This is the prototype vector of the target intent corresponding to the true label of the current sample. For the first in the intent prototype library The prototype vector of an intention, This is a temperature coefficient used to adjust the smoothness of the distribution. The total number of intention prototypes;

[0093] In this embodiment, a constraint entity auxiliary classifier is added after the constraint space sub-layer to calculate the constraint auxiliary supervision loss. This loss prompts the constrained space sublayer to focus on extracting restricted entity features, specifically expressed as:

[0094] ;

[0095] in, This represents the actual labels of entities subject to hard business constraints, such as farmer names or poultry breeds; it also represents the auxiliary classification layer connected to the constraint space sublayers. To constrain the feature vectors output by the spatial sublayer;

[0096] To ensure the independence of temporal features, this embodiment adds an independent temporal-assisted classifier after the temporal-space sublayer and calculates the temporal-assisted supervision loss. The gradient of the loss function is directly backpropagated to the weight matrix, causing the weight matrix to focus on encoding date and time features, specifically as follows:

[0097] ;

[0098] in, The length of the input sequence. For the first The token is a real-time entity tag, such as B-TIME, I-TIME, etc. This represents an independent linear classification layer connected to the temporal-space sublayer. This indicates the conditional probability of predicting the label;

[0099] In this embodiment, a preference entity auxiliary classifier is added after the preference space sub-layer to calculate the preference auxiliary supervision loss. This loss prompts the preference space sublayer to focus on extracting subjective preference features such as ride-sharing participants and priority, specifically expressed as:

[0100] ;

[0101] in, Indicates the true label of the preferred entity. This indicates an auxiliary classification layer connected to the preference space sub-layer. The feature vector output by the preference space sublayer;

[0102] S4: Construct a multi-domain knowledge base, including a customer information base, a farmer information base, and a poultry information base. Perform parallel text retrieval of the customer information base, farmer information base, and poultry information base, and generate knowledge feature vectors containing confidence through hybrid matching.

[0103] In this embodiment, the customer information database is used to store basic business information related to customers, including at least: customer unique identifier (ID), customer standard name, historical delivery time preference, customer geographic location coordinates and customer status information. This database is used to provide accurate entity link targets when parsing carpooling or specifying customer intent.

[0104] In this embodiment, the farmer information database is used to store information on the farming entities participating in the supply of poultry. The content includes: farmer's unique identifier, farmer's name, real-time inventory status, list of available poultry breeds, available quantity, and dynamic business information such as the age of the poultry. This database supports the system in verifying whether the farmer has the ability to fulfill the contract when parsing the intention of a specified farmer.

[0105] In this embodiment, the poultry information database is used to store basic information about poultry breeds, including basic attributes such as standard names of poultry breeds, breed types, and specifications; and an alias mapping dictionary that stores colloquial aliases, abbreviations, or common misspellings of breeds.

[0106] In this embodiment, a matching confidence score is calculated for each knowledge domain, and a hybrid matching process is performed, including exact matching and fuzzy matching based on character overlap (edit distance). The highest matching score is selected as the matching confidence score for that domain.

[0107] Then, alias resolution is performed. For the poultry information database, an additional pre-defined alias mapping dictionary is used. This dictionary stores the mapping relationship between non-standard names and standard IDs. For a matched alias, its pre-defined mapping confidence is extracted as the mapping confidence for alias resolution. ;

[0108] The matching confidence scores from the above-mentioned fields are numerically concatenated in a preset order to generate a knowledge feature vector. , is represented as:

[0109] ;

[0110] in, , , These represent the maximum matching confidence levels for customer, farmer, and poultry standard names, respectively. The mapping confidence level representing alias resolution;

[0111] S5: Utilizing intent tendency as a gating signal to dynamically filter knowledge features, and concatenating the filtered knowledge features with decoupled features to generate fused semantic features, this addresses the noise injection problem introduced by external knowledge bases. Through a dynamic switching mechanism based on intent recognition results, irrelevant knowledge retrieval results are accurately filtered, achieving selective fusion of knowledge features. Specifically, this includes:

[0112] like Figure 3 As shown, the intention activation distribution vector and decoupling enhancement features are obtained. Enhanced features from decoupling Separate the corresponding semantic subspace features from the data. Then calculate the gating coefficient. , is represented as:

[0113] ;

[0114] in, It is the Sigmoid activation function. Represents the element-wise multiplication operation of vectors. This indicates the intention to activate the distribution vector. Indicates a nonlinear projection layer. Represents semantic subspace features;

[0115] In this embodiment, For the corresponding intent master gating signal, if the activation value of a certain intent category in the intent activation distribution vector exceeds the threshold, it is determined that the intent exists. According to the preset mapping relationship between intent categories and knowledge base, for example, the user intention is mapped to the user information database, and the carpooling intention is mapped to the customer information database, and then the corresponding knowledge base master activation signal is generated. This ensures that the matching information in the knowledge base is only paid attention to when the business scenario requires it, such as when it is confirmed as a carpooling task; otherwise, it is directly blocked.

[0116] In this embodiment, For the corresponding content-driven refined gating signal vector, this embodiment uses a nonlinear projection layer MLP to calculate the content confidence that the subspace feature semantically contains a specific entity, and generates a content refined gating signal.

[0117] This embodiment employs a dual-factor mechanism that combines intent-driven and content-driven approaches to generate gating coefficients that control specific knowledge base channels. For each knowledge channel, the gating coefficient of that dimension will approach 1 only when the intent is activated and the content features match; otherwise, it will approach 0.

[0118] In this embodiment, the gating coefficient With knowledge feature vector Perform element-wise multiplication to obtain the gated knowledge representation. , is represented as:

[0119] ;

[0120] Representing knowledge With decoupling enhancement features The features are concatenated and fused to generate fused semantic features that include external knowledge verification. ;

[0121] S6: Intent recognition and entity extraction are performed based on fused semantic features, outputting intent probability distribution and entity sequence, specifically including:

[0122] For fusion semantic features Perform mapping to obtain the basic classification score. , and the intended activation distribution vector Perform a weighted summation to obtain the final score. , is represented as:

[0123] ;

[0124] in, As a balance coefficient, it is preferably 0.5 in this embodiment;

[0125] Final score After processing by the Softmax function, the final intent probability distribution is output, which is used to determine the global business intent of the user input text, such as specifying a customer or carpooling delivery.

[0126] To improve the accuracy of intent recognition in scenarios with few samples, this embodiment adopts a decision mechanism that fuses classifier prediction with prototype similarity. This mechanism ensures that reasonable intent inference can still be made based on prototype similarity even when features are ambiguous.

[0127] This embodiment uses a linear classification layer to fuse semantic features. Each token in the process is categorized, and specific key business entities are extracted.

[0128] Thanks to the dynamic knowledge gating mechanism in this embodiment, when the input entity contains typos, semantic features are fused. The system has already incorporated high-confidence matching signals obtained from standard entity computation. This explicit injection of external knowledge features enables corrections based on strong prior signals from the knowledge base even when literal features are ambiguous, thereby significantly improving the accuracy of entity boundary recognition and classification.

[0129] Optionally, this embodiment also includes calculating the confidence level of the fused semantic features and outputting an uncertainty score result;

[0130] In this embodiment, confidence scores can be calculated on the fused semantic features based on a multilayer perceptron to capture the cognitive uncertainty in parsing the current instruction. This uncertainty mainly comes from two conflict scenarios: First, the conflict between semantics and knowledge. BERT semantics strongly implies a certain intention, such as specifying a farmer, but the knowledge base fails to retrieve the corresponding valid entity, such as the low confidence score of fuzzy matching. Second, semantic ambiguity. The input text itself contains ambiguity or is not within the scope of the intent defined by the system, causing the feature vector to be located near the classification boundary in the semantic space.

[0131] In this embodiment, the fused semantic features are compressed into a single scalar value through nonlinear mapping to obtain the score value. , is represented as:

[0132] ;

[0133] in, , , , All of these are trainable parameters. This represents the activation function. This represents the Sigmoid function, and its output value is... This represents the degree of uncertainty. The closer the value is to 1, the lower the confidence in the current analysis result; the closer it is to 0, the more reliable the analysis result is.

[0134] In this embodiment, the results of intent recognition and entity extraction are combined, and fields are filled according to a predefined JSON template to transform the results of intent recognition and entity extraction into machine-readable structured instructions. At this stage, the scoring value is verified again. If the rating value If the score is below the safety threshold (e.g., 0.2), the structured instructions are sent directly to the vehicle dispatching system or downstream tasks; if the score value is below the safety threshold (e.g., 0.2), the structured instructions are sent directly to the vehicle dispatching system or downstream tasks. If the value exceeds the safety threshold, a manual review request will be generated, and the extracted suspected entities will be presented as candidate suggestions to human customer service to assist in rapid decision-making.

[0135] Example 2

[0136] This embodiment provides a parsing system for large-scale, multi-type customer order data, used to implement the parsing method for large-scale, multi-type customer order data in Embodiment 1, including: a data preprocessing module, a feature extraction module, a decoupling module, an intent activation distribution vector output module, a multi-domain knowledge base construction module, a knowledge feature vector generation module, a semantic subspace feature extraction module, a dynamic knowledge gating module, an intent recognition module, and an entity extraction module;

[0137] In this embodiment, the data preprocessing module is used to extract the remarks and requirements text from the order data and perform data preprocessing;

[0138] In this embodiment, the feature extraction module is used to convert the preprocessed annotation requirement text into a sequence of minimal semantic units and extract the hidden layer feature vector.

[0139] In this embodiment, the decoupling module is used to decompose the hidden layer feature vector into multiple business semantic features and output decoupling enhancement features;

[0140] In this embodiment, the intent activation distribution vector output module is used to calculate the similarity between the semantic features of the intent and the preset intent prototype, and output the intent activation distribution vector.

[0141] In this embodiment, the multi-domain knowledge base construction module is used to construct a multi-domain knowledge base;

[0142] In this embodiment, the knowledge feature vector generation module is used to calculate the matching confidence for each knowledge domain and concatenate them to generate a knowledge feature vector;

[0143] In this embodiment, the semantic subspace feature extraction module is used to separate the corresponding semantic subspace features from the decoupled enhancement features;

[0144] In this embodiment, the dynamic knowledge gating module is used to calculate the gating coefficient based on the semantic subspace features and the intent activation distribution vector. The gating coefficient is multiplied element-wise with the knowledge feature vector to obtain the knowledge representation. The knowledge representation is then concatenated and fused with the decoupled enhancement features to obtain the fused semantic features.

[0145] In this embodiment, the intent recognition module is used to recognize intent based on fused semantic features and output the intent probability distribution;

[0146] In this embodiment, the entity extraction module is used to extract entities based on fused semantic features and output a sequence of business entities.

[0147] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for resolving large-scale multi-category customer order data, characterized in that, Includes the following steps: Extract the notes and requirements text from the order data and perform data preprocessing; The preprocessed annotation requirement text is transformed into a sequence of the smallest semantic units, and the hidden layer feature vector is extracted. The hidden layer feature vector is decomposed into multiple business semantic features, outputting decoupled enhanced features. The similarity between the intent semantic features and the preset intent prototype is calculated, and the intent activation distribution vector is output, specifically including: The hidden layer feature vector is decomposed into multiple business semantic dimensions through multiple parallel linear mapping layers, including action space sub-layer, constraint space sub-layer, time space sub-layer and preference space sub-layer. The features output from each linear mapping layer are concatenated, and the concatenated features are residually connected with the hidden layer feature vectors to obtain decoupled enhanced features. The action space sublayer outputs an intention semantic feature, performs similarity calculation on the intention semantic feature and a preset intention prototype, and outputs an intention activation distribution vector after function processing. The function processing generates an intention activation distribution vector. Construct a multi-domain knowledge base, calculate the matching confidence for each knowledge domain, and concatenate them to generate a knowledge feature vector; Separate the corresponding semantic subspace features from the decoupled enhancement features, calculate the gating coefficient based on the semantic subspace features and the intent activation distribution vector, perform element-wise multiplication of the gating coefficient and the knowledge feature vector to obtain the knowledge representation, and concatenate the knowledge representation with the decoupled enhancement features to obtain the fused semantic features; Intent recognition and entity extraction are performed based on fused semantic features, outputting intent probability distribution and business entity sequence.

2. The method for resolving large-scale multi-category customer order data according to claim 1, wherein, The preprocessed annotation text is transformed into a sequence of minimal semantic units, and the hidden layer feature vector is extracted, specifically including: The annotation requirement text is transformed into a sequence of minimal semantic units using a word segmenter, and the hidden layer feature vector is extracted based on the self-attention mechanism of a multi-layer Transformer encoder.

3. The method of claim 1, wherein, The action space sub-layer calculates the intent classification loss and the prototype comparison loss. The intent classification loss uses the cross-entropy loss function to distinguish intent categories, and the prototype comparison loss is used to maximize the semantic similarity between action features and corresponding intent prototypes. The constraint space sub-layer is connected to a constraint entity auxiliary classifier, which calculates the constraint auxiliary supervision loss. The temporal-space sublayer is connected to a temporal-assisted classifier to calculate the temporal-assisted supervised loss. The preference space sub-layer is connected to a preference entity auxiliary classifier, which calculates the preference auxiliary supervision loss.

4. The method for resolving large-scale multi-category customer order data according to claim 1, wherein, Separate the corresponding semantic subspace features from the decoupling enhancement features, and calculate the gating coefficients based on the semantic subspace features and the intent activation distribution vector, as follows: ； in, It is the Sigmoid activation function. Represents the element-wise multiplication operation of vectors. This indicates the intention to activate the distribution vector. Indicates a nonlinear projection layer. Represents semantic subspace features. This represents the gating coefficient.

5. The method for resolving large-scale multi-category customer order data according to claim 1, wherein, Intent recognition based on fused semantic features specifically includes: The fused semantic features are mapped to obtain the basic classification score. The basic classification score is then weighted and summed with the intent activation distribution vector to obtain the final score. The final score is then processed by the Softmax function to output the final intent probability distribution.

6. The method for resolving large-scale multi-category customer order data according to claim 1, wherein, Entity extraction is performed based on fused semantic features, outputting a sequence of business entities, specifically including: The business entity sequence is extracted by classifying each smallest semantic unit in the fused semantic features through a linear classification layer.

7. The method of claim 1, wherein, It also includes a confidence calculation step for the fused semantic features, which compresses the fused semantic features into a single scalar value through nonlinear mapping to obtain a score value; The results of intent recognition and entity extraction are converted into machine-readable structured instructions. The score is verified. If the score is lower than a preset security threshold, the structured instructions are sent to downstream tasks; otherwise, a review request is generated.

8. A resolution system for large-scale multi-category customer order data, characterized by, The method for parsing large-scale, multi-type customer order data as described in any one of claims 1-7 includes: a data preprocessing module, a feature extraction module, a decoupling module, an intent activation distribution vector output module, a multi-domain knowledge base construction module, a knowledge feature vector generation module, a semantic subspace feature extraction module, a dynamic knowledge gating module, an intent recognition module, and an entity extraction module. The data preprocessing module is used to extract the remarks and requirements text from the order data and perform data preprocessing. The feature extraction module is used to convert the preprocessed annotation requirement text into a sequence of smallest semantic units and extract the hidden layer feature vector. The decoupling module is used to decompose the hidden layer feature vector into multiple business semantic features and output decoupling enhancement features, specifically including: The hidden layer feature vector is decomposed into multiple business semantic dimensions through multiple parallel linear mapping layers, including action space sub-layer, constraint space sub-layer, time space sub-layer and preference space sub-layer. The features output from each linear mapping layer are concatenated, and the concatenated features are residually connected with the hidden layer feature vectors to obtain decoupled enhanced features. The intent activation distribution vector output module is used to calculate the similarity between the semantic features of the intent and the preset intent prototype, and outputs the intent activation distribution vector, specifically including: The action space sub-layer outputs intent semantic features, and calculates the similarity between these features and a preset intent prototype. The function generates an intention activation distribution vector after processing; The multi-domain knowledge base construction module is used to construct a multi-domain knowledge base; The knowledge feature vector generation module is used to calculate the matching confidence for each knowledge domain and concatenate them to generate a knowledge feature vector. The semantic subspace feature extraction module is used to separate the corresponding semantic subspace features from the decoupled enhancement features; The dynamic knowledge gating module is used to calculate the gating coefficient based on the semantic subspace features and the intent activation distribution vector. The gating coefficient is multiplied element-wise with the knowledge feature vector to obtain the knowledge representation. The knowledge representation is then concatenated with the decoupled enhancement features to obtain the fused semantic features. The intent recognition module is used to recognize intents based on fused semantic features and output an intent probability distribution. The entity extraction module is used to extract entities based on fused semantic features and output a sequence of business entities.

Images