Social security digital employee data processing and model building method based on cloud edge collaboration

Abstract

The present application relates to the technical field of digital employee data processing, in particular to a social security digital employee data processing and model building method based on cloud edge collaboration, which specifically comprises the following steps: collecting multi-source social security data and grabbing policy texts; constructing training samples based on the collected multi-source social security data and policy texts, and dividing them into a training set, a validation set and a test set in proportion; constructing a social security digital employee question and answer model based on a neural network architecture of an encoder and a decoder, inputting the training set into the model for processing by the encoder and the decoder to generate an answer sequence; training the model, iteratively training the model multiple times until the model ends training when a preset iteration stopping condition is met, selecting the best model from the validation set as the optimal model; and inputting the test set into the optimal model to generate an answer sequence. The present application can improve processing efficiency, enhance data processing accuracy and reduce operating costs by constructing a social security digital employee question and answer model.

Description

Technical Field

[0001] This invention relates to the field of digital employee data processing technology, and in particular to a method for processing and building models of social security digital employee data based on cloud-edge collaboration. Background Technology

[0002] With social development and changes in population structure, social security services face challenges such as a continuously increasing number of insured individuals, constantly updated policies, and increasingly complex business processes. Traditional social security processing relies heavily on manual operation, which is inefficient, prone to human error, and fails to meet the public's demand for timely and accurate social security information inquiries and service processing. Against this backdrop, leveraging advanced information technology to enhance the intelligence level of social security services has become an important development direction. However, existing technologies have many shortcomings in processing social security data and generating accurate answers, such as incomplete data collection, inaccurate semantic understanding, low efficiency in processing long texts, and a lack of strict adherence to policy rules. These shortcomings limit the development of intelligent social security services and cannot fully meet the complex needs of social security operations.

[0003] The existing methods have the following specific problems: Conventional word segmentation methods tend to fragment specialized terms, leading to semantic fragmentation and affecting the accuracy of understanding specialized terms and long-tail expressions in the social security field. In addition, there is a lack of effective mechanisms to dynamically adjust the weights of embeddings at different granularities, making it difficult to fully integrate semantic information at multiple granularities such as characters, words, and terms, thus limiting the model's ability to model complex semantics. Social security policy texts have long-range dependencies across paragraphs and hierarchical structures of chapters, sections, and clauses. However, conventional absolute position encoding in existing technologies is difficult to model long-distance positional relationships, while fixed-pattern sparse attention strategies cannot adapt to the hierarchical logical chain of policies. This makes it easy for the model to lose important semantic information and logical structure when processing social security policy texts, resulting in inaccurate and incomplete understanding. When generating social security answers, the decoders of conventional technologies usually lack hard constraint mechanisms on policy rules, relying only on soft rule injection, which easily produces logical contradictions and answers that do not meet policy requirements. This seriously affects the practicality and reliability of the model and cannot meet the strict requirements of social security business for answer accuracy and policy consistency.

[0004] Therefore, this invention proposes a cloud-edge collaborative method for processing and building models of social security digital employee data to solve the above problems. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by developing a cloud-edge collaborative method for processing and building models of social security digital employee data. The social security digital employee question-and-answer model constructed by this invention can meet the stringent requirements of social security business for the accuracy of answers and policy consistency, and has practicality and reliability.

[0006] The technical solution of this invention to solve the technical problem is a method for processing and building models of social security digital employee data based on cloud-edge collaboration, including the following steps:

[0007] S1. Collect multi-source social security data through cloud platform and edge computing nodes in collaboration. The collected data is incrementally synchronized to the cloud structured data table on a regular basis through API interface. For missing years of insurance, the average data of adjacent regions of the same insurance type is used to fill the missing field. The policy interpretation documents updated by the government website are crawled in real time through distributed crawler. The policy text is parsed according to the "chapter-section-clause" level.

[0008] S2. Based on the collected multi-source social security data and policy texts, training samples are constructed. The sample generation logic adopts the question-answer pair extraction method, parses the "policy clause-applicable conditions-implementation standard" triplet, generates standard question-answer templates, and divides the training samples into training set, validation set and test set according to the proportion.

[0009] S3. A social security digital employee question-answering model is constructed using a neural network architecture based on encoders and decoders. The model has an encoder-decoder structure. The encoder includes an input layer, a position encoding layer, an attention layer, and an output layer. The decoder includes an input layer, a logic enhancement layer, a normalization layer, and an output layer. The training set is input into the model and processed by the encoder and decoder to generate a sequence of answers.

[0010] S4. Define a loss function to train the model and update the model parameters. The model is trained iteratively multiple times until the preset iteration stopping condition is met, at which point the model ends the training. The model with the best performance is selected as the optimal model through the validation set.

[0011] S5. Input the test set into the optimal model, perform social security digital employee Q&A, and generate an answer sequence.

[0012] The specific process of multi-granularity semantic fusion operation at the encoder's input layer is as follows:

[0013] For the input text Perform character-level, word-level, and term-level embedding fusion. This represents the input text, which contains a question and answer, and is represented as [CLS]question[SEP]answer[SEP]. [CLS] represents the start of the sequence, and [SEP] represents the separator. The specific steps are as follows:

[0014] 1) Calculate character-level embeddings:

[0015] Based on the input text, each character is mapped to a dense vector using a character embedding function, generating a character-level embedding matrix;

[0016] 2) Calculate word-level embeddings:

[0017] The input text is segmented to obtain a word sequence, and then each word sequence is mapped to a dense vector through a word embedding function to generate a word-level embedding matrix.

[0018] 3) Computational term-level embeddings:

[0019] Based on the social security terminology database, the system identifies professional terms in the input text using a terminology lookup function and directly retrieves the corresponding term embedding vectors from the terminology database to generate a term-level embedding matrix.

[0020] 4) Calculate the gating vector:

[0021] The character-level embedding matrix, word-level embedding matrix, and term-level embedding matrix corresponding to the same position are concatenated, and then the gate vector is calculated through linear transformation and Sigmoid activation function.

[0022] 5) Computational fusion embedding:

[0023] The word-level embedding matrix and the term-level embedding matrix are projected onto the same dimensional space as the character-level embedding matrix. Then, the projected word-level embedding, projected term-level embedding, and character-level embedding corresponding to the same position are concatenated. The gate vector at that position is used to weight the concatenated vector to obtain the fused embedding vector at that position. , Indicates the first The fused embedding vector at each position.

[0024] The specific process of relative position encoding operation in the encoder's position encoding layer is as follows:

[0025] A semantically aware relative positional encoding mechanism is adopted, and the specific steps are as follows:

[0026] 1) Calculate the semantic decay coefficient:

[0027] The fused embedding vectors of the two positions are concatenated, and then the semantic decay coefficient is calculated through linear transformation and activation function to reflect the degree of semantic association between the two positions. When the semantic association is high, the decay coefficient approaches zero.

[0028] 2) Calculate the relative position offset:

[0029] Multiply the semantic attenuation coefficient between two positions by their relative position distance and take the negative value to obtain the relative position bias value, and then obtain the relative position bias matrix.

[0030] 3) Generate semantic awareness attention scores:

[0031] The query matrix and key matrix of the input sequence are obtained through linear transformation, and the standard attention score matrix is ​​calculated. Then, the calculated relative position bias matrix is ​​added to the standard attention score matrix to obtain the semantic-aware relative position attention score matrix.

[0032] The specific process of performing hierarchical coefficient attention enhancement operation on the encoder's attention layer is as follows:

[0033] To reduce computational cost and maintain the integrity of the policy logic chain, a dynamic attention mask is constructed based on the policy structure. The specific steps are as follows:

[0034] 1) Hierarchical boundary resolution:

[0035] The hierarchical boundaries of policy texts are automatically identified by matching the numbered parts of the policy text using regular expressions. The numbering system is represented as "Chapter {Chapter {Section {Chapter ...

[0036] 2) Structured attention mask construction:

[0037] Based on the hierarchical structure of the policy text, a binary mask matrix is ​​generated for each level. The chapter-level mask stipulates that attention connections can only be established between positions belonging to the same chapter; the section-level mask stipulates that connections can only be established between positions belonging to the same section; and the clause-level mask stipulates that connections can only be established between positions belonging to the same clause. The three level masks are then added together to obtain the structured attention mask. In attention calculation, connections are only allowed between positions within the same level. The calculation formula is as follows:

[0038] ,

[0039] in, Represents a structured attention mask;

[0040] This represents a chapter-level mask, and the mask logic rules are as follows: , Indicates the chapter-level mask. Line number Column elements;

[0041] This represents a section-level mask, and the mask logic rules are as follows: , Indicates the clause-level mask. Line number Column elements;

[0042] This represents a clause-level mask, and the mask logic rules are as follows: , Indicates the clause-level mask. Line number Column elements;

[0043] 3) Sparse attention computation:

[0044] The structured attention mask is added to the scaled standard attention score matrix, and then the sparse attention weight matrix is ​​calculated using the Softmax function. The sparse attention weight matrix has non-zero weights only between the same level position pairs allowed by the structured attention mask.

[0045] The specific process of feature output operation in the encoder's output layer is as follows:

[0046] The outputs of relative position attention and hierarchical sparse attention are integrated, and the positional information and hierarchical structure information are aggregated to obtain the encoder output matrix. The specific steps are as follows:

[0047] 1) Attention mechanism fusion:

[0048] The relative position attention score matrix and the hierarchical sparse attention weight matrix are concatenated along the feature dimension. The attention gating vector is calculated by linear transformation and sigmoid activation function. Then, the attention gating vector is used to perform a weighted summation of the relative position attention score matrix and the hierarchical sparse attention weight matrix to obtain the fused attention weight matrix.

[0049] 2) Context representation generation:

[0050] The attention value matrix is ​​obtained by linearly transforming the fused embedding vector and the projection weight matrix. Then, the attention value matrix is ​​weighted and summed using the fused attention weight matrix and added to the fused embedding vector. Finally, the sum is normalized to obtain the context representation matrix output by the encoder.

[0051] The specific process of feature input operation at the input layer of the decoder is as follows:

[0052] The contextual representation output by the encoder is fused with the target sequence embedding, thereby semantically associating the input with the target sequence. The specific steps are as follows:

[0053] 1) Target sequence embedding:

[0054] The character embedding function is used to map each character of the target sequence to a dense vector to generate a character-level embedding matrix. Then, the positional embedding is calculated by the positional encoding function and superimposed on the character-level embedding to construct the decoder input embedding matrix.

[0055] 2) Mask self-attention calculation:

[0056] The decoder input embedding matrix is ​​processed using a masked multi-head self-attention mechanism, and the lower triangular mask matrix is ​​used to restrict each position to only pay attention to itself and the positions before it.

[0057] 3) Encoder-decoder attention fusion:

[0058] Attention fusion is performed based on the standard multi-head attention mechanism. The standard multi-head attention mechanism uses the masked self-attention hidden state as the query matrix, the context representation output by the encoder as the key matrix and value matrix, calculates the cross-attention weights, and fuses the encoder and decoder information through a weighted summation method to generate the attention hidden state matrix.

[0059] The specific process of the logic enhancement layer of the decoder performing logic enhancement dynamic constraint operations is as follows:

[0060] Hard constraints are injected into differentiable logic reasoning units to ensure that the output conforms to policy logic. The specific steps are as follows:

[0061] 1) Construction of the logical rule base:

[0062] A set of logical rules is predefined from the policy text to form a logical rule base. Each rule consists of an antecedent predicate logical expression about the input sequence and a consequent predicate logical expression about the output sequence. The two are logically implied. The logical rule base is represented as follows:

[0063] ,

[0064] in, Represents a logical rule base, containing Rules; Indicates the first Rule identifier; The antecedent of the rule is about the input sequence. The predicate logic expression; The consequent of the rule is about the output sequence. The predicate logic expression; This indicates a logical implication relationship; Input the sequence, i.e., the policy text, to the encoder; The decoder outputs a sequence, which is the answer text.

[0065] 2) Logical attention calculation:

[0066] Based on the hidden state of the decoder at the current time step, the attention weight of each rule in the logical rule base at that time step is calculated by linear transformation, hyperbolic tangent function and dot product operation of the rule attention vector. This can be used to dynamically filter the subset of rules most relevant to the current generation context.

[0067] 3) Generation of differentiable logic constraints:

[0068] The context representation of the encoder is processed using the antecedent function. The satisfaction probability of the antecedent of each rule in the logical rule base is calculated. Then, the satisfaction probability is multiplied element by element with the embedding vector of the corresponding rule. The results are then weighted and summed using the rule attention weights calculated at the current time step to obtain the logical constraint vector at that time step. This transforms the symbolic logical rules into differentiable constraint representations.

[0069] 4) Enhanced gated residuals:

[0070] The hidden state at the current time step of the decoder is concatenated with the logical constraint vector. The residual enhancement gating vector is calculated through linear transformation and sigmoid activation function. Then, the logical constraint vector is projected into the hidden space and weighted with the residual enhancement gating vector. Finally, the weighted constraint vector is added to the original hidden state as the residual to obtain the logical enhancement hidden state matrix.

[0071] The specific process of layer normalization processing performed by the encoder's normalization layer is as follows:

[0072] A layer normalization operation is performed on the logic-enhanced hidden state matrix. The distribution of the hidden states is adjusted through normalization, which stabilizes the training process and enhances the gradient flow, generating a layer-normalized hidden state matrix.

[0073] The specific process of the decoder's output layer performing the answer sequence output operation is as follows:

[0074] The normalized hidden state matrix is ​​mapped to the vocabulary space to generate an unnormalized score vector, which is then converted into a word probability distribution using the Softmax function. The specific steps are as follows:

[0075] 1) Score vector calculation:

[0076] By outputting the projection weight matrix and bias vector, a linear transformation is performed on the layer normalized hidden state vector to map the hidden state to the vocabulary space and generate an unnormalized score vector.

[0077] 2) Probability distribution calculation:

[0078] The Softmax function is used to process the unnormalized score vector, and the word probability distribution at the current time step is calculated. This distribution represents the probability of each candidate word being generated given the input sequence and the generated sequences, thus guiding the sequence output. The formula for calculating the word probability distribution is as follows:

[0079] ,

[0080] In the formula, Represents a given input sequence and generated sequences Current word under conditions The probability distribution; Indicates time step Words to be generated; Indicates time step The previously generated word sequence; Indicates the encoder input sequence;

[0081] 3) Sequence generation:

[0082] a) If in the training phase:

[0083] The cross-entropy loss is calculated based on the word probability distribution and the true label. The model parameters are then optimized using the cross-entropy loss function, which is calculated as follows:

[0084] ,

[0085] in, Indicates time step The true label; Indicates the length of the target sequence; Represents cross-entropy loss; This indicates that the model performs well under given input. and the true historical sequence Predicting real words The probability of; Indicates time step The previous true historical sequence; This represents the logarithmic function, with a default base of 10.

[0086] b) If in the reasoning stage:

[0087] The sequence is generated step by step by selecting the word with the highest probability using an autoregressive method until the end marker [SEP] is output, thus completing the generation of the answer sequence. The calculation formula is as follows:

[0088] ,

[0089] in, This is an index operation to retrieve the maximum value.

[0090] The training process of the model is as follows:

[0091] S4.1 Define the loss function:

[0092] Define a multi-task loss function, which includes weighted cross-entropy loss, logical rule loss and boundary alignment loss. Optimize the generation quality through cross-entropy loss, ensure that the output meets the hard logical constraints of the policy rule base through logical rule loss, force key policy values ​​to be within a preset reasonable range through boundary alignment loss, and jointly improve the semantic accuracy and logical consistency of the answer through the multi-task loss function.

[0093] The logical rule loss is defined based on each rule in the logical rule base. The score of the rule being satisfied and the score of the rule being violated are calculated. The logical loss component of each rule is obtained based on the logarithmic function. Then, the average of all rules is taken to finally generate the logical rule loss value, thereby quantifying the degree to which the model output sequence violates the hard logical constraints in the logical rule base of the policy.

[0094] Extract values ​​from a preset set of key values ​​from the model output sequence, determine whether they exceed a preset reasonable range, and then force the key policy values ​​to be within the preset reasonable range.

[0095] S4.2 Define the model parameter update method:

[0096] An adaptive moment estimation optimizer was used to update the model parameters, with an initial learning rate of 0.0001, which was decayed to 90% of the original value every 10 training epochs.

[0097] The gradients of cross-entropy loss, logistic rule loss and boundary alignment loss in the multi-task loss function are calculated independently, and a gradient pruning strategy is implemented. When the gradient L2 norm exceeds the threshold of 1.0, scaling is performed.

[0098] Based on the model update method, the standard error backpropagation algorithm is adopted. The multi-task loss function value is calculated through forward propagation, the gradient of each parameter is obtained through backpropagation, and the Adam optimizer parameter update rule is executed after gradient pruning. This process is repeated iteratively.

[0099] S4.3 Define the stopping condition for iterative training:

[0100] The main stopping condition of the model is to trigger early stopping when the value of the multi-task loss function on the validation set decreases by less than 0.1% for 8 consecutive epochs.

[0101] The maximum training iteration epoch is set to 100 times. Each iteration epoch saves a snapshot of the parameters of the top 3 models with the highest accuracy on the validation set. Finally, the model with the highest policy logic consistency score on the validation set is selected as the optimal model for deployment.

[0102] The effects described in the invention are merely those of the embodiments, and not all the effects of the invention. The above technical solutions have the following advantages or beneficial effects:

[0103] In the encoder input layer, this invention adopts a three-level embedding fusion approach of character-level, word-level, and term-level embeddings, which can avoid the semantic fragmentation problem caused by the fragmentation of professional terms by conventional word segmentation methods, thereby improving semantic integrity and understanding accuracy. At the same time, by dynamically adjusting the weights of different granular embeddings through a gating mechanism, the modeling ability of professional terms and long-tail expressions can be further enhanced.

[0104] In response to the long-range dependencies and hierarchical structure of social security policy texts, this invention employs a semantically aware relative position encoding mechanism and a hierarchical sparse attention mechanism. The former enables highly correlated position pairs to maintain high attention weights even when they are far apart, while the latter constructs a dynamic attention mask based on the policy structure, reducing computational load while maintaining the integrity of the policy logic chain, which can effectively improve the model's ability to understand and model social security policy texts.

[0105] The decoder employs a differentiable logic reasoning unit to construct a logic rule base. Through operations such as logic attention computation and gating residual enhancement, hard constraints are transformed into differentiable constraint representations and injected into the decoding process. This enables the generated answer to strictly follow social security policy rules, avoiding the logical contradictions that are easily generated by conventional decoders due to the lack of hard constraint mechanisms, thereby ensuring the policy logic consistency of the answer.

[0106] In summary, the cloud-edge collaborative method for processing and building models of social security digital employee data proposed in this invention can significantly improve processing efficiency and speed, greatly enhance data accuracy and consistency, effectively reduce operating costs and optimize resource allocation, strengthen risk prevention and compliance assurance, improve service quality and decision support capabilities, and provide an intelligent, precise, efficient, and secure question-and-answer model for social security digital employees. Attached Figure Description

[0107] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0108] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0109] Figure 2 This is a comparison chart of the accuracy of the model of this invention and existing models on the social security question-and-answer test set.

[0110] Figure 3 This is a comparison chart of the experimental results of multi-particle-size embedding ablation.

[0111] Figure 4 This is a comparison chart showing the logical consistency of the model of this invention with other models in terms of social security rules.

[0112] Figure 5 This is a comparison chart of the inference speed of the model of this invention and existing models. Detailed Implementation

[0113] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0114] Example 1

[0115] A method for processing and building models of social security digital employee data based on cloud-edge collaboration includes the following steps:

[0116] S1. Collect multi-source social security data through cloud platform and edge computing nodes in collaboration. The collected data is incrementally synchronized to the cloud structured data table on a regular basis through API interface. For missing years of insurance, the average data of adjacent regions of the same insurance type is used to fill the missing field. The policy interpretation documents updated by the government website are crawled in real time through distributed crawler. The policy text is parsed according to the "chapter-section-clause" level.

[0117] The cloud-based data sources are the social security government cloud platform's database of basic information of insured persons, database of payment records, database of benefit assessment, and database of policies and regulations. The edge-based data sources are the on-site business processing records, self-service machine consultation logs, and remote video conversation transcripts collected by the terminal equipment in the social security service hall.

[0118] Cloud-based structured data tables include details of insured individuals' contributions and pension payment records;

[0119] Edge nodes can use a speech recognition module to convert consultation dialogues into text streams, which are then anonymized and uploaded to the edge data pool.

[0120] The data covers the entire lifecycle of five major insurance types: pension, medical, work injury, unemployment, and maternity insurance. It includes unstructured texts such as the original policy clauses, business handling rules, and frequently asked questions knowledge base. The data spans no less than 10 years to ensure the continuity of historical policies.

[0121] S2. Based on the collected multi-source social security data and policy texts, training samples are constructed. The sample generation logic adopts the question-answer pair extraction method, parses the "policy clause-applicable conditions-implementation standard" triplet, generates standard question-answer templates, and divides the training samples into training set, validation set and test set according to the proportion. The training sample division strategy is shown in Table 1.

[0122] For example, the clause "pension = basic pension + personal account pension" is interpreted as a question-and-answer pair [How to calculate pension? = basic part plus personal account part].

[0123] Table 1 Training Sample Splitting Strategy

[0124]

[0125] S3. A social security digital employee question-answering model is constructed using a neural network architecture based on encoders and decoders. The model has an encoder-decoder structure. The encoder includes an input layer, a position encoding layer, an attention layer, and an output layer. The decoder includes an input layer, a logic enhancement layer, a normalization layer, and an output layer. The training set is input into the model and processed by the encoder and decoder to generate a sequence of answers.

[0126] S4. Define a loss function to train the model and update the model parameters. The model is trained iteratively multiple times until the preset iteration stopping condition is met, at which point the model ends the training. The model with the best performance is selected as the optimal model through the validation set.

[0127] S5. Input the test set into the optimal model, perform social security digital employee Q&A, and generate an answer sequence.

[0128] In a specific implementation, the process of multi-granularity semantic fusion operation at the encoder's input layer is as follows:

[0129] In the field of social security, specialized terminology is fragmented after being segmented by word segmentation tools, resulting in semantic fragmentation. Conventional word segmentation methods cannot preserve the integrity of the terminology, affecting the accuracy of semantic understanding. Therefore, this invention addresses this issue by processing the input text... By fusing character-level, word-level, and terminology-level embeddings, we can avoid fragmenting technical terms and improve semantic integrity. This represents the input text, which contains a question and answer, and is represented as [CLS]question[SEP]answer[SEP]. [CLS] represents the start of the sequence, and [SEP] represents the separator. The specific steps are as follows:

[0130] 1) Calculate character-level embeddings:

[0131] Based on the input text, each character is mapped to a dense vector using a character embedding function, generating a character-level embedding matrix. The calculation formula is as follows:

[0132] ,

[0133] in, This represents a character-level embedding matrix, where each row corresponds to the embedding vector of a single character. This represents a character embedding function that maps each character to a dense vector;

[0134] In natural language processing tasks, the sequence start token [CLS] is used to aggregate global information, and the separator token [SEP] is used to distinguish the question and answer parts. The question and answer are separated by position embeddings and special tokens. During model training, the model adopts an encoder-decoder architecture. The encoder processes the entire sequence to learn the context representation, and the decoder identifies the question part via [SEP] and uses it as input to generate the answer sequence;

[0135] 2) Calculate word-level embeddings:

[0136] The input text is tokenized to obtain a word sequence, and then each word sequence is mapped to a dense vector through a word embedding function to generate a word-level embedding matrix. The calculation formula is as follows:

[0137] ,

[0138] where, represents the word-level embedding matrix, and each row corresponds to the embedding vector of a word; represents the operation of the word segmentation algorithm based on the maximum matching method; represents the word embedding function that maps each word to a dense vector;

[0139] Assume the input text is "housing fund balance";

[0140] Set the character embedding dimension to 3. The character-level embedding matrix is represented as , where each row corresponds to the embedding vector of a character, and "公" corresponds to , "积" corresponds to ;

[0141] After word segmentation, it is ["housing fund", "balance"]. Set the word embedding dimension to 3, then the word-level embedding matrix is represented as , where each row corresponds to the embedding vector of a word, "housing fund" corresponds to , "balance" corresponds to .

[0142] 3) Calculate term-level embeddings:

[0143] Based on the social security term library, professional terms are identified in the input text through a term lookup function, and the corresponding term embedding vectors are directly retrieved from the term library to generate a term-level embedding matrix. The calculation formula is as follows:

[0144] ,

[0145] where, The social security terminology database, such as the social security terminology database containing professional terms such as "number of months for pension payment", is automatically extracted from the domain corpus using the TF-IDF (Term-Frequency-Inverse-Document-Frequency) algorithm; This represents a term lookup function that identifies and retrieves term embeddings from text. This represents a term-level embedding matrix, where each row corresponds to the embedding vector of a term;

[0146] Assuming the input text contains the term "deemed contribution years," and its embedding vector is obtained directly from the social security terminology database, with a word embedding dimension of 3, the term-level embedding matrix might be: ;

[0147] 4) Calculate the gating vector:

[0148] The character-level embedding matrix, word-level embedding matrix, and term-level embedding matrix corresponding to the same position are concatenated. Then, a gate vector is calculated using a linear transformation and the sigmoid activation function. The gate vector can be used to dynamically adjust the contribution weights of embeddings of different granularities in subsequent fusion. The calculation formula is as follows:

[0149] ,

[0150] in, Indicates the first The character-level embedding vector at position i represents the character-level embedding vector at position j in the sequence. Embedding of one character; Indicates the first The word-level embedding vector at the i-th position represents the word-level embedding vector at the i-th position in the sequence. The embedding of each word needs to be aligned to the character position; Indicates the location The term-level embedding vector represents the first term in the sequence. The embedding of each term must be aligned to the character position; This represents a vector concatenation operation, which concatenates three embedded vectors into a single vector. The concatenation dimension is the sum of the dimensions of the three embedded vectors. This is the gated weight matrix, used for linear transformation; This is the gated bias vector; The Sigmoid activation function has an output range of 1. , representing the weights of embedding at different granularities; For the first Gating vectors at each position;

[0151] 5) Computational fusion embedding:

[0152] The word-level embedding matrix and the term-level embedding matrix are respectively projected into the same dimensional space as the character-level embedding matrix. Then, the projected word-level embedding, the projected term-level embedding, and the character-level embedding corresponding to the same position are concatenated, and the concatenated vector is weighted using the gating vector at that position to obtain the fused embedding vector at that position. , denotes the fused embedding vector at the -th position, which can effectively integrate the semantic information at the character, word, and term granularities. The calculation formula is as follows:

[0153] ,

[0154] where, denotes the word-level projection matrix, which projects the word-level embedding vector into a space with the same dimension as the character-level embedding vector, facilitating subsequent fusion operations; denotes the term-level projection matrix, which projects the term-level embedding vector into a space with the same dimension as the character-level embedding vector, facilitating subsequent fusion operations; is the element-wise multiplication;

[0155] It should be noted that the mapping methods of the character embedding function and the word embedding function are implemented using pre-trained models. For example, the character embedding function uses a pre-trained BERT model to map each character into a dense vector space of a fixed dimension. For the character "人", it may be mapped to a 10-dimensional vector, which can capture the semantic features of the character "人". The term injects domain prior knowledge through a term library and combines a gating mechanism to dynamically allocate the weights of the character, word, and term-level embeddings, which can effectively avoid semantic fragmentation caused by word segmentation and improve the modeling ability for professional terms and long-tail expressions. In addition, the term represents the projected word embedding vector, the

[0156] term represents the projected term embedding vector, and the

[0157] term represents the concatenated vector integrating multi-granularity information, thereby achieving the dynamic fusion of character, word, and term three-level semantics and avoiding the problem of semantic fragmentation caused by the fragmentation of professional terms.

[0158] In the specific implementation manner, the specific process of the relative position encoding operation in the position encoding layer of the encoder is as follows:

[0157] There are long-range dependency relationships across paragraphs in social security policy texts, such as reimbursement conditions scattered in multiple chapters. Conventional absolute position encoding is difficult to model the long-distance position correlation. Therefore, the present invention adopts a semantic-aware relative position encoding mechanism, which can make position pairs with strong correlation maintain high attention weights even at long distances. The specific steps are as follows:

[0158] 1) Calculate the semantic decay coefficient:

[0159] The fused embedding vectors from the two positions are concatenated, and then a semantic decay coefficient is calculated using a linear transformation and activation function to reflect the semantic correlation between the two positions. When the semantic correlation is high, the decay coefficient approaches zero. The semantic decay coefficient can dynamically adjust the intensity of decay at subsequent positions. The calculation formula is as follows:

[0160] ,

[0161] in, Indicates the first The fused embedding vector at position i represents the th position in the sequence. Semantic features of each position; Indicates the first The fused embedding vector at position i represents the th position in the sequence. Semantic features of each position; This represents a vector concatenation operation, which concatenates two vectors into a single vector. The semantic decay weight matrix represents the weights used to calculate semantic relevance. This indicates a modified linear unit activation function. This indicates that the output range of the modified linear unit activation function is... This ensures that the attenuation coefficient is non-negative. Indicates the first The position and the first The semantic attenuation coefficient between positions can be used to dynamically adjust the position attenuation intensity. The position and the first When the meanings of two positions are highly related, such as "years of contribution" and "retirement age" being highly related, Approaching 0;

[0162] 2) Calculate the relative position offset:

[0163] Multiplying the semantic attenuation coefficient between two positions by their relative position distance and taking the negative value yields the relative position bias value, which in turn produces the relative position bias matrix. This matrix can be used to adjust the correlation between two positions in attention calculations. The formula for calculating the relative position bias value is as follows:

[0164] ,

[0165] in, Indicates the first The position and the first The relative distance between two positions can be represented, for example, by the absolute value of the index difference between the two positions; Indicates the first The position to the first The relative position offset of each position is a scalar;

[0166] It should be noted that the relative position offset value During the calculation process, The negative sign before an item controls the direction of relevance, indicating that the greater the distance or the lower the semantic relevance, the stronger the attention weight decay. That is, when the semantics are highly relevant... Approaching 0, and Approaching 0 to reduce the position decay effect;

[0167] 3) Generate semantic awareness attention scores:

[0168] The query matrix and key matrix of the input sequence are obtained through linear transformation, and the standard attention score matrix is ​​calculated. Then, the calculated relative position bias matrix is ​​added to the standard attention score matrix to obtain the semantic-aware relative position attention score matrix. The calculation formula is as follows:

[0169] ,

[0170] in, The query matrix is ​​obtained from the input sequence through a linear transformation; The key matrix is ​​obtained from the input sequence through a linear transformation. Key matrix transpose; This is the relative position offset matrix. The relative position offset matrix is ​​the first Line number Column elements; This is the relative position attention score matrix;

[0171] It should be noted that the fusion embedding vector is defined as... That is, the first The fused embedding vector at each position It is a fused embedding vector The The elements are fused with the embedded vector as the input sequence, and the query matrix is ​​obtained through two different linear transformation layers. Bond matrix ,and The term is a standard attention score matrix, used to calculate the correlation between any two positions.

[0172] In a specific implementation, the process of performing hierarchical coefficient attention enhancement operation on the encoder's attention layer is as follows:

[0173] Because social security policies have a chapter-section-clause hierarchical structure, user problems often require cross-level reasoning. For example, cross-regional medical insurance reimbursement involves multiple policy chapters. Conventional global attention calculations are computationally expensive and prone to noise, while fixed-pattern sparse attention strategies using local windows cannot adapt to the hierarchical logical chain of policies. Therefore, this invention constructs a dynamic attention mask based on the policy structure to reduce computational cost and maintain the integrity of the policy logical chain. The specific steps are as follows:

[0174] 1) Hierarchical boundary resolution:

[0175] The hierarchical boundaries of policy texts can be automatically identified by matching the numbered sections of the policy text using regular expressions. The numbering system is represented as "Chapter No. {Chapter No.} Section No. {Article No.}", and the regular expression that matches the numbering system is "Chapter No. (\d+) Section No. (\d+) Article". (\d+) is a regular expression pattern that matches one or more numeric characters, (\d) matches the digits 0-9, and (+) indicates one or more repetitions. This can automatically identify the start and end indexes of chapters, sections, and articles without manual annotation.

[0176] It should be noted that the purpose of hierarchical boundary parsing is to automatically identify the boundary position indexes of chapters, sections, and clauses in policy texts, provide a basis for hierarchical division for the construction of structured attention masks, and ensure that structured attention masks can accurately reflect the hierarchical structure of policy documents.

[0177] 2) Structured attention mask construction:

[0178] Based on the hierarchical structure of the policy text, a binary mask matrix is ​​generated for each level. The chapter-level mask stipulates that attention connections can only be established between positions belonging to the same chapter; the section-level mask stipulates that connections can only be established between positions belonging to the same section; and the clause-level mask stipulates that connections can only be established between positions belonging to the same clause. The three level masks are then added together to obtain the structured attention mask. In attention calculation, connections are only allowed between positions within the same level. The calculation formula is as follows:

[0179] ,

[0180] in, For structured attention masks;

[0181] This represents a chapter-level mask, defining the attention connectivity between chapters. The mask logic rules are as follows: , For chapter-level masking Line number Column elements;

[0182] This represents a section-level mask, defining the attention connectivity within a section. The mask logic rules are as follows: , For the clause-level mask Line number Column elements;

[0183] This represents a clause-level mask, defining the attention connectivity within a clause. The mask logic rules are as follows: , For the clause-level mask Line number Column elements;

[0184] 3) Sparse attention computation:

[0185] The structured attention mask is added to the scaled standard attention score matrix, and then the sparse attention weight matrix is ​​calculated using the Softmax function. The sparse attention weight matrix has non-zero weights only between pairs of positions at the same level allowed by the structured attention mask. The calculation formula is as follows:

[0186] ,

[0187] in, This represents the dimension of the key vector, which determines the size of the vector dimension in the attention mechanism. For example, when setting it, it is assigned a value based on the quotient of the hidden layer dimension of the model divided by the number of attention heads. For the Softmax function; It is a sparse attention weight matrix, which only retains non-zero weights within the same level.

[0188] It should be noted that the structured attention mask utilizes the inherent chapter-section-clause hierarchy of policy documents to achieve structural prior guidance, constrain the scope of attention, and enable the model to focus more on the content of relevant levels, thereby improving the understanding and modeling ability of policy logic. For example, "reimbursement ratio" only focuses on the "medical insurance benefits" chapter, thereby suppressing cross-chapter noise interference.

[0189] When calculating the sparse attention weight matrix, "same level" refers to the same chapter, section, or clause. During the calculation, only the weights between positions belonging to the same level are retained, while the weights of other positions across levels are set to zero to reduce the amount of computation and avoid noise interference from across levels.

[0190] When calculating the sparse attention weight matrix, the policy condition logic chain is fragmented by avoiding the fixed pattern of the sliding window, so as to ensure semantic integrity. For example, the logical relationship of "continuous payment ≥ 15 years" needs to be fully preserved. Through dynamic attention mask, the model can maintain the integrity of these condition logic chains, so as to understand the text that conforms to the policy logic more accurately.

[0191] To preserve non-zero weights within the same level, when calculating the sparse attention weight matrix, when the first... The position and the first When the positions belong to the same level ,otherwise, Therefore, in the calculation process of the Softmax function, When used as input to the Softmax function, it causes the output weight at the corresponding position to approach 0, thus retaining only the non-zero weights within the same level, while suppressing cross-level weights.

[0192] In a specific implementation, the specific process of feature output operation in the encoder's output layer is as follows:

[0193] Because social security policy texts contain complex and diverse semantic information, as well as rich hierarchical structures, conventional methods often focus only on single feature extraction or simple feature combinations, failing to effectively integrate location information and hierarchical structure information. Therefore, this invention integrates the outputs of relative positional attention and hierarchical sparse attention, aggregating location information and hierarchical structure information to obtain the encoder output matrix. The specific steps are as follows:

[0194] 1) Attention mechanism fusion:

[0195] The relative position attention score matrix and the hierarchical sparse attention weight matrix are concatenated along the feature dimension. An attention gating vector is then calculated using a linear transformation and the sigmoid activation function. Finally, the attention gating vector is used to perform a weighted summation of the relative position attention score matrix and the hierarchical sparse attention weight matrix to obtain the fused attention weight matrix. The calculation formula is as follows:

[0196] ,

[0197] ,

[0198] in, This indicates that the attention mechanism fuses the weight matrix for linear transformation. This represents the attention mechanism fusion bias vector; This represents the relative position attention score matrix. Sparse attention weight matrix Concatenate along the feature dimension; This represents the fused attention weight matrix, expressed as attention scores based on relative positions. and hierarchical sparse attention weights We obtain the attention score based on the dimension and relative position. same; The attention-gated vector is output through the Sigmoid function, with a range of... Dynamically adjust relative position attention score and hierarchical sparse attention weights Contribution weight;

[0199] 2) Context representation generation:

[0200] The attention value matrix is ​​obtained by linearly transforming the fused embedding vector and the projection weight matrix. Then, the attention value matrix is ​​weighted and summed using the fused attention weight matrix, and then added to the fused embedding vector. Finally, the sum is normalized to obtain the context representation matrix output by the encoder, which can effectively integrate relative position information and hierarchical structure information. The calculation formula is as follows:

[0201] ,

[0202] in, Represents the attention value matrix. , This represents the attention value projection weight matrix, used for linear transformation; The context representation matrix representing the encoder output integrates and fuses attention and location information; Represents the fused embedding vector; Presentation layer normalization operation.

[0203] In a specific implementation, the specific process of feature input operation at the input layer of the decoder is as follows:

[0204] The social security question-and-answer task requires the model to accurately understand the input question and generate an answer that conforms to policy logic. Conventional methods, when fusing the encoder output with the target sequence, often use simple concatenation or weighted summation, which cannot fully capture the complex semantic relationship between the input and the target sequence. Therefore, this invention fuses the contextual representation of the encoder output with the target sequence embedding, thereby establishing a semantic relationship between the input and the target sequence. The specific steps are as follows:

[0205] 1) Target sequence embedding:

[0206] The character embedding function maps each character of the target sequence to a dense vector, generating a character-level embedding matrix. Then, the positional embedding is calculated using a positional encoding function and superimposed on the character-level embedding to construct the decoder input embedding matrix. The calculation formula is as follows:

[0207] ,

[0208] in, Represents the target sequence; This represents the decoder input embedding matrix; This represents a character embedding function that maps each character to a dense vector; This represents the absolute position encoding function, which is specifically implemented by calculating the position embedding using sine and cosine functions.

[0209] 2) Mask self-attention calculation:

[0210] The decoder input embedding matrix is ​​processed using a masked multi-head self-attention mechanism. By using a lower triangular mask matrix to restrict each position to only focus on itself and its preceding positions, information leakage can be prevented, and the internal dependencies of the target sequence can be effectively captured. The calculation formula is as follows:

[0211] ,

[0212] in, This indicates the operation of the mask multi-head self-attention mechanism; The masked self-attention hidden state matrix can capture the internal dependencies of the target sequence;

[0213] It should be noted that, in implementing the masked multi-head self-attention mechanism, a lower triangular mask matrix is ​​added to the standard multi-head attention mechanism. And the lower triangular mask matrix The Line number The elements of the column are When the first The row index value of the row is greater than or equal to the first row. When using the column index value of a column, ,otherwise This ensures that each position of the decoder can only focus on itself and the positions before it, preventing information leakage;

[0214] 3) Encoder-decoder attention fusion:

[0215] Attention fusion is performed based on the standard multi-head attention mechanism. The standard multi-head attention mechanism uses the masked self-attention hidden state as the query matrix, and the context representation output by the encoder as the key and value matrices. Cross-attention weights are calculated, and encoder and decoder information is fused using a weighted summation method to generate the attention hidden state matrix. This enables semantic association between the input sequence and the target sequence. The calculation formula is as follows:

[0216] ,

[0217] in, This represents the standard multi-head attention mechanism; This represents the attention hidden state matrix.

[0218] In a specific implementation, the specific process of the logic enhancement layer of the decoder performing logic enhancement dynamic constraint operations is as follows:

[0219] The answers to social security questions must strictly adhere to policy rules. Because conventional decoders lack hard constraint mechanisms, injecting soft rules can easily lead to logical contradictions. Therefore, this invention injects hard constraints through differentiable logic reasoning units, ensuring the output conforms to policy logic. The specific steps are as follows:

[0220] 1) Construction of the logical rule base:

[0221] A set of logical rules is predefined from the policy text to form a logical rule base. Each rule consists of an antecedent predicate logical expression about the input sequence and a consequent predicate logical expression about the output sequence. The two are logically implied. The logical rule base is represented as follows:

[0222] ,

[0223] in, Represents a logical rule base, containing Rules; Indicates the first Rule identifier; The antecedent of the rule is about the input sequence. The predicate logic expression; The consequent of the rule is about the output sequence. The predicate logic expression; This indicates a logical implication relationship; Input the sequence, i.e., the policy text, to the encoder; The decoder outputs a sequence, which is the answer text.

[0224] 2) Logical attention calculation:

[0225] Based on the hidden state of the decoder at the current time step, the attention weight of each rule in the logical rule base at that time step is calculated through linear transformation, hyperbolic tangent function, and dot product operation of the rule attention vector. This weight can be used to dynamically filter the subset of rules most relevant to the current generation context. The calculation formula is as follows:

[0226] ,

[0227] in, Indicates time step On the rules Attention weights; Indicates the decoder at time step The hidden states are the attention hidden state matrix. The OK; Represents the logical attention weight matrix; Represents the implicit dimension of the rule, and represents the dimension of the implicit representation in the rule attention mechanism. (Setting...) ; Indicates the first Rule identifier Attention vector; express transpose; Represents the natural exponential function; Represents the hyperbolic tangent function; Indicates the first Rule identifier Attention vector; Indicates the first Rule identifier; express transpose;

[0228] 3) Generation of differentiable logic constraints:

[0229] The encoder's context representation is processed using an antecedent function. The probability of satisfying the antecedent of each rule in the logical rule base is calculated. Then, this probability of satisfaction is multiplied element-wise by the embedding vector of the corresponding rule's consequent. The results are then weighted and summed using the rule attention weights calculated at the current time step to obtain the logical constraint vector for that time step. This transforms the symbolic logical rules into differentiable constraint representations. The calculation formula is as follows:

[0230] ,

[0231] in, For time step The constraint vector; The antecedent function transforms the symbolic logic expression of the rule antecedent into a differentiable operation, specifically implemented using a multilayer perceptron. The input is the context representation matrix output by the encoder. The output is a scalar value representing the probability that the antecedent is satisfied, with a range of 1 / 2π. ; As the consequent embedding vector, the rule consequent is mapped to a vector of fixed dimension;

[0232] 4) Enhanced gated residuals:

[0233] The hidden state at the current time step of the decoder is concatenated with the logical constraint vector. A residual enhancement gating vector is then calculated using a linear transformation and the sigmoid activation function. This logical constraint vector is then projected into the hidden space and weighted using the residual enhancement gating vector. Finally, the weighted constraint vector is added to the original hidden state as a residual to obtain the logical enhancement hidden state matrix. This allows for dynamic adjustment of the injection strength of constraint information. The calculation formula is as follows:

[0234] ,

[0235] ,

[0236] in, This represents the residual enhancement gated weight matrix, used for linear transformation; Represents the residual enhancement gated bias vector; Indicates time step The logic-enhanced hidden state matrix integrates constraint information; Indicates time step The residual enhancement gated vector is output through the Sigmoid activation function, with a range of... Dynamically adjust the constraint strength; This represents the constraint projection weight matrix, which projects the constraint vectors... Projected into the hidden space;

[0237] It should be noted that in the process of generating differentiable logic constraints, the symbolic rules are transformed into differentiable operations through an antecedent function, achieving soft injection of hard constraints. The antecedent function uses a neural network to approximate logical operations, outputting the probability of satisfying the rule antecedent, ensuring gradient propagation during training, thereby enforcing the policy rules when generating the answer; dynamic attention filtering uses attention weights. Calculated based on the current hidden state, according to the hidden state. Selecting a subset of relevant rules avoids interference from irrelevant constraints, allowing high-weight rules to be activated and low-weight rules to be suppressed, thus dynamically adapting to the context of the generation step; the gated residual augmentation operation enhances the gated vector through residual augmentation. Adaptive adjustment of constraint strength to ensure compatibility with contextual semantics; residual enhancement gating vector. As a Sigmoid gate vector, a weighted constraint vector The residual connection, when constraints conflict with semantics, the residual enhancement gating vector Approaching 0 weakens injection, while increasing it strengthens it, achieving flexible integration of hard constraints; the logically enhanced hidden state vector is obtained by stacking the logically enhanced hidden states of all time steps. This serves as the input to the normalization layer to maintain the integrity of the sequence structure.

[0238] In a specific implementation, the layer normalization process performed by the encoder's normalization layer is as follows:

[0239] A layer normalization operation is performed on the logistic augmentation hidden state matrix. This normalization process adjusts the distribution of the hidden states, stabilizes the training process, and enhances the gradient flow, generating a layer-normalized hidden state matrix. The calculation formula is as follows:

[0240] ,

[0241] in, The hidden state matrix of the representation layer is normalized, which stabilizes the training process and improves the gradient flow. Presentation layer normalization operation.

[0242] In a specific implementation, the output layer of the decoder performs the answer sequence output operation as follows:

[0243] The normalized hidden state matrix is ​​mapped to the vocabulary space to generate an unnormalized score vector, which is then converted into a word probability distribution using the Softmax function. The specific steps are as follows:

[0244] 1) Score vector calculation:

[0245] By outputting the projection weight matrix and bias vector, a linear transformation is performed on the layer-normalized hidden state vector, mapping the hidden state to the vocabulary space to generate an unnormalized score vector. The calculation formula is as follows:

[0246] ,

[0247] in, This represents the output projection weight matrix, which maps the hidden states to the vocabulary space. This represents the output projection bias vector; Indicates time step The unnormalized score vector; Indicates time step The layer-normalized hidden state vector;

[0248] 2) Probability distribution calculation:

[0249] The Softmax function is used to process the unnormalized score vector, and the word probability distribution at the current time step is calculated. This distribution represents the probability of each candidate word being generated given the input sequence and the generated sequences, thus guiding the sequence output. The formula for calculating the word probability distribution is as follows:

[0250] ,

[0251] In the formula, Represents a given input sequence and generated sequences Current word under conditions The probability distribution; Indicates time step Words to be generated; Indicates time step The previously generated word sequence; Indicates the encoder input sequence;

[0252] 3) Sequence generation:

[0253] a) If in the training phase:

[0254] The cross-entropy loss is calculated based on the word probability distribution and the true label. The model parameters are then optimized using the cross-entropy loss function, which is calculated as follows:

[0255] ,

[0256] in, Indicates time step The true label, i.e., the target word; Indicates the length of the target sequence; Represents cross-entropy loss; This indicates that the model performs well under given input. and the true historical sequence Predicting real words The probability of; Indicates time step The previous true historical sequence; This represents the logarithmic function, with a default base of 10.

[0257] b) If in the reasoning stage:

[0258] The sequence is generated step by step by selecting the word with the highest probability using an autoregressive method until the end marker [SEP] is output, thus completing the generation of the answer sequence. The calculation formula is as follows:

[0259] ,

[0260] in, This is a maximum value indexing operation, which selects the word with the highest probability as the current output.

[0261] In a specific implementation, the model training process is as follows:

[0262] S4.1 Define the loss function:

[0263] A multi-task loss function is defined, comprising a weighted average of cross-entropy loss, logical rule loss, and boundary alignment loss. Cross-entropy loss optimizes the generation quality; logical rule loss ensures the output meets the hard logical constraints of the policy rule base; boundary alignment loss forces key policy values ​​to remain within a preset reasonable range; and the multi-task loss function is used in conjunction to improve the semantic accuracy and logical consistency of the answer. The calculation formula is as follows:

[0264] ,

[0265] in, For multi-task loss functions; These are the weighting coefficients for the logical rule loss; Loss due to logical rules; These are the weighting coefficients for the boundary alignment loss; For boundary alignment loss;

[0266] The logical rule loss is defined based on each rule in the logical rule base. The scores for satisfied and violated rules are calculated, and the logical loss component for each rule is obtained using a logarithmic function. The average of all rules is then used to generate the final logical rule loss value. This value quantifies the degree to which the model's output sequence violates the hard logical constraints in the logical rule base. The calculation formula is as follows:

[0267] ,

[0268] in, Loss due to logical rules; This represents the total number of rules in the rule base. For rules The satisfied fraction represents the degree of satisfaction of differentiable logic; For rules The score that is violated represents the degree of violation of differentiable logic;

[0269] The system extracts values ​​from a pre-defined set of key values ​​from the model output sequence and determines whether they exceed a pre-defined reasonable range. This forces the key policy values ​​to remain within the pre-defined reasonable range, preventing the generation of numerical results that are detached from the actual policy. The calculation formula is as follows:

[0270] ,

[0271] in, For boundary alignment loss; This represents the numerical values ​​extracted from the output sequence. It belongs to the preset set of key values. For example, numerical values It refers to the number of years of contributions; This indicates the numerical values ​​extracted from the output sequence. Less than the lower bound The penalty items at that time; This indicates the numerical values ​​extracted from the output sequence. Greater than the upper bound The penalty items at that time;

[0272] It should be noted that the logical rule loss is based on the rule base to calculate the satisfaction and violation of rules, ensuring that the answer meets the antecedent-consequence logical relationship of the social security policy; the boundary alignment loss is determined by extracting key values ​​of the output sequence to determine whether it exceeds the boundary. For each value, if it is lower than the preset lower boundary, the lower boundary difference is calculated; if it is higher than the preset upper boundary, the upper boundary difference is calculated. Finally, the differences of all out-of-bounds values ​​are summed to constrain the values ​​to be within the reasonable range of the policy.

[0273] S4.2 Define the model parameter update method:

[0274] An adaptive moment estimation optimizer was used to update the model parameters, with an initial learning rate of 0.0001, which was decayed to 90% of the original value every 10 training epochs.

[0275] The gradients of cross-entropy loss, logistic rule loss and boundary alignment loss in the multi-task loss function are calculated independently, and a gradient pruning strategy is implemented. When the gradient L2 norm exceeds the threshold of 1.0, scaling is performed.

[0276] Based on the model update method, the standard error backpropagation algorithm is adopted. 32 question-answer pairs are randomly sampled from the training set as a mini-batch. The multi-task loss function value is calculated through forward propagation, and the gradient of each parameter is obtained through backpropagation. After gradient clipping, the Adam optimizer parameter update rule is executed, and this process is repeated iteratively.

[0277] S4.3 Define the stopping condition for iterative training:

[0278] The main stopping condition of the model is to trigger early stopping when the value of the multi-task loss function on the validation set decreases by less than 0.1% for 8 consecutive epochs.

[0279] The maximum training iteration epoch is set to 100 times. Each iteration epoch saves a snapshot of the parameters of the top 3 models with the highest accuracy on the validation set. Finally, the model with the highest policy logic consistency score on the validation set is selected as the optimal model for deployment.

[0280] Example 2

[0281] To compare the overall performance of different models on the social security question-answering task, the model of this invention is compared with the traditional bidirectional long short-term memory network, the standard transformer model, and the basic version of the language representation model based on bidirectional transformer pre-training. The characteristics of each model are shown in Table 2.

[0282] Table 2 Performance Comparison of the Invention Model and Existing Models

[0283]

[0284] The test set uses real user inquiries collected from the edge terminal. Examples of the test set are shown in Table 3.

[0285] Table 3. Sample Test Set

[0286]

[0287] The evaluation metric is accuracy, and the calculation method is as follows: ;

[0288] Depend on Figure 2 The experimental results show that the model of this invention achieves a significantly higher accuracy than existing models (standard neural network Transformer model, BiLSTM bidirectional long short-term memory model, BERT-base model), indicating that the multi-granularity embedding mechanism fully preserves the semantics of technical terms, enabling the model to answer practical questions more accurately.

[0289] Example 3

[0290] The role of each component in the multi-granularity embedded architecture was analyzed using training curves. A systematic ablation study was conducted, and the configuration of the ablation comparison experiment is shown in Table 4.

[0291] Table 4 Ablation Comparison Experiment Configuration Table

[0292]

[0293] The validation set features, proportions, and validation starting points are set as shown in Table 5.

[0294] Table 5 Validation Set Settings

[0295]

[0296] The experiment compares the performance differences between the complete model and simplified versions using missing term embeddings, missing term gating mechanisms, and character embeddings only. Figure 3 The experimental results show that the complete model is significantly better than other configurations in terms of training efficiency and final accuracy. The domain knowledge directly injected into the terminology database and the three-level embedding weights dynamically adjusted by the gating mechanism effectively solve the problem of compound terms being incorrectly fragmented, enabling the model to grasp the deep semantics of policy texts more quickly.

[0297] Example 4

[0298] Heatmaps were used to assess the impact of different constraint mechanisms on the ability to comply with policy rules. The experiment focused on five core social security rules, including eligibility determination, amount calculation, and time limit requirements. The performance of the unconstrained model, the soft constraint model, and the hard constraint mechanism of this invention were compared. The rule category comparison settings are shown in Table 6.

[0299] Table 6. Rule Category Comparison Settings

[0300]

[0301] The logical consistency score is evaluated as follows: ;

[0302] like Figure 4 The experimental results show that the model of this invention maintains excellent logical consistency across all rule categories. For example, when dealing with pension calculation problems, this unit can force the model to output a policy formula that conforms to "basic pension + personal account pension" and automatically focus on relevant rule subsets through dynamic rule attention to avoid interference from irrelevant rules such as "time limit for applying for maternity allowance".

[0303] Example 5

[0304] The efficiency advantage of the model in processing long texts was verified by comparing inference time. The inference delay under social security policy texts of different lengths was measured in the experiment. The efficiency comparison dimensions are shown in Table 6.

[0305] Table 6. Efficiency Comparison of the Model of this Invention with Other Models

[0306]

[0307] The test environment configuration is shown in Table 7.

[0308] Table 7 Test Environment Configuration Table

[0309]

[0310] Depend on Figure 5 The experimental results show that, compared with existing models (standard neural network Transformer model, BiLSTM bidirectional long short-term memory model, BERT-base model), the model of this invention maintains high accuracy, and in particular, it significantly outperforms traditional models in processing long texts such as "medical insurance reimbursement policy" with 500 characters. This indicates that the hierarchical sparse attention mechanism, based on the dynamic attention mask constructed from the policy chapter structure, greatly reduces redundant computation and achieves the best balance between efficiency and performance.

[0311] Although the specific embodiments of the invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the invention. Based on the technical solutions of the invention, various modifications or variations that can be made by those skilled in the art without creative effort are still within the scope of protection of the invention.

Claims

1. A method for processing and building models of social security digital employee data based on cloud-edge collaboration, characterized in that, Includes the following steps: S1. Collect multi-source social security data collaboratively through cloud platform and edge computing nodes, and synchronize the collected data to cloud structured data tables incrementally on a regular basis through API interface. Fill the missing insurance years field with the average data of adjacent regions of the same insurance type; use distributed crawler to crawl the updated policy interpretation documents of government websites in real time, and parse the policy text according to the "chapter-section-clause" level. S2. Based on the collected multi-source social security data and policy texts, training samples are constructed. The sample generation logic adopts the question-answer pair extraction method, parses the "policy clause-applicable conditions-implementation standard" triplet, generates standard question-answer templates, and divides the training samples into training set, validation set and test set according to the proportion. S3. A social security digital employee question-answering model is constructed using a neural network architecture based on encoders and decoders. The model has an encoder-decoder structure. The encoder includes an input layer, a position encoding layer, an attention layer, and an output layer. The decoder includes an input layer, a logic enhancement layer, a normalization layer, and an output layer. The training set is input into the model and processed by the encoder and decoder to generate a sequence of answers. The specific process of multi-granularity semantic fusion operation at the encoder's input layer is as follows: For the input text Perform character-level, word-level, and term-level embedding fusion. This represents the input text, which contains a question and answer, and is represented as [CLS]question[SEP]answer[SEP]. [CLS] represents the start of the sequence, and [SEP] represents the separator. The specific steps are as follows: 1) Calculate character-level embeddings: Based on the input text, each character is mapped to a dense vector using a character embedding function, generating a character-level embedding matrix; 2) Calculate word-level embeddings: The input text is segmented to obtain a word sequence, and then each word sequence is mapped to a dense vector through a word embedding function to generate a word-level embedding matrix. 3) Computational term-level embeddings: Based on the social security terminology database, the system identifies professional terms in the input text using a terminology lookup function and directly retrieves the corresponding term embedding vectors from the terminology database to generate a term-level embedding matrix. 4) Calculate the gating vector: The character-level embedding matrix, word-level embedding matrix, and term-level embedding matrix corresponding to the same position are concatenated, and then the gate vector is calculated through linear transformation and Sigmoid activation function. 5) Computational fusion embedding: The word-level embedding matrix and the term-level embedding matrix are projected onto the same dimensional space as the character-level embedding matrix. Then, the projected word-level embedding, projected term-level embedding, and character-level embedding corresponding to the same position are concatenated. The gate vector at that position is used to weight the concatenated vector to obtain the fused embedding vector at that position. , Indicates the first The fused embedding vector at each position; The specific process of relative position encoding operation in the encoder's position encoding layer is as follows: A semantically aware relative positional encoding mechanism is adopted, and the specific steps are as follows: 1) Calculate the semantic decay coefficient: The fused embedding vectors of the two positions are concatenated, and then the semantic decay coefficient is calculated through linear transformation and activation function to reflect the degree of semantic association between the two positions. When the semantic association is high, the decay coefficient approaches zero. 2) Calculate the relative position offset: Multiply the semantic attenuation coefficient between two positions by their relative position distance and take the negative value to obtain the relative position bias value, and then obtain the relative position bias matrix. 3) Generate semantic awareness attention scores: The embedded vectors are fused as the input sequence. The query matrix and key matrix of the input sequence are obtained through linear transformation. The standard attention score matrix is ​​calculated. Then, the calculated relative position bias matrix is ​​added to the standard attention score matrix to obtain the semantic-aware relative position attention score matrix. S4. Define a loss function to train the model and update the model parameters. The model is trained iteratively multiple times until the preset iteration stopping condition is met, at which point the model ends the training. The model with the best performance is selected as the optimal model through the validation set. S5. Input the test set into the optimal model, perform social security digital employee Q&A, and generate an answer sequence.

2. The method for processing and building models of social security digital employee data based on cloud-edge collaboration as described in claim 1, characterized in that, The specific process of performing hierarchical coefficient attention enhancement operation on the encoder's attention layer is as follows: To reduce computational cost and maintain the integrity of the policy logic chain, a dynamic attention mask is constructed based on the policy structure. The specific steps are as follows: 1) Hierarchical boundary resolution: The hierarchical boundaries of policy texts are automatically identified by matching the numbered parts of the policy text using regular expressions. The numbering system is represented as "Chapter {Chapter {Section {Chapter ... 2) Structured attention mask construction: Based on the hierarchical structure of the policy text, a binary mask matrix is ​​generated for each level. The chapter-level mask stipulates that attention connections are only established between positions belonging to the same chapter; the section-level mask stipulates that connections are only established between positions belonging to the same section; and the clause-level mask stipulates that connections are only established between positions belonging to the same clause. The three level masks are then added together to obtain the structured attention mask. During attention calculation, connections are only allowed between positions within the same level. The calculation formula is as follows: ， in, Represents a structured attention mask; This represents a chapter-level mask, and the mask logic rules are as follows: , Indicates the chapter-level mask. Line number Column elements; This represents a section-level mask, and the mask logic rules are as follows: , Indicates the clause-level mask. Line number Column elements; This represents a clause-level mask, and the mask logic rules are as follows: , Indicates the clause-level mask. Line number Column elements; 3) Sparse attention computation: The structured attention mask is added to the scaled standard attention score matrix, and then the sparse attention weight matrix is ​​calculated using the Softmax function. The sparse attention weight matrix has non-zero weights only between the same level position pairs allowed by the structured attention mask.

3. The method for processing and building models of social security digital employee data based on cloud-edge collaboration as described in claim 2, characterized in that, The specific process of feature output operation in the encoder's output layer is as follows: The outputs of relative position attention and hierarchical sparse attention are integrated, and the positional information and hierarchical structure information are aggregated to obtain the encoder output matrix. The specific steps are as follows: 1) Attention mechanism fusion: The relative position attention score matrix and the hierarchical sparse attention weight matrix are concatenated along the feature dimension. The attention gating vector is calculated by linear transformation and sigmoid activation function. Then, the attention gating vector is used to perform a weighted summation of the relative position attention score matrix and the hierarchical sparse attention weight matrix to obtain the fused attention weight matrix. 2) Context representation generation: The attention value matrix is ​​obtained by a linear transformation that fuses the embedding vector and the projected weight matrix. The calculation formula is as follows: ， in, Represents the attention value matrix. Represents the fused embedding vector. This represents the attention value projection weight matrix, used for linear transformation; Then, the attention value matrix is ​​weighted and summed using the fused attention weight matrix, and then added to the fused embedding vector. Finally, the sum is normalized to obtain the context representation matrix output by the encoder.

4. The method for processing and building models of social security digital employee data based on cloud-edge collaboration according to claim 3, characterized in that, The specific process of feature input operation at the input layer of the decoder is as follows: The contextual representation output by the encoder is fused with the target sequence embedding, thereby semantically associating the input with the target sequence. The specific steps are as follows: 1) Target sequence embedding: The character embedding function is used to map each character of the target sequence to a dense vector to generate a character-level embedding matrix. Then, the positional embedding is calculated by the positional encoding function and superimposed on the character-level embedding to construct the decoder input embedding matrix. 2) Mask self-attention calculation: The decoder input embedding matrix is ​​processed using a masked multi-head self-attention mechanism, and the lower triangular mask matrix is ​​used to restrict each position to only pay attention to itself and the positions before it. 3) Encoder-decoder attention fusion: Attention fusion is performed based on the standard multi-head attention mechanism. The standard multi-head attention mechanism uses the masked self-attention hidden state as the query matrix, the context representation output by the encoder as the key matrix and value matrix, calculates the cross-attention weights, and fuses the encoder and decoder information through a weighted summation method to generate the attention hidden state matrix.

5. The method for processing and building models of social security digital employee data based on cloud-edge collaboration according to claim 4, characterized in that, The specific process of the logic enhancement layer of the decoder performing logic enhancement dynamic constraint operations is as follows: Hard constraints are injected into differentiable logic reasoning units to ensure that the output conforms to policy logic. The specific steps are as follows: 1) Construction of the logical rule base: A set of logical rules is predefined from the policy text to form a logical rule base. Each rule consists of an antecedent predicate logical expression about the input sequence and a consequent predicate logical expression about the output sequence. The two are logically implied. The logical rule base is represented as follows: ， in, Represents a logical rule base, containing Rules; Indicates the first Rule identifier; The antecedent of the rule is about the input sequence. The predicate logic expression; The consequent of the rule is about the output sequence. The predicate logic expression; This indicates a logical implication relationship; Input the sequence, i.e., the policy text, to the encoder; The decoder outputs a sequence, which is the answer text. 2) Logical attention calculation: Based on the hidden state of the decoder at the current time step, the attention weight of each rule in the logical rule base at that time step is calculated by linear transformation, hyperbolic tangent function and dot product operation of rule attention vector, and used to dynamically filter the subset of rules most relevant to the current generation context. 3) Generation of differentiable logic constraints: The context representation of the encoder is processed using the antecedent function. The satisfaction probability of the antecedent of each rule in the logical rule base is calculated. Then, the satisfaction probability is multiplied element by element with the embedding vector of the corresponding rule. The results are then weighted and summed using the rule attention weights calculated at the current time step to obtain the logical constraint vector at that time step. This transforms the symbolic logical rules into differentiable constraint representations. 4) Enhanced gated residuals: The hidden state at the current time step of the decoder is concatenated with the logical constraint vector. The residual enhancement gating vector is calculated through linear transformation and sigmoid activation function. Then, the logical constraint vector is projected into the hidden space and weighted with the residual enhancement gating vector. Finally, the weighted constraint vector is added to the original hidden state as the residual to obtain the logical enhancement hidden state matrix.

6. The method for processing and building models of social security digital employee data based on cloud-edge collaboration according to claim 5, characterized in that, The specific process of layer normalization processing performed by the encoder's normalization layer is as follows: A layer normalization operation is performed on the logic-enhanced hidden state matrix. The distribution of the hidden states is adjusted through normalization, which stabilizes the training process and enhances the gradient flow, generating a layer-normalized hidden state matrix.

7. The method for processing and building models of social security digital employee data based on cloud-edge collaboration as described in claim 6, characterized in that, The specific process of the decoder's output layer performing the answer sequence output operation is as follows: The normalized hidden state matrix is ​​mapped to the vocabulary space to generate an unnormalized score vector, which is then converted into a word probability distribution using the Softmax function. The specific steps are as follows: 1) Score vector calculation: By outputting the projection weight matrix and bias vector, a linear transformation is performed on the layer normalized hidden state vector to map the hidden state to the vocabulary space and generate an unnormalized score vector. 2) Probability distribution calculation: The Softmax function is used to process the unnormalized score vector, and the word probability distribution at the current time step is calculated. This distribution represents the probability of each candidate word being generated given the input sequence and the generated sequences, thus guiding the sequence output. The formula for calculating the word probability distribution is as follows: ， In the formula, Represents the encoder input sequence and generated sequences Current word under conditions The probability distribution; Indicates time step The unnormalized score vector; Indicates time step Words to be generated; Indicates time step The previously generated word sequence; Indicates the encoder input sequence; 3) Sequence generation: a) If in the training phase: The cross-entropy loss is calculated based on the word probability distribution and the true label. The model parameters are then optimized using the cross-entropy loss function, which is calculated as follows: ， in, Indicates time step The true label; Indicates the length of the target sequence; Represents cross-entropy loss; This indicates that the model performs well under given input. and the true historical sequence Predicting real words The probability of; Indicates time step The previous true historical sequence; This represents the logarithmic function, with a default base of 10. b) If in the reasoning stage: The sequence is generated step by step by selecting the word with the highest probability using an autoregressive method until the end marker [SEP] is output, thus completing the generation of the answer sequence. The calculation formula is as follows: ， in, This is an index operation to retrieve the maximum value.

8. The method for processing and building models of social security digital employee data based on cloud-edge collaboration according to claim 7, characterized in that, The training process of the model is as follows: S4.1 Define the loss function: Define a multi-task loss function, which includes weighted cross-entropy loss, logical rule loss and boundary alignment loss. Optimize the generation quality through cross-entropy loss, ensure that the output meets the hard logical constraints of the policy rule base through logical rule loss, force key policy values ​​to be within a preset reasonable range through boundary alignment loss, and jointly improve the semantic accuracy and logical consistency of the answer through the multi-task loss function. The logical rule loss is defined based on each rule in the logical rule base. The score of the rule being satisfied and the score of the rule being violated are calculated. The logical loss component of each rule is obtained based on the logarithmic function. Then, the average of all rules is taken to finally generate the logical rule loss value, thereby quantifying the degree to which the model output sequence violates the hard logical constraints in the logical rule base of the policy. Extract values ​​from a preset set of key values ​​from the model output sequence, determine whether they exceed a preset reasonable range, and then force the key policy values ​​to be within the preset reasonable range. S4.2 Define the model parameter update method: An adaptive moment estimation optimizer was used to update the model parameters, with an initial learning rate of 0.0001, which was decayed to 90% of the original value every 10 training epochs. The gradients of cross-entropy loss, logistic rule loss and boundary alignment loss in the multi-task loss function are calculated independently, and a gradient pruning strategy is implemented. When the gradient L2 norm exceeds the threshold of 1.0, scaling is performed. Based on the model update method, the standard error backpropagation algorithm is adopted. The multi-task loss function value is calculated through forward propagation, the gradient of each parameter is obtained through backpropagation, and the Adam optimizer parameter update rule is executed after gradient pruning. This process is repeated iteratively. S4.3 Define the stopping condition for iterative training: The main stopping condition of the model is to trigger early stopping when the value of the multi-task loss function on the validation set decreases by less than 0.1% for 8 consecutive epochs. The maximum training iteration epoch is set to 100 times. Each iteration epoch saves a snapshot of the parameters of the top 3 models with the highest accuracy on the validation set. Finally, the model with the highest policy logic consistency score on the validation set is selected as the optimal model for deployment.

Images