A domain classification method for embedding complex legal question and answer in frequency domain

Abstract

This invention discloses a domain classification method for complex legal question-and-answer questions embedded in the frequency domain, comprising: acquiring the original legal question text; concatenating the original legal question text with preset category labels to obtain a joint input sequence; converting the joint input sequence into a frequency domain feature sequence representing the text structure features through a fast Fourier transform; inputting the frequency domain feature sequence into an encoder to extract text feature representations and label feature representations; and calculating the confidence scores of the text feature representations belonging to each category label through a pre-trained scoring function to obtain the sub-case classification results.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and more specifically to a domain classification method for complex legal question answering embedded in the frequency domain. Background Technology

[0002] With the deepening application of artificial intelligence technology in the legal field, legal question-answering systems based on Large Language Models (LLM) have become important tools to assist judges, lawyers, and legal researchers. However, while traditional text embedding methods (such as BERT and Sentence-BERT) can capture local semantics, they lack the ability to model complex patterns such as logical structures and causal relationships in long texts, making it difficult to effectively handle long-range dependencies and logical constraints in legal texts. Complex legal cases have rigorous logical structures and hierarchical expression systems, often involving cross-paragraph and even cross-chapter references and echoes. Taking contract law as an example, the interpretation of a clause may require reference to the general principles in the general provisions, and such long-range dependencies often exceed the effective range of traditional self-attention mechanisms.

[0003] Complex legal question-and-answer sub-case classification refers to the process of understanding complex queries involving multi-domain legal knowledge as a series of sub-cases with clear legal significance using natural language processing technology, and then accurately classifying and reasoning about each sub-case. This chapter will combine text vectorization embedded in the frequency domain and text classification techniques with joint label-text processing to achieve complex legal question-and-answer sub-case classification embedded in the frequency domain. Existing text classification methods treat these texts as linear sequences, ignoring their structured information. In addition, existing multi-label classification methods often employ independent prediction strategies, which leads to gaps in the relationship between labels and text and ignores the rich semantic relationships between legal labels.

[0004] In complex legal cases, the shortcomings of current text classification technology become more apparent, with issues such as limited semantic understanding and broken semantic connections between text tags leading to a decline in classification performance in complex legal scenarios. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and address the problems of complex logical structures and weak label associations in complex legal cases. In the process of domain classification, this invention provides a domain classification method for complex legal questions and answers that embeds the frequency domain. By introducing technologies such as frequency domain and label-text joint processing, large models can perform better when facing the domain classification problem of complex legal questions and answers.

[0006] The objective of this invention is achieved through the following technical solution: A domain classification method for complex legal question answering embedded in the frequency domain, comprising: Obtain the original legal question text, and concatenate the preset category labels with the original legal question text to obtain a joint input sequence; The joint input sequence is converted into a frequency domain feature sequence that characterizes the text structure features by using a fast Fourier transform. The frequency domain feature sequence is input into the encoder to extract text feature representation and label feature representation; The confidence scores of the text feature representations belonging to each category label are calculated using a pre-trained scoring function to obtain the sub-case classification results.

[0007] Furthermore, the specific process of concatenating the preset category labels with the original legal question text to obtain the joint input sequence is as follows: First, match the pre-defined category labels with the original legal issue text. Concatenate into a joint input : ; Where C represents the number of category labels, which is the total number of all possible category labels in the dataset; For combined input After word segmentation, the token sequence is obtained: ; Where L is the sequence length; Each token vector It is converted into a vector through the embedding layer. : ; Where D is the embedding dimension; Stack all the token vectors in order to obtain the joint input sequence X: .

[0008] Furthermore, the step of converting the joint input sequence into a frequency domain feature sequence representing text structural features through Fast Fourier Transform includes: Obtain the initial semantic vector sequence after the embedding layer of the pre-trained model. Where L is the sequence length and D is the dimension; a fast Fourier transform is performed on each feature dimension to obtain the spectrum matrix. : ; Take the amplitude spectrum as the frequency domain feature sequence : .

[0009] Furthermore, the confidence score reflecting the text feature representation belonging to the corresponding category is obtained by calculating the score function using a two-layer feedforward neural network.

[0010] Furthermore, the encoder adopts a uni-encoder architecture, specifically a bidirectional Transformer encoder.

[0011] Furthermore, the classification model that performs the method is pre-trained in the following manner: Construct a total loss function that includes classification loss, frequency domain reconstruction loss, and label relationship modeling loss; Based on the multi-task learning framework, the pre-trained model is fine-tuned end-to-end using the total loss function to obtain the classification model.

[0012] Furthermore, the total loss function is specifically as follows: ; in, The classification loss is calculated using binary cross-entropy; The frequency domain reconstruction loss is calculated using mean square error; Modeling loss for label relationships computed using graph contrastive learning; hyperparameters and The weights used to balance the various loss terms.

[0013] Preferably, the present invention also provides a domain classification device for complex legal question-and-answer formats embedded in the frequency domain, comprising: The embedding module is used to obtain the original legal question text and concatenate the preset category labels with the original legal question text to obtain a joint input sequence; The frequency domain transformation module is used to convert the joint input sequence into a frequency domain feature sequence that characterizes the text structure features through a fast Fourier transform. The joint processing module is used to input the frequency domain feature sequence into the encoder to extract text feature representation and label feature representation; The classification module is used to calculate the confidence scores of text feature representations belonging to each category label using a pre-trained scoring function, and obtain the sub-case classification results.

[0014] Preferably, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the domain classification method for complex legal question answering embedded in the frequency domain.

[0015] Preferably, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the domain classification method for complex legal question answering embedded in the frequency domain.

[0016] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: 1. This invention applies Fast Fourier Transform (FFT) to text vectorization, effectively capturing the logical structure and long-range dependencies of periodic patterns in legal texts, enhancing the model's ability to understand the hidden logical structures in complex legal texts. It overcomes local semantic limitations and strengthens the logical perception within legal texts.

[0017] 2. By adopting a label-text joint encoding (uni-encoder architecture), the disconnect between labels and text in existing multi-label classification is broken, and deep information interaction at three levels—label-text, label-label, and text-text—is realized. This significantly improves the multi-label prediction quality in complex scenarios, realizes deep semantic associations, and enhances classification accuracy.

[0018] 3. It achieved an F1 score of 98.1% on the Legal Split Cases dataset, providing accurate sub-case classification and reasoning for complex legal queries; it has high-precision industrial application value. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the present invention. Detailed Implementation

[0020] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0021] Example 1 See Figure 1 This embodiment provides a domain classification method for complex legal question answering embedded in the frequency domain. First, a pre-trained model is obtained, and a frequency domain transformation and label-text joint processing mechanism are embedded in the pre-trained model. A multi-task learning framework is adopted, and the pre-trained model is trained based on a total loss function that includes classification loss, frequency domain reconstruction loss and label relationship modeling loss to obtain a sub-case classification model. The pre-trained model is an extension of the DeBERTa-v3+BERT architecture, introducing frequency domain transformation and a label-text joint processing mechanism. After fine-tuning, the fltclass classification model is trained to achieve fine-grained classification and sub-case classification of complex legal questions and answers. Specifically, it includes the following steps: First, obtain the original legal question text, and then concatenate the preset category labels with the original legal question text to obtain the joint input sequence; Secondly, the joint input sequence is converted into a frequency domain feature sequence that characterizes the text structure features through Fast Fourier Transform; this process can extract structural features and periodic patterns in legal texts, and enhance the model's ability to perceive long-range dependencies and logical relationships.

[0022] Next, the frequency domain feature sequence is input into the encoder to extract text feature representation and label feature representation. In this embodiment, the encoder adopts a uni-encoder architecture based on GLiClass, specifically a bidirectional Transformer encoder. It is used to concatenate labels and text for joint encoding, realizing information interaction at three levels: label-text, label-label, and text-text, which significantly improves the accuracy and semantic relevance of multi-label classification.

[0023] Finally, the confidence scores of the text feature representations belonging to each category label are calculated using a pre-trained scoring function to obtain the sub-case classification results.

[0024] Preferably, the principle of frequency domain transformation is as follows: The core idea of ​​the Fourier transform can be summarized in one sentence: any complex function or signal can be decomposed into a superposition of a series of simple sine waves (Sin) and cosine waves (Cos) with different frequencies and amplitudes.

[0025] Time Domain: This refers to the most common form of signals we see in everyday life. The horizontal axis represents time, and the vertical axis represents amplitude (intensity). It shows the strength of a signal at a specific point in time, but hides its frequency components.

[0026] Frequency Domain: A view of the signal after Fourier transform. The horizontal axis represents frequency, and the vertical axis represents amplitude (or energy). It shows which frequencies make up the signal and the intensity of each frequency.

[0027] The Fourier transform is a generalization of the Fourier series, used to process aperiodic signals. It transforms time-domain signals... Convert to frequency domain signal The formula is: ; Its inverse transform (recovering from the frequency domain to the time domain) is: ; in Called the "spectrum," it is a complex number that contains frequencies of 10 ... The amplitude and phase information of the components. Its modulus. This indicates the amplitude (intensity) of the frequency. This is Euler's formula. Its application is ingenious. It cleverly uses a complex exponential function to simultaneously represent both sine and cosine components. This can be understood as the integral being used to calculate the original time-domain signal. With frequency The similarity between complex sine waves.

[0028] If the similarity is high (i.e., the frequency is the dominant component of the signal), the integral result... The modulus is very large; if the similarity is very low (i.e., the signal does not contain that frequency), the integral result will be small. The modulus is close to zero.

[0029] Any function can be represented as a superposition of a series of sine and cosine waves. By integrating the signals and calculating the correlation between the signal and complex sine waves of different frequencies, the spectrum is obtained, decomposing the complex whole into simpler components. In short, the Fourier transform is like a pair of "frequency eyes," able to see through the simple structure behind any complex signal, thus enabling more effective processing and understanding of them.

[0030] Faced with complex legal cases that semantic models struggle to handle, this "frequency eye" provides a completely new perspective from the time domain to the frequency domain. Therefore, this embodiment captures the periodic structure and long-range dependency patterns of legal texts through Fourier transform.

[0031] Preferably, the specific implementation process of the above sub-case classification is as follows: First, match the pre-defined category labels with the original legal issue text. Concatenate into a joint input : ; Where C represents the number of category labels, which is the total number of all possible category labels in the dataset; It consists of C tags and the original legal issue text. The combined input is composed of multiple parts.

[0032] For combined input After word segmentation, the token sequence is obtained: ; Where L is the sequence length; Each token vector It is converted into a vector through the embedding layer. : ; Where D is the embedding dimension; Stack all the token vectors in order to obtain the joint input sequence (matrix form) X: .

[0033] The position of the token corresponding to row index i (sequence dimension); Column index j corresponds to the embedding dimension (feature dimension); For each column of X (i.e., the j-th feature dimension), it is a sequence of real numbers of length L: ; Performing a Discrete Fourier Transform (DFT) on each such sequence yields a complex spectrum sequence. To improve computational efficiency, the Fast Fourier Transform (FFT) algorithm is actually used.

[0034] Define the DFT transformation formula : ; in .

[0035] Performing the above transformation on all D dimensions yields the complex spectrum matrix: ; Each row corresponds to a frequency component. Each column corresponds to one dimension of the original embedding, and each element is a complex number: a+jb, containing amplitude and phase information.

[0036] To preserve the structural strength features of the text and reduce computational complexity, typically only the amplitude spectrum (modulus) is retained: ; The real-number frequency domain feature sequence is obtained: ; Although the dimension remains L×D, the meaning has changed from "time domain token position" to "frequency component index".

[0037] The above describes the entire process of transforming the initial semantic vector of the original legal issue text into a frequency domain feature sequence, thus realizing multi-level legal knowledge representation: 1. Lexical level: Capturing semantic information of legal terms through word embedding; 2. Syntactic level: Capturing the structural features of legal texts through frequency domain analysis; 3. Semantic hierarchy: Capture the deep semantic relationships of legal concepts through joint encoding.

[0038] Then, joint encoding is performed using a bidirectional Transformer encoder: ; in, This is the hidden output state of the encoder.

[0039] Then, text feature representations are extracted respectively. and label feature representation : ; The scoring function employs a learnable MLP: ; fractional function This is a two-layer fully connected feedforward neural network. The activation function is ReLU, and the output layer has no activation function, directly mapping the vector to a scalar confidence score. This confidence score reflects which category text T belongs to. The probability magnitude of each category. The final output is the score for all categories k=1,2,...,C. Form a fractional vector [s1, s2, ..., This allows us to obtain the final prediction result.

[0040] The above describes the process of joint tag-text processing, which realizes innovative tag interaction modeling and achieves three levels of interaction: 1. Tag-text interaction establishes fine-grained associations between tags and text through a cross-attention mechanism; 2. Tag-to-tag interaction: Capturing hierarchical relationships between legal concepts through a self-attention mechanism between tags; 3. Text-to-text interaction: Maintaining the ability of traditional self-attention mechanisms to model internal text relationships, capturing hierarchical relationships between legal concepts through self-attention mechanisms between tags.

[0041] Specifically, in this embodiment, the training of the fltclass classification model adopts an end-to-end multi-task learning framework: a unified multi-task learning objective is proposed, organically combining frequency domain feature learning and label prediction, with the total loss function being: ; Hyperparameters and The weights used to balance the various loss terms.

[0042] in, The classification loss is represented by binary cross-entropy, which measures the difference between the predicted result and the true label. The frequency domain reconstruction loss is represented by the MSE loss, ensuring that important structural information is not lost during the frequency domain transformation. Specifically, the text features output by the encoder are represented... After passing through a lightweight inverse Fourier transform layer (learnable inverse transform or fixed IDFT), the original temporal semantic vector sequence is reconstructed. and the initial joint input sequence Calculate the mean square error: The loss for label relationship modeling is represented by constructing a relationship graph through attention weights between category labels and using graph contrastive learning loss to enhance the semantic association of labels. Construct a label co-occurrence graph (based on Point Mutual Information (PMI)), perform random dropout on the label embeddings, and then use InfoNCE loss. For any label i, its representation in the two enhanced views is as follows: The core objective of contrastive learning is to make the representations of the same label in two views mutually attractive (positive sample pairs), and the representations of different labels mutually repulsive (negative sample pairs).

[0043] Using InfoNCE loss, sum and average over all labels: in: τ is the cosine similarity function; τ>0 is the temperature hyperparameter, which controls the smoothness of the distribution (in this embodiment, τ=0.1, τ=0.1). To verify the enhancement effect of the frequency domain embedding module on long texts, this paper divides the Legal Split Cases test set into three levels according to text length: short text (less than 30 tokens), medium text (30 to 60 tokens), and long text (more than 60 tokens). The F1 score of each model at each level is then calculated. Table 1. F1 scores of different models on short, medium, and long texts. Table 2. Co-occurrence F1 scores (CoF1) of different models on key label pairs As shown in Table 1, the F1 scores of all models decrease with increasing text length, indicating that long texts are more difficult to classify. The baseline model BERT has an F1 score of only 85.2 on long texts, a decrease of 4.9 percentage points compared to its performance on short texts. Although DeBERTa outperforms BERT overall, it still lags behind short texts by 3.1 percentage points on long texts.

[0044] Adding the frequency domain embedding module (+FFT) alone improves the model's F1 score on short texts by only 0.3 points compared to DeBERTa (92.8 vs 92.5), while achieving a significant 2.1% improvement on long texts (91.5 vs 89.4). This result strongly demonstrates that the frequency domain module can effectively capture long-range dependencies and logical structures in long texts, thus compensating for the shortcomings of traditional Transformers in handling ultra-long sequences.

[0045] In summary, the F1 score stratified by text length clearly reveals the core contribution of the frequency domain embedding module: it enables the model to more accurately grasp the logical structure of long texts containing complex dependencies, thereby significantly improving classification performance. This finding is highly consistent with the theoretical expectations of frequency domain analysis, validating the effectiveness of our proposed method in modeling the structure of legal texts.

[0046] To verify the effectiveness of the label-text joint processing module in modeling label dependencies, this paper selects four sets of label pairs with strong semantic associations from the Legal SplitCases dataset: (domestic violence, child custody), (property division, child custody), (loan guarantee, inheritance issues), and (traffic accident, personal injury compensation). The co-occurrence F1 scores of each model on these label pairs are calculated, and the results are shown in Table 2.

[0047] As shown in Table 2, the baseline models BERT and DeBERTa performed poorly on the label co-occurrence task, with average CoF1 scores of 82.7 and 85.1, respectively, indicating that independent prediction strategies struggle to capture the intrinsic relationships between labels. After introducing the label-text joint processing module (+label-textClass), the average CoF1 significantly improved to 94.2, demonstrating that this module effectively models label dependencies through a self-attention mechanism between labels.

[0048] Table 3 Comparison of experimental data for different models on the Legal Split Cases Dataset To validate the frequency domain embedding module and the label-text joint processing module, basic evaluation criteria tests were performed on different models on the Legal SplitCases Dataset.

[0049] As shown in Table 3, the proposed full model (fltclass) achieved optimal performance across all evaluation metrics, with a precision of 97.8%, a recall of 98.5%, and an F1 score of 98.1%. Compared to the traditional BERT baseline model (F1=96.5), our method delivers a significant improvement of 7.8 percentage points, demonstrating the effectiveness of the embedded frequency domain information and label-text joint processing strategy in legal text classification tasks.

[0050] Example 2 Based on the same inventive concept, this application also provides a domain classification device for complex legal question-and-answer formats embedded in the frequency domain, which can be used to implement the method described in the above embodiments, specifically including the following: The embedding module is used to acquire the original legal issue text and perform word segmentation and embedding processing to obtain an initial semantic vector sequence; The frequency domain transformation module is used to convert the initial semantic vector sequence into a frequency domain feature sequence representing the text structure features through a fast Fourier transform. The joint processing module is used to concatenate the preset category labels with the frequency domain feature sequence to construct a unified joint input sequence; input the joint input sequence into the encoder for joint encoding, and extract text feature representation and label feature representation respectively; The classification module is used to concatenate the text feature representation with each of the label feature representations, calculate the probability score of the text feature representation belonging to each category label through a scoring function, and obtain the sub-case classification result.

[0051] Preferably, embodiments of this application also provide a specific implementation of an electronic device capable of implementing all steps in the domain classification method for embedding complex legal questions and answers in the above embodiments. The electronic device specifically includes the following: Processor, memory, communications interface, and bus; The processor, memory, and communication interface communicate with each other via a bus; the communication interface is used to realize information transmission between server-side devices, metering devices, and user-side devices.

[0052] The processor is used to call a computer program in memory. When the processor executes the computer program, it implements all the steps in the domain classification method for complex legal question answering embedded in the frequency domain in the above embodiments.

[0053] Embodiments of this application also provide a computer-readable storage medium capable of implementing all steps of a domain classification method for complex legal question-and-answer embedded in the frequency domain as described in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the domain classification method for complex legal question-and-answer embedded in the frequency domain as described in the above embodiments.

[0054] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, hardware + program embodiments are relatively simple in description because they are fundamentally similar to method embodiments; relevant parts can be referred to the descriptions in the method embodiments.

[0055] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0057] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0058] This invention is not limited to the embodiments described above. The above description of specific embodiments is intended to illustrate and explain the technical solutions of this invention. The specific embodiments described above are merely illustrative and not restrictive. Without departing from the spirit and scope of the claims, those skilled in the art can make many specific modifications based on the teachings of this invention, and these modifications all fall within the scope of protection of this invention.

Claims

1. A domain classification method for complex legal question-and-answer systems embedded in the frequency domain, characterized in that, include: Obtain the original legal question text, and concatenate the preset category labels with the original legal question text to obtain a joint input sequence; The joint input sequence is converted into a frequency domain feature sequence that characterizes the text structure features by using a fast Fourier transform. The frequency domain feature sequence is input into the encoder to extract text feature representation and label feature representation; The confidence scores of the text feature representations belonging to each category label are calculated using a pre-trained scoring function to obtain the sub-case classification results.

2. The domain classification method according to claim 1, characterized in that, The specific process of concatenating the preset category labels with the original legal question text to obtain the joint input sequence is as follows: First, match the pre-defined category labels with the original legal issue text. Concatenate into a joint input : ； Where C represents the number of category labels, which is the total number of all possible category labels in the dataset; For combined input After word segmentation, the token sequence is obtained: ； Where L is the sequence length; Each token vector It is converted into a vector through the embedding layer. : ； Where D is the embedding dimension; Stack all the token vectors in order to obtain the joint input sequence X: 。 3. The domain classification method according to claim 1, characterized in that, The step of converting the joint input sequence into a frequency domain feature sequence representing text structural features through Fast Fourier Transform includes: Obtain the initial semantic vector sequence after the embedding layer of the pre-trained model. Where L is the sequence length and D is the dimension; a fast Fourier transform is performed on each feature dimension to obtain the spectrum matrix. : ; Take the amplitude spectrum as the frequency domain feature sequence : .

4. The domain classification method according to claim 1, characterized in that, The confidence score, which reflects the text feature representation belonging to the corresponding category, is obtained by calculating the score function and using a two-layer feedforward neural network.

5. The domain classification method according to claim 1, characterized in that, The encoder adopts a uni-encoder architecture, specifically a bidirectional Transformer encoder.

6. The domain classification method according to any one of claims 1 to 5, characterized in that, The classification model that performs the method is pre-trained in the following manner: Construct a total loss function that includes classification loss, frequency domain reconstruction loss, and label relationship modeling loss; Based on the multi-task learning framework, the pre-trained model is fine-tuned end-to-end using the total loss function to obtain the classification model.

7. The domain classification method according to claim 6, characterized in that, The total loss function is specifically as follows: ； in, The classification loss is calculated using binary cross-entropy; The frequency domain reconstruction loss is calculated using mean square error; Modeling loss for label relationships computed using graph contrastive learning; hyperparameters and The weights used to balance the various loss terms.

8. A domain classification device for complex legal questions and answers embedded in the frequency domain, characterized in that, include: The embedding module is used to obtain the original legal question text and concatenate the preset category labels with the original legal question text to obtain a joint input sequence; The frequency domain transformation module is used to convert the joint input sequence into a frequency domain feature sequence that characterizes the text structure features through a fast Fourier transform. The joint processing module is used to input the frequency domain feature sequence into the encoder to extract text feature representation and label feature representation; The classification module is used to calculate the confidence scores of text feature representations belonging to each category label using a pre-trained scoring function, and obtain the sub-case classification results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the domain classification method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the domain classification method according to any one of claims 1 to 7.

Images