A method and device for identifying semantic equivalence of requirements

Abstract

The present invention discloses a method and device for identifying semantic equivalence of requirements. Specifically, for any anchor sentence pair containing two segments of unlabeled requirement text, the requirement text is first augmented using multiple text data augmentation strategies to generate three augmented sentence pairs. To reduce data augmentation noise, a sentence pair mixing strategy is further proposed to merge the three augmented sentence pairs into the anchor sentence pair's corresponding positive sentence pair, obtaining the anchor sentence pair and its corresponding positive sentence pair. A text interaction module is then pre-trained using contrastive learning loss. Finally, the contrastive learning pre-training process is fine-tuned using binary cross-entropy loss in a limited supervision scenario. This method effectively addresses the problem of sentence polysemy. The data augmentation strategy, combined with the sentence pair mixing strategy, can reduce data augmentation noise and align the comparison objects with the contrastive learning loss.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a method and device for identifying semantic equivalence of requirements. Background Art

[0002] Requirement Semantic Equivalence Identification (RSEI) aims to measure the semantic equivalence between two requirement texts. The goal of RSEI is to determine whether two texts are semantically equivalent. As a specialized text matching task, RSEI serves as a fundamental building block in many natural language processing applications, supporting numerous downstream tasks such as question-answering systems, dialogue systems, and intelligent customer service. With RSEI, intelligent systems can return identical responses for equivalent texts, which not only reduces user information search time but also significantly reduces overall system operating costs.

[0003] At the same time, in fields such as military and software engineering, the way requirements are described varies from person to person, and the same requirement, when proposed by different users, may take many different forms. Repeated modeling and analysis of the same requirement will lead to a waste of time and resources, delaying fleeting strategic and business opportunities. Therefore, in systems such as demand-resource matching and demand analysis, for requirements described in textual form, semantic equivalence recognition can serve as a fundamental module for many requirement-related applications. It can discover equivalent requirements and directly return the analysis results (responses / resources) of existing requirements, which can significantly save system operating time and costs. Essentially, if the source and target requirements express the same intent, or if they can be satisfied with the same responses / resources, they can be considered semantically equivalent.

[0004] In recent years, with the rise of deep learning methods, large-scale pre-trained language models (PLMs), as essential components in neural networks, can provide good text initialization embeddings for semantic equivalence recognition tasks. Despite the remarkable success of current methods, these data-intensive models do not work well in low-resource, real-world scenarios. In other words, in real-world scenarios, massive amounts of well-annotated data are difficult to obtain, resulting in insufficient training of model parameters. Therefore, to supplement the lack of supervisory signals, existing semantic equivalence recognition research has introduced contrastive learning, clustering semantically similar texts as positive samples and separating semantically different texts as negative samples to obtain acceptable text semantic representations.

[0005] At the same time, a lot of research has been conducted for a long time to eliminate the common polysemy phenomenon in natural language to clarify the true semantics of the text. However, polysemous sentences caused by limited contextual information and polysemous words have been ignored in the field of demand semantic equivalence recognition. The same word can express different semantics in different contexts. For example, "apple" means both fruit and a technology company. Similarly, the same semantics can also be expressed by different words. For example, "taxi" and "taxi" both mean taxis. Figure 1 For example, in Example #1 and Example #2, the source requirement, "Apple prices must adjust according to market demand," is equivalent to its corresponding target requirements: "Apple product prices must adjust when market demand changes" and "As a seasonal fruit, apple prices must match market demand." However, these two target requirements are semantically inconsistent, as shown in Example #3. This paradox reflects the potential for ambiguity in the semantics of requirements. Equivalence is relative, not absolute, and its true meaning must be determined by the objects of comparison and the context in which they are used.

[0006] To alleviate the limited supervisory signal problem caused by the difficulty in obtaining massive amounts of labeled data, existing methods widely use contrastive learning, comparing large-scale unlabeled anchor samples with their positive and negative counterparts to enhance the semantic information contained in text representations. Despite significant progress, they still cannot accurately capture the polysemy phenomenon in polysemous sentences. Summary of the Invention

[0007] In order to solve the above-mentioned sentence polysemy problem, the present invention proposes a method for identifying semantic equivalence of requirements, which is specifically implemented by using the pairwise contrastive learning (PairContrast) method. The identification method is specifically as follows:

[0008] Given any two unlabeled requirement texts as anchor sentence pairs For source demand, To meet the target requirements, use random shuffling strategy to expand and The augmented source demand and augmented target requirements And through interaction, we can get augmented sentence pairs and

[0009] right Encode and get R S 、R T 、R S′ 、R T′ ;

[0010] RS 、R T 、R S′ 、R T′ Use the soft attention alignment mechanism to model and obtain and Interaction representation of and and Interaction representation of and and Interaction representation of and and Interaction representation of and

[0011] Interaction Representation and and and and Aggregate and get anchor sentence pairs The aggregate representation r st , the aggregate representation of the three augmented sentence pairs r st′ , r s′t and r s′t′ , and mix the aggregate representations of the three augmented sentence pairs to obtain the integrated representation r′ of the three augmented sentence pairs st ;

[0012] In the unsupervised pre-training phase, a linear mapper is used to transform the aggregate representation r st and integrated representation r′ st Mapped into different vector spaces, in the supervised fine-tuning stage, the aggregate representation r is represented by a predictor. st Mapped into binary equivalence labels.

[0013] Furthermore, the Encode, specifically:

[0014]

[0015]

[0016]

[0017]

[0018] in,(·) i represents the output of the i-th layer in PLM, avg is the average value, D an and D au They represent the Dropout operations on the required text features in the anchor sentence pairs and augmented sentence pairs respectively.

[0019] Furthermore, the interaction and and and and Perform aggregation, specifically:

[0020]

[0021]

[0022]

[0023]

[0024] Where ⊙ represents element-wise multiplication.

[0025] Furthermore, the integrated representation r′ st for

[0026] r′ st =mixup([r st′ ; r s′t ; r s′t′ ])

[0027] Among them, mixup represents the sentence pair mixing function.

[0028] Furthermore, the sentence pair mixing function may be any one of an averaging strategy and a maximization strategy.

[0029] Furthermore, a linear mapper is used to represent the aggregate r st and integrated representation r′ st Mapped into different vector spaces, specifically:

[0030] r st =Linear(r st ),

[0031] r′ st =Linear(r′ st )

[0032] Among them, Linear is a linear mapper.

[0033] Furthermore, the aggregate representation r is represented by a predictor. st Mapped to binary equivalence labels, specifically:

[0034]

[0035] in, is a binary equivalence label, MLP represents a multilayer perceptron with two linear layers and one ReLu layer, and the sigmoid activation function maps the predicted equivalence to the range of [0, 1].

[0036] In the unsupervised pre-training phase, this method uses a contrastive learning loss to maximize the consistency between the representations of anchor and augmented sentence pairs. In the supervised fine-tuning phase, a binary cross-entropy loss is used to fine-tune the unsupervised pre-training process under limited labeled data.

[0037] The present invention also proposes a demand semantic equivalence recognition device, which includes a sentence pair augmentation module, an encoding module, an interaction representation calculation module, an aggregation module, a mapping module and a prediction module.

[0038] The sentence pair augmentation module is given any two unlabeled demand texts as anchor sentence pairs. For source demand, To meet the target requirements, use random shuffling strategy to expand and The augmented source demand and augmented target requirements And through interaction, we can get augmented sentence pairs and

[0039] The encoding module Encode and get R S 、R T 、R S′ 、R T′ ;

[0040] The interaction represents the computational module, S 、R T 、R S′ 、R T′ Use the soft attention alignment mechanism to model and obtain and Interaction representation of and and Interaction representation of and and Interaction representation of and and Interaction representation of and

[0041] The aggregation module represents the interaction and and and and Aggregate and get anchor sentence pairs The aggregate representation r st , the aggregate representation of the three augmented sentence pairs r st′ , r s′t and r s′t′ , and mix the aggregate representations of the three augmented sentence pairs to obtain the integrated representation r′ of the three augmented sentence pairs st ;

[0042] The mapping module uses a linear mapper to transform the aggregate representation r in the unsupervised pre-training stage. st and integrated representation r′ st Mapping into different vector spaces;

[0043] The prediction module, in the supervised fine-tuning stage, uses the predictor to represent the aggregate r st Mapped into binary equivalence labels.

[0044] The proposed method is experimentally tested on two public real-world datasets, and the results show that it can achieve state-of-the-art performance compared with other competitive methods. The main contributions can be summarized as follows:

[0045] 1. The recognition method proposed in this invention can effectively solve the problem of multiple meanings in a sentence;

[0046] 2. The present invention adopts a data augmentation strategy, combined with a sentence pair mixing strategy, which can reduce data augmentation noise and match the comparison objects with the contrastive learning loss;

[0047] 3. A large number of experiments were conducted on two benchmark datasets to verify the superiority and robustness of the method proposed in this invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0049] Figure 1 Identify instances for semantic equivalence of requirements.

[0050] Figure 2 This is the model architecture of PairContrast.

[0051] Figure 3 The performance comparison of the PairContrast model with representative baselines on different datasets with different supervision sizes.

[0052] Figure 4 Performance of paired control variants with different hybrid strategies in terms of accuracy and F1 score on BQ-Corpus and LCQMC. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0054] The following first defines the issues and symbols that may be involved in the embodiment:

[0055] The requirement semantic equivalence recognition task aims to predict the semantic equivalence of two requirement texts, namely the source requirement and target needs In the limited supervision scenario, and Represent large-scale unlabeled and limited labeled sentence pairs, respectively, where That is, the scale of unlabeled sentence pairs is much larger than that of labeled sentence pairs. This limited supervision scenario is closer to reality because well-labeled data is expensive and difficult to obtain. Based on the above notation, the semantic equivalence recognition problem of requirements is expressed as and The prediction function is learned in order to accurately estimate the equivalence of unknown demand sentence pairs. The above process can be expressed as:

[0056]

[0057] in, Source demand and target needs The prediction equivalence between them; F(·) is a well-designed demand semantic equivalence recognition method, which is trained so that Able to approach true equivalence labels

[0058] Figure 2 The detailed model architecture of the semantic equivalence identification method proposed in this invention is shown in Figure 2. The method mainly uses sentence-pair-level contrastive learning to solve the polysemy problem. The method is specifically as follows:

[0059] The first step is sentence pair augmentation. For an anchor sentence pair consisting of two demand texts, namely the source demand and target needs First, we augment the anchor source demand by applying a random shuffling strategy and anchor target requirements The augmented source demand and augmented target requirements In this case, in addition to the anchor sentence In addition to the paired text comparison scenario, we can also obtain augmented sentence pairs for three paired text comparison scenarios by considering the interaction between different demand texts, namely: and

[0060] In the second step, the anchor sentence pair and the three augmented sentence pairs are encoded to obtain the PLM representation of the anchor sentence pair and the augmented sentence pair. The present invention averages the last two layers of the PLM to obtain the augmented source demand. and augmented target requirements By encoding, different layers of PLM can reflect different levels of semantics, syntax and other information. Through the above enhanced sentence pair augmentation strategy, high-quality augmented sentence pairs can be obtained for comparison. and augmented sentence pairs The encoding formula is described as follows:

[0061]

[0062]

[0063]

[0064]

[0065] in,(·) i represents the output of the i-th layer in PLM; D an Sum = D au They represent the Dropout operations on the required text features in the anchor sentence pairs and augmented sentence pairs respectively.

[0066] The third step is to obtain the interactive representation through the interactor. After obtaining the PLM representation of the anchor sentence pair and the augmented sentence pair, that is: R S 、R T 、R S′ and R T′ , using the interactor to mine the implicit interactions in each pair of text comparison scenarios. The present invention uses the soft attention alignment mechanism to model the interaction between two demand texts. Taking the following anchor sentence pair as an example, the formula of the interactor is described as follows:

[0067] A t =R S ·R T T ,

[0068]

[0069]

[0070]

[0071]

[0072]

[0073]

[0074] where R T T and A t T It is R T and A t The transpose of A t It is R S and R T The attention weights between ⊙ and ⊙ represent element-wise multiplication. and yes and Alignment representation. and yes and In addition, mean and max represent the average pooling and maximum pooling operations respectively. and In the comparison scenario Down and The interactive representation of the three augmented sentence pairs obtained in the encoder is modeled by the interactor as well as And so on.

[0075] The fourth step is to obtain the aggregated representation of anchor sentence pairs and the integrated representation of augmented sentence pairs through an aggregator. An aggregator is used to aggregate the interactive representations of the source and target requirements, and then input them into the classifier of the downstream task for equivalence prediction. Taking the anchor sentence pair below as an example, the formula for the interaction representation aggregation process is as follows:

[0076]

[0077] Among them, r st Anchor sentence pair The aggregate representation of the three augmented sentence pairs is r st′ , r s′t , r s′t′ It can also be calculated by the above formula:

[0078]

[0079]

[0080]

[0081] Where ⊙ represents element-wise multiplication.

[0082] Since each anchor sentence pair has three augmented sentence pairs, and the influence and effect of each augmented sentence pair as a comparison sample is uncertain, in order to reduce data augmentation noise, the aggregated representations of the three augmented sentence pairs are mixed to obtain an integrated augmented sentence pair as the positive sentence pair corresponding to the anchor sample, which helps to alleviate the overconfidence bias of the model. This process can be formulated as:

[0083] r′ st =mixup([r st′ ; r s′t ; r s′t′ ]),

[0084] Among them, r′ st It is an integrated representation of augmented sentence pairs; mixup represents a sentence pair mixing function that can be adjusted according to actual application scenarios, such as averaging, maximization or other strategies.

[0085] The fifth step is to obtain the final representation of the anchor sentence pair and the augmented sentence pair before contrastive learning through the mapper. In the unsupervised pre-training stage, a linear mapper is used to map the semantic representation into different vector spaces. The formula is as follows:

[0086] r st =Linear(r st ),

[0087] r′ st =Linear(r′ st ),

[0088] Among them, r st and r′ st They are the final representations of the anchor sentence pair and the augmented sentence pair before contrastive learning.

[0089] In the sixth step, the aggregate representation of the input anchor sentence pair is mapped to a binary equivalence label through the predictor. In the supervised fine-tuning stage, the predictor is used to map the aggregate representation of the input sentence pair to a binary equivalence label. The formula is as follows:

[0090]

[0091] Here, MLP represents a multilayer perceptron with two linear layers and one ReLu layer. The sigmoid activation function maps prediction equivalence to the range of [0, 1].

[0092] The following is the training process and loss function:

[0093] In the unsupervised pre-training stage, N unlabeled demand sentence pairs are randomly sampled from the dataset to construct a training batch, and 2N representations are generated through the operations described in the above model architecture, namely N anchor sentence pair representations and N augmented sentence pair representations.

[0094] The present invention uses contrastive learning loss to maximize the consistency between the representation of anchor sentence pairs and their corresponding augmented sentence pairs, and to minimize the inconsistency between the representation of anchor sentence pairs and other sentence pairs in the same training batch. Therefore, the contrastive learning loss formula is as follows:

[0095]

[0096] Where s(·) represents the cosine similarity function, which is used to measure the consistency between representations; 1 j≠i ∈{0, 1} is an indicator; τ is a temperature hyperparameter that scales the size of the similarity.

[0097] In the supervised fine-tuning stage, with limited annotated demand sentence pairs The binary cross entropy loss is used to fine-tune the model parameters pre-trained by sentence-level contrastive learning to obtain the optimal number of training steps. Each training instance consists of a source requirement A target requirement and its corresponding true equivalence label The training goal of this stage is to minimize the following binary cross entropy loss:

[0098]

[0099] in, is the true equivalence label of the i-th training instance; is the prediction equivalence of the PairContrast model predicted for the i-th training instance.

[0100] The present invention also proposes a demand semantic equivalence recognition device, which includes a sentence pair augmentation module, an encoding module, an interaction representation calculation module, an aggregation module, a mapping module and a prediction module.

[0101] The sentence pair augmentation module is given any two unlabeled demand texts as anchor sentence pairs. For source demand, To meet the target requirements, use random shuffling strategy to expand and The augmented source demand and augmented target requirements And through interaction, we can get augmented sentence pairs and

[0102] The encoding module Encode and get R S 、R T 、R S′ 、R T′ ;

[0103] The interaction represents the computational module, S 、R T 、R S′ 、R T′ Use the soft attention alignment mechanism to model and obtain and Interaction representation of and and Interaction representation of and and Interaction representation of and and Interaction representation of and

[0104] The aggregation module represents the interaction and and and and Aggregate and get anchor sentence pairs The aggregate representation r st , the aggregate representation of the three augmented sentence pairs r st′ , r s′t and r s′t′ , and mix the aggregate representations of the three augmented sentence pairs to obtain the integrated representation r′ of the three augmented sentence pairs st ;

[0105] The mapping module uses a linear mapper to transform the aggregate representation r in the unsupervised pre-training stage. st and integrated representation r′ st Mapping into different vector spaces;

[0106] The prediction module, in the supervised fine-tuning stage, uses the predictor to represent the aggregate r st Mapped into binary equivalence labels.

[0107] The following is the experimental setup and experimental verification:

[0108] Datasets and evaluation metrics: Extensive experiments are conducted on two publicly available datasets, namely the Chinese Corpus of User Information Needs in the Banking Domain (BQ-Corpus) and the Large-Scale Chinese Information Needs Equivalence Recognition Corpus (LCQMC), to evaluate the performance of the proposed method. Specifically, BQ-Corpus and LCQMC contain 120,000 and 260,068 real sentence pairs with manual annotations, respectively. In the unsupervised pre-training stage, all available sentence pairs in each dataset are used for contrastive learning pre-training. In the supervised fine-tuning stage, limited annotated data is used to simulate real-world scenarios. Specifically, 10% of the training sentence pairs are randomly sampled from BQ-Corpus and LCQMC to create limited supervision scenarios, that is, BQ-Corpus and LCQMC have 10,000 and 23,876 annotated instances as fine-tuning samples, respectively. The detailed statistics of the two datasets are summarized in Table 1.

[0109] Table 1 Dataset statistics

[0110]

[0111]

[0112] This method treats the requirement semantic equivalence recognition task as a binary classification problem, where equivalence is classified as positive (value 1) or negative (value 0). Therefore, the performance of the requirement semantic equivalence recognition model is evaluated using commonly used evaluation metrics for classification tasks, namely accuracy and F1 score.

[0113] Implementation details: All experiments in this work were implemented using the deep learning framework PyTorch and run on a server with an NVIDIA RTX 3090 GPU. The pre-trained language model used in this paper is BERT-Base-Chinese. Specifically, the AdamW optimizer was used to train the model for up to 50 epochs when adjusting the model parameters. The maximum text length was set to 50, the batch size for both pre-training and fine-tuning was set to 128, the dropout was set to 0.2 and 0.3, respectively, and the initial learning rate for the pre-training phase was set to 3e-5; the initial learning rates for fine-tuning on BQ-Corpus and LCQMC were 4e-5 and 8e-5, respectively, and the temperature τ in the contrastive learning loss was set to 0.1. The pre-training process was stopped when the Spearman correlation coefficient between the ground-truth equivalence label and the sentence pair similarity score did not improve further.

[0114] Main experimental results: The effectiveness of this method is tested by comparing it with the actual performance of other competitive semantic equivalence recognition baseline models on public datasets, namely: DA, ABCNN, ESIM, BIMPM, EnhancedRCNN, BERT base , ConSERT and SimCSE. Table 2 shows the comparison of the proposed sentence pair-level contrastive learning method (PairContrast) with all baseline models in terms of accuracy and F1 value.

[0115] Table 2 Accuracy of all models

[0116]

[0117] As shown in Table 2, among all baselines, the PLM-based model (i.e., BERT base , ConSERT, and SimCSE), models without PLM perform relatively poorly on both datasets. This suggests that PLM is crucial for incorporating prior knowledge into the requirement semantic equivalence identification task when the number of labeled samples is limited. A careful inspection of Table 2 reveals that SimCSE outperforms other baseline models in both evaluation metrics on both datasets, making it the most competitive baseline because it benefits from a carefully designed contrastive learning objective for large-scale unlabeled data and is able to capture semantic information from unlabeled samples.

[0118] For the method proposed in the present invention, the experimental results given in Table 2 clearly show the effectiveness of the method proposed in the present invention in the task of semantic equivalence recognition of requirements in scenarios with limited labeled samples. Specifically, on the BQ-Corpus dataset, PairContrast achieves an absolute improvement of 3.16 and 2.10 in accuracy and F1 value relative to the best baseline SimCSE, respectively. Similar phenomena can also be observed on the LCQMC dataset, namely: PairContrast achieves gains of 3.27 and 1.47 in accuracy and F1 value compared to SimCSE, respectively. The difference between PairContrast and the most advanced baseline model SimCSE is that it performs comparative learning on sentence pairs by considering the interactions between requirement texts, rather than comparing single sentences. In summary, the phenomena observed in the overall performance of the model indicate that in limited supervision scenarios, the performance of the requirement semantic equivalence recognition model can be improved by using sentence pair-level comparative learning.

[0119] Ablation Experiments: To explore our proposed method in depth, we conducted extensive ablation experiments to better understand what makes PairContrast perform well. Specifically, we removed key components of PairContrast and compared their accuracy with the full PairContrast model. The performance comparison results are shown in Table 3, where the model variants w / o-pretraining directly fine-tune on a limited number of labeled sentence pairs; w / o-shuffling and w / o-differ-layer respectively remove the random shuffling strategy and the different PLM layer reading strategy; the variant w / o-projector removes the mapper module.

[0120] Table 3 Accuracy of PairContrast model variants with different components removed on BQ-Corpus and LCQMC

[0121] Model variants BQ-Corpus LCQMC PairContrast 79.64 82.85 w / o pretraining 73.74(-5.90) 79.26(-3.59) w / o shuffling 76.83(-2.81) 80.76(-2.09) w / o-differ-layer 76.71(-2.93) 79.98(-2.87) w / o-projector 76.55(-3.09) 80.59(-2.26)

[0122] As shown in Table 3, on both datasets, accuracy drops significantly when each key component of PairContrast is removed. Specifically, the most striking observation in Table 3 is that removing the pre-training stage (i.e., the w / o pretraining variant) leads to the most significant performance drop, with drops of up to 5.90% and 3.59% on the BQ-Corpus and LCQMC datasets, respectively. This is reasonable, as the limited number of labeled examples for supervised learning in the real-world scenarios simulated in this paper makes it difficult to directly fine-tune the model to achieve optimal parameters. Furthermore, it is clear that the mapper plays a crucial role in the PairContrast model, as the w / o projector variant suffers a drop in accuracy almost second only to the w / o pretraining variant. Furthermore, the comparative results demonstrate the effectiveness of our proposed enhanced data augmentation mechanism, namely, random shuffling and different PLM layer read strategies. Considering all of the aforementioned observations, we can conclude that the essential components of our proposed PairContrast model work together to give our proposed sentence-pair-level contrastive learning a certain advantage in the task of demand semantic equivalence recognition.

[0123] Effect of Supervision Scale: Here, we extensively examine the performance of PairContrast and representative baseline models at different supervision scales, namely, the best non-PLM baseline EnhancedRCNN, the PLM-based baseline BERT base and the best CL-based baseline SimCSE. In the supervised fine-tuning stage, we mainly consider four limited supervision scenarios with different scales, i.e., {0.0001, ····, 0.1} times the full training set. Figure 3 To compare the performance of the PairContrast model with representative baselines on different datasets with different supervision scales, Figure 3 a and Figure 3 b shows the comparison results of the accuracy of each model on the BQ-Corpus and LCQMC datasets. Figure 3 As shown, it can be observed that PairContrast outperforms other baseline models in all limited supervision scenarios. Specifically, we find that BERT base The model outperforms the Enhanced RCNN model in all scenarios, except for a supervision scale of 0.0001, where 10 instances are sampled from the BQ-Corpus training set and 23 instances are sampled from the LCQMC training set for fine-tuning. This is because the success of PLM relies heavily on fine-tuning on large-scale labeled data. When the scale of labeled data is particularly small, PLM fine-tuning is significantly affected. Therefore, as the supervision scale increases, PLM's powerful capabilities become increasingly apparent.

[0124] It is worth noting that when using contrastive learning to train BERT on large-scale unlabeled data base After the model is pre-trained, namely the SimCSE model and the PairContrast model, we can Figure 4 This proves that semantic equivalence recognition in demand under limited supervision scenarios does benefit from the introduction of contrastive learning. In addition, by comparing the performance of the SimCSE model and the PairContrast model at different supervision scales, we also found that the SimCSE model achieved the greatest improvement when the supervision scale was 0.01. However, when the supervision scale was 0.1, our proposed PairContrast model performed better than BERT. base The model achieves the largest improvement. Given that under the same supervision scale, the SimCSE model only uses different sentences in the dataset for pre-training, while the PairContrast model uses all sentence pairs, we speculate that this may be because there are a large number of repeated sentences in both datasets, and the ratio of unlabeled data to labeled data will affect the performance of contrastive learning.

[0125] Impact of Mixing Strategies: We further analyze the impact of different sentence pair mixing strategies on the performance of the PairContrast model by considering three commonly used feature fusion methods, namely, average, maximum, and splicing strategies. We use linear layers in the splicing strategy to keep the feature dimensions of the spliced ​​augmented sentence pairs consistent with those of the anchor sentence pairs. Figure 4 The performance of model variants with different mixing strategies on two datasets is shown in . Figure 4 In the first and third groups, it can be clearly observed that when the average strategy is applied on the BQ-Corpus dataset, PairContrast achieves the best performance in terms of accuracy and F1 value. The performance of the maximum strategy and the splicing strategy on BQ-Corpus is relatively poor. For the LCQMC dataset, we found that the PairContrast model achieved the best performance when the splicing strategy was applied. It is worth noting that on the LCQMC dataset, the performance produced by the average strategy is roughly the same as that of the splicing strategy. About Figure 4 The results shown in the figure show that for the sentence pair mixing strategy, there is no doubt that the maximum method may not be the best choice for the sentence pair-level contrastive learning we proposed. At the same time, considering that the splicing strategy achieves similar performance to the average strategy, but introduces additional linear layer parameters, thereby increasing the computational complexity and time cost of the PairContrast model, we prefer to use the average sentence pair mixing strategy to fuse the feature representations of the three augmented sentence pairs.

[0126] Error Analysis: We carefully examine cases where the PairContrast model makes incorrect predictions and analyze in detail the factors that limit the model’s performance.

[0127] Table 4 Model accuracy of sentence-pair level contrastive learning using unlabeled data from different sources in the unsupervised pre-training phase

[0128]

[0129] First, we observe that the source of unlabeled sentence pairs has a huge impact on the performance of the model. For example, the performance of the PairContrast model on the LCQMC dataset can benefit from the mixture of unlabeled sentence pairs from the two datasets, while the performance on the BQ-Corpus dataset will be impaired. The specific results are shown in Table 4. We speculate that the possible reasons are due to the imbalance of data volume and domain bias. This is because the number of vocabulary and different sentences in the BQ-Corpus dataset is very small compared to the LCQMC dataset. In addition, the BQ-Corpus dataset is a limited domain dataset focused on the banking field, while LCQMC is a more general open domain dataset. These factors together lead to the performance of the PairContrast model using mixed sentence pairs for unsupervised pre-training to decline on the BQ-Corpus dataset, while the performance on the LCQMC dataset is improved.

[0130] Furthermore, determining the threshold for equivalence is difficult even for experts. In this regard, we observed a significant number of incorrect cases with predicted equivalence close to the default threshold of 0.5, suggesting that determining the equivalence between two requirement texts is difficult. This may be due to the fact that the category bias in sentence-pair-level contrastive learning is much more severe than that in sentence-level contrastive learning. More precisely, since there are only two categories of sentence pairs in the semantic equivalence detection task—equivalent or not—sentence-pair-level contrastive learning is more likely to separate sentence pairs of the same category in the training batch than to separate individual sentences. This is because sentences are much more diverse than sentence pairs.

[0131] While embodiments of the present invention have been described above, these embodiments do not exhaustively describe all details and do not limit the invention to the specific embodiments described. Numerous modifications and variations are possible based on the above description. These embodiments are selected and described in detail herein to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to utilize the present invention and its modifications and uses. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for identifying semantic equivalence of requirements, characterized in that: The method comprises: Given any two unlabeled requirement texts as anchor sentence pairs For source demand, To meet the target requirements, use random shuffling strategy to expand and The augmented source demand and augmented target requirements And through interaction, we can get augmented sentence pairs and right Encode and get R S 、R T 、R S' 、R T' ; R S 、R T 、R S' 、R T' Use the soft attention alignment mechanism to model and obtain and Interaction representation of and and Interaction representation of and and Interaction representation of and and Interaction representation of and Interaction Representation and and and and Aggregate and get anchor sentence pairs The aggregate representation r st and the aggregate representation r of the three augmented sentence pairs st' , r s't and r s't' , and mix the aggregate representations of the three augmented sentence pairs to obtain the integrated representation r' of the three augmented sentence pairs st ; In the unsupervised pre-training phase, a linear mapper is used to transform the aggregate representation r st and the integrated representation r' st Mapped into different vector spaces, in the supervised fine-tuning stage, the aggregate representation r is represented by a predictor. st Mapped into binary equivalence labels.

2. The identification method according to claim 1, characterized in that The pair Encode, specifically: in,(·) i represents the output of the i-th layer in PLM, avg is the average value, D an and D au They represent the Dropout operations on the required text features in the anchor sentence pairs and augmented sentence pairs respectively.

3. The identification method according to claim 1, characterized in that The pair interaction represents and and and and Perform aggregation, specifically: Where ⊙ represents element-wise multiplication.

4. The identification method according to claim 1, wherein: The integrated representation r' st for r′ st =mixup([r st' ;r s't ;r s't' ]) Among them, mixup represents the sentence pair mixing function.

5. The identification method according to claim 4, characterized in that: The sentence pair mixing function is any one of an averaging strategy and a maximization strategy.

6. The identification method according to claim 1, characterized in that Use a linear mapper to represent the aggregate r st and the integrated representation r' st Mapped into different vector spaces, specifically: r st =Linear(r st ), r′ st =Linear(r′ st ) Among them, Linear is a linear mapper.

7. The identification method according to claim 1, characterized in that: Use the predictor to represent the aggregate st Mapped to binary equivalence labels, specifically: in, is a binary equivalence label, MLP represents a multilayer perceptron with two linear layers and one ReLu layer, and the sigmoid activation function maps the predicted equivalence to the range of [0,1].

8. The identification method according to any one of claims 1 to 7, characterized in that: During the unsupervised pre-training stage, a contrastive learning loss is used to maximize the consistency between the representations of anchor and augmented sentence pairs.

9. The identification method according to any one of claims 1 to 7, characterized in that: In the supervised fine-tuning stage, binary cross entropy loss is used to fine-tune the unsupervised pre-training process under limited labeled data scenarios.

10. A device for identifying semantic equivalence of requirements, characterized in that: The recognition device includes a sentence pair augmentation module, an encoding module, an interaction representation calculation module, an aggregation module, a mapping module and a prediction module. The sentence pair augmentation module is given any two unlabeled demand texts as anchor sentence pairs. For source demand, To meet the target requirements, use random shuffling strategy to expand and The augmented source demand and augmented target requirements And through interaction, we can get augmented sentence pairs and The encoding module Encode and get R S 、R T 、R S' 、R T' ; The interaction represents the computational module, S 、R T 、R S' 、R T' Use the soft attention alignment mechanism to model and obtain and Interaction representation of and and Interaction representation of and and Interaction representation of and and Interaction representation of and The aggregation module represents the interaction and and and and Aggregate and get anchor sentence pairs The aggregate representation r st , the aggregate representation of the three augmented sentence pairs r st' , r s't and r s't' , and mix the aggregate representations of the three augmented sentence pairs to obtain the integrated representation r' of the three augmented sentence pairs st ; The mapping module uses a linear mapper to transform the aggregate representation r in the unsupervised pre-training stage. st and the integrated representation r' st Mapping into different vector spaces; The prediction module, in the supervised fine-tuning stage, uses the predictor to represent the aggregate r st Mapped into binary equivalence labels.

Images