Vector-based semantic analysis method and device, equipment and storage medium
Through pre -processing and input language models of policy text, using semantic similarity training models are used to solve the problem of low degree of accuracy of policy text analysis in existing technology, and to achieve higher semantic analysis accuracy.
Patent Information
- Application Number
- CN202411424936.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-12
- Publication Date
- 2025-01-28
AI Technical Summary
The prior art has low accuracy when conducting semantic analysis of policy texts, especially when facing policy texts with complex structures and rigorous words, it is easy to misjudgment to determine semantic similarity based on semantic vector differences alone.
By obtaining the text of the policy to analyze, standardize pre -processing, obtain the target policy text, and enter it to the pre -training target language model. Data, optimize the similarity of the semantic vector.
The accuracy of semantic analysis of policy text has been improved, so that the model has the ability to judge text similarity through the similarity between semantic vectors, and reduce misjudgment.
Smart Images
Figure CN119378559A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of natural language processing, and in particular to a vector-based semantic analysis method, apparatus, device and storage medium. Background Art
[0002] With the continuous development of natural language processing technology and machine learning technology, semantic vectors have become an important tool for understanding and analyzing text data. Among them, semantic vectors convert text content into vectors in high-dimensional space through mathematical expressions, thereby capturing the semantic similarities and differences between texts.
[0003] At present, the commonly used semantic vector optimization strategy mainly focuses on minimizing the difference between the semantic vector predicted by the semantic analysis model and the true semantic vector, and simply determines whether the texts are similar based on the difference between the vectors themselves. This method is prone to misjudgment when facing policy texts with complex structures and rigorous wording. In other words, the method of judging the semantic similarity of two policy texts based solely on the difference between their respective semantic vectors is low in accuracy.
[0004] Therefore, how to improve the accuracy of semantic analysis of policy texts is a problem that needs to be solved urgently. Summary of the invention
[0005] The main purpose of this application is to provide a vector-based semantic analysis method, device, equipment and storage medium, aiming to improve the accuracy of semantic analysis of policy texts.
[0006] To achieve the above object, the present application provides a vector-based semantic analysis method, the method comprising:
[0007] In response to the semantic analysis request, obtaining the policy text to be analyzed;
[0008] Performing standardization preprocessing on the policy text to be analyzed to obtain a target policy text;
[0009] The target policy text is input into a target language model to obtain a semantic vector of the target policy text, wherein the target language model is trained using the initial semantic vectors of the two sample policy texts of the sample pair as model input data and using label data constructed based on the true semantic similarity between the two sample policy texts as model training labels, and the semantic vector represents the semantic information of the target policy text.
[0010] In one embodiment, the step of performing standardization preprocessing on the policy text to be analyzed to obtain the target policy text includes:
[0011] Performing data cleaning on the policy text to be analyzed to obtain a clean policy text;
[0012] The cleaning policy text is segmented and encoded to obtain a target policy text, wherein the data format of the target policy text is a data format that can be understood by the target language model.
[0013] In one embodiment, the method further comprises:
[0014] Acquire a sample data set, wherein the sample data set includes a plurality of sample data, each of the sample data includes a sample pair, and the true semantic similarity between two sample policy texts of the sample pair;
[0015] Determine the initial semantic vectors of the two sample policy texts in each piece of sample data, and construct label data based on the true semantic similarity in each piece of sample data;
[0016] The initial semantic vectors corresponding to the sample data are used as model input data, and the label data corresponding to the sample data are used as model training labels to train the first pre-trained language model to obtain the target language model.
[0017] In one embodiment, the sample pair is a positive sample pair or a negative sample pair;
[0018] One of the positive sample pairs includes two positive sample policy texts and a first true semantic similarity between the two positive sample policy texts, wherein the first true semantic similarity is greater than a first similarity threshold;
[0019] One negative sample pair includes two sections of negative sample policy texts and a second true semantic similarity between the two sections of the negative sample policy texts, wherein the second true semantic similarity is less than a second similarity threshold, and the second similarity threshold is less than the first similarity threshold.
[0020] In one embodiment, the step of determining the initial semantic vectors of the two sample policy texts in each piece of sample data includes:
[0021] Inputting two sections of the sample policy text in the sample data into a second pre-trained language model respectively to obtain a plurality of subword vectors of the two sections of the sample policy text, wherein the second pre-trained language model has a language understanding function;
[0022] An initial semantic vector of each of the sample policy texts is determined based on the multiple subword vectors of each of the two sample policy texts.
[0023] In one embodiment, the step of determining the initial semantic vector of each of the sample policy texts based on the multiple subword vectors of each of the two sample policy texts comprises:
[0024] Determine a classification subword vector among the multiple subword vectors of the sample policy text, wherein the classification subword vector represents semantic information of the sample policy text;
[0025] The classified subword vector is used as the initial semantic vector of the sample policy text.
[0026] In one embodiment, the step of determining the initial semantic vector of each of the sample policy texts based on the multiple subword vectors of each of the two sample policy texts comprises:
[0027] Determining an average vector of a plurality of said subword vectors of said sample policy text;
[0028] The average value vector is used as the initial semantic vector of the sample policy text.
[0029] In addition, to achieve the above-mentioned purpose, the present application also provides a vector-based semantic analysis device, and the vector-based semantic analysis device includes:
[0030] An acquisition module, used for acquiring the policy text to be analyzed in response to a semantic analysis request;
[0031] A preprocessing module, used for performing standardized preprocessing on the policy text to be analyzed to obtain a target policy text;
[0032] A semantic analysis module is used to input the target policy text into a target language model to obtain a semantic vector of the target policy text, wherein the target language model is trained using the initial semantic vectors of the two sample policy texts of the sample pair as model input data and using label data constructed based on the true semantic similarity between the two sample policy texts as model training labels, and the semantic vector represents the semantic information of the target policy text.
[0033] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer-readable storage medium, and a program for implementing a vector-based semantic analysis method is stored on the computer-readable storage medium. The program for implementing a vector-based semantic analysis method is executed by a processor to implement the steps of the vector-based semantic analysis method as described above.
[0034] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned vector-based semantic analysis method when executed by a processor.
[0035] The present application provides a vector-based semantic analysis method. The present application first obtains a policy text to be analyzed in response to a semantic analysis request, then performs standardized preprocessing on the obtained policy text to be analyzed to obtain a target policy text, and inputs the target policy text into a target language model to obtain a semantic vector for characterizing the semantic information of the target policy text, wherein the target language model is trained using an initial semantic vector of each of two sample policy texts in a sample pair as model input data, and using label data constructed based on the true semantic similarity between the two sample policy samples as model training labels.
[0036] In summary, after receiving the semantic analysis request triggered by the user, the present application obtains the policy text to be analyzed specified by the semantic analysis request, performs standardized preprocessing on the policy text to be analyzed to obtain the target policy text, and then inputs the target policy text into the target language model to obtain the semantic vector of the target policy text. Since the pre-trained target language model has the ability to focus on semantic similarity, compared with the traditional semantic vector optimization strategy that focuses on minimizing text differences and ignores semantic similarity, thereby leading to misjudgment, when using samples to train the language model, the present application uses two initial semantic vectors constructed based on two policy texts as model input data, and uses the label data constructed by the semantic similarity between the two policy texts as model training labels. In this way, the present application trains the model by using the true semantic similarity of the policy text pair as label data. The trained model learns the expression habits and true semantics of the text in the government field, and optimizes the similarity of the semantic vector based on the true semantic similarity, so that the model has the ability to judge whether the texts are actually similar through the similarity between the two semantic vectors, thereby improving the accuracy of the semantic vector finally output, that is, improving the accuracy of the semantic analysis of the policy text. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0039] Figure 1 This is a flow chart of the first embodiment of the vector-based semantic analysis method of the present application;
[0040] Figure 2 A schematic diagram of a language model training process involved in an embodiment of a vector-based semantic analysis method of the present application;
[0041] Figure 3 This is a schematic diagram of the module structure of the vector-based semantic analysis device of this application;
[0042] Figure 4 Schematic diagram of the device structure of the hardware operating environment involved in the vector-based semantic analysis method in the embodiment of the present application.
[0043] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0044] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.
[0045] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.
[0046] The main solution of the present application is: in response to a semantic analysis request, obtaining a policy text to be analyzed; performing standardized preprocessing on the policy text to be analyzed to obtain a target policy text; inputting the target policy text into a target language model to obtain a semantic vector of the target policy text, wherein the target language model is trained using the initial semantic vectors of the two sample policy texts of the sample pair as model input data and using label data constructed based on the true semantic similarity between the two sample policy texts as model training labels, and the semantic vector represents the semantic information of the target policy text.
[0047] At present, the commonly used semantic vector optimization strategy mainly focuses on minimizing the difference between the semantic vector predicted by the semantic analysis model and the true semantic vector, and simply determines whether the texts are similar based on the difference between the vectors themselves. This method is prone to misjudgment when facing policy texts with complex structures and rigorous wording. In other words, the method of judging the semantic similarity of two policy texts based solely on the difference between their respective semantic vectors is low in accuracy.
[0048] After receiving a semantic analysis request triggered by a user, the present application obtains the policy text to be analyzed specified by the semantic analysis request, performs standardized preprocessing on the policy text to be analyzed to obtain the target policy text, and then inputs the target policy text into the target language model to obtain the semantic vector of the target policy text. Since the pre-trained target language model has the ability to focus on semantic similarity, compared with the traditional semantic vector optimization strategy that focuses on minimizing text differences and ignores semantic similarity, thereby leading to misjudgment, when using samples to train the language model, the present application uses two initial semantic vectors constructed based on two sections of policy text as model input data, and uses label data constructed by the semantic similarity between the two sections of policy text as model training labels. In this way, the present application trains the model by using the true semantic similarity of the policy text pair as label data, and the trained model learns the expression habits and true semantics of the text in the government field, and optimizes the similarity of the semantic vector based on the true semantic similarity, so that the model has the ability to judge whether the texts are actually similar through the similarity between the two semantic vectors, thereby improving the accuracy of the semantic vector finally output, that is, improving the accuracy of the semantic analysis of the policy text.
[0049] It should be noted that the execution subject of the method in each embodiment of the vector-based semantic analysis method of the present application can be a semantic analysis system, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a vector-based semantic analysis device that can realize the above functions, etc., and this embodiment does not specifically limit this. The following takes the semantic analysis system as an example of the execution subject to illustrate this embodiment and the following embodiments.
[0050] Based on this, the present application proposes a vector-based semantic analysis method of the first embodiment, please refer to Figure 1 The vector-based semantic analysis method includes steps S10 to S30:
[0051] Step S10, in response to the semantic analysis request, obtaining the policy text to be analyzed;
[0052] It should be noted that the semantic analysis request is used to instruct the semantic analysis system to perform the semantic analysis function. Specifically, the embodiment of the present application does not limit the triggering method of the semantic analysis request. For example, in one feasible implementation, the semantic analysis system includes a user operation interface, which provides an input box for entering text and a virtual button for issuing a semantic analysis request. After the user enters the policy text into the input box, the semantic analysis request can be triggered by clicking the virtual button.
[0053] After the semantic analysis system receives the semantic analysis request triggered by the user, in response to the semantic analysis request, the policy text pointed to by the semantic analysis request (hereinafter referred to as the policy text to be analyzed for distinction) is obtained.
[0054] Step S20, performing standardization preprocessing on the policy text to be analyzed to obtain a target policy text; it should be noted that after obtaining the policy text to be analyzed, it is necessary to perform standardization preprocessing on the policy text to be analyzed to obtain a policy text that meets the input requirements of the target model (hereinafter referred to as the target policy text for distinction).
[0055] In this embodiment, the step S20 may include:
[0056] Step S201, performing data cleaning on the policy text to be analyzed to obtain a clean policy text;
[0057] In a specific implementation, non-alphanumeric characters, punctuation marks, etc. in the text are removed by regular expressions or text cleaning functions to obtain a cleaned policy text (hereinafter referred to as clean policy text for distinction) to reduce noise and improve the purity of the text.
[0058] Step S202, performing word segmentation and encoding processing on the cleaning policy text to obtain a target policy text, wherein the data format of the target policy text is a data format that can be understood by the target language model.
[0059] In a specific implementation, a tokenizer is used to segment the clean policy text into tokens, which can be complete words or more fine-grained sub-word units, and classification tags (such as BERT's [CLS] token) are added at the beginning and end of the text sequence, and separation tags (such as BERT's [SEP] token) are added at appropriate positions in the text sequence. That is, the word segmentation result may contain complete words, sub-word units, and special tags in BERT. Each word block after word segmentation is mapped to a corresponding integer identifier (token ID) for model recognition, and for tasks containing multiple text fragments (such as questions and answers in question-answering tasks), a unique segment identifier (segmentID) is assigned to each fragment so that the model can distinguish different texts, and then a binary array of the same length as the input sequence is generated to mark the location source of the real word, and finally the target policy text is obtained. It should be noted that the tokenizer can be a rule-based tokenizer, a statistical-based tokenizer, or a tokenizer trained by a deep learning method. In addition, the data format of the target policy text is an input format that the target model can understand.
[0060] Step S30, inputting the target policy text into a target language model to obtain a semantic vector of the target policy text, wherein the target language model is trained using the initial semantic vectors of the two sample policy texts in the sample pair as model input data and using label data constructed based on the true semantic similarity between the two sample policy texts as model training labels, and the semantic vector represents the semantic information of the target policy text.
[0061] It should be noted that two policy texts with similar or unrelated topics (hereinafter referred to as sample policy texts for distinction) are selected as sample pairs, and the initial semantic vectors of each sample policy text are constructed based on the two sample policy texts in the sample pair, wherein the initial semantic vector is a semantic vector obtained by semantically analyzing the sample policy text using a general pre-trained language model (such as BERT, GPT, etc.); and label data is constructed based on the true semantic similarity between the two sample policy texts in a sample pair; the initial semantic vectors corresponding to the two sample policy texts in the sample pair are used as model input data, and the label data is used as the model training label to train the pre-trained language model to obtain the target language model. It should be understood that the target language model has a stronger ability to understand the text in the government field and to determine the semantic similarity of policy texts than the general pre-trained language model.
[0062] In this embodiment, the target policy text is input into the target language model to obtain a semantic vector of the target policy text, wherein the semantic vector represents the semantic information of the target policy text.
[0063] It should be noted that after obtaining the semantic vector of the target policy text, the semantic vector is converted into text form to obtain the semantic information corresponding to the semantic vector, and an association relationship between the semantic information and the target policy text is constructed, so that when the user performs a government text retrieval, the corresponding semantic information is matched based on the search text input by the user, and the target policy text associated with the semantic information is pushed to the user for viewing.
[0064] After receiving a semantic analysis request triggered by a user, the embodiment of the present application obtains the policy text to be analyzed specified by the semantic analysis request, performs standardized preprocessing on the policy text to be analyzed to obtain the target policy text, and then inputs the target policy text into the target language model to obtain the semantic vector of the target policy text. Since the pre-trained target language model has the ability to focus on semantic similarity, compared with the traditional semantic vector optimization strategy that focuses on minimizing text differences and ignores semantic similarity, thereby leading to misjudgment, when using samples to train the language model, the present application uses two initial semantic vectors constructed based on two sections of policy text as model input data, and uses label data constructed by the semantic similarity between the two sections of policy text as model training labels. In this way, the embodiment of the present application trains the model by using the real semantic similarity of the policy text pair as label data, and the trained model learns the expression habits and real semantics of the text in the government field, optimizes the similarity of the semantic vector based on the real semantic similarity, so that the model has the ability to judge whether the texts are actually similar through the similarity between the two semantic vectors, thereby improving the accuracy of the semantic vector finally output, that is, improving the accuracy of the semantic analysis of the policy text.
[0065] In addition, compared to the traditional method of manually annotating policy texts and inputting the annotated policy texts into a general pre-trained language model to obtain the semantic vector of the policy text, the embodiment of the present application does not need to consume a large amount of human resources to perform text annotation work, thereby improving the efficiency of semantic analysis.
[0066] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. On this basis, the vector-based semantic analysis method also includes:
[0067] Step A10, obtaining a sample data set, wherein the sample data set includes a plurality of sample data, each of the sample data includes a sample pair, and the true semantic similarity between two sample policy texts of the sample pair;
[0068] Before training the pre-trained language model, a sample data set for model training is obtained, wherein the sample data set includes a plurality of sample data, and each sample data includes a sample pair and the true semantic similarity between two sample policy texts in the sample pair.
[0069] In this embodiment, the sample pair is a positive sample pair or a negative sample pair;
[0070] One of the positive sample pairs includes two positive sample policy texts and a first true semantic similarity between the two positive sample policy texts, wherein the first true semantic similarity is greater than a first similarity threshold;
[0071] One negative sample pair includes two sections of negative sample policy texts and a second true semantic similarity between the two sections of the negative sample policy texts, wherein the second true semantic similarity is less than a second similarity threshold, and the second similarity threshold is less than the first similarity threshold.
[0072] In a specific embodiment, the sample data in the sample data set is positive sample data or negative sample data, that is, the sample pair in the sample data is a positive sample pair or a negative sample pair. It should be understood that the sample data set includes multiple positive sample data and multiple negative sample data. Among them, the positive sample pair in a positive sample data includes two policy texts (hereinafter referred to as positive sample policy texts for distinction) and the real semantic similarity between the two positive sample policy texts (hereinafter referred to as the first real semantic similarity for distinction), and the first real semantic similarity is greater than the first similarity threshold, and the first similarity threshold is a preset lower limit value representing the high similarity between the two policy texts; the negative sample pair in a negative sample data includes two policy texts (hereinafter referred to as negative sample policy texts for distinction) and the real semantic similarity between the two negative sample policy texts (hereinafter referred to as the second real semantic similarity for distinction), and the second real semantic similarity is less than the second similarity threshold, and the second similarity threshold is a preset upper limit value representing the low similarity between the two policy texts.
[0073] Step A20, determining the initial semantic vectors of the two sample policy texts in each piece of sample data, and constructing label data based on the true semantic similarity in each piece of sample data;
[0074] It should be noted that the initial semantic vector of the sample policy text is a semantic vector obtained after a general pre-trained language model performs semantic analysis on the sample policy text. The initial semantic vector represents the pre-trained language model's understanding of the sample policy text.
[0075] An initial semantic vector of each sample policy text is constructed based on the two sample policy texts in each sample data, that is, each sample data includes two initial semantic vectors, and label data is constructed based on the true semantic similarity of the two sample policy texts in each sample data.
[0076] Step A30, using each of the initial semantic vectors corresponding to each of the sample data as model input data, and using the label data corresponding to each of the sample data as model training labels, to train the first pre-trained language model to obtain the target language model.
[0077] After obtaining the initial semantic vectors and label data constructed based on each sample data, the initial semantic vectors corresponding to each sample data are used as model input data, and the label data corresponding to each sample data are used as model training labels, and the pre-trained language model (hereinafter referred to as the first pre-trained language model for distinction) is trained to obtain the target language model. In this way, the initial semantic vectors of similar policy texts are closer in the vector space, while the initial semantic vectors of unrelated policy texts are farther apart in the vector space.
[0078] In one feasible implementation, a loss function suitable for the first pre-trained language model is selected, and the model parameters of the first pre-trained language model are optimized with the goal of maximizing the similarity between positive sample pairs and minimizing the similarity between negative sample pairs until the loss function converges to obtain a target language model. The loss function may be a contrast loss function or a triplet loss function. After obtaining the target language model, the target language model is verified to verify the effectiveness of the semantic vector obtained by the optimized target language model in the field of policy consulting.
[0079] In this embodiment, the step A20 may include:
[0080] Step A201, respectively inputting two sections of the sample policy text in the sample data into a second pre-trained language model to obtain a plurality of subword vectors of the two sections of the sample policy text, wherein the second pre-trained language model has a language understanding function;
[0081] For each piece of sample data, the two sample policy texts in each piece of sample data are input into the pre-trained language model (hereinafter referred to as the second pre-trained language model for distinction), and the sample policy text is semantically analyzed by the second pre-trained language model to obtain multiple subword vectors of the sample policy text, i.e., the model output content. It should be noted that the subword vector is the vector representation of the token output at each layer.
[0082] Step A202: determining an initial semantic vector of each of the sample policy texts based on the multiple subword vectors of each of the two sample policy texts.
[0083] Based on the multiple subword vectors of each sample policy text output by the second pre-trained language model, an initial semantic vector of each sample policy text is determined.
[0084] In a feasible implementation, the step A202 may include:
[0085] Step A2021, determining a classification subword vector among the multiple subword vectors of the sample policy text, wherein the classification subword vector represents semantic information of the sample policy text;
[0086] Step A2022: Use the classified subword vector as the initial semantic vector of the sample policy text.
[0087] In this embodiment, the classification subword vector among the multiple subword vectors of each sample policy text is used as the initial semantic vector of the sample policy text, wherein it should be noted that the classification subword vector represents the semantic information of the sample policy text.
[0088] Exemplarily, when the second pre-trained language model is a BERT model, the token pointed to by the classification subword vector is a [CLS] token, which is located at the beginning of each input sequence, and its vector representation is designed to summarize the semantic information of the entire sequence.
[0089] In another feasible implementation, the step A202 may include:
[0090] Step A2023, determining an average vector of a plurality of said subword vectors of said sample policy text;
[0091] Step A2024, using the average value vector as the initial semantic vector of the sample policy text.
[0092] In this embodiment, the average value of multiple subword vectors of each sample policy text is calculated to obtain an average value vector, and the average value vector is used as the initial semantic vector of the sample policy text.
[0093] For example, Figure 2 The figure shows a schematic diagram of the language model training process, which includes obtaining a sample data set containing multiple positive sample data and multiple negative sample data; determining the initial semantic vector of each sample policy text in the sample data; and training the pre-trained model based on the initial semantic vector corresponding to each sample data and the true semantic similarity until the loss function converges to obtain the target language model.
[0094] In this way, the embodiment of the present application trains the pre-trained language model by using the initial semantic vectors of each sample policy text constructed based on two sample policy texts as model input data, and using the label data constructed based on the true semantic similarity between the two sample policy texts as model training labels to obtain a target language model. The trained target language model has a stronger policy text understanding ability and text similarity judgment ability, thereby improving the accuracy of semantic analysis of policy texts.
[0095] The present application also provides a vector-based semantic analysis device, see Figure 3 , the vector-based semantic analysis device comprises:
[0096] An acquisition module 10, for acquiring a policy text to be analyzed in response to a semantic analysis request;
[0097] A preprocessing module 20 is used to perform standardization preprocessing on the policy text to be analyzed to obtain a target policy text;
[0098] The semantic analysis module 30 is used to input the target policy text into a target language model to obtain a semantic vector of the target policy text, wherein the target language model is trained using the initial semantic vectors of the two sample policy texts of the sample pair as model input data and using label data constructed based on the true semantic similarity between the two sample policy texts as model training labels, and the semantic vector represents the semantic information of the target policy text.
[0099] Optionally, the preprocessing module 20 is further used for:
[0100] Performing data cleaning on the policy text to be analyzed to obtain a clean policy text;
[0101] The cleaning policy text is segmented and encoded to obtain a target policy text, wherein the data format of the target policy text is a data format that can be understood by the target language model.
[0102] Optionally, the vector-based semantic analysis device further includes a model training module, and the model training module is used to:
[0103] Acquire a sample data set, wherein the sample data set includes a plurality of sample data, each of the sample data includes a sample pair, and the true semantic similarity between two sample policy texts of the sample pair;
[0104] Determine the initial semantic vectors of the two sample policy texts in each piece of sample data, and construct label data based on the true semantic similarity in each piece of sample data;
[0105] The initial semantic vectors corresponding to the sample data are used as model input data, and the label data corresponding to the sample data are used as model training labels to train the first pre-trained language model to obtain the target language model.
[0106] Optionally, the sample pair is a positive sample pair or a negative sample pair;
[0107] One of the positive sample pairs includes two positive sample policy texts and a first true semantic similarity between the two positive sample policy texts, wherein the first true semantic similarity is greater than a first similarity threshold;
[0108] One negative sample pair includes two sections of negative sample policy texts and a second true semantic similarity between the two sections of the negative sample policy texts, wherein the second true semantic similarity is less than a second similarity threshold, and the second similarity threshold is less than the first similarity threshold.
[0109] Optionally, the model training module is also used to:
[0110] The step of determining the initial semantic vectors of the two sample policy texts in each piece of sample data includes:
[0111] Inputting two sections of the sample policy text in the sample data into a second pre-trained language model respectively to obtain a plurality of subword vectors of the two sections of the sample policy text, wherein the second pre-trained language model has a language understanding function;
[0112] An initial semantic vector of each of the sample policy texts is determined based on the multiple subword vectors of each of the two sample policy texts.
[0113] Optionally, the model training module is also used to:
[0114] Determine a classification subword vector among the multiple subword vectors of the sample policy text, wherein the classification subword vector represents semantic information of the sample policy text;
[0115] The classified subword vector is used as the initial semantic vector of the sample policy text.
[0116] Optionally, the model training module is also used to:
[0117] Determining an average vector of a plurality of said subword vectors of said sample policy text;
[0118] The average value vector is used as the initial semantic vector of the sample policy text.
[0119] The vector-based semantic analysis device provided in the embodiment of the present application adopts the vector-based semantic analysis method in the above embodiment, which can solve the technical problem of how to improve the accuracy of semantic analysis of policy texts. Compared with the prior art, the beneficial effects of the vector-based semantic analysis device provided in the embodiment of the present application are the same as the beneficial effects of the vector-based semantic analysis method provided in the above embodiment, and the other technical features in the vector-based semantic analysis device are the same as the features disclosed in the above embodiment method, which will not be repeated here.
[0120] The present application provides a vector-based semantic analysis device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the vector-based semantic analysis method in the above-mentioned embodiment one.
[0121] Reference below Figure 4 , which shows a schematic diagram of the structure of a vector-based semantic analysis device suitable for implementing the embodiment of the present application. The vector-based semantic analysis device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The vector-based semantic analysis device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0122] like Figure 4As shown, the vector-based semantic analysis device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the vector-based semantic analysis device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the vector-based semantic analysis device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a vector-based semantic analysis device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.
[0123] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.
[0124] The vector-based semantic analysis device provided by the present application adopts the vector-based semantic analysis method in the above-mentioned embodiment, which can solve the technical problem of how to improve the accuracy of semantic analysis of policy texts. Compared with the prior art, the beneficial effects of the vector-based semantic analysis device provided by the present application are the same as the beneficial effects of the vector-based semantic analysis method provided by the above-mentioned embodiment, and the other technical features in the vector-based semantic analysis device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.
[0125] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0126] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0127] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the vector-based semantic analysis method in the above-mentioned embodiment.
[0128] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.
[0129] The computer-readable storage medium may be included in the vector-based semantic analysis device; or may exist independently without being assembled into the vector-based semantic analysis device.
[0130] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a vector-based semantic analysis device, the vector-based semantic analysis device: obtains a policy text to be analyzed in response to a semantic analysis request; performs standardized preprocessing on the policy text to be analyzed to obtain a target policy text; inputs the target policy text into a target language model to obtain a semantic vector of the target policy text, wherein the target language model is trained using the initial semantic vectors of the two sample policy texts of the sample pair as model input data and using label data constructed based on the true semantic similarity between the two sample policy texts as model training labels, and the semantic vector represents the semantic information of the target policy text.
[0131] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0132] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0133] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.
[0134] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned vector-based semantic analysis method, and can solve the technical problem of how to improve the accuracy of semantic analysis of policy texts. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the vector-based semantic analysis method provided in the above-mentioned embodiment, and will not be elaborated here.
[0135] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the vector-based semantic analysis method as described above when executed by a processor.
[0136] The computer program product provided by this application can improve the accuracy of semantic analysis of policy texts. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of this application are the same as the beneficial effects of the vector-based semantic analysis method provided by the above embodiment, which will not be repeated here.
[0137] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.
Claims
1. A vector-based semantic analysis method, characterized in that: The method comprises: In response to a semantic analysis request, obtaining a policy text to be analyzed; Performing standardization preprocessing on the policy text to be analyzed to obtain a target policy text; The target policy text is input into a target language model to obtain a semantic vector of the target policy text, wherein the target language model is trained using the initial semantic vectors of the two sample policy texts of the sample pair as model input data and using label data constructed based on the true semantic similarity between the two sample policy texts as model training labels, and the semantic vector represents the semantic information of the target policy text.
2. The method according to claim 1, characterized in that The step of performing standardization preprocessing on the policy text to be analyzed to obtain the target policy text includes: Performing data cleaning on the policy text to be analyzed to obtain a clean policy text; The cleaning policy text is segmented and encoded to obtain a target policy text, wherein the data format of the target policy text is a data format that can be understood by the target language model.
3. The method according to claim 1, characterized in that The method further comprises: Acquire a sample data set, wherein the sample data set includes a plurality of sample data, each of the sample data includes a sample pair, and the true semantic similarity between two sample policy texts of the sample pair; Determine the initial semantic vectors of the two sample policy texts in each piece of sample data, and construct label data based on the true semantic similarity in each piece of sample data; The initial semantic vectors corresponding to the sample data are used as model input data, and the label data corresponding to the sample data are used as model training labels to train the first pre-trained language model to obtain the target language model.
4. The method according to claim 3, characterized in that The sample pair is a positive sample pair or a negative sample pair; One of the positive sample pairs includes two positive sample policy texts and a first true semantic similarity between the two positive sample policy texts, wherein the first true semantic similarity is greater than a first similarity threshold; One negative sample pair includes two sections of negative sample policy texts and a second true semantic similarity between the two sections of the negative sample policy texts, wherein the second true semantic similarity is less than a second similarity threshold, and the second similarity threshold is less than the first similarity threshold.
5. The method according to claim 3, characterized in that The step of determining the initial semantic vectors of the two sample policy texts in each piece of sample data includes: Inputting two sections of the sample policy text in the sample data into a second pre-trained language model respectively to obtain a plurality of subword vectors of the two sections of the sample policy text, wherein the second pre-trained language model has a language understanding function; An initial semantic vector of each of the sample policy texts is determined based on the multiple subword vectors of each of the two sample policy texts.
6. The method according to claim 5, characterized in that The step of determining the initial semantic vector of each of the sample policy texts based on the multiple subword vectors of each of the two sample policy texts comprises: Determine a classification subword vector among the multiple subword vectors of the sample policy text, wherein the classification subword vector represents semantic information of the sample policy text; The classified subword vector is used as the initial semantic vector of the sample policy text.
7. The method according to claim 5, characterized in that The step of determining the initial semantic vector of each of the sample policy texts based on the multiple subword vectors of each of the two sample policy texts comprises: Determining an average vector of a plurality of said subword vectors of said sample policy text; The average value vector is used as the initial semantic vector of the sample policy text.
8. A vector-based semantic analysis device, characterized in that: The vector-based semantic analysis device comprises: An acquisition module, used for acquiring the policy text to be analyzed in response to a semantic analysis request; A preprocessing module, used for performing standardized preprocessing on the policy text to be analyzed to obtain a target policy text; A semantic analysis module is used to input the target policy text into a target language model to obtain a semantic vector of the target policy text, wherein the target language model is trained using the initial semantic vectors of the two sample policy texts of the sample pair as model input data and using label data constructed based on the true semantic similarity between the two sample policy texts as model training labels, and the semantic vector represents the semantic information of the target policy text.
9. A vector-based semantic analysis device, characterized in that: The vector-based semantic analysis device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the vector-based semantic analysis method as described in any one of claims 1 to 7.
10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the vector-based semantic analysis method according to any one of claims 1 to 7 are implemented.