Positioning model training method, text positioning method and related device
By constructing hierarchical title text data and training a target localization model, and using feature vectors and similarity thresholds to determine positive and negative samples, the problem of title localization in complex long texts is solved, and the localization efficiency of document review is improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN SECURITIES INFORMATION
- Filing Date
- 2023-03-10
- Publication Date
- 2026-05-12
AI Technical Summary
Existing technologies struggle to effectively locate titles or paragraphs related to review issues within complex, long texts, especially when title features are not obvious or scattered. The amount of annotation required is large and difficult to exhaustively count, and negative samples cannot be determined through simple complements.
By acquiring review question data and title text data, hierarchical title text data is constructed, a target localization model is trained, and positive and negative samples are determined using feature vectors and similarity thresholds. The localization model is then optimized to improve localization efficiency.
It enables efficient location of titles related to review issues in complex and long texts, reducing the amount of annotation and improving the efficiency of location during the document review process.
Smart Images
Figure CN116227430B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a localization model training method, a text localization method, and related equipment. Background Technology
[0002] In recent years, with the rapid development of the internet and the rapid growth of text information, intelligent review has become an inevitable trend in the face of massive document review pressure. Intelligent review obtains samples through corpus annotation, and then uses model algorithms to train and judge to obtain the answer. However, for documents to be reviewed, manually annotating all paragraphs is a huge workload. Existing solutions usually first locate the source, then narrow down a candidate range, and finally annotate the candidate range.
[0003] For example, regular expression matching algorithms can be used to match document titles, and a candidate range can be obtained by locating the titles to reduce the workload of annotation. However, for some review questions, the features of the titles are not obvious or are scattered, making it difficult to locate the titles using regular expression matching algorithms. In addition, in the location of titles in long documents, the target paragraph of a review question may appear under multiple titles, and how to obtain a candidate set of these possible titles becomes the first problem encountered. For annotators, "possible title locations" are difficult to annotate, partly because the amount of annotation is huge, and partly because it is difficult to exhaustively list them all. In common text binary classification problems, after positive samples are labeled, unlabeled samples automatically become negative samples, but for document title location, it is impossible to determine negative samples by simply using the complement set.
[0004] Therefore, in document review scenarios, how to identify the headings or paragraphs in complex, long texts that are relevant to the review questions has become a pressing technical problem. Summary of the Invention
[0005] This application provides a localization model training method, a text localization method, and related equipment for determining model samples and locating text based on the trained model.
[0006] A first aspect of this application provides a method for training a localization model, the method comprising:
[0007] Obtain the review issue data and title text data, wherein the title text data corresponds to each title level;
[0008] Based on the audit question data, obtain the hierarchical title text data from the title text data, wherein the hierarchical title text data is title text data under multiple title levels;
[0009] Obtain the title text data under the target title level from the hierarchical title text data, wherein the title text data under the target title level includes at least the hierarchical title text data under the target title level;
[0010] The target title text data under the target title level is used as training samples to train the initial localization model until the preset conditions are met, and then the training stops to obtain the target localization model.
[0011] Optionally, obtaining the hierarchical title text data from the title text data based on the review question data includes:
[0012] Obtain the paragraph text data corresponding to the title text data, and annotate the paragraph text data and title text data according to the review question data to obtain the annotated paragraph text data and the annotated title text data;
[0013] A text corpus is constructed based on the title text data and the paragraph text data, wherein the paragraph text data corresponds to different title levels in the title text data.
[0014] The title level corresponding to any paragraph text data in the text corpus is concatenated to obtain the title text data.
[0015] Obtain the corresponding level heading text data based on the labeled paragraph text data.
[0016] Optionally, obtaining the target title text data under the target title level from the title text data includes:
[0017] All the hierarchical title text data are input into a target language model to calculate a first feature vector of all the hierarchical title text data, wherein the target language model is used to obtain the feature vector;
[0018] Based on the first feature vectors corresponding to all the hierarchical title text data, calculate the average feature vector of the hierarchical title text data, and the first similarity between all the first feature vectors and the average feature vector;
[0019] A target similarity threshold is determined based on all the first similarities;
[0020] The target heading level of the paragraph text data is located based on the labeled paragraph text data, wherein the target heading level is the heading level corresponding to the paragraph text data;
[0021] Obtain the title text data of all title levels under the target title level, and construct the title text data under the target title level based on the title text data of all title levels.
[0022] Optionally, determining the target similarity threshold based on all the first similarities includes:
[0023] Calculate the average of all the first similarities to obtain the initial similarity threshold;
[0024] The initial similarity threshold is calculated together with a preset value to obtain the target similarity threshold.
[0025] Optionally, the step of using the target title text data under the target title level as training samples to train the initial localization model until a preset condition is met to stop training, thereby obtaining the target localization model, includes:
[0026] Construct a target dataset based on the target title text data;
[0027] The target title text data in the target dataset is used as training samples to train the initial localization model to obtain the target localization model.
[0028] Optionally, the step of using the target title text data in the target dataset as training samples to train the initial localization model to obtain the target localization model includes:
[0029] Set all labeled hierarchical title text data in the target dataset as positive samples;
[0030] All the title text data in the target dataset, excluding the labeled hierarchical title text data, are input into the target language model to obtain multiple second feature vectors;
[0031] Calculate the second similarity between any second feature vector and the average feature vector, and determine the title text data corresponding to the second similarity that is less than the target similarity threshold as a negative sample;
[0032] The positive and negative samples are used as training samples to train the initial localization model to obtain the target localization model.
[0033] A second aspect of this application provides a localization model training system, comprising:
[0034] The acquisition unit is used to acquire audit issue data and title text data, wherein the title text data corresponds to each title level;
[0035] The acquisition unit is further configured to acquire hierarchical title text data in the title text data based on the audit question data, wherein the hierarchical title text data is title text data under multiple title levels;
[0036] The acquisition unit is further configured to acquire target title text data under the target title level from the title text data, wherein the target title text data includes at least the hierarchical title text data under the target title level;
[0037] The training unit is used to train the initial positioning model using the target title text data under the target title level as training samples until the preset conditions are met, and then stop training to obtain the target positioning model.
[0038] A third aspect of this application provides a text location method, including:
[0039] Acquire the target text data to be processed and the review issue data, wherein the target text data includes at least the target title text data;
[0040] Obtain a pre-trained target localization model, wherein the target localization model is the target localization model described in the first aspect;
[0041] The audit question data and the target title text data are input into the target positioning model to obtain the prediction result of the audit question data output by the target positioning model, wherein the prediction result includes at least the positioning title text data.
[0042] Optionally, the target text data further includes target paragraph text data, and the process of obtaining the target text data to be processed includes:
[0043] Obtain the target text data;
[0044] The target text data is parsed to obtain the target title text data and the target paragraph text data, wherein the target paragraph text data corresponds to different title levels in the target title text data.
[0045] A text corpus is constructed based on the target title text data and the target paragraph text data.
[0046] Optionally, after inputting the review question data and the target title text data into the target positioning model to obtain the prediction result of the review question data output by the target positioning model, the method further includes:
[0047] Obtain the positioning paragraph text data corresponding to the positioning title text data, and construct a text library for the initial judgment model based on the positioning paragraph text data;
[0048] Based on any of the audit issue data, the label data for positive samples is determined as the first label data, and the label data for negative samples is determined as the second label data;
[0049] The initial judgment model is used to judge and train any of the audit question data and the positive sample to output the first judgment label data, and the first judgment label data is made to approach the first label data.
[0050] The initial judgment model is used to judge and train any of the audit issue data and the negative sample to output the second judgment label data, and the second judgment label data is made to approach the second label data.
[0051] When the first judgment label data or the second judgment label data satisfies the first label data or the second label data, it is determined that the initial judgment model training is completed and the target judgment model is formed.
[0052] The target judgment model is used to judge the located paragraph text data to obtain the target paragraph content data corresponding to the located paragraph text data, wherein the target paragraph content data is the review reference answer of the review question data.
[0053] A fourth aspect of this application provides a text positioning system, including:
[0054] The acquisition unit is used to acquire target text data to be processed and review issue data, wherein the target text data includes at least target title text data;
[0055] The acquisition unit is further configured to acquire a pre-trained target localization model, wherein the target localization model is the target localization model described in the first aspect;
[0056] An input unit is used to input the review question data and the target title text data into the target positioning model to obtain the prediction result of the review question data output by the target positioning model, wherein the prediction result includes at least the positioning title text data.
[0057] The system provided in the fourth aspect of this application is used to perform the text positioning method described in the third aspect.
[0058] The fifth aspect of this application provides a localization model training device, comprising:
[0059] Central processing unit, memory, input / output interfaces, wired or wireless network interfaces, and power supply;
[0060] The memory is either a short-term storage memory or a persistent storage memory;
[0061] The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the method described in the first aspect.
[0062] A sixth aspect of this application provides a text positioning device, comprising:
[0063] Central processing unit, memory, input / output interfaces, wired or wireless network interfaces, and power supply;
[0064] The memory is either a short-term storage memory or a persistent storage memory;
[0065] The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the method described in the third aspect.
[0066] A fourth aspect of this application provides a computer-readable storage medium, characterized in that the computer-readable storage medium includes instructions that, when executed on a computer, cause the computer to perform the method described in the first aspect or the third aspect.
[0067] As can be seen from the above technical solutions, the embodiments of this application have the following advantages: The text localization method proposed in this application first acquires the target text data to be processed and the review question data; then, it acquires a pre-trained target localization model; finally, it inputs the review question data and the target title text data into the target localization model to obtain the prediction result of the review question data output by the target localization model. Therefore, the position of the title related to the review question can be determined in the entire text data, and the trained localization model can be used by subsequent users, maximizing the localization efficiency in the document review process. Attached Figure Description
[0068] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings.
[0069] Figure 1 This is a flowchart illustrating a localization model training method disclosed in an embodiment of this application;
[0070] Figure 2This is a flowchart illustrating a text location method disclosed in an embodiment of this application;
[0071] Figure 3 This is a flowchart illustrating another text location method disclosed in an embodiment of this application;
[0072] Figure 4 This is a schematic diagram of the structure of a localization model training system disclosed in an embodiment of this application;
[0073] Figure 5 This is a schematic diagram of the structure of a text positioning system disclosed in an embodiment of this application;
[0074] Figure 6 This is a schematic diagram of the structure of a positioning model training device disclosed in an embodiment of this application;
[0075] Figure 7 This is a schematic diagram of the structure of a text positioning device disclosed in an embodiment of this application. Detailed Implementation
[0076] In existing technologies, it is difficult for annotators to label "possible title locations" due to both the sheer volume of annotations and the difficulty in exhaustively identifying them. In common text binary classification problems, once positive samples are labeled, unlabeled samples automatically become negative samples. However, for document title localization, it is impossible to determine negative samples through simple complements.
[0077] Meanwhile, in the question-and-answer domain, negative samples are identified by training similarity models and methods that do not include extracted segments. However, document review is different from the question-and-answer scenario, and it is not possible to distinguish between positive and negative samples by extracting segments.
[0078] Therefore, in document review scenarios, determining the candidate set of annotations for complex and long texts and identifying negative samples for the model using limited annotations have become pressing technical problems. To address this, this application proposes a text localization method to determine model samples and locate text based on a trained model.
[0079] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0080] Please see Figure 1 , Figure 1 This is a flowchart illustrating a localization model training method disclosed in an embodiment of this application. It includes steps 101-104.
[0081] 101. Obtain the review issue data and title text data.
[0082] For each specific question and each specific document (or article), the location of the question within the document is uncertain. Therefore, a localization model is needed to determine the position of the question within the document. This embodiment proposes a localization model training method to train a localization model, thereby facilitating the location of the title within the document containing any specific question.
[0083] In one specific embodiment, multiple sets of review question data and multiple sets of text data can be obtained. The text data includes at least title text data and paragraph text data. It is easy to understand that the review question data refers to the question the user is searching for, while the text data can be understood as documents or articles, etc. The specific format of the text data is not restricted here. Furthermore, a set of text data may contain one or more review question data sets; this is not specifically limited here. Since the review question data and title text data are primarily used for training the localization model, the following description will use these two types of data.
[0084] In one specific embodiment, multiple sets of review question data and multiple sets of title text data are obtained. In this embodiment, a set of title text data may contain multiple segments of title text data, and a set of title text data may also correspond to one or more review question data. The specific content and quantity of the title text data and review question data are not limited here, and will not be elaborated further. For ease of understanding and description, only the title text data and paragraph text data of the same set will be described in detail below.
[0085] 102. Obtain the hierarchical title text data from the title text data based on the audit issue data.
[0086] The title text data includes various heading levels, such as first-level heading, second-level heading, or third-level heading. Specifically, the heading levels mentioned above can be understood as the hierarchical structure of headings in the document.
[0087] Therefore, the hierarchical heading text data within the heading text data can be obtained from the review issue data. The hierarchical heading text data refers to the heading text data under a specific heading level.
[0088] In one specific embodiment, the annotator can annotate a small number of paragraphs in the file based on each review issue data. Then, using the aforementioned file parsing technology, the annotated paragraphs and their corresponding heading text data at all levels can be obtained. By using delimiters to concatenate the heading text data at all levels of the paragraph, the hierarchical heading text data (i.e., the hierarchical heading text data described above) can be obtained.
[0089] It should be noted that the concatenation of heading text data at all levels can be understood as follows: a paragraph is a level 1 heading (AA); a level 2 heading (BB); and a level 3 heading (CC). Therefore, concatenating AABBCC with a concatenation character will be considered as concatenating heading text at all levels.
[0090] 103. Obtain the title text data of the target title level from the hierarchical title text data.
[0091] Once all the hierarchical title text data is obtained, the target title level of the problem can be determined based on the review question data. Then, all the title text data under that target title level can be used as a dataset for subsequent training samples.
[0092] In one specific embodiment, after obtaining the paragraph text data marked in the above steps, all first-level headings of that paragraph text data can be located. Thus, the heading text data of all levels under the first-level headings is taken as the dataset, which is the heading text data under the target heading level. It is easy to understand that, since the above description refers to the hierarchical heading text data, which includes the heading text data of each heading level, the heading text data under the first-level heading level is a subset of the hierarchical heading text data. For ease of understanding and description, this will not be elaborated further below.
[0093] 104. Use the title text data under the target title level as training samples to train the initial positioning model until the preset conditions are met, then stop training to obtain the target positioning model.
[0094] Based on the title text data of the target title level obtained in step 103 above, in one specific embodiment, the title text data containing the labeled paragraph text data is used as positive samples, and the remaining title text data in the dataset that meets the preset requirements is used as negative samples. To facilitate the subsequent training of the primary localization model, the remaining title text data that meets the preset requirements can also be input into a language model to determine the average value vector and threshold of the title text data. For ease of understanding and description, this will be elaborated upon later. Using the aforementioned positive and negative samples as training samples, the initial localization model is trained. Training can stop once the output values in the initial localization model samples meet the preset conditions, thus obtaining the target localization model. The specific training process will not be elaborated upon here.
[0095] Based on the above embodiments, in another specific embodiment, the preset condition can be that the evaluation index in the positioning model reaches a preset condition. For example, it can be known that the index reaches the preset condition when the accuracy or recall rate meets the business requirements.
[0096] The positioning model training method proposed in this embodiment can train the positioning model based on the review question and the title text in the text, which improves the feasibility of the solution. At the same time, it can also avoid other review problems such as the inability of regular expression matching algorithms to locate the title, thus improving the model training effect.
[0097] Please see Figure 2 , Figure 1 This is a flowchart illustrating a text localization method disclosed in an embodiment of this application. It includes steps 201-203.
[0098] 201. Obtain the target text data to be processed and the audit issue data.
[0099] Before analyzing any article and searching for answer paragraphs related to the review question, it is necessary to first identify the article or document to be processed and the corresponding review question.
[0100] In one specific embodiment, it is necessary to first obtain the target text data to be processed and the review question data. It is easy to understand that the target text data described in this embodiment is the article or document described above, and its file format can be WORD or PDF, etc. Specifically, there are no restrictions on the format or file type of the target text data here. Furthermore, the review question data can be understood as the review questions that users need to find or determine. For example, in a company's prospectus (target text data), users may want to know the company's revenue for this year; this revenue information can be understood as the review question data.
[0101] Based on the above embodiments, in another embodiment, the article or document to be processed can be parsed into title text and content paragraph text respectively. Each content paragraph text needs to correspond one-to-one with its multi-level heading text. It is easy to understand that the title text described in this embodiment is the target title text data described above, and the content paragraph text is the target paragraph text data described above. For ease of understanding, this will not be elaborated further. It is also important to understand that step 201 mainly parses the text in an article or document into title and non-title content. Each non-title content, i.e., content paragraph, can find its title and parent title (e.g., this paragraph is a paragraph under AA level 1 heading / BB level 2 heading / CC level 3 heading).
[0102] 202. Obtain the pre-trained target localization model.
[0103] Since the review question data and target text data have been determined, the localization model can be trained based on the target text data and review question data to obtain the trained target localization model, thereby locating the title of the review question data in the parsed target text data.
[0104] In one specific embodiment, the target localization model can also be trained by a machine learning algorithm on multiple sets of text data training samples. It is easy to understand that these text data training samples can be the target text data mentioned above, or text data from other documents or articles; details will not be elaborated here. It is also easy to understand that the text data described above will include corresponding title text data and paragraph text data, i.e., title text and content paragraph text.
[0105] Based on the above embodiments, it can be understood that for each specific review question and each specific document, the trained localization model is used to locate the corresponding title text of the review question within the parsed document, and then the candidate set of annotations is obtained through the title text localization. In other words, in application, the obtained target localization model can locate the required position (title) within a document. The target localization model is used for article localization to filter the candidate set. Furthermore, the candidate set of annotations refers to the fact that after locating the title text, the title text and the paragraphs it contains are all potential answers to the review question; therefore, the title text and the paragraphs it contains become the candidate set of annotations.
[0106] In one embodiment, the annotation and training method for the localization model involves first annotating a small amount of paragraph text related to the question in a text corpus, and then locating the title from the annotated paragraph text to construct a title text dataset for training the localization model. The text corpus can be divided into two sub-corpora for the model. One sub-corpus contains only hierarchical titles, which are obtained from the title text after title localization. The other sub-corpus contains at least title text and paragraph text, which are obtained after paragraph identification.
[0107] 203. Input the audit question data and target title text data into the target positioning model to obtain the prediction results of the audit question data output by the target positioning model.
[0108] After obtaining the trained target localization model, the title text prediction of the audit question data can be performed.
[0109] In one specific embodiment, the audit question data and the target title text data can be input into the target positioning model to obtain the prediction results related to the audit question data output by the target positioning model. The result corresponds to the positioning title text data, that is, the title text where the answer to the audit question data is located.
[0110] This embodiment proposes a text localization method that first acquires the target text data to be processed and the review question data; then, it acquires a pre-trained target localization model, which is trained using deep learning on multiple sets of text data training samples, each set including the title text data; the review question data and the target title text data are input into the target localization model to obtain the prediction result of the review question data output by the target localization model. Thus, the position of the title related to the review question can be determined within the entire text data, and the trained localization model can be used by subsequent users, maximizing the localization efficiency in the document review process.
[0111] For a detailed description of the text localization method and localization model training method proposed in the embodiments of this application, please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a flowchart illustrating another text localization method disclosed in an embodiment of this application. It includes steps 301-310.
[0112] 301. Obtain the target text data and parse it to obtain the target title text data and the target paragraph text data.
[0113] In this embodiment, step 301 is the same as the aforementioned Figure 1Step 201 is similar, and will not be elaborated here.
[0114] 302. Mark the title text data and paragraph text data corresponding to the audit issue data based on the audit issue data.
[0115] It should be noted beforehand that steps 302-309 can be understood as the training process for the localization model. Specifically, it can be understood as the training process for... Figure 1 A detailed description of steps 101-104 is provided below. Specifically, a text corpus can be constructed based on title text data and paragraph text data. This text corpus can be understood as a database containing at least title text and content paragraph text. It is easy to understand that title text and content paragraph text can be either the title text data and paragraph text data mentioned above, or the target title text data and target paragraph text data mentioned above; no specific limitation is made here. For ease of description, the text data in the text corpus will be described in detail below as either paragraph text data or title text data.
[0116] Then, the heading levels corresponding to any paragraph in the text corpus can be concatenated to obtain the hierarchical heading text data of the paragraph text data to be annotated. It is easy to understand that this is because each content paragraph text needs to correspond one-to-one with its multi-level heading text.
[0117] In one specific embodiment, delimiters can be used on each paragraph text (i.e., paragraph text data) in the text corpus to concatenate all levels of heading text for that paragraph text, thereby obtaining the hierarchical heading text of the paragraph text to be annotated, and constructing a heading text library accordingly. Specifically, for all levels of heading text, for example, if a paragraph text is a paragraph under AA (first-level heading), BB (second-level heading), and CC (third-level heading), then AABBCC can be connected with a concatenation character, thus being regarded as a concatenation of all levels of heading text. Correspondingly, the hierarchical heading text is the hierarchical heading text data described above. For ease of understanding and description, this will not be elaborated further hereafter.
[0118] Based on the above embodiments, it is easy to understand that each article or document will have many paragraphs, and each paragraph has its own first-level, second-level, third-level, and fourth-level headings (heading text). At this time, the heading texts corresponding to all paragraphs can be extracted (in one embodiment, duplicate headings will be deduplicated), and these heading texts can be concatenated using concatenation characters as the corpus / text to be predicted for the localization model, and a heading text library can be constructed based on this hierarchical heading text.
[0119] Of course, in another specific embodiment, the title text can be annotated on the document first. After the annotation is completed, the document is parsed to obtain the title text. Then, the title text and paragraph text are concatenated to form a text corpus. The specific order of annotation and concatenation is not restricted here and can be adjusted according to user needs.
[0120] 303. Construct a text corpus based on title text data and paragraph text data, and concatenate the title levels corresponding to any paragraph text data in the text corpus.
[0121] Correspondingly, step 303 and the aforementioned step 302 are not executed in any particular order, which will not be elaborated here.
[0122] In one specific embodiment, a small amount of paragraph text data corresponding to each review question can be annotated within the text corpus constructed in the preceding steps. Specifically, this involves manually annotating a small segment of text data to identify the paragraph text that can answer the review question (i.e., the annotated paragraph text data). It is self-explanatory that this will not be elaborated upon further.
[0123] 304. Obtain the corresponding level heading text data based on the labeled paragraph text data, and input all level heading text data into the target language model to calculate the first feature vector of all level heading text data. Then, calculate the average feature vector of the level heading text data and the first similarity between all first feature vectors and the average feature vector based on the first feature vectors corresponding to all level heading text data.
[0124] Based on the labeled paragraph text data, the corresponding title text data is obtained. At this point, the title text data is the hierarchical title text data corresponding to the labeled paragraph text data in step 302. Then, all the hierarchical title text data obtained at this stage are input into the target language model to calculate the feature vector of that hierarchical title text data, which is the first feature vector. Since there are multiple hierarchical title text data, there will be multiple feature vectors in the first feature vector. Finally, the average value of the feature vectors corresponding to all hierarchical title text data is calculated based on each feature vector in the first feature vector, which is the average feature vector. Therefore, the similarity between the two, i.e., the first similarity, can be determined based on the average feature vector and the first feature vector.
[0125] In one specific embodiment, the heading text for each level, concatenated with delimiters, is obtained from the annotated paragraph text. This heading text is then input into the target language model to calculate its feature vector. The average of all the obtained feature vectors is then calculated and used as the average feature vector. The language model can then be used to calculate the similarity between each feature vector and the average feature vector.
[0126] It is easy to understand that the target language model can be trained by oneself or obtained from open source. It is mainly used to obtain feature vectors. The specific method of obtaining the target language model is not restricted here, and will not be elaborated on further.
[0127] 305. Determine the target similarity threshold based on all first similarities.
[0128] In one specific embodiment, after obtaining the first similarity determined by each first feature vector and the average feature vector, an average value is calculated based on all the first similarities. This average value can be understood as the similarity threshold, which is the initial similarity threshold described above. Then, this initial similarity threshold is compared with a preset value to obtain the target similarity threshold. Specifically, an empirical constant value can be subtracted from or added to the initial similarity threshold, and the difference is used as the final threshold, i.e., the target similarity threshold.
[0129] It should be noted that this empirical constant value can typically be 0.1, 0.2, etc. If the similarity threshold is 0.9, it can be fine-tuned according to the actual situation, setting the final threshold to 0.89, 0.91, etc. The specific value (preset value) of the empirical constant is not restricted here, and will not be elaborated further later.
[0130] 306. Locate the first heading level of the paragraph text data based on the labeled paragraph text data, then obtain the heading text data of all heading levels corresponding to the first heading level, and construct the target dataset based on all heading text data.
[0131] Based on the annotated paragraph text data obtained in step 303, the first heading level corresponding to that paragraph text data can be located, where the first heading level represents the heading level where the paragraph text data is located. Then, based on the first heading level, the heading text data of all levels under that heading level are obtained and used as the target dataset.
[0132] In one embodiment, the annotated paragraph text data can be located to all first-level headings (i.e., the first heading level) of that paragraph text, and then the heading text of all levels of the first-level headings can be taken as the dataset (i.e., the target dataset). It should be noted that the first-level headings can be understood as including an extremely long document (or article), such as an article with a dozen chapters and hundreds of pages. In this case, it is generally believed that a review question can be found in one or two chapters, so the first-level headings are initially selected, i.e., 1-2 chapters are fixed. Of course, in another embodiment, other numbers of chapters can also be fixed according to the actual screening requirements. This is not limited here and will not be elaborated on further.
[0133] It is important to understand that the title text data contained in the target dataset at this time includes not only the title text data of all levels, but also the title text data of each level concatenated with delimiters in step 306 above.
[0134] 307. Set all labeled hierarchical title text data in the target dataset as positive samples, and input all title text data in the target dataset except for the labeled hierarchical title text data into the target language model to obtain multiple second feature vectors.
[0135] The title text data of each level concatenated with delimiters in the target dataset of step 306 above (i.e., all labeled title text data of each level) are used as positive samples. Then, the title text data of the target dataset excluding the labeled title text data is input into the target language model (i.e. the unlabeled title text) to obtain the second feature vector corresponding to the remaining title text data.
[0136] In one specific embodiment, the labeled title texts of each level concatenated with delimiters in the target dataset are used as positive samples. The remaining title texts in the target dataset are then concatenated with delimiters, and the feature vector, i.e. the second feature vector, is obtained using the target language model.
[0137] 308. Calculate the second similarity between any second feature vector and the average feature vector, and determine the title text data corresponding to the second similarity less than the target similarity threshold as a negative sample.
[0138] Once the second feature vectors of the remaining title text data in the target dataset are obtained, the corresponding second similarity can be calculated with the average feature vector. At this point, the title text data corresponding to the second similarity score below the target similarity threshold can be determined as negative samples.
[0139] In one specific embodiment, the second similarity between all second feature vectors and the average feature vector is calculated, and then the title text with a second similarity less than the target similarity threshold (final threshold) is taken as a negative sample.
[0140] In another specific embodiment, other title texts that are neither classified as positive nor negative samples can also be discarded. The specific deletion method will not be elaborated here.
[0141] 309. Use positive and negative samples as training samples to train the initial localization model to obtain the target localization model.
[0142] After obtaining the positive and negative samples in step 308, these positive and negative samples can be used as training samples to train the initial localization model, thereby obtaining the target localization model.
[0143] In one specific embodiment, a dataset can be constructed from positive and negative samples to train a title localization model, and this dataset can be used as corpus to train the localization model.
[0144] It should be noted that steps 302-309 are not in any particular order to be executed with step 301. In other words, the target localization model can be trained first, and then the target text data can be used for prediction. That is, the target text data can be used as training samples for training. There are no specific restrictions here.
[0145] 310. Input the audit question data and target title text data into the target positioning model to obtain the prediction results of the audit question data output by the target positioning model.
[0146] In this embodiment, step 310 is the same as described above. Figure 1 Step 203 is similar and will not be elaborated here. However, it should be noted that at this point, the title text data from the parsed and constructed document can be input into the trained target localization model to predict the title text of the review question data, thereby obtaining the localized title text data (or localized title text) and a candidate set. It is easy to understand that the candidate set can store the localized title and paragraph text data that are associated with any review question data.
[0147] In another specific embodiment, the positioning title text and candidate set can also be used for annotation and subsequent applications by other users, and no specific restrictions are imposed here.
[0148] It's also important to clarify beforehand that step 310 is the application stage after training the target localization model. It can be used to locate possible title texts as a candidate set within the entire article. However, it's crucial to understand that when entering the target localization model, the article (or document, etc.) needs to be parsed into title text or paragraph text. Then, the title texts at all levels are obtained, concatenated, and used as the text to be predicted. In this way, the localization model (i.e., the target localization model) can obtain the located title text. To facilitate understanding of the above steps, the following example illustrates this. For example:
[0149] A prospectus typically has 500-1000 pages and roughly 10 chapters. When you want to know if a company has disclosed its revenue for the year (a key audit question), the answer might appear in the "Company Overview," "Financial Status," or other sections. Of course, each prospectus contains many other questions; the above is just one example.
[0150] Specifically, if you need to find the answer to a question in the prospectus, 1,000 pages of content is too much. So you can first define 1-2 chapters using "first-level headings". But even one chapter is too much. If you put an entire chapter directly into the model for judgment, the effect will not be good and the speed will be too slow. So you need to find some paragraphs (a few or even a dozen) that may answer the question first.
[0151] Steps 302-309 first locate several headings using the positioning model (the more clearly the headings are subdivided, the more precise the positioning, making it easier to determine the more specific headings and under which headings the heading belongs).
[0152] In summary, after performing steps 301-310, the title text (i.e., the title containing the answer to the review question and its position) can be obtained. Simultaneously, since the title text within the text has been found, all paragraph texts under that title can also be identified. Then, a corresponding candidate set can be constructed based on the paragraph text for subsequent annotation and application by users. Please refer to the steps for obtaining the review reference answer below. These are the steps that can be performed after step 310.
[0153] Once the title text is located, the next step is to find the paragraph text and its content beneath it, which represents the answer to the review question. Specifically, first, the corresponding paragraph text data is obtained, and a text library for the initial judgment model is constructed based on this data. Then, for any review question, the label data for positive samples is determined as the first label data, and the label data for negative samples is determined as the second label data. Next, the initial judgment model is trained on any review question and positive sample to output the first judgment label data, ensuring that the first judgment label data approximates the first label data. Simultaneously, the initial judgment model can also be trained on any review question and negative sample to output the second judgment label data, ensuring that the second judgment label data approximates the second label data. In other words, both positive and negative samples are used in the initial judgment model for training, with the goal of making the first and second judgment labels approximate the first and second labels. The specific training steps are not detailed here.
[0154] When either the first or second judgment label data satisfies the first or second label data, the initial judgment model is considered to have completed training, forming the target judgment model. Finally, the target paragraph text data is judged based on the target judgment model to obtain the target paragraph content data corresponding to the target paragraph text data, where the target paragraph content data serves as the reference answer for the review question data.
[0155] Based on the above embodiments, in another embodiment, all paragraphs under the located title are extracted as the text library for the judgment model. For each review question, the label of the positive sample is determined as the first label (first label data), and the label of the negative sample is determined as the second label (second label data). It is important to understand that the positive and negative samples at this stage are used to train the question judgment model, and their purpose is different from the positive and negative samples used to train the location model mentioned earlier. Then, the review question and the positive sample are concatenated, and then judged by an encoder or discriminator to determine whether the obtained judgment label tends towards the first label. Simultaneously, the review question and the negative sample are also concatenated, and then judged by an encoder or discriminator to determine whether the obtained judgment label tends towards the second label. This process continues until the judgment label based on the positive sample approaches the first label, and the judgment label based on the negative sample approaches the second label. At this point, the trained and optimized encoder and discriminator are determined as the judgment model. Finally, the trained judgment model is used to judge the paragraph text of the document to obtain the review reference answer, which is the paragraph containing the final determined answer to the review question.
[0156] It should be noted that positive or negative sample concatenation is used because a document (or article) may contain multiple review questions. Each review question can be concatenated with a paragraph, and the concatenated text can be input into the judgment model to obtain the judgment answer of each paragraph for each review question.
[0157] The text localization method and localization model training method proposed in this embodiment not only propose an automatic method for determining the similarity threshold, but also a method for determining negative samples for complex localization problems. Furthermore, in the review problem where the regular matching algorithm cannot locate the title, the title is located by inferring from a small number of labeled samples, thereby improving the labeling efficiency and model training effect of intelligent review.
[0158] In addition, after locating the title text, a judgment model proposed in this embodiment can be used to locate the specific paragraph text, thereby ultimately determining the title position and paragraph position related to the review issue.
[0159] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0160] If the plan involves sensitive information (such as user information or corporate information), it should state that the collection, use, and processing of sensitive information must comply with the laws, regulations, and standards of the relevant countries and regions, and must be carried out with the permission or consent of the relevant entities (such as users or enterprises).
[0161] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a localization model training system disclosed in an embodiment of this application.
[0162] Acquisition unit 401 is used to acquire audit issue data and title text data, wherein the title text data corresponds to each title level;
[0163] The acquisition unit 401 is also used to acquire hierarchical title text data in the title text data based on the audit issue data, wherein the hierarchical title text data is title text data under multiple title levels;
[0164] The acquisition unit 401 is also used to acquire target title text data under the target title level from the hierarchical title text data, wherein the title text data under the target title level includes at least the hierarchical title text data under the target title level.
[0165] Training unit 402 is used to train the initial localization model with target title text data under the target title level as training samples until the preset conditions are met and training stops, thus obtaining the target localization model.
[0166] For example, the system further includes: a construction unit 403 and a labeling unit 404;
[0167] The annotation unit 404 is used to obtain the paragraph text data corresponding to the title text data, and to annotate the paragraph text data and title text data according to the review issue data, so as to obtain the annotated paragraph text data and the annotated title text data.
[0168] Construction unit 403 is used to construct a text corpus based on title text data and paragraph text data, wherein there is a correspondence between different title levels in the paragraph text data and the title text data.
[0169] The acquisition unit 401 is specifically used to concatenate the title levels corresponding to any paragraph text data in the text corpus in order to obtain the title text data.
[0170] The acquisition unit 401 is also used to acquire the corresponding level title text data based on the labeled paragraph text data.
[0171] For example, the system further includes: a calculation unit 405, a determination unit 406, and a positioning unit 407;
[0172] The calculation unit 405 is used to input all hierarchical title text data into the target language model to calculate the first feature vector of all hierarchical title text data, wherein the target language model is used to obtain the feature vector;
[0173] The calculation unit 405 is also used to calculate the average feature vector of the hierarchical title text data and the first similarity between all the first feature vectors and the average feature vector based on the first feature vectors corresponding to all the hierarchical title text data.
[0174] Determining unit 406 is used to determine the target similarity threshold based on all first similarities;
[0175] The positioning unit 407 is used to locate the target heading level of the paragraph text data according to the labeled paragraph text data, wherein the target heading level is the heading level corresponding to the paragraph text data;
[0176] The acquisition unit 401 is specifically used to acquire the title text data of all title levels under the target title level, and to construct the target title text data based on the title text data of all title levels.
[0177] For example, the system includes:
[0178] The calculation unit 405 is specifically used to calculate the average of all first similarities to obtain the initial similarity threshold;
[0179] The calculation unit 405 is also used to calculate the initial similarity threshold and the preset value to obtain the target similarity threshold.
[0180] For example, the system includes:
[0181] Construction unit 403 is specifically used to construct the target dataset based on the target title text data;
[0182] Training unit 402 is specifically used to train the initial localization model using the target title text data in the target dataset as training samples to obtain the target localization model.
[0183] For example, the system further includes: a setting unit 408;
[0184] Setting unit 408 is used to set all labeled hierarchical title text data in the target dataset as positive samples;
[0185] The acquisition unit 401 is specifically used to input all title text data in the target dataset, excluding the labeled hierarchical title text data, into the target language model to obtain multiple second feature vectors;
[0186] The calculation unit 405 is specifically used to calculate the second similarity between any second feature vector and the average feature vector, and to determine the title text data corresponding to the second similarity that is less than the target similarity threshold as a negative sample.
[0187] Training unit 402 is specifically used to train the initial localization model using positive and negative samples as training samples to obtain the target localization model.
[0188] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a text positioning system disclosed in an embodiment of this application.
[0189] The acquisition unit 501 is used to acquire target text data to be processed and review issue data, wherein the target text data includes at least target title text data;
[0190] The acquisition unit 501 is also used to acquire a pre-trained target localization model, wherein the target localization model is the target localization model described in the first aspect;
[0191] Input unit 502 is used to input the audit question data and target title text data into the target positioning model to obtain the prediction result of the audit question data output by the target positioning model, wherein the prediction result includes at least the positioning title text data.
[0192] For example, the target text data also includes target paragraph text data, and the system further includes: construction unit 503;
[0193] Acquisition unit 501 is specifically used to acquire target text data;
[0194] The acquisition unit 501 is also used to parse the target text data to obtain the target title text data and the target paragraph text data, wherein the target paragraph text data and the different title levels in the target title text data have a corresponding relationship;
[0195] Construction unit 503 is used to construct a text corpus based on the target title text data and the target paragraph text data.
[0196] For example, the system further includes: a labeling unit 504, a training unit 505, and a determination unit 506;
[0197] The acquisition unit 501 is also used to acquire the positioning paragraph text data corresponding to the positioning title text data, and to construct the text library of the initial judgment model based on the positioning paragraph text data;
[0198] The labeling unit 504 is used to determine the label data of the positive sample as the first label data based on any audit issue data, and to determine the label data of the negative sample as the second label data;
[0199] Training unit 505 is used to train the initial judgment model on any audit question data and positive sample to output the first judgment label data and make the first judgment label data approach the first label data.
[0200] The training unit 505 is also used to train the initial judgment model on any audit question data and negative samples to output the second judgment label data and make the second judgment label data approach the second label data.
[0201] The determining unit 506 is used to determine that the initial judgment model training is completed and a target judgment model is formed when the first judgment label data or the second judgment label data satisfies the first label data or the second label data.
[0202] The acquisition unit 501 is also used to judge the located paragraph text data according to the target judgment model in order to obtain the target paragraph content data corresponding to the located paragraph text data, wherein the target paragraph content data is the review reference answer of the review question data.
[0203] Please refer to the following: Figure 6 The schematic diagram of a localization model training device disclosed in this application includes:
[0204] Central processing unit 601, memory 605, input / output interface 604, wired or wireless network interface 603, and power supply 602;
[0205] Memory 605 is either a short-term storage memory or a persistent storage memory;
[0206] The central processing unit 601 is configured to communicate with the memory 605 and execute instructions stored in the memory 605 to perform the aforementioned operations. Figure 1 The localization model training method in the illustrated embodiment.
[0207] Please refer to the following: Figure 7 The structural schematic diagram of a text positioning device disclosed in this application includes:
[0208] Central processing unit 701, memory 705, input / output interface 704, wired or wireless network interface 703, and power supply 702;
[0209] Memory 705 is either a short-term storage memory or a persistent storage memory;
[0210] The central processing unit 701 is configured to communicate with the memory 705 and execute instructions stored in the memory 705 to perform the aforementioned operations. Figure 2 or Figure 3 The text positioning method in the illustrated embodiment.
[0211] This application also provides a chip system, characterized in that the chip system includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected via a circuit, and the at least one processor is used to run computer programs or instructions to perform the aforementioned... Figures 1 to 3 The method in the illustrated embodiment.
[0212] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0213] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.
[0214] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0215] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0216] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
Claims
1. A method for training a localization model, characterized in that, The method includes: Obtain the review issue data and title text data, wherein the title text data corresponds to each title level; Obtaining hierarchical title text data from the title text data based on the review question data, wherein the hierarchical title text data comprises title text data under multiple title levels; wherein, obtaining the hierarchical title text data from the title text data based on the review question data includes: obtaining paragraph text data corresponding to the title text data, and annotating the title text data and paragraph text data according to the review question data to obtain annotated paragraph text data and annotated title text data; constructing a text corpus based on the title text data and the paragraph text data, wherein the paragraph text data corresponds to different title levels in the title text data; concatenating the title level corresponding to any paragraph text data in the text corpus to obtain hierarchical title text data; obtaining the corresponding hierarchical title text data according to the annotated paragraph text data; Obtain the title text data under the target title level from the hierarchical title text data, wherein the title text data under the target title level includes at least the hierarchical title text data under the target title level; The target title text data under the target title level is used as training samples to train the initial localization model until the preset conditions are met, at which point the training stops, and the target localization model is obtained.
2. The localization model training method according to claim 1, characterized in that, The step of obtaining the title text data under the target title level from the hierarchical title text data includes: All the hierarchical title text data are input into a target language model to calculate a first feature vector of all the hierarchical title text data, wherein the target language model is used to obtain the feature vector; Based on the first feature vectors corresponding to all the hierarchical title text data, calculate the average feature vector of the hierarchical title text data, and the first similarity between all the first feature vectors and the average feature vector; A target similarity threshold is determined based on all the first similarities; The target heading level of the paragraph text data is located based on the labeled paragraph text data, wherein the target heading level is the heading level corresponding to the paragraph text data; Obtain the title text data of all title levels under the target title level, and construct the title text data under the target title level based on the title text data of all title levels.
3. The localization model training method according to claim 2, characterized in that, The step of determining the target similarity threshold based on all the first similarities includes: Calculate the average of all the first similarities to obtain the initial similarity threshold; The initial similarity threshold is calculated together with a preset value to obtain the target similarity threshold.
4. The localization model training method according to claim 2, characterized in that, The step of using the target title text data under the target title level as training samples to train the initial localization model until a preset condition is met to stop training, thereby obtaining the target localization model, includes: Construct a target dataset based on the target title text data; The target title text data in the target dataset is used as training samples to train the initial localization model to obtain the target localization model.
5. The localization model training method according to claim 4, characterized in that, The step of using the target title text data in the target dataset as training samples to train the initial localization model to obtain the target localization model includes: Set all labeled hierarchical title text data in the target dataset as positive samples; All the title text data in the target dataset, excluding the labeled hierarchical title text data, are input into the target language model to obtain multiple second feature vectors; Calculate the second similarity between any second feature vector and the average feature vector, and determine the title text data corresponding to the second similarity that is less than the target similarity threshold as a negative sample; The positive and negative samples are used as training samples to train the initial localization model to obtain the target localization model.
6. A text localization method, characterized in that, The method includes: Acquire the target text data to be processed and the review issue data, wherein the target text data includes at least the target title text data; Obtain a pre-trained target localization model, wherein the target localization model is the target localization model described in any one of claims 1 to 5; The audit question data and the target title text data are input into the target positioning model to obtain the prediction result of the audit question data output by the target positioning model, wherein the prediction result includes at least the positioning title text data.
7. The text positioning method according to claim 6, characterized in that, The target text data also includes target paragraph text data, and the process of obtaining the target text data to be processed includes: Obtain the target text data; The target text data is parsed to obtain the target title text data and the target paragraph text data, wherein the target paragraph text data corresponds to different title levels in the target title text data. A text corpus is constructed based on the target title text data and the target paragraph text data.
8. A localization model training system, characterized in that, The system includes: The acquisition unit is used to acquire audit issue data and title text data, wherein the title text data corresponds to each title level; The acquisition unit is further configured to acquire hierarchical title text data from the title text data based on the review question data, wherein the hierarchical title text data comprises title text data under multiple title levels; wherein, acquiring the hierarchical title text data from the title text data based on the review question data includes: acquiring paragraph text data corresponding to the title text data, and annotating the title text data and paragraph text data based on the review question data to acquire annotated paragraph text data and annotated title text data; constructing a text corpus based on the title text data and the paragraph text data, wherein the paragraph text data corresponds to different title levels in the title text data; concatenating the title level corresponding to any paragraph text data in the text corpus to acquire hierarchical title text data; and acquiring the corresponding hierarchical title text data based on the annotated paragraph text data. The acquisition unit is further configured to acquire title text data under the target title level from the hierarchical title text data, wherein the title text data under the target title level includes at least the hierarchical title text data under the target title level; The training unit is used to train the initial positioning model with the target title text data under the target title level as training samples until the preset conditions are met, and then stop training to obtain the target positioning model.
9. A physical device, characterized in that, The device includes: Central processing unit, memory, input / output interfaces, wired or wireless network interfaces, and power supply; The memory is either a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the localization model training method according to any one of claims 1 to 5 or the text localization method according to any one of claims 6 to 7.