Structured text information generation method and device, electronic equipment and storage medium

Abstract

This invention provides a method, apparatus, electronic device, and storage medium for generating structured text information. By performing word segmentation on the text information to be structured, a first sub-word is determined for the text information to be structured. First feature tensor information for the first sub-word is generated using the first text quantity information and the first text tensor information. Structured text information for the text information to be structured is generated based on the first feature tensor information. This achieves a sub-word-based approach, effectively avoiding the problem of multiple structures interspersed in the output sub-words, thereby improving the efficiency of text structuring.

Description

Technical Field

[0001] This invention relates to the field of text structuring technology, and in particular to a method for generating structured text information, a device for generating structured text information, an electronic device, and a computer-readable storage medium. Background Technology

[0002] NLP (Natural Language Processing) is a branch of artificial intelligence that uses computers to understand, process, and utilize human language. In NLP, text structuring transforms natural text into text with different segments, each with a corresponding category. These segments are often referred to as structural segments in text structuring. Text structuring primarily involves breaking down natural text into multiple structural segments and then distinguishing the categories corresponding to these segments.

[0003] The accuracy of text structuring determines the efficiency of natural language processing. Therefore, improving the efficiency of text structuring has become a technical problem that needs to be overcome by those skilled in the art. Summary of the Invention

[0004] The present invention provides a method, apparatus, electronic device, and computer-readable storage medium for generating structured text information, in order to solve the problem of how to improve the efficiency of text structuring.

[0005] This invention discloses a method for generating structured text information, comprising:

[0006] Obtain the text information to be structured, and determine the first text quantity information and the first text tensor information for the text information to be structured;

[0007] Perform word segmentation on the text information to be structured to determine the first sub-word for the text information to be structured;

[0008] Using the first text quantity information and the first text tensor information, a first feature tensor information for the first subword is generated;

[0009] Based on the first feature tensor information, structured text information is generated for the text information to be structured.

[0010] Optionally, the method is applied to a pre-trained multilingual model BERT, which includes a word embedding layer and an encoding layer. The step of generating first feature tensor information for the first subword using the first text quantity information and the first text tensor information includes:

[0011] The word embedding layer dimension information for the word embedding layer and the encoding layer dimension information for the encoding layer are determined using the first text tensor information.

[0012] Based on the word embedding layer dimension information, the encoding layer dimension information, and the first text quantity information, a first feature tensor information for the first sub-word is generated.

[0013] Optionally, the step of generating structured text information for the text information to be structured based on the first feature tensor information includes:

[0014] Perform a dimensionality reduction operation on the first feature tensor information to generate the second feature tensor information;

[0015] Based on the second feature tensor information, structured text information is generated for the text information to be structured.

[0016] Optionally, the multilingual model BERT includes a connection layer, and the step of performing dimensionality reduction on the first feature tensor information to generate the second feature tensor information includes:

[0017] Determine the connection layer dimension information for the connection layer;

[0018] Based on the connection layer dimension information, the multilingual model Bert is used to perform dimensionality reduction on the first feature tensor information to generate the second feature tensor information.

[0019] Optionally, the text information to be structured includes multiple first structural segments, and the step of generating structured text information for the text information to be structured based on the second feature tensor information includes:

[0020] The anchor point of the first structural segment is determined based on the second feature tensor;

[0021] Based on the anchor points of the structural segments, determine the start and end position information of the first structural segment for the first structural segment;

[0022] Based on the start and end positions, determine the first structural segment category information for the first structural segment;

[0023] Based on the anchor point of the first structural segment, the start and end position information of the first structural segment, and the category information of the first structural segment, the structured text information for the text information to be structured is determined.

[0024] Optionally, it also includes:

[0025] Obtain the labeled text information and determine the second text quantity information for the labeled text information;

[0026] Determine the second text tensor information for the annotated text information;

[0027] Using the second text quantity information and the second text tensor information, a third feature tensor information for the labeled text information is generated;

[0028] Perform a dimensionality reduction operation on the third feature tensor information to generate the fourth feature tensor information;

[0029] The multilingual model Bert is trained based on the fourth feature tensor information.

[0030] Optionally, the labeled text information includes multiple second structural segments, and the step of training the multilingual model BERT based on the fourth feature tensor information includes:

[0031] The annotated text information is segmented to determine the second sub-word for the annotated text information, and the sub-word sequence vector used to express the second sub-word; the sub-word sequence vector has corresponding sequence vector length information;

[0032] Determine text tag information for the labeled text information; the text tag information includes the start and end position information of the second structural segment for the second structural segment, and the category information of the second structural segment;

[0033] The fourth feature tensor information is used to determine the anchor point prediction value for the second sub-word, the structural segment start and end position information prediction value for the second structural segment, and the structural segment category score prediction value for the second structural segment;

[0034] The target anchor point score label value is determined using the sequence vector length information and the anchor point prediction value;

[0035] The starting position label value of the target structural segment is determined using the start and end position information of the second structural segment and the predicted value of the start and end position information of the structural segment.

[0036] The target structural segment category score label value is determined using the second structural segment category information and the predicted structural segment category score.

[0037] The multilingual model Bert is trained using the target anchor point score label value, the target structural segment start position label value, and the target structural segment category score label value.

[0038] This invention also discloses a structured text information generation device, comprising:

[0039] The unstructured text information acquisition module is used to acquire the unstructured text information and determine the first text quantity information and the first text tensor information for the unstructured text information;

[0040] The first sub-word determination module is used to perform word segmentation on the text information to be structured and determine the first sub-word for the text information to be structured.

[0041] The first feature tensor information generation module is used to generate first feature tensor information for the first sub-word using the first text quantity information and the first text tensor information.

[0042] The structured text information generation module is used to generate structured text information for the text information to be structured based on the first feature tensor information.

[0043] Optionally, the method is applied to a pre-trained multilingual model BERT, which includes a word embedding layer and an encoding layer. The first feature tensor information generation module, using the first text quantity information and..., is used for:

[0044] The word embedding layer dimension information for the word embedding layer and the encoding layer dimension information for the encoding layer are determined using the first text tensor information.

[0045] Based on the word embedding layer dimension information, the encoding layer dimension information, and the first text quantity information, a first feature tensor information for the first sub-word is generated.

[0046] Optionally, the structured text information generation module is used for:

[0047] Perform a dimensionality reduction operation on the first feature tensor information to generate the second feature tensor information;

[0048] Based on the second feature tensor information, structured text information is generated for the text information to be structured.

[0049] Optionally, the multilingual model BERT includes a connection layer, and the structured text information generation module is used for:

[0050] Determine the connection layer dimension information for the connection layer;

[0051] Based on the connection layer dimension information, the multilingual model Bert is used to perform dimensionality reduction on the first feature tensor information to generate the second feature tensor information.

[0052] Optionally, the text information to be structured includes multiple first structure segments, and the structured text information generation module is used for:

[0053] The anchor point of the first structural segment is determined based on the second feature tensor;

[0054] Based on the anchor points of the structural segments, determine the start and end position information of the first structural segment for the first structural segment;

[0055] Based on the start and end positions, determine the first structural segment category information for the first structural segment;

[0056] Based on the anchor point of the first structural segment, the start and end position information of the first structural segment, and the category information of the first structural segment, the structured text information for the text information to be structured is determined.

[0057] Optionally, it also includes:

[0058] The annotation text information acquisition module is used to acquire annotation text information and determine the second text quantity information for the annotation text information;

[0059] The second text tensor information determination module is used to determine the second text tensor information for the labeled text information;

[0060] The third feature tensor information generation module is used to generate third feature tensor information for the labeled text information using the second text quantity information and the second text tensor information.

[0061] The fourth feature tensor information generation module is used to perform a dimensionality reduction operation on the third feature tensor information to generate the fourth feature tensor information;

[0062] The BERT multilingual model training module is used to train the BERT multilingual model based on the fourth feature tensor information.

[0063] Optionally, the multilingual model BERT training module is used for:

[0064] The annotated text information is segmented to determine the second sub-word for the annotated text information, and the sub-word sequence vector used to express the second sub-word; the sub-word sequence vector has corresponding sequence vector length information;

[0065] Determine text tag information for the labeled text information; the text tag information includes the start and end position information of the second structural segment for the second structural segment, and the category information of the second structural segment;

[0066] The fourth feature tensor information is used to determine the anchor point prediction value for the second sub-word, the structural segment start and end position information prediction value for the second structural segment, and the structural segment category score prediction value for the second structural segment;

[0067] The target anchor point score label value is determined using the sequence vector length information and the anchor point prediction value;

[0068] The starting position label value of the target structural segment is determined using the start and end position information of the second structural segment and the predicted value of the start and end position information of the structural segment.

[0069] The target structural segment category score label value is determined using the second structural segment category information and the predicted structural segment category score.

[0070] The multilingual model Bert is trained using the target anchor point score label value, the target structural segment start position label value, and the target structural segment category score label value.

[0071] This invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0072] The memory is used to store computer programs;

[0073] When the processor executes a program stored in the memory, it implements the method described in the embodiments of the present invention.

[0074] This invention also discloses a computer-readable storage medium storing instructions that, when executed by one or more processors, cause the processors to perform the methods described in this invention.

[0075] The embodiments of the present invention have the following advantages:

[0076] In this embodiment of the invention, by performing word segmentation on the text information to be structured, a first sub-word is determined for the text information to be structured; a first feature tensor information for the first sub-word is generated using the first text quantity information and the first text tensor information; and structured text information for the text information to be structured is generated based on the first feature tensor information. This achieves the goal of using sub-words as the granularity, effectively avoiding the problem of multiple structures interspersed in the output sub-words, thereby improving the efficiency of text structuring. Attached Figure Description

[0077] Figure 1 This is a flowchart of the steps of a structured text information generation method provided in an embodiment of the present invention;

[0078] Figure 2 This is a schematic diagram of a text structured process for the BERT model provided in an embodiment of the present invention;

[0079] Figure 3 This is a schematic diagram of a structured text provided in an embodiment of the present invention;

[0080] Figure 4This is a flowchart of another method for generating structured text information provided in this embodiment of the invention;

[0081] Figure 5 This is a flowchart of the steps of a model training method provided in an embodiment of the present invention;

[0082] Figure 6 This is a schematic diagram of the model training process provided in an embodiment of the present invention;

[0083] Figure 7 This is a structural block diagram of a structured text information generation device provided in an embodiment of the present invention;

[0084] Figure 8 This is a hardware structure block diagram of an electronic device provided in various embodiments of the present invention. Detailed Implementation

[0085] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0086] In practical applications, relevant text structuring methods mainly include prior template matching methods based on predefined rule bases and deep learning methods based on pre-trained language models. The main differences between the two lie in their computational requirements and accuracy. The former requires no computational power but has lower accuracy; the latter requires some computational power but offers significantly improved accuracy.

[0087] To ensure accuracy, deep learning methods based on pre-trained language models are clearly a better choice. The main implementation approaches for deep learning methods based on pre-trained language models include text classification and named entity recognition.

[0088] The core of text classification-based technologies is to use sentences as the structuring granularity. This involves structuring the entire text using a pre-trained language model, primarily by segmenting the text into sentences, inputting these sentences into the pre-trained language model to extract feature information for each sentence, classifying these features, and finally determining the category for each sentence. While using sentences as the structuring granularity offers advantages such as ease of maintenance and high structuring accuracy, this approach is limited because sentences represent a coarse-grained granularity, whereas the granularity of text structuring varies across different scenarios. Therefore, this approach cannot accurately handle fine-grained text structuring tasks. In some scenarios, a finer granularity than that of sentences is required for text processing.

[0089] For example, in the process of structuring the medical text "[Chief Complaint] Diagnosed with small cell lung cancer more than 1.5 years ago. [Present Illness] The patient sought medical attention at a local hospital due to recurrent cough for more than 1 month. On August 17, 2016, a lung CT scan showed: a solid nodule in the lower lobe of the right lung, enlarged lymph nodes in the right hilum, and a small amount of pleural effusion in the right side. On August 29, 2016, the patient underwent bronchoscopic biopsy under general anesthesia with central ultrasound guidance (EBUS-TBNA), and the surgery was successful.", the actual output of structuring the entire text using a pre-trained language model, with sentences as the structuring granularity, is shown below:

[0090] 1. [Chief Complaint] Diagnosed with small cell lung cancer more than 1.5 years ago;

[0091] 2. [Present Illness] The patient sought medical attention at a local hospital due to a persistent cough that had lasted for more than a month;

[0092] 3. Lung CT scan on August 17, 2016 showed:

[0093] 4. Solid nodule in the lower lobe of the right lung;

[0094] 5. Enlarged lymph nodes in the right hilum;

[0095] 6. Small amount of pleural effusion on the right side;

[0096] 7. On August 29, 2016, under general anesthesia and with central ultrasound guidance;

[0097] 8. EBUS-TBNA (Emboscopic Blastocyst Biopsy)

[0098] 9. The surgery was successful.

[0099] In practical applications, the text structuring of medical texts focuses on sub-word information such as "lesion location", "medication time", and "disease diagnosis". Text structuring requires sub-words as the structuring granularity. If a text classification-based approach is used to handle this type of task, it will cause a certain degree of error.

[0100] The core idea behind named entity recognition (NIR) is to use sub-words as the granularity of structuring. A pre-trained language model is used to structure the entire text, primarily through word segmentation to obtain multiple sub-words. These sub-words are then input into the pre-trained language model for entity classification, ultimately determining the category for each sub-word. While using sub-words as the granularity of structuring can handle various fine-grained structuring tasks, it cannot consider the structural continuity of the text. This can lead to the problem of multiple structure interleaving, where various types of sub-words are interspersed, causing sub-words that should belong to the same structural segment to be split.

[0101] For example, in the process of text structuring the medical text information "EBUS-TBNA performed under general anesthesia on August 29, 2016, and the surgery was successful," the relevant technology uses sub-words as the structuring granularity and a pre-trained language model to structure the entire text. The actual output result is shown below:

[0102] "2016-08-29";

[0103] "exist";

[0104] General anesthesia;

[0105] "Downward";

[0106] "central";

[0107] "Ultrasound";

[0108] "guide";

[0109] "Down";

[0110] "Bronchi";

[0111] "mirror";

[0112] "Puncture biopsy";

[0113] “(”;

[0114] “EBUS-TBNA”;

[0115] “),;

[0116] "Operation";

[0117] "smoothly".

[0118] The expected output of text structuring for this medical text information is:

[0119] "2016-08-29";

[0120] General anesthesia;

[0121] “Downward central ultrasound guidance”;

[0122] "Bronchoscopic biopsy (EBUS-TBNA)".

[0123] Therefore, the output results are for "bronchus"; "endoscopy"; "puncture biopsy"; "("; "EBUS-TBNA";

[0124] There are multiple structures interspersed in “),”.

[0125] In the above examples, in scenarios with small granularity of structure, many structural categories, and a large number of words in a single structural text, the entities in a single structural text usually belong to the same category. However, using named entity recognition methods can result in the intermingling of multiple structural types, causing a single structural text to be split into multiple structures. For example, "bronchoscopy-TBNA" can be divided into "bronchus", "scope", "bronchoscopy-TBNA", and "EBUS-TBNA", thus affecting the accuracy of text structuring.

[0126] In summary, text structuring methods in related technologies have at least the following problems: 1. In the implementation approach based on text classification, the granularity of structuring cannot be adaptively adjusted according to the task scenario, resulting in a large error in the judgment of fine-grained structures, which directly affects the accuracy; 2. In the implementation approach based on named entity recognition, the problem of multiple types of structures interspersed cannot be handled, which disrupts the continuity within structural segments and affects the accuracy.

[0127] Comparing the two approaches, the text classification-based approach has a coarse granularity of structuring but ensures continuity within structural segments, while the named entity recognition-based approach has a fine granularity of structuring but may disrupt continuity within structural segments. Essentially, they represent a contradiction between dynamic structuring granularity and intra-segment continuity. When structuring text, reducing the fine granularity allows for wider adaptability but increases the proportion of multiple structural types, affecting intra-segment continuity. Conversely, increasing the fine granularity reduces the proportion of multiple structural types but introduces significant errors in low-granularity structuring scenarios. Therefore, the fundamental problem is the inability to balance "dynamic structuring granularity" and "intra-segment continuity," resulting in low accuracy in text structuring.

[0128] In view of this, this invention addresses the problem of low accuracy in text structuring in existing technologies by providing a text structuring method. 1. By optimizing the decoder structure, the sub-word features in the decoder's output tensor are increased. For example, the output tensor can have a shape of (1, seq_len, 3 + target_num), meaning that for each sub-word, it includes 3 + target_num features. Based on this output tensor, multiple judgments can be made for each sub-word. For example, this may include: determining whether each sub-word might be a structural segment anchor point; if so, identifying the start and end positions of the structural segment; and determining the structural segment category. Furthermore, by determining whether each sub-word is a structural segment anchor point and the corresponding start and end positions of the structural segment, multiple structural segments and their start and end positions can be obtained. The length of each structural segment can be determined based on the structural segment anchor point and can be of different lengths, balancing the dynamic structuring granularity and intra-segment continuity of text structuring, thereby improving the model's prediction accuracy. 2. For decoder optimization, the text labels in the training set data can be optimized accordingly. Each sub-word's text label can include whether it is a structural segment anchor point, the start and end positions of the corresponding structural segment, and its category. Furthermore, the loss function design can be optimized. The redesigned loss function includes structural segment anchor point score loss, structural segment start and end position loss, and structural segment category score loss. The total loss is obtained by weighted summation of these losses. This design of both text labels and the loss function improves the model's training performance. 3. Since structural segments in professional texts can include multiple sub-words of different types (e.g., "bronchoscopy biopsy" includes "bronchus," "scope," and "biopsy"), dynamic structuring granularity while ensuring structural segment continuity avoids splitting structural segments into multiple sub-words, providing more accurate reference indicators and higher reference value for business scenarios.

[0129] Reference Figure 1 The diagram illustrates a flowchart of a structured text information generation method provided in an embodiment of the present invention, which may specifically include the following steps:

[0130] Step 101: Obtain the text information to be structured, and determine the first text quantity information and the first text tensor information for the text information to be structured;

[0131] Step 102: Perform word segmentation on the text information to be structured to determine the first sub-word for the text information to be structured;

[0132] Step 103: Generate first feature tensor information for the first sub-word using the first text quantity information and the first text tensor information;

[0133] Step 104: Generate structured text information for the text information to be structured based on the first feature tensor information.

[0134] In practical applications, the embodiments of the present invention can be applied to natural language processing devices. For example, natural language processing devices can be used to perform natural language processing on business orders in the field of design communication.

[0135] In a specific implementation, the embodiments of the present invention can obtain text information to be structured. The text information to be structured can be any text information that needs to be structured. For example, it can be text information in a business form entered by the user.

[0136] After obtaining the text information to be structured, the embodiments of the present invention can determine the first text quantity information for the text information to be structured. For example, if the input text information to be structured is a single text, the first text quantity information can be batch_size = 1; if the input text information to be structured is N texts, the first text quantity information batch_size = N.

[0137] In practical applications, the multilingual model BERT is an overall self-encoding language model.

[0138] In a specific implementation, embodiments of the present invention can perform word segmentation on the text information to be structured to determine the first sub-word for the text information to be structured. For example, the multilingual model BERT can be used to perform word segmentation on the text information to be structured, thereby generating the first sub-word for the text information to be structured.

[0139] In this embodiment of the invention, for each piece of text to be structured, a text tensor corresponding to each piece of text to be structured can be obtained. That is, this embodiment of the invention can determine the first text tensor information for the text information to be structured. For example, after obtaining the text information to be structured, its text tensor information is determined to be a one-dimensional tensor of (1, seq_len), where seq_len can represent the length of the sub-word list after the text to be structured is segmented, that is, the number of sub-words in the text information to be structured. For example, the BERT model's word segmenter can be pre-set. By performing word segmentation on each piece of text information to be structured through the BERT model, a one-dimensional vector of shape (1, seq_len) can be obtained. By tensorizing the one-dimensional vectors corresponding to each piece of text information to be structured output by the BERT model's word segmenter, the text tensor corresponding to each piece of text information to be structured is obtained, that is, the first text tensor information.

[0140] After determining the first text tensor information, embodiments of the present invention can generate first feature tensor information for the first sub-word using the first text quantity information and the first text tensor information. For example, the text tensors corresponding to each piece of text information to be structured can be input into the word embedding layer and encoding layer of the BERT model. After performing feature extraction on the sub-words through the BERT model, the first feature tensor information is obtained. For example, the tensor shape of the first feature tensor information can be (N, seq_len, hidden_dim), where N can be the first text quantity information, and hidden_dim represents the word embedding layer dimension and encoding layer dimension in the BERT model. Figure 2 As shown, Figure 2 This is a schematic diagram of a text structuring process for the BERT model provided in an embodiment of the present invention. The encoded output is (1, seq_len, hidden_dim). Because the input text to be structured is a single text, the first text quantity information is "1".

[0141] After determining the first feature tensor information for the first subword, this embodiment of the invention can generate structured text information for the text information to be structured based on the first feature tensor information.

[0142] For example, the first feature tensor information can be reduced in dimensionality by the decoder of the BERT model to obtain the second feature tensor information. The second feature tensor can also be used to filter the structural segment anchor points, determine the start and end positions of the structural segments, and obtain the structural segment category. Finally, based on the start and end positions and the corresponding structural segment categories of the multiple structural segments, the structured text information can be determined.

[0143] Of course, the above examples are merely illustrative. Those skilled in the art can also use other methods to generate structured text information for the text information to be structured based on the first feature tensor information. In this regard, the embodiments of the present invention do not impose any limitations.

[0144] In this embodiment of the invention, by performing word segmentation on the text information to be structured, a first sub-word is determined for the text information to be structured; a first feature tensor information for the first sub-word is generated using the first text quantity information and the first text tensor information; and structured text information for the text information to be structured is generated based on the first feature tensor information. This achieves the goal of using sub-words as the granularity, effectively avoiding the problem of multiple structures interspersed in the output sub-words, thereby improving the efficiency of text structuring.

[0145] Based on the above embodiments, modified embodiments of the above embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in the modified embodiments.

[0146] In an optional embodiment of the present invention, the method is applied to a pre-trained multilingual model BERT, the multilingual model BERT including a word embedding layer and an encoding layer, and the step of generating first feature tensor information for the first subword using the first text quantity information and the first text tensor information includes:

[0147] The word embedding layer dimension information for the word embedding layer and the encoding layer dimension information for the encoding layer are determined using the first text tensor information.

[0148] Based on the word embedding layer dimension information, the encoding layer dimension information, and the first text quantity information, a first feature tensor information for the first sub-word is generated.

[0149] In specific implementations, this invention can be applied to a pre-trained multilingual model BERT. In practical applications, the multilingual model BERT can include a word embedding layer and an encoding layer. This invention can input the text tensors corresponding to each piece of text information to be structured into the word embedding layer and encoding layer of the BERT model. After feature extraction of the sub-words by the BERT model, the first feature tensor information can be obtained. For example, the tensor shape of the first feature tensor information can be (N, seq_len, hidden_dim), where N can be the first text quantity information, and hidden_dim represents the dimensional information of the word embedding layer and the encoding layer in the BERT model. Figure 2 As shown, Figure 2 This is a schematic diagram of a text structuring process for the BERT model provided in an embodiment of the present invention. The encoded output is (1, seq_len, hidden_dim). Because the input text to be structured is a single text, the first text quantity information is "1".

[0150] In this embodiment of the invention, word embedding layer dimension information for the word embedding layer and encoding layer dimension information for the encoding layer are determined by the first text tensor information; based on the word embedding layer dimension information, the encoding layer dimension information, and the first text quantity information, a first feature tensor information for the first sub-word is generated, thereby improving the generation efficiency of the first feature tensor information and further improving the efficiency of text structuring.

[0151] In an optional embodiment of the present invention, the step of generating structured text information for the text information to be structured based on the first feature tensor information includes:

[0152] Perform a dimensionality reduction operation on the first feature tensor information to generate the second feature tensor information;

[0153] Based on the second feature tensor information, structured text information is generated for the text information to be structured.

[0154] In practical implementation, dimensionality reduction is the reduction of the dimension of features, which can effectively compress data and accelerate the learning model. In this embodiment of the invention, before structuring text information, dimensionality reduction can be performed on the first feature tensor to generate second feature tensor information, thereby further improving the efficiency of text structuring.

[0155] In an optional embodiment of the present invention, the multilingual model BERT includes a connection layer, and the step of performing a dimensionality reduction operation on the first feature tensor information to generate the second feature tensor information includes: determining the connection layer dimension information for the connection layer; and performing a dimensionality reduction operation on the first feature tensor information using the multilingual model BERT based on the connection layer dimension information to generate the second feature tensor information.

[0156] For example, a fully connected layer of dimension (hidden_dim, 3 + target_num) can be used to reduce the dimensionality of the first feature tensor output by the encoder of the BERT model, resulting in the second feature tensor. Specifically, taking a single piece of text to be structured as an example, such as... Figure 2 As shown, the encoder output of the BERT model has a first feature tensor shape of (1, seq_len, hidden_dim). Starting from the dimension of seq_len (dim=0), the feature of dimension (1, hidden_dim) corresponding to each first word can be compressed to (1, 3+target_num) through a fully connected layer of dimension (hidden_dim, 3+target_num), finally obtaining the second feature tensor of shape (1, seq_len, 3+target_num). Here, target_num can represent the number of candidate structured categories. The meaning of this second feature tensor is that the prediction result of each first word is a one-dimensional feature vector of length 3+target_num, where the first three features are used to represent "the confidence score of the first word as a structural segment anchor point", "the distance between the starting position of the structural segment and the anchor point if the first word is the anchor point", and "the distance between the ending position of the structural segment and the anchor point if the first word is the anchor point", respectively. Figure 2 As shown, the first feature 201, the second feature 202, the third feature 203, and the fourth feature 204 can be feature vectors. The first three features in the compressed output of the fully connected layer correspond to the fourth feature 204, the third feature 203, and the second feature 202, respectively. Finally, the target_num features represent the "confidence scores for each structured category, assuming the first word is the anchor point". Figure 2 For example, the last target_num features in the compressed output of the fully connected layer correspond to the first feature 201. For instance, if there are five structured categories, then target_num is 5, and the last five feature values ​​can be (0.4, 0.1, 0.9, 0.8, 1.2).

[0157] Of course, the above examples are merely illustrative, and those skilled in the art can generate second feature tensor information with any other arbitrary values. Therefore, the embodiments of the present invention do not impose any limitations on this.

[0158] In an optional embodiment of the present invention, the text information to be structured includes a plurality of first structural segments, and the step of generating structured text information for the text information to be structured based on the second feature tensor information includes:

[0159] The anchor point of the first structural segment is determined based on the second feature tensor;

[0160] Based on the anchor points of the structural segments, determine the start and end position information of the first structural segment for the first structural segment;

[0161] Based on the start and end positions, determine the first structural segment category information for the first structural segment;

[0162] Based on the anchor point of the first structural segment, the start and end position information of the first structural segment, and the category information of the first structural segment, the structured text information for the text information to be structured is determined.

[0163] For example, the first structural segment anchor point can be determined as follows: The sigmoid function is an S-shaped function commonly used in biology. Whether the first word can serve as an anchor point for a structural segment needs to be determined jointly by the anchor point probability and the maximum structural segment category probability. Specifically, the anchor point probability and the maximum structural segment category probability for the first word can be determined, and their product can be calculated. Specifically, the last target_num features of each first word in the second feature tensor information and the first feature of each first word can be processed using the sigmoid function. That is, the "confidence score of each first word as a structural segment anchor point" and the "confidence score of the structural segment corresponding to each first word belonging to each structured category, assuming each first word is an anchor point" can be processed using the sigmoid function. Figure 2 As shown, the features at the corresponding index positions in the fourth feature 204 and the first feature 201 are processed by sigmoid to compress the confidence score range to 0-1, and expressed as a probability. This is equivalent to obtaining, after sigmoid processing, "the probability that each first word is an anchor point of a structural segment" and "the probability that, assuming each first word is an anchor point, the structural segment corresponding to each first word belongs to each structured category." Then, "the probability that, assuming each first word is an anchor point, the structural segment corresponding to each first word belongs to each structured category" is maximized to obtain the maximum probability corresponding to each first word, which serves as the maximum structured category probability for each first word. Figure 2 As shown, it can be max(sigmoid(first feature 201)). Finally, the matrix dot product of "the probability that each first word is a structural segment anchor point" and "the maximum structured category probability of each first word" can be used to obtain the anchor point probability that each first word is a structural segment anchor point. Then, the anchor point probability is compared with a preset probability threshold, and the first word whose anchor point probability is greater than the probability threshold is determined as the structural segment anchor point. The probability threshold can be set to 0.5. This probability threshold can be obtained through experiments. The inventors of this invention found in experiments that the improved BERT model performs best at a probability threshold of 0.5. Figure 2 As shown, the comparison method for each first sub-word can be to determine whether max(sigmoid(first feature 201))×sigmoid(fourth feature 204) is greater than the preset probability threshold of 0.5. If so, the first sub-word can be the anchor point of the first structural segment, and the structural segment prediction result corresponding to the first sub-word can be output, that is, the structured text information. The structured text information can include N structural segments. The structural segment prediction result can include the start position, end position, and structural category corresponding to the N structural segments respectively. The structural segment prediction result can be obtained by calculating the start and end positions of the structural segments and the structural segment category.

[0164] For example, the start and end position information of the first structural segment can be determined as follows: After filtering the structural segment anchor points, determine the anchor point indices in the second feature tensor information. Assuming the first word is the structural segment anchor point, the anchor point index in the second feature tensor is 0; if the third first sub-word is the structural segment anchor point, the anchor point index in the second feature tensor is 2. Then, based on the second and third features corresponding to each anchor point index in the second feature tensor—namely, "the distance between the start position of each structural segment anchored by the first sub-word and the anchor point itself" and "the distance between the end position of each structural segment anchored by the first sub-word and the anchor point itself"—subtract the second feature corresponding to each anchor point index from the anchor point index to obtain "the start position of the structural segment corresponding to each structural segment anchor point." Finally, add the third feature corresponding to each anchor point index to obtain "the end position of the structural segment corresponding to each structural segment anchor point," which is the start and end position information of the first structural segment. It should be noted that the BERT model in this embodiment of the invention can obtain the distance between the start and end positions of the structural segment and the anchor word through the decoder, and then calculate the start and end positions through the distance and the position of the anchor word. However, if the decoder directly obtains the start and end positions of the structural segment, the BERT model in this embodiment of the invention can additionally learn the relative relationship between the start and end positions and the anchor, which reduces the convergence speed and affects the performance.

[0165] For example, the first structural segment category information can be determined as follows: First, determine the anchor index corresponding to the structural segment anchor points obtained after anchor point filtering (the acquisition of structural segment categories does not have an execution order relationship with the calculation of the start and end positions of structural segments; therefore, the anchor index can be acquired again). Second, determine the maximum structured category probability corresponding to each anchor index. Third, determine the structural segment category corresponding to each anchor index based on the maximum structured category probability corresponding to each anchor index; that is, the first structural segment category information. For example, as follows... Figure 2 As shown, the structural segment category corresponding to each anchor index can be determined as [first feature 201.index(max(first feature 201))]. First feature 201 can be the last target_num features in a one-dimensional tensor with shape (1, 3 + target_num) corresponding to each anchor index. Figure 2 As shown, the first feature 201 in a column of information in the second feature tensor, where max(first feature 201) can be the maximum structured class probability corresponding to each anchor index, and index(max(first feature 201)) is the class index of the maximum structured class probability corresponding to each anchor index, thus obtaining the structured class corresponding to the class index in the first feature 201.

[0166] For example, structured text information can be determined as follows: based on the first feature and the last target_num features of each first sub-word in the second feature tensor information, it can be determined whether each first sub-word is an anchor point; based on the second and third features in the one-dimensional feature vector corresponding to the first sub-word determined as an anchor point, the start and end positions of multiple structural segments are determined; and based on the last target_num features in the one-dimensional feature vector corresponding to the first sub-word determined as an anchor point, the structural segment categories corresponding to multiple structural segments are determined. For example... Figure 2 As shown, structured text information can include the start and end positions of each structure segment and the category of the structure segment.

[0167] In another example, such as Figure 3 As shown, Figure 3 This is a schematic diagram of structured text provided in an embodiment of the present invention. Taking the text information to be structured, "2016-08-29 under general anesthesia, bronchoscopic biopsy under central ultrasound guidance (EBUS-TBNA), surgery successful.", word segmentation processing yields "2016-08-29" / "under" / "general anesthesia" / "under" / "central" / "ultrasound" / "guided" / "under" / "bronchus" / "scope" / "biopsy" / "( / "EBUS-TBNA / )" / "surgery" / "successfully". Through word detection confidence prediction and detection start and end position prediction, we can obtain " The confidence level for "2016-08-29" is 0.7, with start and end positions (0, 1), and the word category is spatiotemporal description. The confidence level for "general anesthesia" is 0.8, with start and end positions (2, 3), and the word category is qualifier. The confidence level for "central," "ultrasound," and "guided" is 0.8, with start and end positions (4, 7), and the word category is qualifier. The confidence level for "bronchus," "endoscopy," and "bronchial biopsy" / "( / "EBUS-TBNA / )" is 0.9, with start and end positions (8, 14), and the word category is detection method. The confidence level for "surgery" and "successful" is 0.6, with start and end positions (11, 16), and the word category is other. The actual expected output is "2016-08-29," "general anesthesia," "central ultrasound guidance," and "bronchoscopic biopsy EBUS-TBNA," and the structured text information matches the expected output.

[0168] The structured text in this embodiment of the invention includes the start and end positions of multiple structural segments and word classification. The word classification can be the category of structural segment. The structured text also includes the confidence level corresponding to each structural segment. The confidence level is used to characterize the accuracy of the structural segment and can be the anchor point probability obtained in the structural segment anchor point screening.

[0169] In this embodiment of the invention, a first structural segment anchor point is determined based on the second feature tensor; based on the structural segment anchor point, the start and end position information of the first structural segment is determined; based on the start and end positions, the first structural segment category information is determined; and based on the first structural segment anchor point, the first structural segment start and end position information, and the first structural segment category information, structured text information for the text information to be structured is determined, further improving the accuracy of the structured text information, thereby further enhancing the structured text information.

[0170] To enable those skilled in the art to better understand the embodiments of the present invention, an example is used below to illustrate the embodiments of the present invention.

[0171] refer to Figure 4 , Figure 4 This is a flowchart of another method for generating structured text information provided in this embodiment of the invention.

[0172] S401: Obtain the input text information to be structured, and determine the first text quantity information of the text information to be structured.

[0173] If the input text information to be structured is a single text, then the batch_size = 1. If the input text information to be structured is N texts, then the batch_size = N.

[0174] S402: Each piece of text information to be structured is segmented using the BERT model's word segmenter. For each piece of text information to be structured, the first text tensor information corresponding to each piece of text information to be structured can be obtained.

[0175] Here, the text tensor is a one-dimensional tensor with a shape of (1, seq_len), where seq_len represents the length of the sub-word list after the text information to be structured is segmented, i.e., the number of sub-words in the text information to be structured. The BERT model's segmenter is pre-set. By segmenting each piece of text information to be structured through the BERT model, a one-dimensional vector with a shape of (1, seq_len) can be obtained. By tensorizing the one-dimensional vectors corresponding to each piece of text information to be structured output by the BERT model's segmenter, the first text tensor information corresponding to each piece of text information to be structured is obtained.

[0176] The BERT model used in this solution is an improved version of the BERT model, for example. Figure 2The input text information to be structured can be a single text. One of the main inventive points of this application is to optimize the decoding layer (decoder) of the BERT model, while other parts can use the structure in the existing BERT model.

[0177] S403: Input the text tensors corresponding to each piece of text information to be structured into the word embedding layer and encoding layer in the BERT model, and after feature extraction, obtain the first feature tensor information.

[0178] The tensor shape of the first feature tensor information can be (N, seq_len, hidden_dim), where N is the number of text information to be structured, and hidden_dim represents the dimension of the word embedding layer and the encoding layer in the BERT model. For example... Figure 2 As shown, Figure 2 The encoded output is (1, seq_len, hidden_dim) because the input text information to be structured is a single text.

[0179] S404: The second feature tensor information is obtained by performing feature dimensionality reduction on the first feature tensor information through the decoder of the BERT model.

[0180] Optionally, feature dimensionality reduction includes using a fully connected layer of dimension (hidden_dim, 3 + target_num) to reduce the dimensionality of the first feature tensor output by the encoder of the BERT model, obtaining the second feature tensor. Specifically, taking a single piece of text information to be structured as an example, such as... Figure 2 As shown, the first feature tensor output by the encoder of the BERT model has a tensor shape of (1, seq_len, hidden_dim). Starting from the dimension of seq_len (dim=0), the feature of each word with a dimension of (1, hidden_dim) is compressed to (1, 3+target_num) through a fully connected layer with a dimension of (hidden_dim, 3+target_num), finally obtaining the second feature tensor with a shape of (1, seq_len, 3+target_num).

[0181] Here, `target_num` represents the number of candidate structured categories. The meaning of this second feature tensor information is that the prediction result for each sub-word is a one-dimensional feature vector of length 3 + `target_num`, where the first three features are used to characterize "the confidence score of the sub-word as a structural segment anchor point", "the distance between the starting position of the structural segment and the anchor point if the sub-word is an anchor point", and "the distance between the ending position of the structural segment and the anchor point if the sub-word is an anchor point". Figure 2For example, the first three features in the compressed output of the fully connected layer correspond to the fourth feature 204, the third feature 203, and the second feature 202, respectively. The final `target_num` features represent the "confidence scores for each structured category, assuming the subword is the anchor point". Figure 2 For example, the last target_num features in the compressed output of the fully connected layer correspond to the first feature 201. For instance, if there are five structured categories, then target_num is 5, and the last five feature values ​​can be (0.4, 0.1, 0.9, 0.8, 1.2).

[0182] S405: Based on the second feature tensor information, perform structural segment anchor point selection.

[0183] Optionally, the anchor point selection for structural segments includes: whether a word can serve as an anchor point for a structural segment needs to be determined jointly by the anchor point probability and the maximum structural segment category probability. Specifically, the product of the anchor point probability and the maximum structural segment category probability can be determined. This can be achieved by applying a sigmoid function to the last `target_num` features of each word and the first feature of each word in the second feature tensor information. This is equivalent to applying a sigmoid function to both the "confidence score of each word as a structural segment anchor point" and the "confidence score of the structural segment corresponding to each word belonging to each structured category, assuming each word is an anchor point." Figure 2 The fourth feature 204 and the features at the corresponding index positions in the first feature 201 are processed by sigmoid to compress the confidence score range to 0-1, i.e., represented in probabilistic form. This is equivalent to obtaining, after sigmoid processing, "the probability that each subword is an anchor point of a structural segment" and "the probability that, assuming each subword is an anchor point, the corresponding structural segment belongs to each structured category." Then, "the probability that, assuming each subword is an anchor point, the corresponding structural segment belongs to each structured category" is maxed to obtain the maximum probability for each subword, which serves as the maximum structured category probability for each subword. Figure 2 As shown, it can be max(sigmoid(first feature 201)). Finally, the matrix dot product of "the probability of each word being a structural segment anchor point" and "the maximum structured category probability of each word" is used to obtain the anchor point probability of each word being a structural segment anchor point. Then, the anchor point probability is compared with a preset probability threshold, and words with anchor point probabilities greater than the probability threshold are determined as structural segment anchor points. The probability threshold can be set to 0.5. This probability threshold was obtained through experiments. At a probability threshold of 0.5, the improved BERT model performs best in practice. Figure 2As shown, the comparison formula for each sub-word can be max(sigmoid(first feature 201))×sigmoid(fourth feature 204)>0.5? If yes, then the sub-word is a structural segment anchor point, and the structural segment prediction result (i.e., structured text) corresponding to the sub-word can be output. It can include N structural segments. The structural segment prediction result can include the start position, end position, and structural category of each of the N structural segments. The structural segment prediction result is obtained by calculating the start and end positions of the structural segments and the structural segment category. The specific calculation method is described below.

[0184] S406: Calculate the start and end positions of the structural segments based on the second feature tensor information.

[0185] Optionally, the calculation of the start and end positions of the structural segment may include: determining the anchor index of the structural segment anchor obtained after the structural segment anchor filtering in the second feature tensor information. For example, if the first word is the structural segment anchor, the anchor index in the second feature tensor information is 0; if the third sub-word is the structural segment anchor, the anchor index in the second feature tensor information is 2. Then, based on the second and third features corresponding to each anchor index in the second feature tensor information, namely "the distance between the start position of the structural segment with each sub-word as the anchor and the sub-word anchor" and "the distance between the end position of the structural segment with each sub-word as the anchor and the sub-word anchor", subtract the second feature corresponding to each anchor index from each anchor index to obtain "the start position of the structural segment corresponding to each structural segment anchor". Add the third feature corresponding to each anchor index to each anchor index to obtain "the end position of the structural segment corresponding to each structural segment anchor". It should be noted that the improved BERT model obtains the distance between the start and end positions of the structural segment and the anchor word through the decoder, and then calculates the start and end positions based on the distance and the position of the anchor word. However, if the decoder directly obtains the start and end positions of the structural segment, the BERT model needs to additionally learn the relative relationship between the start and end positions and the anchor, which reduces the convergence speed and affects the performance.

[0186] S407: Based on the information of this second feature tensor, obtain the structural segment category.

[0187] Optionally, the structure segment category is obtained by: determining the anchor index corresponding to the structure segment anchor points obtained after anchor point filtering (the structure segment category acquisition can be performed without any order of execution with the calculation of the start and end positions of the structure segments, so the anchor index can be obtained again), determining the maximum structured category probability corresponding to each anchor index, and determining the structure segment category corresponding to each anchor index based on the maximum structured category probability corresponding to each anchor index. For example, Figure 2As shown, the structural segment category corresponding to each anchor index can be determined as [first feature 201.index(max(first feature 201))]. First feature 201 is the last target_num features in a one-dimensional tensor with shape (1, 3 + target_num) corresponding to each anchor index. Figure 2 The first feature 201 is in a column of the second feature tensor information, where max(first feature 201) can be the maximum structured class probability corresponding to each anchor index, and index(max(first feature 201)) is the class index of the maximum structured class probability corresponding to each anchor index, so as to obtain the structured class corresponding to the class index in the first feature 201.

[0188] S408. Based on the start and end positions of the multiple structural segments and their respective structural segment categories, determine the structured text.

[0189] Specifically, based on the first feature and the last `target_num` features of each word in the second feature tensor information, it can be determined whether each word is an anchor point. Based on the second and third features in the one-dimensional feature vector corresponding to the anchored word, the start and end positions of multiple structural segments are determined. Finally, based on the last `target_num` features in the one-dimensional feature vector corresponding to the anchored word, the structural segment categories of the multiple structural segments are determined. For example... Figure 2 As shown, structured text can include the start and end positions of each structure segment and the segment category. In one example, such as... Figure 3 As shown, the structured text includes the start and end positions of multiple structural segments and word classifications. The word classification can be the category of the structural segment. The structured text also includes the confidence score corresponding to each structural segment. This confidence score is used to characterize the accuracy of the structural segment and can be the anchor point probability obtained in the structural segment anchor point selection.

[0190] In an optional embodiment of the present invention, it further includes:

[0191] Obtain the labeled text information and determine the second text quantity information for the labeled text information;

[0192] Determine the second text tensor information for the annotated text information;

[0193] Using the second text quantity information and the second text tensor information, a third feature tensor information for the labeled text information is generated;

[0194] Perform a dimensionality reduction operation on the third feature tensor information to generate the fourth feature tensor information;

[0195] The multilingual model Bert is trained based on the fourth feature tensor information.

[0196] Since the present invention has optimized the decoder part of the BERT model, in specific implementations, the present invention may also need to optimize the training process of the improved BERT model accordingly.

[0197] Unlike text information that needs to be structured, labeled text information can be text information used to train the BERT model.

[0198] This invention can acquire labeled text information and determine second text quantity information for the labeled text information; determine second text tensor information for the labeled text information; generate third feature tensor information for the labeled text information using the second text quantity information and the second text tensor information; perform dimensionality reduction operation on the third feature tensor information to generate fourth feature tensor information; and train the multilingual model Bert based on the fourth feature tensor information.

[0199] The process of determining the second text quantity information and the second text tensor information, as well as generating the third feature tensor information and the fourth feature tensor information, is basically similar to S401-S404, so the description is relatively simple. For relevant parts, please refer to the descriptions in S401-S404.

[0200] In this embodiment of the invention, by acquiring labeled text information and determining second text quantity information for the labeled text information; determining second text tensor information for the labeled text information; generating third feature tensor information for the labeled text information using the second text quantity information and the second text tensor information; performing dimensionality reduction operation on the third feature tensor information to generate fourth feature tensor information; and training the multilingual model BERT based on the fourth feature tensor information, a prerequisite is provided for using the BERT model to structure unstructured text information to generate structured text information, thereby ensuring the efficiency of generating structured text information.

[0201] In an optional embodiment of the present invention, the labeled text information includes a plurality of second structural segments, and the step of training the multilingual model BERT based on the fourth feature tensor information includes:

[0202] The annotated text information is segmented to determine the second sub-word for the annotated text information, and the sub-word sequence vector used to express the second sub-word; the sub-word sequence vector has corresponding sequence vector length information;

[0203] Determine text tag information for the labeled text information; the text tag information includes the start and end position information of the second structural segment for the second structural segment, and the category information of the second structural segment;

[0204] The fourth feature tensor information is used to determine the anchor point prediction value for the second sub-word, the structural segment start and end position information prediction value for the second structural segment, and the structural segment category score prediction value for the second structural segment;

[0205] The target anchor point score label value is determined using the sequence vector length information and the anchor point prediction value;

[0206] The starting position label value of the target structural segment is determined using the start and end position information of the second structural segment and the predicted value of the start and end position information of the structural segment.

[0207] The target structural segment category score label value is determined using the second structural segment category information and the predicted structural segment category score.

[0208] The multilingual model Bert is trained using the target anchor point score label value, the target structural segment start position label value, and the target structural segment category score label value.

[0209] To enable those skilled in the art to better understand the embodiments of the present invention, an example is provided below to illustrate how the embodiments of the present invention train the BERT model.

[0210] refer to Figure 5 , Figure 5 This is a flowchart of the steps of a model training method provided in an embodiment of the present invention.

[0211] S501: Obtain the input annotation text information.

[0212] The process for obtaining the labeled text information can be the same as that for obtaining the text information to be structured, and will not be repeated here. For ease of explanation, it is assumed that the number of input labeled text is 1. Therefore, the second text quantity information of the labeled text information will not be mentioned in subsequent steps.

[0213] S502: Obtain the text label information of the annotated text information.

[0214] The text label information may include the start and end positions of each second structural segment and the category information of the second structural segment.

[0215] S503: By processing the labeled text through the BERT model's word segmenter, word embedding layer, and encoding layer, the third feature tensor information corresponding to the labeled text information can be obtained.

[0216] S504 uses the decoder of the BERT model to perform feature dimensionality reduction on the third feature tensor information to obtain the fourth feature tensor information.

[0217] Unlike generating structured text information through the BERT model, during the training process of the BERT model, an additional list of sub-words obtained after word segmentation of the annotated text can be retained. For example... Figure 6 As shown, Figure 6 This is a schematic diagram of the model training process provided in an embodiment of the present invention; after being processed by the word segmenter of the BERT model, a sub-word sequence vector for the second sub-word can be obtained. The tensor shape of the sub-word sequence vector can be a one-dimensional vector of (1, seq_len), where seq_len can be the length of the sub-word list obtained by word segmentation of the labeled text.

[0218] S505: Based on the fourth feature tensor information, for each second sub-word, determine the predicted value of the anchor point corresponding to each second sub-word, the predicted value of the start and end position information of the second structural segment, and the predicted value of the structural segment category score.

[0219] Compared to generating structured text information through the BERT model, the training process for the BERT model does not require the selection of structural segment anchor points. Instead, it retains the predicted output of each sub-word in the seq_len dimension. This allows for maximizing the use of the BERT model's output for loss calculation, improving the accuracy of loss calculation, and further enhancing the model's training performance.

[0220] Optionally, for each second sub-word, the anchor point probability of each sub-word as a structural segment anchor point can be determined, and the anchor point probability is used as the anchor point score prediction value. The specific implementation method can be roughly the same as S405, and will not be described in detail here.

[0221] Optionally, for each second sub-word, the sub-word index of each second sub-word in the second feature tensor information can be determined. Then, based on the second feature and the third feature corresponding to each sub-word index in the second feature tensor information, the second feature corresponding to each sub-word index is subtracted from each sub-word index to obtain the "starting position of the structural segment corresponding to each structural segment anchor point", which is used as the predicted value of the starting position of the structural segment. The third feature corresponding to each sub-word index is added to each sub-word index to obtain the "ending position of the structural segment corresponding to each structural segment anchor point", which is used as the predicted value of the ending position of the structural segment.

[0222] Optionally, for each second sub-word, the sub-word index of each second sub-word in the second feature tensor information can be determined, the maximum structured category probability corresponding to each sub-word index can be determined, and the predicted value of the structure segment category score of the structure segment corresponding to each sub-word index can be determined based on the maximum structured category probability corresponding to each sub-word index.

[0223] S506: Based on the text label information of the annotated text information, determine the target anchor point score label value, the target structure segment start position label value, and the target structure segment category score label value corresponding to each second sub-word.

[0224] The target anchor score label value can include 1 and 0. The target anchor score label value can be used to characterize whether the subword is an anchor. 1 indicates that the second subword is an anchor, and 0 indicates that the second subword is not an anchor. The target structural segment start position label value can include the structural segment start position label value and the structural segment end position label value. The start position label value of the second structural segment can be used to characterize the start position of the structural segment corresponding to the anchor when the second sub-word is the anchor. If the second sub-word is not the anchor, the structural segment start position label value is 0. The structural segment end position label value is used to characterize the end position of the structural segment corresponding to the anchor when the second sub-word is the anchor. If the second sub-word is not the anchor, the structural segment end position label value is 0. The structural segment category score label value includes 1 and 0. The target structural segment category score label value can be used to characterize whether the second structural segment corresponding to the second sub-word belongs to the structural segment category when the second sub-word is the anchor. 1 indicates that the second sub-word belongs to the structural segment category, and 0 indicates that the sub-word does not belong to the structural segment category. If the second sub-word is not the anchor, the structural segment category label of each corresponding second structural segment is 0.

[0225] Optionally, the target anchor point score label, the target structure segment start position label, and the target structure segment category score label can be determined in the following ways.

[0226] S5061: Zero Matrix Initialization. Create four zero matrices with shapes (1, seq_len), (1, seq_len), (1, seq_len), and (1, seq_len, target_num), respectively, to record the anchor score label value, the start position label value, the end position label value, and the class score label value of each sub-word's structure segment. These four rectangles can be called the anchor_target matrix, start_target matrix, end_target matrix, and class_target matrix, respectively.

[0227] S5062: Obtain the index of the structural segment anchor point of each structural segment in the sub-word list in the text tag information.

[0228] Based on the text tag information obtained in S502 and the subword list obtained in S503, locate the index of the first second subword at the beginning of each second structural segment in the subword list, and determine the index as the structural segment anchor point of each structural segment, so as the index in the subword list.

[0229] Optionally, the position of the structural segment anchor can be added to the text tag. The position of the structural segment anchor can be the first second word at the beginning of the structural segment, or any other second word in the second structural segment.

[0230] S5063: Obtain the index of the termination position of each structural segment in the text tag information in the subword list.

[0231] Based on the text tag information obtained in S502 and the subword list obtained in S503, locate the index of the second-to-last subword in the subword list for each second structure segment.

[0232] S5064: Determine the tag value matrix corresponding to each second structural segment.

[0233] Generate a label zero matrix of shape (1, target_num). Based on the text label information obtained in S502 and the subword list obtained in S503, determine the index of the true category of each second structural segment in the target_num structural segment categories, and then set the value of the corresponding position in the label zero matrix to 1. For example, if category 3 is the fourth category in target_num, then the label of category 3 can be [0, 0, 0, 1, 0, 0, ...]. The length of this matrix is ​​target_num, which is used to represent that the probability of this structural segment in the true category is 1, and the probability in other structural segment categories is 0; the probability of each non-anchor subword in each category is 0.

[0234] S5065: Determine the anchor_target matrix corresponding to the labeled text information.

[0235] Based on the index of the anchor point of each second structural segment in the word list, in the zero matrix of shape (1, seq_len) generated in S2061 above, determine the corresponding position in the seq_len dimension, and set the 0 at the corresponding position to 1. This is used to represent that the anchor score label value of the second sub-words that can be used as structural segment anchors is 1, while the anchor score label value of other second sub-words is 0. The anchor_target matrix corresponding to the labeled text information can be obtained.

[0236] S5066: Determine the start_target matrix corresponding to the labeled text information.

[0237] In the text tag information of this invention embodiment, the structural segment anchor point of each second structural segment can be the first second sub-word of each second structural segment. Therefore, the index of the structural segment anchor point in the sub-word list can also be the index of the starting position of the second structural segment in the list.

[0238] Based on the index of the anchor point of each second structural segment in the sub-word list, in the zero matrix of shape (1, seq_len) generated by S2061, determine the corresponding position in the seq_len dimension, and set the 0 at the corresponding position to the index of the anchor point of that structural segment in the sub-word list. This is used to characterize that, except for the second sub-word that can be used as a structural segment anchor, the starting position label value of the structural segment is the index corresponding to the anchor point, and the starting position label value of the structural segment of other sub-words is 0. The start_target matrix corresponding to the annotated text can be obtained.

[0239] If the first second subword of the second structural segment is not the structural segment anchor, then the index of the first second subword of each second structural segment in the subword list, and the index of the structural segment anchor of each second structural segment in the subword list can be obtained. Based on the index of the structural segment anchor of each second structural segment in the subword list, the corresponding position in the seq_len dimension of the zero matrix of shape (1, seq_len) generated by S2061 can be determined, and the 0 at the corresponding position can be set to the index of the first subword in the subword list.

[0240] S5067: Determine the end_target matrix corresponding to the labeled text information.

[0241] Based on the index of the anchor point of each second structural segment in the subword list, in the zero matrix of shape (1, seq_len) generated by S2061, determine the corresponding position in the seq_len dimension, and set the 0 at the corresponding position to the index of the last first second subword in the subword list. This is used to characterize that, except for the second subword that can be used as a structural segment anchor with the structural segment termination position label value corresponding to the index of the last first second subword, the anchor point score label value of other second subwords is 0. The end_target matrix corresponding to the annotated text can be obtained.

[0242] S5068: Determine the class_target matrix corresponding to the labeled text information.

[0243] Based on the index of the anchor point of each second structural segment in the word list, in the zero matrix of shape (1, seq_len, target_num) generated by S2061, determine the corresponding position of each structural segment anchor point in the seq_len dimension, and replace the (1, target_num) zero matrix at the corresponding position with the label value matrix corresponding to the structural segment of that structural segment anchor point, thereby obtaining the class_target matrix corresponding to the annotated text.

[0244] S507: Using the target anchor point score label value, the target structural segment start position label value, and the target structural segment category score label value, calculate the structural segment anchor point score loss, structural segment start and end position loss, and structural segment category score loss, and obtain the total loss after weighted summation.

[0245] Optionally, calculating the anchor score loss for a structural segment may include: using the cross-entropy loss function to calculate the loss error between the anchor score label value (anchor_target) and the anchor score prediction value (anchor_pred) corresponding to each structural segment anchor.

[0246] Optionally, the loss at the start and end positions of the structural segment can be the IOU loss. Calculating the loss can include: determining the loss based on the predicted start position value, start position label value, end position prediction value, and end position label value corresponding to each structural segment anchor point. Specifically, the calculation formula can be as follows:

[0247] Intersection max =ax(start) pred ,tart target )#(1)

[0248] Intersection min =Min(end) pred ,nd target )#(2)

[0249] Union max =Min(start) pred ,tart target )#(3)

[0250] Union min =ax(end) pred ,nd target )#(4)

[0251] Intersection = Intersection max-ntersection min #(5)

[0252] Union = Union max -nion min #(6)

[0253]

[0254] Intersection max It could be the starting coordinates of the most intersecting part.

[0255] Intersection min It can be the ending coordinate of the intersection, Union max It can be the starting coordinate of the union part, Union min It can be the terminating coordinate of the union part, Intersection can be the intersection distance, and Union can be the union distance.

[0256] Optionally, calculating the segment class score loss may include: using the cross-entropy loss function to calculate the loss error of the segment class score label value (class_target) and the segment class score prediction value (class_pred) for each segment anchor point.

[0257] Optionally, the weights for the weighted sum can be set freely; in this case, an arithmetic mean is used.

[0258] S508: Based on the total loss, update the parameters of the BERT model through backpropagation to train the multilingual model BERT.

[0259] Specifically, the gradient descent algorithm is used to update the parameters of BERT word embeddings, BERT encoding layers, and fully connected layers to optimize the model performance.

[0260] It should be noted that if there are N input labeled texts, each labeled text and the BERT model can be trained through S502 to S508. Each time S502 to S508 is executed, the BERT model can be trained for one round. Each labeled text can be trained for multiple rounds. After M rounds of training, the improved BERT model can be obtained.

[0261] In this embodiment of the invention, word segmentation is performed on the annotated text information to determine a second sub-word for the annotated text information and a sub-word sequence vector for expressing the second sub-word; the sub-word sequence vector has corresponding sequence vector length information; text tag information for the annotated text information is determined; the text tag information includes the start and end position information of the second structural segment for the second structural segment, and the category information of the second structural segment; the fourth feature tensor information is used to determine the anchor point prediction value for the second sub-word, the predicted value of the start and end position information of the structural segment for the second structural segment, and the predicted value of the structural segment category score for .... The target anchor point score label value is determined by using the sequence vector length information and the anchor point prediction value; the target structural segment start position label value is determined by using the second structural segment start and end position information and the predicted value of the structural segment start and end position information; the target structural segment category score label value is determined by using the second structural segment category information and the predicted value of the structural segment category score; the target anchor point score label value, the target structural segment start position label value, and the target structural segment category score label value are used to train the multilingual model BERT, thereby realizing the pre-training process of the BERT model, thus ensuring that the BERT model can generate structured text information based on the text information to be structured.

[0262] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0263] Reference Figure 7 The diagram illustrates a structural block diagram of a structured text information generation device provided in an embodiment of the present invention, which may specifically include the following modules:

[0264] The unstructured text information acquisition module 701 is used to acquire unstructured text information and determine the first text quantity information and the first text tensor information for the unstructured text information.

[0265] The first sub-word determination module 702 is used to perform word segmentation on the text information to be structured and determine the first sub-word for the text information to be structured.

[0266] The first feature tensor information generation module 703 is used to generate first feature tensor information for the first sub-word using the first text quantity information and the first text tensor information.

[0267] The structured text information generation module 704 is used to generate structured text information for the text information to be structured based on the first feature tensor information.

[0268] Optionally, the method is applied to a pre-trained multilingual model BERT, which includes a word embedding layer and an encoding layer. The first feature tensor information generation module, using the first text quantity information and..., is used for:

[0269] The word embedding layer dimension information for the word embedding layer and the encoding layer dimension information for the encoding layer are determined using the first text tensor information.

[0270] Based on the word embedding layer dimension information, the encoding layer dimension information, and the first text quantity information, a first feature tensor information for the first sub-word is generated.

[0271] Optionally, the structured text information generation module is used for:

[0272] Perform a dimensionality reduction operation on the first feature tensor information to generate the second feature tensor information;

[0273] Based on the second feature tensor information, structured text information is generated for the text information to be structured.

[0274] Optionally, the multilingual model BERT includes a connection layer, and the structured text information generation module is used for:

[0275] Determine the connection layer dimension information for the connection layer;

[0276] Based on the connection layer dimension information, the multilingual model Bert is used to perform dimensionality reduction on the first feature tensor information to generate the second feature tensor information.

[0277] Optionally, the text information to be structured includes multiple first structure segments, and the structured text information generation module is used for:

[0278] The anchor point of the first structural segment is determined based on the second feature tensor;

[0279] Based on the anchor points of the structural segments, determine the start and end position information of the first structural segment for the first structural segment;

[0280] Based on the start and end positions, determine the first structural segment category information for the first structural segment;

[0281] Based on the anchor point of the first structural segment, the start and end position information of the first structural segment, and the category information of the first structural segment, the structured text information for the text information to be structured is determined.

[0282] Optionally, it also includes:

[0283] The annotation text information acquisition module is used to acquire annotation text information and determine the second text quantity information for the annotation text information;

[0284] The second text tensor information determination module is used to determine the second text tensor information for the labeled text information;

[0285] The third feature tensor information generation module is used to generate third feature tensor information for the labeled text information using the second text quantity information and the second text tensor information.

[0286] The fourth feature tensor information generation module is used to perform a dimensionality reduction operation on the third feature tensor information to generate the fourth feature tensor information;

[0287] The BERT multilingual model training module is used to train the BERT multilingual model based on the fourth feature tensor information.

[0288] Optionally, the multilingual model BERT training module is used for:

[0289] The annotated text information is segmented to determine the second sub-word for the annotated text information, and the sub-word sequence vector used to express the second sub-word; the sub-word sequence vector has corresponding sequence vector length information;

[0290] Determine text tag information for the labeled text information; the text tag information includes the start and end position information of the second structural segment for the second structural segment, and the category information of the second structural segment;

[0291] The fourth feature tensor information is used to determine the anchor point prediction value for the second sub-word, the structural segment start and end position information prediction value for the second structural segment, and the structural segment category score prediction value for the second structural segment;

[0292] The target anchor point score label value is determined using the sequence vector length information and the anchor point prediction value;

[0293] The starting position label value of the target structural segment is determined using the start and end position information of the second structural segment and the predicted value of the start and end position information of the structural segment.

[0294] The target structural segment category score label value is determined using the second structural segment category information and the predicted structural segment category score.

[0295] The multilingual model Bert is trained using the target anchor point score label value, the target structural segment start position label value, and the target structural segment category score label value.

[0296] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0297] In addition, this invention also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described structured text information generation method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0298] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described structured text information generation method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0299] Figure 8 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0300] The electronic device 800 includes, but is not limited to, components such as: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, a processor 810, and a power supply 811. Those skilled in the art will understand that... Figure 8 The electronic device structures shown are not intended to limit the electronic device. An electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. In embodiments of the present invention, the electronic device includes, but is not limited to, mobile phones, tablet computers, laptops, PDAs, in-vehicle terminals, wearable devices, and pedometers.

[0301] It should be understood that, in this embodiment of the invention, the radio frequency unit 801 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink data from the base station and processes it with the processor 810; additionally, it transmits uplink data to the base station. Typically, the radio frequency unit 801 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, etc. Furthermore, the radio frequency unit 801 can also communicate with networks and other devices through a wireless communication system.

[0302] Electronic devices provide users with wireless broadband internet access through network module 802, such as helping users send and receive emails, browse web pages, and access streaming media.

[0303] The audio output unit 803 can convert audio data received by the radio frequency unit 801 or the network module 802 or stored in the memory 809 into audio signals and output them as sound. Furthermore, the audio output unit 803 can also provide audio output related to specific functions performed by the electronic device 800 (e.g., call signal reception sound, message reception sound, etc.). The audio output unit 803 includes a speaker, a buzzer, and a receiver, etc.

[0304] Input unit 804 is used to receive audio or video signals. Input unit 804 may include a graphics processing unit (GPU) 8041 and a microphone 8042. The GPU 8041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on display unit 806. The image frames processed by GPU 8041 can be stored in memory 809 (or other storage medium) or transmitted via radio frequency unit 801 or network module 802. Microphone 8042 can receive sound and process such sound into audio data. The processed audio data can be converted into a format that can be transmitted to a mobile communication base station via radio frequency unit 801 in telephone call mode.

[0305] The electronic device 800 also includes at least one sensor 805, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 8061 according to the ambient light level, and the proximity sensor can turn off the display panel 8061 and / or backlight when the electronic device 800 is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used to identify the posture of the electronic device (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. The sensor 805 may also include a fingerprint sensor, pressure sensor, iris sensor, molecular sensor, gyroscope, barometer, hygrometer, thermometer, infrared sensor, etc., which will not be described in detail here.

[0306] The display unit 806 is used to display information input by the user or information provided to the user. The display unit 806 may include a display panel 8061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0307] User input unit 807 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of electronic devices. Specifically, user input unit 807 includes a touch panel 8071 and other input devices 8072. Touch panel 8071, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 8071). Touch panel 8071 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 810, which receives and executes commands from the processor 810. In addition, touch panel 8071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. Besides touch panel 8071, user input unit 807 may also include other input devices 8072. Specifically, other input devices 8072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, joysticks, etc., which will not be described in detail here.

[0308] Furthermore, the touch panel 8071 can cover the display panel 8061. When the touch panel 8071 detects a touch operation on or near it, it transmits the information to the processor 810 to determine the type of touch event. Subsequently, the processor 810 provides corresponding visual output on the display panel 8061 based on the type of touch event. Although in Figure 8 In this embodiment, the touch panel 8071 and the display panel 8061 are two independent components to realize the input and output functions of the electronic device. However, in some embodiments, the touch panel 8071 and the display panel 8061 can be integrated to realize the input and output functions of the electronic device. The specific implementation is not limited here.

[0309] Interface unit 808 serves as an interface for connecting external devices to electronic device 800. For example, external devices may include a wired or wireless headphone port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 808 can be used to receive input from external devices (e.g., data, power, etc.) and transmit the received input to one or more components within electronic device 800, or it can be used to transmit data between electronic device 800 and external devices.

[0310] The memory 809 can be used to store software programs and various data. The memory 809 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 809 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0311] The processor 810 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 809, and by calling data stored in the memory 809, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. The processor 810 may include one or more processing units; preferably, the processor 810 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 810.

[0312] The electronic device 800 may also include a power supply 811 (such as a battery) for supplying power to various components. Preferably, the power supply 811 is logically connected to the processor 810 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system.

[0313] In addition, the electronic device 800 includes some functional modules not shown, which will not be described in detail here.

[0314] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0315] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0316] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

[0317] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0318] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0319] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0320] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0321] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0322] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0323] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for generating structured text information, characterized in that, include: Obtain the text information to be structured, and determine the first text quantity information and the first text tensor information for the text information to be structured; Perform word segmentation on the text information to be structured to determine the first sub-word for the text information to be structured; Using the first text quantity information and the first text tensor information, a first feature tensor information for the first subword is generated; Based on the first feature tensor information, structured text information is generated for the text information to be structured; The step of generating structured text information for the text information to be structured based on the first feature tensor information includes: Perform a dimensionality reduction operation on the first feature tensor information to generate the second feature tensor information; Based on the second feature tensor information, structured text information is generated for the text information to be structured; The text information to be structured includes multiple first structural segments, and the step of generating structured text information for the text information to be structured based on the second feature tensor information includes: The anchor point of the first structural segment is determined based on the second feature tensor; Based on the anchor points of the structural segments, determine the start and end position information of the first structural segment for the first structural segment; Based on the start and end positions, determine the first structural segment category information for the first structural segment; Based on the anchor point of the first structural segment, the start and end position information of the first structural segment, and the category information of the first structural segment, the structured text information for the text information to be structured is determined.

2. The method according to claim 1, characterized in that, The method is applied to a pre-trained multilingual model BERT, which includes a word embedding layer and an encoding layer. The step of generating first feature tensor information for the first subword using the first text quantity information and the first text tensor information includes: The word embedding layer dimension information for the word embedding layer and the encoding layer dimension information for the encoding layer are determined using the first text tensor information. Based on the word embedding layer dimension information, the encoding layer dimension information, and the first text quantity information, a first feature tensor information for the first sub-word is generated.

3. The method according to claim 2, characterized in that, The multilingual model BERT includes a connection layer, and the step of performing dimensionality reduction on the first feature tensor information to generate the second feature tensor information includes: Determine the connection layer dimension information for the connection layer; Based on the connection layer dimension information, the multilingual model Bert is used to perform dimensionality reduction on the first feature tensor information to generate the second feature tensor information.

4. The method according to claim 2, characterized in that, Also includes: Obtain the labeled text information and determine the second text quantity information for the labeled text information; Determine the second text tensor information for the annotated text information; Using the second text quantity information and the second text tensor information, a third feature tensor information for the labeled text information is generated; Perform a dimensionality reduction operation on the third feature tensor information to generate the fourth feature tensor information; The multilingual model Bert is trained based on the fourth feature tensor information.

5. The method according to claim 4, characterized in that, The labeled text information includes multiple second structural segments, and the step of training the multilingual model Bert based on the fourth feature tensor information includes: The annotated text information is segmented to determine the second sub-word for the annotated text information, and the sub-word sequence vector used to express the second sub-word; the sub-word sequence vector has corresponding sequence vector length information; Determine text tag information for the labeled text information; the text tag information includes the start and end position information of the second structural segment for the second structural segment, and the category information of the second structural segment; The fourth feature tensor information is used to determine the anchor point prediction value for the second sub-word, the structural segment start and end position information prediction value for the second structural segment, and the structural segment category score prediction value for the second structural segment; The target anchor point score label value is determined using the sequence vector length information and the anchor point prediction value; The starting position label value of the target structural segment is determined using the start and end position information of the second structural segment and the predicted value of the start and end position information of the structural segment. The target structural segment category score label value is determined using the second structural segment category information and the predicted structural segment category score. The multilingual model Bert is trained using the target anchor point score label value, the target structural segment start position label value, and the target structural segment category score label value.

6. A structured text information generation device, characterized in that, include: The unstructured text information acquisition module is used to acquire the unstructured text information and determine the first text quantity information and the first text tensor information for the unstructured text information; The first sub-word determination module is used to perform word segmentation on the text information to be structured and determine the first sub-word for the text information to be structured. The first feature tensor information generation module is used to generate first feature tensor information for the first sub-word using the first text quantity information and the first text tensor information. A structured text information generation module is used to generate structured text information for the text information to be structured based on the first feature tensor information; The structured text information generation module is used for: Perform a dimensionality reduction operation on the first feature tensor information to generate the second feature tensor information; Based on the second feature tensor information, structured text information is generated for the text information to be structured; The text information to be structured includes multiple first structure segments, and the structured text information generation module is used for: The anchor point of the first structural segment is determined based on the second feature tensor; Based on the anchor points of the structural segments, determine the start and end position information of the first structural segment for the first structural segment; Based on the start and end positions, determine the first structural segment category information for the first structural segment; Based on the anchor point of the first structural segment, the start and end position information of the first structural segment, and the category information of the first structural segment, the structured text information for the text information to be structured is determined.

7. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method as described in any one of claims 1-5.

8. A computer-readable storage medium having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1-5.

Images