A text semantic model training method, a text semantic recognition method and device

Abstract

This invention relates to the field of natural language processing technology and discloses a text semantic model training method, a text semantic recognition method, and an apparatus. This invention is applicable to long text semantic recognition scenarios. Based on the start position identifier, character content identifier, character end position identifier, non-character identifier, text length, and total number of target segments for each character in each target segment in the target segment set, feature training is performed. This can effectively avoid severing the semantic connection between context and avoid the phenomenon of model error propagation, thereby improving the model accuracy. At the same time, it does not require excessive hardware and software resources for text data encoding, thus optimizing costs and enhancing practicality.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a text semantic model training method, a text semantic recognition method, and an apparatus. Background Technology

[0002] Long texts typically refer to those exceeding 1000 characters. Directly encoding long texts and feeding them into neural networks for training can lead to problems such as vanishing and exploding gradients. Simultaneously, the length limitations of pre-trained models and the high hardware requirements of excessively long texts make training on long text tasks challenging. Therefore, most related techniques segment long texts according to pre-defined lengths, such as segmenting them by punctuation marks indicating sentence endings (periods, exclamation marks, etc.) or directly segmenting them into fixed-length fragments. This segmentation method treats the context as independent segments, artificially fragmenting the semantic meaning of the context. Fixed-length segmentation can lead to information loss, especially at the boundaries of text segments. If a semantic unit (such as a sentence or paragraph) is segmented into two segments, the model may struggle to capture the complete context, resulting in a bias in the model's learning of each segment.

[0003] Furthermore, most existing semantic fragment extraction schemes are designed for short text scenarios, and the extracted semantic fragments are generally short in length. Generally, current mainstream pre-trained models perform well for short text encoding; simultaneously, the short length of semantic fragments allows the semantic fragment extraction task to be simply viewed as a named entity recognition task. Currently, there are many high-performing named entity recognition models available, so these can be used, thus reducing the difficulty of the task. However, existing solutions for short text and short semantic fragments are not suitable for long text semantic fragment extraction. First, at the boundaries of text fragments, the model may perform poorly due to a lack of context, especially when a semantic unit spans two fragments. Second, the position of the text segmentation can affect the results. If the segmentation point is exactly in the middle of an important semantic unit, the model may fail to understand the complete semantic meaning. Additionally, direct encoding schemes require significant hardware resources, resulting in extremely high costs and making them impractical. Summary of the Invention

[0004] In view of this, the present invention provides a text semantic model training method, a text semantic recognition method and apparatus to solve the problems of inaccurate recognition of long text semantics and high cost of hardware encoding.

[0005] According to a first aspect, embodiments of this disclosure provide a text semantic model training method, the method comprising:

[0006] Multiple target text data are acquired, and each target text data is segmented into segments to obtain multiple segments of the target text data. The target text data is long text data.

[0007] Select a subset of fragments from multiple segments as the target fragment set for each target text data;

[0008] Obtain the characteristics of the target fragment set. The characteristics of the target fragment set include: the characteristics of each target fragment in the target fragment set, and the characteristics of each target fragment include: the start position identifier of each character, the character content identifier, the end position identifier of the character, the non-character identifier, the text length, and the total number of target fragments.

[0009] The features of the target fragment set are input into the text semantic recognition model to obtain the text semantic recognition result corresponding to the features of the target fragment set, as well as the loss value between the text semantic recognition result corresponding to the features of the target fragment set and the true label of the target text data. The loss value is used to update the parameter values ​​of the text semantic recognition model.

[0010] The text semantic model training method in this embodiment is applicable to long text semantic recognition scenarios. It performs feature training based on the start position identifier, character content identifier, character end position identifier, non-character identifier, text length, and total number of target segments for each character in each target segment set. This effectively avoids severing the semantic connection between contexts and avoids the propagation of model errors, thereby improving the model accuracy. At the same time, it does not require excessive hardware and software resources for text data encoding, thus optimizing costs and enhancing practicality.

[0011] In some optional implementations, the target text data is segmented to obtain multiple segments of the target text data, including:

[0012] Obtain multiple initial text data;

[0013] Multiple initial text data are cleaned to obtain multiple target text data;

[0014] Obtain the maximum text length from the text lengths corresponding to multiple target text data;

[0015] The target text data is segmented according to a preset text length to obtain multiple segments of the target text data. The number of segments is determined based on the maximum text length and the preset text length.

[0016] The embodiments disclosed herein clean and segment text data, which not only adapts to text recognition with different contextual segments, but also helps to improve the accuracy of text semantic recognition.

[0017] In some optional implementations, the text semantic model training method in this disclosure further includes: for each target segment in the target segment set, randomly selecting a target number of characters from the target segment, wherein the target number is determined according to the number of characters included in the target segment and a preset selection ratio; and labeling each character in the target number of characters to form a start position identifier, a character content identifier, a character end position identifier, and a non-character identifier for each character.

[0018] In some optional implementations, the features of the target fragment set are input into a text semantic recognition model to obtain the semantic recognition results corresponding to the features of the target fragment set, including:

[0019] The features of the target fragment set are encoded using a feature encoding network in the text semantic recognition model to obtain the feature vector of the target fragment set; the feature encoding network is the XLNet encoding network.

[0020] The feature decoding network in the text semantic recognition model is used to decode the features of the encoded target fragment set to obtain the semantic recognition result corresponding to the features of the target fragment set. The feature decoding network is a conditional random field network.

[0021] The embodiments of this disclosure encode and decode the features of the target fragment set through a feature encoding network, effectively handling the semantic relationships of the context, thereby reducing error propagation during the model learning process and improving the accuracy of extracting semantic fragments from long texts.

[0022] In some optional implementations, the parameter values ​​of the text semantic recognition model are updated through the following steps:

[0023] Based on the loss value between the text semantic recognition result corresponding to the features of the target fragment set and the real label of the target text data, the text semantic recognition model is trained cyclically until the preset number of training iterations is reached to end the training, so as to obtain the trained text semantic recognition model.

[0024] The embodiments disclosed herein improve the recognition accuracy of the text semantic recognition model by continuously iterating and updating the parameters of the text semantic recognition model.

[0025] According to a second aspect, embodiments of this disclosure provide a text semantic recognition method, the method comprising:

[0026] The text data to be recognized is obtained and segmented into multiple segments. The text data to be recognized is long text data.

[0027] Select a subset of fragments from multiple segments to form the set of fragments to be identified in the text data to be identified.

[0028] Obtain the features of the set of segments to be identified. The features of the set of segments to be identified include: the features of each segment to be identified in the set of segments to be identified. The features of each segment to be identified include: the start position identifier of each character, the character content identifier, the end position identifier of the character, the non-character identifier, the text length, and the total number of target segments.

[0029] The features of the set of fragments to be identified are input into the text semantic recognition model in the first aspect or any embodiment of the first aspect to obtain the text semantic recognition result corresponding to the text data to be identified.

[0030] According to a third aspect, embodiments of this disclosure provide a text semantic model training apparatus, the apparatus comprising:

[0031] The data acquisition module is used to acquire multiple target text data and segment each target text data into multiple segments, where the target text data is long text data;

[0032] The sample selection module is used to select a subset of fragments from multiple segments as the target fragment set for the target text data.

[0033] The feature acquisition module is used to acquire the features of the target fragment set. The features of the target fragment set include: the features of each target fragment in the target fragment set; the features of each target fragment include: the start position identifier of each character, the character content identifier, the end position identifier of the character, the non-character identifier, the text length, and the total number of target fragments.

[0034] The feature training module is used to input the features of the target fragment set into the text semantic recognition model, obtain the text semantic recognition result corresponding to the features of the target fragment set, and the loss value between the text semantic recognition result corresponding to the features of the target fragment set and the true label of the target text data. The loss value is used to update the parameter values ​​of the text semantic recognition model.

[0035] According to a fourth aspect, embodiments of this disclosure provide a computer device, including:

[0036] The memory and the processor are interconnected and communicate with each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the text semantic model training method in the first aspect or any embodiment of the first aspect, or the text semantic recognition method in the second aspect.

[0037] According to a fifth aspect, embodiments of this disclosure provide a computer-readable storage medium storing computer instructions, which are used to cause a computer to perform the text semantic model training method of the first aspect or any embodiment of the first aspect, or the text semantic recognition method of the second aspect.

[0038] In a sixth aspect, embodiments of this disclosure provide a computer program product, including computer instructions for causing a computer to execute the text semantic model training method in the first aspect or any embodiment of the first aspect, or the text semantic recognition method in the second aspect. Attached Figure Description

[0039] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating a text semantic model training method according to an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram illustrating the segmentation of text according to text length according to an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of long text data segmentation according to an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram illustrating the segmentation of multiple text segments based on the maximum text length and a preset text length according to an embodiment of the present invention;

[0044] Figure 5 This is a schematic diagram illustrating the annotation features of each character in a target number of characters according to an embodiment of the present invention;

[0045] Figure 6 This is a schematic diagram of the encoding and decoding process according to an embodiment of the present invention;

[0046] Figure 7 This is a simplified flowchart illustrating the text semantic model training method according to an embodiment of the present invention;

[0047] Figure 8 A flowchart illustrating a text semantic recognition method according to an embodiment of the present invention;

[0048] Figure 9 This is a simplified flowchart illustrating the text semantic recognition method according to an embodiment of the present invention;

[0049] Figure 10 This is a structural block diagram of the text semantic model training device according to an embodiment of the present invention;

[0050] Figure 11 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] According to embodiments of the present invention, a text semantic model training method is provided, which can be applied in scenarios requiring high accuracy, such as information retrieval, text analysis, or document element recognition, for example, in rigorous vertical fields like medicine and law. In these vertical fields, information extraction needs to ensure accuracy, and existing text training methods for short text scenarios are not suitable for long text semantic recognition scenarios requiring high accuracy. Therefore, this disclosure provides a text semantic model training method applicable to long text scenarios. It should be noted that the steps shown in the flowcharts can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0053] This embodiment provides a text semantic model training method, which can be used on computer devices and mobile terminals, such as mobile phones and tablets. Figure 1 This is a flowchart of a text semantic model training method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0054] Step S101: Obtain multiple target text data and segment each target text data to obtain multiple segments of the target text data. The target text data is long text data.

[0055] Step S102: Select a subset of segments from multiple segments as the target segment set for each target text data.

[0056] Specifically, multiple target text datasets for training are prepared and data cleaning is performed. The target text datasets consist of many sample batches of varying lengths, with each batch containing a fixed number of samples (batch_size). Each batch is passed in one time, with dimensions [batch_size, dmax]. For each batch, as follows... Figure 2As shown, the text is segmented according to its length, that is, the vector of dimension [batch_size, dmax] is divided into S segments of dimension [batch_size, L]. A predetermined number of segments (corresponding to some segments mentioned above) are selected from the segmented segments to form the target segment set for each target text data.

[0057] In this disclosure, a segment refers to a small portion of natural language text that has a specific meaning or context. It is typically a phrase, sentence, or fragment of text, with variable length, containing specific semantic information that can be used to understand the meaning of the text or express specific concepts. In natural language processing, extracting segments aims to capture important semantic content from text for further tasks such as semantic analysis, information retrieval, and question-answering systems. Current solutions generally perform poorly with long texts; therefore, improving the accuracy of training semantic segments from long texts is of great significance.

[0058] In some optional implementations, the target text data is segmented to obtain multiple segments of the target text data, including:

[0059] Step a1: Obtain multiple initial text data.

[0060] Step a2 involves cleaning multiple initial text data to obtain multiple target text data.

[0061] Step a3: Obtain the maximum text length from the text lengths corresponding to multiple target text data.

[0062] Step a4: The target text data is segmented according to the preset text length to obtain multiple segments of the target text data. The number of segments is determined based on the maximum text length and the preset text length.

[0063] Specifically, multiple initial text data are prepared for training, and after cleaning, multiple target text data are obtained. The maximum length d among these N data is then calculated. max Since the long text exceeds the readable length, it needs to be segmented into multiple segments. Therefore, it is necessary to statistically analyze the maximum length of the data, such as... Figure 3 The diagram illustrates the segmentation of long text data, dividing a piece of text into multiple segments according to a preset text length. The preset text length for each segment is set to L, such as L = 256 or 512, etc. The number of segments S is d. max / L is the integer value. For example, Figure 4 As shown, for example, when L is 256, when d max When d is 512, S is 2, meaning the long text is divided into two segments; when d... max When S is 3, the last segment that is less than 256 can be filled with padding bits.

[0064] In this embodiment of the disclosure, the preset text length can be flexibly set, and different lengths L can be selected for training in each training process.

[0065] Step S103: Obtain the features of the target fragment set. The features of the target fragment set include: the features of each target fragment in the target fragment set, and the features of each target fragment include: the start position identifier of each character, the character content identifier, the character end position identifier, the non-character identifier, the text length, and the total number of target fragments.

[0066] As an example, for each target segment in the target segment set, a target number of characters are randomly selected from the target segment, wherein the target number is determined according to the number of characters included in the target segment and the preset selection ratio; each character in the target number of characters is labeled to form a start position identifier, a character content identifier, a character end position identifier, and a non-character identifier for each character.

[0067] Specifically, for each target segment in the target segment set, the labeling space is set to BIEO. BIEO is a common sequence labeling scheme. B (Begin) represents the beginning of an entity (corresponding to the start position of each character in the above examples); I (Inside) represents other words inside the entity (corresponding to the character content in the above examples), following a label starting with B; E (End) represents the end of the entity (corresponding to the end position of the character in the above examples), usually the last word of the entity; O (Outside) represents words that are not any entity (corresponding to the non-character identifiers in the above examples). In this task, semantic segments of different lengths are treated as entities of different lengths, and the start and end positions of semantic segments are labeled using BIEO tags to extract features of semantic segments in the text. Figure 5 As shown, for example, each target segment in the target segment set contains 0-11 characters. A target number of characters are randomly selected from the target segments. The target number is determined based on the number of characters included in the target segment and a preset selection ratio. For example, from... Figure 5 Characters at positions 1-5 and 8-10 are selected and labeled. This labeling process yields target segment features with B, I, E, and O tags. Therefore, the two text segments at positions 1-5 and 8-10 can be extracted as semantic segments within this text. This method yields a labeled target segment dataset D with dimensions N×d, where D contains N segment data points, and d represents the length of each target segment.

[0068] Furthermore, the features of the target fragments also include text length and the total number of target fragments. Multiple target text data are divided into many sample batches of different lengths. The number of samples in each batch is a fixed batch_size. Each time a batch is passed in, the dimension is [batch_size, dmax]. For each batch, such as... Figure 2 As shown, the text is segmented along the length dimension, that is, the vector of dimension [batch_size, dmax] is divided into S segments of dimension [batch_size, L]. A predetermined number of segments are selected from the segmented segments to form the target segment set for each target text data. The target segment set of each target text data is labeled with B, I, E, and O after feature annotation.

[0069] Step S104: Input the features of the target fragment set into the text semantic recognition model to obtain the text semantic recognition result corresponding to the features of the target fragment set, and the loss value between the text semantic recognition result corresponding to the features of the target fragment set and the real label of the target text data. The loss value is used to update the parameter values ​​of the text semantic recognition model.

[0070] In some optional implementations, the features of the target fragment set are input into a text semantic recognition model to obtain the semantic recognition results corresponding to the features of the target fragment set, including:

[0071] Step b1: Encode the features of the target fragment set using the feature encoding network in the text semantic recognition model to obtain the feature vector of the target fragment set; the feature encoding network is the XLNet encoding network.

[0072] Step b2: The feature decoding network in the text semantic recognition model is used to decode the features of the encoded target fragment set to obtain the semantic recognition result corresponding to the features of the target fragment set. The feature decoding network is a conditional random field network.

[0073] Specifically, based on step S103 above, the features of each target fragment set are fed into the XLNet encoding network for encoding. Figure 2 In this process, the input vector of the first vector T in each batch with a preset text length of L is randomly initialized, while the input vectors of the second to Sth vectors T are the last output of the previous stage. This method achieves semantic continuity of context, thereby enabling more accurate extraction of semantic segments between adjacent paragraphs.

[0074] like Figure 6As shown, each batch of S T vectors is encoded by the XLNet encoding network, resulting in S vectors of dimension [batch_size, L, emb], where emb represents the dimension of the numerical representation of the characters after encoding by the XLNet feature encoding network. Each of the S text segments is then decoded using a Conditional Random Field (CRF) network in the order of the S text segments. Figure 6 In this process, the decoding space is the BIEO space defined above, and the relationships between labels are constrained using joint probabilities. The final BIEO result is obtained by collecting consecutive BIE labels as extracted semantic fragments. The recognition results are compared with the true labels, using cross-entropy as the loss function.

[0075] As an example, the parameter values ​​of the text semantic recognition model are updated through the following steps:

[0076] Based on the loss value between the text semantic recognition result corresponding to the features of the target fragment set and the real label of the target text data, the text semantic recognition model is trained cyclically until the preset number of training iterations is reached to end the training, so as to obtain the trained text semantic recognition model.

[0077] Specifically, repeat steps S101-S104 above, iteratively training for multiple epochs (corresponding to the preset number of training cycles mentioned above) until the last epoch ends or the loss function fails to converge for multiple consecutive epochs, at which point training stops directly. Here, "multiple consecutive epochs" is a hyperparameter, typically taking values ​​of 5, 10, 20, etc. After training is complete, use a test set to evaluate model performance, including precision, recall, and F1-score.

[0078] In some optional implementations, the text semantic model training method in this disclosure further includes:

[0079] Step c1: Select a subset of segments from multiple segments as the training set, and select a subset of segments from multiple segments as the test set.

[0080] Step c2: Extract features from the training set for each target segment.

[0081] Step c3: Input the features of each target segment in the test set and the features of each target segment in the training set into the text semantic recognition model.

[0082] Step c4: Obtain the text semantic recognition results corresponding to the features of the test set.

[0083] Step c5: Compare the text semantic recognition results corresponding to the features of the test set with the text semantic recognition results corresponding to the features of the training set to obtain the accuracy of the text semantic recognition model.

[0084] Step c6: Adjust the parameters of the text semantic recognition model until the accuracy of the text semantic recognition model meets the requirements, and then you will get the trained text semantic recognition model.

[0085] Specifically, the original dataset D is randomly sampled and split into two parts in a 7:2:1 ratio. 70% of the data is used as the training set to train the wooden bowl text recognition model; 20% of the data is used as the validation set to verify the performance of the current model after each epoch of training (epoch represents one training cycle); and 10% of the data is used as the test set to verify the performance of the final model.

[0086] like Figure 7 The diagram shown is a simplified flowchart of the text semantic model training method in this embodiment of the present disclosure.

[0087] exist Figure 7 Before inputting the S T vectors from each batch into the XLNet encoding network for encoding, it first determines whether each segmented vector is the first segment of the target text data (long text data). If it is, the input vectors are randomly initialized, and the S T vectors are encoded before being encoded using the XLNet encoding network. If it is not the first segment of the target text data (long text data), the last hidden vector of the previous segment is used, and then encoded using the XLNet encoding network. Then, it determines whether each encoded vector is the last segment of the target text data (long text data). If it is, it is encoded again using the XLNet encoding network and then decoded using a conditional random field network.

[0088] In summary, the text semantic model training method in this embodiment is suitable for long text semantic recognition scenarios. Based on the start position identifier, character content identifier, character end position identifier, non-character identifier, text length, and total number of target segments for each character in the target segment set, feature training can effectively avoid severing the semantic connection between context and avoid the propagation of model errors, thereby improving the model accuracy. At the same time, it does not require excessive hardware and software resources for text data encoding, thus optimizing costs and enhancing practicality.

[0089] Based on the same concept, such as Figure 8As shown, this embodiment provides a text semantic recognition method that can be used in computer devices and mobile terminals, such as mobile phones and tablets. The flowchart of the text semantic recognition method according to this embodiment includes the following steps:

[0090] Step S801: Obtain the text data to be recognized and segment the text data to be recognized to obtain multiple segments of the text data to be recognized. The text data to be recognized is long text data.

[0091] Step S802: Select a subset of segments from multiple segments as the set of segments to be identified in the text data to be identified.

[0092] For details, please refer to the above process of segmenting and selecting the target text data; it will not be repeated here.

[0093] Step S803: Obtain the features of the set of segments to be identified. The features of the set of segments to be identified include: the features of each segment to be identified in the set of segments to be identified. The features of each segment to be identified include: the start position identifier of each character, the character content identifier, the character end position identifier, the non-character identifier, the text length, and the total number of target segments.

[0094] For details, please refer to the feature extraction process of the target text data mentioned above, which will not be repeated here. During the process of obtaining features from the set of segments to be identified, if a segment spans two segments, the sequences of the two segments with labeled features are collected together.

[0095] Step S804: Input the features of the set of fragments to be identified into the text semantic recognition model to obtain the text semantic recognition result corresponding to the text data to be identified.

[0096] Specifically, during training, the features of the set of segments to be identified are encoded, and then each segment is decoded using a CRF to obtain the label sequence for each segment, thereby obtaining the text semantic recognition result corresponding to the text data to be identified. For example... Figure 9 The diagram shown is a simplified flowchart of the text semantic recognition method in this embodiment of the present disclosure.

[0097] exist Figure 9Before inputting the S T vectors from each batch into the XLNet encoding network for encoding, it first determines whether each segmented vector is the first segment of the text data to be recognized (long text data). If it is, the input vectors are randomly initialized, and the S T vectors are encoded before being encoded using the XLNet encoding network. If it is not the first segment of the target text data (long text data), the last hidden vector of the previous segment is used, and then encoded using the XLNet encoding network. Then, it determines whether each encoded vector is the last segment of the text data to be recognized (long text data). If it is, it is encoded again using the XLNet encoding network and then decoded using a conditional random field network.

[0098] The text semantic recognition method in this embodiment can be applied to long text semantic recognition scenarios with context segmentation. At the same time, the use of the XLNet pre-trained model for cyclic encoding effectively handles the semantic connections of the context, thereby reducing error propagation during the model learning process and improving the accuracy of long text semantic segment recognition.

[0099] This embodiment also provides a text semantic model training device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0100] This embodiment provides a text semantic model training device, such as Figure 10 As shown, it includes:

[0101] The data acquisition module 101 is used to acquire multiple target text data and segment each target text data to obtain multiple segments of the target text data. The target text data is long text data.

[0102] The sample selection module 102 is used to select a subset of fragments from multiple fragments as the target fragment set of the target text data.

[0103] The feature acquisition module 103 is used to acquire the features of the target fragment set. The features of the target fragment set include: the features of each target fragment in the target fragment set; the features of each target fragment include: the start position identifier of each character, the character content identifier, the end position identifier of the character, the non-character identifier, the text length, and the total number of target fragments.

[0104] The feature training module 104 is used to input the features of the target fragment set into the text semantic recognition model, obtain the text semantic recognition result corresponding to the features of the target fragment set, and the loss value between the text semantic recognition result corresponding to the features of the target fragment set and the true label of the target text data. The loss value is used to update the parameter values ​​of the text semantic recognition model.

[0105] In some optional implementations, the data acquisition module 101 includes:

[0106] The first acquisition submodule is used to acquire multiple initial text data;

[0107] The text cleaning submodule is used to clean multiple initial text data to obtain multiple target text data.

[0108] The second acquisition submodule is used to obtain the maximum text length from the text lengths corresponding to multiple target text data;

[0109] The segmentation module is used to segment the target text data into segments according to a preset text length, resulting in multiple segments of the target text data. The number of segments is determined based on the maximum text length and the preset text length.

[0110] In some optional implementations, the feature acquisition module 103 includes:

[0111] The feature generation submodule is used to randomly select a target number of characters from each target segment in the target segment set. The target number is determined based on the number of characters included in the target segment and a preset selection ratio. Each character in the target number of characters is labeled to form a start position identifier, character content identifier, character end position identifier, and non-character identifier for each character.

[0112] In some alternative implementations, the feature training module 104 includes:

[0113] The feature encoding submodule is used to encode the features of the target fragment set using the feature encoding network in the text semantic recognition model, so as to obtain the feature vector of the target fragment set; the feature encoding network is the XLNet encoding network;

[0114] The feature decoding submodule is used to decode the features of the encoded target fragment set using the feature decoding network in the text semantic recognition model, and obtain the semantic recognition result corresponding to the features of the target fragment set. The feature decoding network is a conditional random field network.

[0115] In some optional implementations, the feature training module 104 updates the parameter values ​​of the text semantic recognition model through the following sub-modules:

[0116] The iterative update submodule is used to train the text semantic recognition model in a loop based on the loss value between the text semantic recognition result corresponding to the features of the target fragment set and the real label of the target text data, until the preset number of training times is reached and the training ends, so as to obtain the trained text semantic recognition model.

[0117] In some optional implementations, embodiments of this disclosure also provide a text semantic recognition device, including:

[0118] The text to be recognized acquisition module is used to acquire the text data to be recognized and to segment the text data to be recognized into multiple segments. The text data to be recognized is long text data.

[0119] The text recognition module is used to select a subset of segments from multiple fragments as the set of segments to be recognized in the text data to be recognized.

[0120] The feature acquisition module is used to acquire the features of the set of segments to be identified. The features of the set of segments to be identified include: the features of each segment to be identified in the set of segments to be identified. The features of each segment to be identified include: the start position identifier of each character, the character content identifier, the end position identifier of the character, the non-character identifier, the text length, and the total number of target segments.

[0121] The feature recognition module is used to input the features of the set of fragments to be recognized into the text semantic recognition model to obtain the text semantic recognition result corresponding to the text data to be recognized.

[0122] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0123] In this embodiment, the text semantic model training device and the text semantic recognition device are presented in the form of functional units. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0124] This invention also provides a computer device having the above-described text semantic model training apparatus and text semantic recognition method.

[0125] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 11As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 11 Take a processor 10 as an example.

[0126] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0127] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0128] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0129] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0130] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0131] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0132] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0133] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for training a text semantic model, characterized in that, The method includes: Multiple target text data are acquired, and each target text data is segmented to obtain multiple segments of the target text data, wherein the target text data is long text data; Select a subset of fragments from the plurality of fragments as the target fragment set for each target text data; The characteristics of the target fragment set are obtained, and the characteristics of the target fragment set include: the characteristics of each target fragment in the target fragment set, and the characteristics of each target fragment include: the start position identifier of each character, the character content identifier, the character end position identifier, the non-character identifier, the text length, and the total number of target fragments; The features of the target fragment set are input into the text semantic recognition model to obtain the text semantic recognition result corresponding to the features of the target fragment set, and the loss value between the text semantic recognition result corresponding to the features of the target fragment set and the true label of the target text data. The loss value is used to update the parameter values ​​of the text semantic recognition model.

2. The method according to claim 1, characterized in that, The target text data is segmented to obtain multiple segments of the target text data, including: Obtain multiple initial text data; Multiple initial text data are cleaned to obtain multiple target text data; Obtain the maximum text length from the text lengths corresponding to the plurality of target text data; The target text data is segmented according to a preset text length to obtain multiple segments of the target text data. The number of the multiple segments is determined based on the maximum text length and the preset text length.

3. The method according to claim 1, characterized in that, Also includes: For each target segment in the target segment set, a target number of characters are randomly selected from the target segment, wherein the target number is determined according to the number of characters included in the target segment and a preset selection ratio; each character in the target number of characters is marked to form a start position identifier, a character content identifier, a character end position identifier, and a non-character identifier for each character.

4. The method according to claim 1, characterized in that, The features of the target fragment set are input into a text semantic recognition model to obtain the semantic recognition results corresponding to the features of the target fragment set, including: The features of the target fragment set are encoded using a feature encoding network in a text semantic recognition model to obtain a feature vector of the target fragment set; the feature encoding network is an XLNet encoding network. The features of the encoded target fragment set are decoded using a feature decoding network in a text semantic recognition model to obtain the semantic recognition result corresponding to the features of the target fragment set. The feature decoding network is a conditional random field network.

5. The method according to claim 1, characterized in that, Update the parameter values ​​of the text semantic recognition model using the following steps: Based on the loss value between the text semantic recognition result corresponding to the features of the target fragment set and the real label of the target text data, the text semantic recognition model is trained cyclically until the preset number of training iterations is reached to end the training, so as to obtain the trained text semantic recognition model.

6. A text semantic recognition method, characterized in that, The method includes: The text data to be identified is obtained, and the text data to be identified is segmented into segments to obtain multiple segments of the text data to be identified. The text data to be identified is long text data. Select a subset of segments from the plurality of segments as the set of segments to be identified in the text data to be identified; The features of the set of segments to be identified are obtained. The features of the set of segments to be identified include: the features of each segment to be identified in the set of segments to be identified. The features of each segment to be identified include: the start position identifier of each character, the character content identifier, the character end position identifier, the non-character identifier, the text length, and the total number of target segments. The features of the set of fragments to be identified are input into the text semantic recognition model of any one of claims 1 to 5 to obtain the text semantic recognition result corresponding to the text data to be identified.

7. A text semantic model training device, characterized in that, The device includes: The data acquisition module is used to acquire multiple target text data and segment each target text data to obtain multiple segments of the target text data, wherein the target text data is long text data; The sample selection module is used to select a subset of segments from the plurality of segments as the target segment set of the target text data; The feature acquisition module is used to acquire the features of the target fragment set. The features of the target fragment set include: the features of each target fragment in the target fragment set, and the features of each target fragment include: the start position identifier of each character, the character content identifier, the end position identifier of the character, the non-character identifier, the text length, and the total number of target fragments. The feature training module is used to input the features of the target fragment set into the text semantic recognition model to obtain the text semantic recognition result corresponding to the features of the target fragment set, and the loss value between the text semantic recognition result corresponding to the features of the target fragment set and the true label of the target text data. The loss value is used to update the parameter values ​​of the text semantic recognition model.

8. A computer device, characterized in that, include: A memory and a processor are interconnected, the memory stores computer instructions, and the processor executes the computer instructions to perform the text semantic model training method of any one of claims 1 to 5, or the text semantic recognition method of claim 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the text semantic model training method of any one of claims 1 to 5, or the text semantic recognition method of claim 6.

10. A computer program product, characterized in that, It includes computer instructions for causing a computer to execute the text semantic model training method of any one of claims 1 to 5, or the text semantic recognition method of claim 6.

Images