Text semantic model training method and device and text semantic recognition method and device

Abstract

The invention relates to the technical field of natural language processing, discloses a text semantic model training method and device and a text semantic recognition method and device, and is suitable for a long text semantic recognition scene. Feature training is carried out based on the starting position identification, the character content identification, the character ending position identification, the non-character identification, the text length and the total number of the target segments of each character of each target segment in the target segment set, so that segmentation of context semantic relation can be effectively avoided, a model error transfer phenomenon is avoided, and the accuracy of the model is improved. Therefore, the model accuracy is improved, and meanwhile, too large hardware and software resources are not needed for text data coding, so that the cost is optimized, and the practicability is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a text semantic model training method, a text semantic recognition method and a device. Background Art

[0002] Long text usually refers to more than 1,000 characters. Directly encoding long text and putting it into neural network training will cause problems such as gradient vanishing and gradient explosion. At the same time, due to the length limit of the pre-trained model and the high requirements of hardware equipment for long text. Taking the above factors into consideration, long text task training is difficult. Therefore, most related technologies divide long text into pre-set lengths, such as by punctuation marks such as periods and exclamation marks that indicate the end of sentences, or directly divide into segments according to fixed lengths. This segmentation method regards the context as an independent segment, artificially splitting the context semantics. Fixed-length segmentation may lead to information loss, especially at the boundaries of text segments. If a semantic unit (such as a sentence or a paragraph) is divided into two segments, the model may find it difficult to capture the complete context. This leads to a certain deviation in the model's learning of each segment.

[0003] In addition, most of the existing semantic segment extraction solutions are aimed at short text scenarios, and the length of the extracted semantic segments is generally short. Generally speaking, the current mainstream pre-training models have better encoding effects for short texts; at the same time, the length of semantic segments is short, and the semantic segment extraction task can be simply regarded as a named entity recognition task. At present, there are many models with good effects for named entity recognition tasks, so named entity recognition models with good effects can be used, thereby reducing the difficulty of the task. However, the existing solutions for short texts and short semantic segments are not suitable for long text semantic segment extraction tasks. First, at the boundary of the text segment, the model may perform poorly due to lack of context, especially when a semantic unit spans two segments; secondly, the location of the text segmentation may affect the results. If the segmentation point is exactly in the middle of an important semantic unit, the model may not be able to understand the complete semantics. In addition, the direct encoding solution requires a lot of hardware resources to implement, resulting in extremely high costs and cannot be applied in practice. Summary of the invention

[0004] In view of this, the present invention provides a text semantic model training method, a text semantic recognition method and a device to solve the problem of being unable to accurately recognize the semantics of long texts and the high cost of hardware encoding.

[0005] According to a first aspect, an embodiment of the present disclosure provides a text semantic model training method, the method comprising:

[0006] Acquire multiple target text data, and segment each target text data into segments to obtain multiple segments of the target text data, where the target text data is long text data;

[0007] Selecting some segments from the multiple segments as a target segment set for each target text data;

[0008] Acquire the features of the target segment set, the features of the target segment set include: the features of each target segment in the target segment set, the features of the target segment include: the start position identifier of each character, the character content identifier, the character end position identifier, the non-character identifier, the text length, and the total number of target segments;

[0009] The features of the target segment set are input into the text semantic recognition model to obtain the text semantic recognition results corresponding to the features of the target segment set, as well as the loss value between the text semantic recognition results corresponding to the features of the target segment set and the true label of the target text data. The loss value is used to update the parameter value of the text semantic recognition model.

[0010] The text semantic model training method in the disclosed embodiment is suitable for long text semantic recognition scenarios. It performs feature training based on the start position identifier, character content identifier, character end position identifier, non-character identifier, text length, and total number of target segments of each character in each target segment in the target segment set. It can effectively avoid severing contextual semantic connections and avoiding model error transmission, thereby improving the model accuracy. At the same time, it does not require excessive hardware and software resources for text data encoding, thereby optimizing costs and enhancing practicality.

[0011] In some optional implementations, segmenting the target text data into segments to obtain multiple segments of the target text data includes:

[0012] Get multiple initial text data;

[0013] Cleaning a plurality of initial text data to obtain a plurality of target text data;

[0014] Obtaining a maximum text length from text lengths corresponding to a plurality of target text data;

[0015] The target text data is segmented into segments according to a preset text length to obtain multiple segments of the target text data, and the number of the multiple segments is determined based on the maximum text length and the preset text length.

[0016] The disclosed embodiment cleans and segments text data, which is not only suitable for processing text recognition in different context segments, but also helps to improve the accuracy of text semantic recognition.

[0017] In some optional embodiments, the text semantic model training method in the embodiment of the present disclosure further includes: for each target segment in the target segment set, randomly selecting a target number of characters from the target segment, wherein the target number is determined according to the number of characters included in the target segment and a preset selection ratio; and separately marking each character in the target number of characters to form a start position identifier, a character content identifier, a character end position identifier, and a non-character identifier for each character.

[0018] In some optional implementations, the features of the target segment set are input into a text semantic recognition model to obtain semantic recognition results corresponding to the features of the target segment set, including:

[0019] The feature encoding network in the text semantic recognition model is used to encode the features of the target segment set to obtain the feature vector of the target segment set; the feature encoding network is the XLNet encoding network;

[0020] The feature decoding network in the text semantic recognition model is used to decode the features of the encoded target segment set to obtain the semantic recognition results corresponding to the features of the target segment set. The feature decoding network is a conditional random field network.

[0021] The disclosed embodiment encodes and decodes the features of a target segment set through a feature encoding network, effectively processing the semantic connection of the context, thereby reducing error propagation in the model learning process and improving the accuracy of long text semantic segment extraction.

[0022] In some optional implementations, the parameter values ​​of the text semantic recognition model are updated through the following steps:

[0023] Based on the loss value between the text semantic recognition results corresponding to the features of the target fragment set and the true label of the target text data, the text semantic recognition model is trained in a loop until the preset training times are reached to obtain a trained text semantic recognition model.

[0024] The disclosed embodiment helps to improve the recognition accuracy of the text semantic recognition model by continuously iteratively updating the parameters of the text semantic recognition model.

[0025] According to a second aspect, an embodiment of the present disclosure provides a text semantic recognition method, the method comprising:

[0026] Acquire text data to be recognized, and segment the text data to be recognized to obtain multiple segments of the text data to be recognized, where the text data to be recognized is long text data;

[0027] Selecting some segments from the multiple segments as a set of segments to be recognized of the text data to be recognized;

[0028] Acquire features of the set of segments to be identified, the features of the set of segments to be identified include: features of each segment to be identified in the set of segments to be identified, the features of the segments to be identified include: a start position identifier of each character, a character content identifier, a character end position identifier, a non-character identifier, a text length, and a total number of target segments;

[0029] The features of the set of segments to be identified are input into the text semantic recognition model in the first aspect or any one of the embodiments of the first aspect to obtain a text semantic recognition result corresponding to the text data to be identified.

[0030] According to a third aspect, an embodiment of the present disclosure provides a text semantic model training device, the device comprising:

[0031] A data acquisition module is used to acquire multiple target text data and segment each target text data into segments to obtain multiple segments of the target text data, where the target text data is long text data;

[0032] A sample selection module, used for selecting some segments from the multiple segments as a target segment set of target text data;

[0033] A feature acquisition module is used to acquire features of a target segment set, wherein the features of the target segment set include: features of each target segment in the target segment set, wherein the features of the target segment include: a start position identifier of each character, a character content identifier, a character end position identifier, a non-character identifier, a text length, and a total number of target segments;

[0034] The feature training module is used to input the features of the target segment set into the text semantic recognition model to obtain the text semantic recognition results corresponding to the features of the target segment set, as well as the loss value between the text semantic recognition results corresponding to the features of the target segment set and the true label of the target text data. The loss value is used to update the parameter value of the text semantic recognition model.

[0035] According to a fourth aspect, an embodiment of the present disclosure provides a computer device, including:

[0036] The memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the text semantic model training method in the first aspect or any embodiment of the first aspect, or the text semantic recognition method in the second aspect by executing the computer instructions.

[0037] According to the fifth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute the text semantic model training method in the first aspect or any embodiment of the first aspect, or the text semantic recognition method in the second aspect.

[0038] In a sixth aspect, an embodiment of the present disclosure provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the text semantic model training method in the first aspect or any implementation of the first aspect, or the text semantic recognition method in the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 is a flowchart of a text semantic model training method according to an embodiment of the present invention;

[0041] Figure 2 is a schematic diagram of dividing text segments according to text length according to an embodiment of the present invention;

[0042] Figure 3 is a schematic diagram of segmenting long text data according to an embodiment of the present invention;

[0043] Figure 4 is a schematic diagram of dividing a plurality of segments based on a maximum text length and a preset text length according to an embodiment of the present invention;

[0044] Figure 5 is a schematic diagram of marking features of each character in a target number of characters according to an embodiment of the present invention;

[0045] Figure 6 is a schematic diagram of the encoding and decoding process according to an embodiment of the present invention;

[0046] Figure 7 It is a simplified flowchart of a text semantic model training method according to an embodiment of the present invention;

[0047] Figure 8 A schematic diagram of a flow chart of a text semantic recognition method according to an embodiment of the present invention;

[0048] Fig. 9 It is a simplified flowchart of the text semantic recognition method according to an embodiment of the present invention;

[0049] Fig.10 is a structural block diagram of a text semantic model training device according to an embodiment of the present invention;

[0050] Fig.11 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0052] According to an embodiment of the present invention, an embodiment of a text semantic model training method is provided, which can be applied in scenarios such as information retrieval or text analysis or paragraph element recognition that require a high degree of accuracy, such as relatively rigorous vertical fields such as medical and legal fields. In these vertical fields, the extraction of information needs to ensure accuracy, and the existing text training methods for short text scenarios are not suitable for long text semantic recognition scenarios that require high accuracy. Therefore, an embodiment of the present disclosure provides a text semantic model training method that can be used for long text scenarios. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0053] In this embodiment, a text semantic model training method is provided, which can be used in computer devices, mobile terminals, such as mobile phones, tablet computers, etc. Figure 1 is a flowchart of a text semantic model training method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0054] Step S101, obtaining a plurality of target text data, and segmenting each target text data into segments to obtain a plurality of segments of the target text data, wherein the target text data is long text data.

[0055] Step S102 : selecting some segments from the multiple segments as a target segment set for each target text data.

[0056] Specifically, prepare multiple target text data for training and complete data cleaning. The multiple target text data are sample batches of different lengths. The number of samples in each batch is a fixed batch_size. Each time a batch is passed in, the dimension is [batch_size, dmax]. For each batch, Figure 2As shown, the text segments are segmented according to the text length, that is, the vector with the dimension [batch_size, dmax] is segmented into S segments with the dimension [batch_size, L]. A preset number of segments (corresponding to the partial segments mentioned above) are selected from the multiple segments after segmentation to form a target segment set for each target text data.

[0057] The fragment in the embodiments of the present disclosure refers to a small part of a natural language text with a specific meaning or context, usually a phrase, sentence or fragment in the text, with an irregular length, containing specific semantic information, which can be used to understand the meaning of the text or express a specific concept. In natural language processing, the purpose of extracting fragments is to capture the important semantic content in the text for further semantic analysis, information retrieval, question-answering systems and other tasks. Existing solutions currently perform poorly in long texts, so it is of great significance to improve the accuracy of training semantic fragments of long texts.

[0058] In some optional implementations, segmenting the target text data into segments to obtain multiple segments of the target text data includes:

[0059] Step a1, obtaining a plurality of initial text data.

[0060] Step a2, cleaning the multiple initial text data to obtain multiple target text data.

[0061] Step a3, obtaining the maximum text length from the text lengths corresponding to the multiple target text data.

[0062] Step a4, segmenting the target text data into segments according to a preset text length to obtain multiple segments of the target text data, wherein the number of the multiple segments is determined based on the maximum text length and the preset text length.

[0063] Specifically, prepare multiple initial text data for training, and obtain multiple target text data after cleaning, and count the maximum length d among these N data max ,Since long text exceeds the length that can be encoded, it needs to be divided into multiple segments, so the maximum length of the data needs to be counted, such as Figure 3 The figure shows a schematic diagram of segmenting long text data, where a text segment is segmented into multiple segments according to the preset text length. The preset text length of each segment is set to L, such as L is 256 or 512. The number of segments S is d max / L rounds up to the integer value. Figure 4 As shown, for example, L is 256, when d max When d is 512, S is 2, which means the long text is divided into two sections. max When it is 513, S is 3, and the part of the last segment that is less than 256 can be filled with padding bits.

[0064] In the disclosed embodiment, the preset text length can be flexibly set, and a different length L can be selected for training in each training process.

[0065] Step S103, obtaining features of the target segment set, the features of the target segment set include: features of each target segment in the target segment set, the features of the target segment include: the start position identifier of each character, the character content identifier, the character end position identifier, the non-character identifier, the text length, and the total number of target segments.

[0066] As an example, for each target segment in the target segment set, a target number of characters are randomly selected from the target segment, where the target number is determined based on the number of characters included in the target segment and a preset selection ratio; each character in the target number of characters is labeled separately to form a start position identifier, a character content identifier, a character end position identifier, and a non-character identifier for each character.

[0067] Specifically, for each target segment in the target segment set, the annotation space is set to BIEO. BIEO is a common sequence annotation scheme, where B (Begin) represents the beginning of an entity (i.e., corresponding to the start position identifier of each character in the above); I (Inside) represents other words inside the entity (i.e., corresponding to the character content identifier in the above), following a label starting with B; E (End) represents the end of the entity (i.e., corresponding to the character end position identifier in the above), usually the last word of the entity; O (Outside) represents a word that is not any entity (i.e., corresponding to the non-character identifier in the above). In this task, semantic segments of different lengths are regarded as entities of different lengths, and the start and end positions of the semantic segments are annotated with BIEO tags, thereby extracting the features of the semantic segments in the text. As Figure 5 As shown, for example, for each target segment in the target segment set, there are 0-11 characters in total, and a target number of characters are randomly selected from the target segment. The target number is determined according to the number of characters included in the target segment and a preset selection ratio. For example, Figure 5 The characters at positions 1-5 and 8-10 are selected and annotated, so as to obtain the target segment features with labels B, I, E, and O. Therefore, the two texts at positions 1-5 and 8-10 can be extracted as semantic segments in this text. In this way, a target segment data set D with annotated information can be obtained, with a dimension of N×d, where the target segment data set D contains N segment data, and d represents the length of each target segment data.

[0068] Furthermore, the characteristics of the target segment also include the text length and the total number of target segments. The multiple target text data are a number of sample batches of different lengths. The number of samples in each batch is a fixed batch_size. A batch is passed in each time, and the dimension is [batch_size, dmax]. For each batch, Figure 2 As shown, the text segments are segmented in the dimension of text length, that is, the vector with dimension [batch_size, dmax] is segmented into S segments with dimension [batch_size, L]. A preset number of segments are selected from the multiple segments after segmentation to form a target segment set for each target text data. The target segment set for each target text data is labeled with B, I, E, and O labels after feature annotation.

[0069] Step S104, input the features of the target segment set into the text semantic recognition model to obtain the text semantic recognition results corresponding to the features of the target segment set, and the loss value between the text semantic recognition results corresponding to the features of the target segment set and the true label of the target text data. The loss value is used to update the parameter value of the text semantic recognition model.

[0070] In some optional implementations, the features of the target segment set are input into a text semantic recognition model to obtain semantic recognition results corresponding to the features of the target segment set, including:

[0071] Step b1, using the feature encoding network in the text semantic recognition model to encode the features of the target segment set to obtain the feature vector of the target segment set; the feature encoding network is the XLNet encoding network.

[0072] Step b2, using the feature decoding network in the text semantic recognition model to decode the features of the encoded target segment set to obtain the semantic recognition results corresponding to the features of the target segment set, the feature decoding network is a conditional random field network.

[0073] Specifically, on the basis of the above step S103, the features of each target segment set are put into the XLNet encoding network for encoding. Figure 2 In the example, the input vector of the first vector T of each batch with a preset text length of L is randomly initialized, and the input vectors of the second to S vectors T are the last output of the previous stage. In this way, contextual semantic continuity is achieved, so that the semantic fragments between the upper and lower segments can be extracted more accurately.

[0074] like Figure 6As shown in the figure, each batch of S T vectors is converted into S vectors of dimension [batch_size, L, emb] after being encoded by the XLNet encoding network, where emb represents the dimension of the numerical representation of the character after being encoded by the XLNet feature encoding network. Each segment is decoded using the Conditional Random Field (CRF) network in the order of the S text segments. Figure 6 In , the decoding space is the BIEO space set above, and the relationship between labels is constrained by using joint probability. Finally, the BIEO result can be obtained, and the continuous BIE labels are collected as the extracted semantic fragments. The recognition result is compared with the real label, and the cross entropy is used as the loss function.

[0075] As an example, update the parameter values ​​of the text semantic recognition model through the following steps:

[0076] Based on the loss value between the text semantic recognition results corresponding to the features of the target fragment set and the true label of the target text data, the text semantic recognition model is trained in a loop until the preset training times are reached to obtain a trained text semantic recognition model.

[0077] Specifically, the above steps S101 to S104 are repeatedly executed, and multiple epochs (corresponding to the preset number of training times mentioned above) are iterated until the last epoch ends or the loss function does not converge for multiple consecutive epochs and the training is stopped directly. The number of consecutive epochs here is a hyperparameter, and the general value is 5, 10, 20, etc. After the training is completed, the test set is used to evaluate the model performance, including precision, recall, and F1-score.

[0078] In some optional implementations, the text semantic model training method in the embodiment of the present disclosure further includes:

[0079] Step c1, selecting some segments from the multiple segments as training sets, and selecting some segments from the multiple segments as test sets.

[0080] Step c2, extracting features of each target segment from the training set and extracting features of each target segment from the training set;

[0081] Step c3, input the features of each target segment in the test set and the features of each target segment in the training set into the text semantic recognition model.

[0082] Step c4, obtaining text semantic recognition results corresponding to the features of the test set.

[0083] Step c5, comparing the text semantic recognition results corresponding to the features of the test set with the text semantic recognition results corresponding to the features of the training set to obtain the accuracy of the text semantic recognition model.

[0084] Step c6, adjusting the parameters of the text semantic recognition model until the accuracy of the text semantic recognition model meets the requirements, thereby obtaining a trained text semantic recognition model.

[0085] Specifically, the original data set D is randomly sampled and split according to the ratio of 7:2:1. 70% of the data is used as a training set to train the Mubo text training recognition model; 20% of the data is used as a validation set to verify the effect of the current model after each epoch of training. Epoch represents a training cycle; 10% of the data is used as a test set to verify the effect of the final model.

[0086] like Figure 7 The figure is a simplified flowchart of the text semantic model training method in the embodiment of the present disclosure.

[0087] exist Figure 7 In the process, before inputting the S T vectors of each batch into the XLNet encoding network for encoding, first determine whether each vector after segmentation is the first segment of the target text data (long text data). If it is the first segment of the target text data (long text data), randomly initialize the input vector and encode the S T vectors, then use the XLNet encoding network for encoding. If it is not the first segment of the target text data (long text data), use the last hidden vector of the previous segment and then use the XLNet encoding network for encoding. Then determine whether each vector after encoding is the last segment of the target text data (long text data). If it is the last segment of the target text data (long text data), encode it using the XLNet encoding network and then decode it through the conditional random field network.

[0088] In summary, the text semantic model training method in the embodiment of the present invention is suitable for long text semantic recognition scenarios. It performs feature training based on the start position identifier, character content identifier, character end position identifier, non-character identifier, text length, and total number of target segments of each character in each target segment in the target segment set. It can effectively avoid severing contextual semantic connections and avoiding model error transmission, thereby improving the model accuracy. At the same time, it does not require excessive hardware and software resources for text data encoding, thereby optimizing costs and enhancing practicality.

[0089] Based on the same concept, Figure 8As shown, in this embodiment, a text semantic recognition method is provided, which can be used in computer devices, mobile terminals, such as mobile phones, tablet computers, etc. According to the flowchart of the text semantic recognition method of an embodiment of the present invention, the process includes the following steps:

[0090] Step S801, obtaining text data to be recognized, and segmenting the text data to be recognized into segments to obtain multiple segments of the text data to be recognized, where the text data to be recognized is long text data.

[0091] Step S802: Select some segments from the multiple segments as a set of segments to be recognized of the text data to be recognized.

[0092] Please refer to the above-mentioned target text data segmentation and selection process for details, which will not be repeated here.

[0093] Step S803, obtaining features of the set of fragments to be identified, the features of the set of fragments to be identified include: features of each fragment to be identified in the set of fragments to be identified, the features of the fragments to be identified include: the start position identifier of each character, the character content identifier, the character end position identifier, the non-character identifier, the text length, and the total number of target fragments.

[0094] For details, please refer to the feature extraction process of the target text data above, which will not be described here. In the process of obtaining the features of the set of segments to be identified, if a cross-segment situation is encountered, the sequences with label features of the two segments are collected.

[0095] Step S804: input the features of the to-be-recognized segment set into a text semantic recognition model to obtain a text semantic recognition result corresponding to the to-be-recognized text data.

[0096] Specifically, during the training process, the features of the set of segments to be identified are encoded, and then each segment is decoded through CRF to obtain the label sequence of each segment, and then the text semantic recognition result corresponding to the text data to be identified is obtained. Fig. 9 , which is a simplified flowchart of the text semantic recognition method in an embodiment of the present disclosure.

[0097] exist Fig. 9In the process, before inputting the S T vectors of each batch into the XLNet encoding network for encoding, first determine whether each vector after segmentation is the first segment of the text data (long text data) to be recognized. If it is the first segment of the text data (long text data) to be recognized, randomly initialize the input vector and encode the S T vectors, then use the XLNet encoding network for encoding. If it is not the first segment of the target text data (long text data), use the last hidden vector of the previous segment and then use the XLNet encoding network for encoding. Then determine whether each vector after encoding is the last segment of the text data (long text data) to be recognized. If it is the last segment of the text data (long text data) to be recognized, then use the XLNet encoding network for encoding and then decode it through the conditional random field network.

[0098] The text semantic recognition method in the embodiment of the present disclosure can be applied to the semantic recognition scenario of long text with context segmentation. At the same time, the XLNet pre-trained model is used for cyclic encoding to effectively process the semantic connection of the context, thereby reducing error propagation in the model learning process and improving the accuracy of long text semantic segment recognition.

[0099] In the present embodiment, a text semantic model training device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0100] This embodiment provides a text semantic model training device, such as Fig.10 As shown, including:

[0101] The data acquisition module 101 is used to acquire a plurality of target text data and segment each target text data into segments to obtain a plurality of segments of the target text data, where the target text data is long text data;

[0102] A sample selection module 102, configured to select some segments from the plurality of segments as a target segment set of target text data;

[0103] The feature acquisition module 103 is used to acquire the features of the target segment set, the features of the target segment set include: the features of each target segment in the target segment set, the features of the target segment include: the start position identifier of each character, the character content identifier, the character end position identifier, the non-character identifier, the text length, and the total number of target segments;

[0104] The feature training module 104 is used to input the features of the target segment set into the text semantic recognition model to obtain the text semantic recognition results corresponding to the features of the target segment set, and the loss value between the text semantic recognition results corresponding to the features of the target segment set and the true label of the target text data. The loss value is used to update the parameter value of the text semantic recognition model.

[0105] In some optional implementations, the data acquisition module 101 includes:

[0106] A first acquisition submodule is used to acquire a plurality of initial text data;

[0107] A text cleaning submodule is used to clean a plurality of initial text data to obtain a plurality of target text data;

[0108] A second acquisition submodule is used to acquire a maximum text length from text lengths corresponding to a plurality of target text data;

[0109] The segmentation submodule is used to segment the target text data into segments according to a preset text length to obtain multiple segments of the target text data, and the number of the multiple segments is determined based on the maximum text length and the preset text length.

[0110] In some optional implementations, the feature acquisition module 103 includes:

[0111] The feature generation submodule is used to randomly select a target number of characters from each target segment in the target segment set, wherein the target number is determined according to the number of characters included in the target segment and a preset selection ratio; and mark each character in the target number of characters respectively, thereby forming a start position identifier, a character content identifier, a character end position identifier, and a non-character identifier for each character.

[0112] In some optional implementations, the feature training module 104 includes:

[0113] The feature encoding submodule is used to encode the features of the target segment set using the feature encoding network in the text semantic recognition model to obtain the feature vector of the target segment set; the feature encoding network is the XLNet encoding network;

[0114] The feature decoding submodule is used to decode the features of the encoded target segment set using the feature decoding network in the text semantic recognition model to obtain the semantic recognition results corresponding to the features of the target segment set. The feature decoding network is a conditional random field network.

[0115] In some optional implementations, the feature training module 104 updates the parameter values ​​of the text semantic recognition model through the following submodules:

[0116] The iterative update submodule is used to cyclically train the text semantic recognition model based on the loss value between the text semantic recognition results corresponding to the features of the target fragment set and the true label of the target text data until the training is terminated after a preset number of training times, so as to obtain a trained text semantic recognition model.

[0117] In some optional implementations, the present disclosure further provides a text semantic recognition device, including:

[0118] The module for acquiring text to be recognized is used to acquire text data to be recognized and segment the text data to be recognized to obtain multiple segments of the text data to be recognized. The text data to be recognized is long text data.

[0119] The to-be-recognized text recognition module is used to select some segments from the multiple segments as a to-be-recognized segment set of the to-be-recognized text data.

[0120] The module for acquiring features to be identified is used to acquire features of a set of fragments to be identified. The features of the set of fragments to be identified include: features of each fragment to be identified in the set of fragments to be identified. The features of the fragments to be identified include: the start position identifier of each character, the character content identifier, the character end position identifier, the non-character identifier, the text length, and the total number of target fragments.

[0121] The feature recognition module to be recognized is used to input the features of the set of fragments to be recognized into the text semantic recognition model to obtain the text semantic recognition results corresponding to the text data to be recognized.

[0122] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0123] The text semantic model training device and text semantic recognition device in this embodiment are presented in the form of functional units, where the units refer to ASIC (Application Specific Integrated Circuit) circuits, processors and memories that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0124] An embodiment of the present invention also provides a computer device having the above-mentioned text semantic model training device and text semantic recognition method.

[0125] See also Fig.11 , Fig.11 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig.11As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.11 A processor 10 is taken as an example.

[0126] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0127] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0128] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0129] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0130] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0131] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0132] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0133] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A text semantic model training method, characterized in that: The method comprises: Acquire multiple target text data, and segment each target text data into segments to obtain multiple segments of the target text data, wherein the target text data is long text data; Selecting some segments from the multiple segments as a target segment set for each target text data; Acquire the features of the target segment set, wherein the features of the target segment set include: the features of each target segment in the target segment set, wherein the features of the target segment include: the start position identifier of each character, the character content identifier, the character end position identifier, the non-character identifier, the text length, and the total number of target segments; The features of the target segment set are input into a text semantic recognition model to obtain text semantic recognition results corresponding to the features of the target segment set, as well as loss values ​​between the text semantic recognition results corresponding to the features of the target segment set and the true labels of the target text data, wherein the loss values ​​are used to update parameter values ​​of the text semantic recognition model.

2. The method according to claim 1, characterized in that The target text data is segmented into segments to obtain multiple segments of the target text data, including: Obtain multiple initial text data; Cleaning a plurality of initial text data to obtain a plurality of target text data; Obtaining a maximum text length from the text lengths corresponding to the plurality of target text data; The target text data is segmented into segments according to a preset text length to obtain a plurality of segments of the target text data, wherein the number of the plurality of segments is determined based on the maximum text length and the preset text length.

3. The method according to claim 1, characterized in that Also includes: For each target segment in the target segment set, a target number of characters are randomly selected from the target segment, wherein the target number is determined according to the number of characters included in the target segment and a preset selection ratio; each character in the target number of characters is marked respectively, thereby forming a start position identifier, a character content identifier, a character end position identifier, and a non-character identifier for each character.

4. The method according to claim 1, characterized in that: Inputting the features of the target segment set into a text semantic recognition model to obtain semantic recognition results corresponding to the features of the target segment set, including: Encoding the features of the target segment set by using a feature encoding network in a text semantic recognition model to obtain a feature vector of the target segment set; the feature encoding network is an XLNet encoding network; The features of the encoded target segment set are decoded using a feature decoding network in a text semantic recognition model to obtain semantic recognition results corresponding to the features of the target segment set, wherein the feature decoding network is a conditional random field network.

5. The method according to claim 1, characterized in that The parameter values ​​of the text semantic recognition model are updated through the following steps: Based on the loss value between the text semantic recognition results corresponding to the features of the target segment set and the true label of the target text data, the text semantic recognition model is trained in a loop until the training is terminated after a preset number of training times, so as to obtain the trained text semantic recognition model.

6. A text semantic recognition method, characterized in that: The method comprises: Acquire text data to be recognized, and segment the text data to be recognized to obtain multiple segments of the text data to be recognized, wherein the text data to be recognized is long text data; Selecting some segments from the multiple segments as a set of segments to be recognized of the text data to be recognized; Acquire features of the set of segments to be identified, wherein the features of the set of segments to be identified include: features of each segment to be identified in the set of segments to be identified, wherein the features of the segment to be identified include: a start position identifier of each character, a character content identifier, a character end position identifier, a non-character identifier, a text length, and a total number of target segments; The features of the set of fragments to be identified are input into the text semantic recognition model of any one of claims 1 to 5 to obtain a text semantic recognition result corresponding to the text data to be identified.

7. A text semantic model training device, characterized in that: The device comprises: A data acquisition module is used to acquire a plurality of target text data and segment each target text data into segments to obtain a plurality of segments of the target text data, wherein the target text data is long text data; A sample selection module, used for selecting some segments from the multiple segments as a target segment set of the target text data; A feature acquisition module, used to acquire features of the target segment set, wherein the features of the target segment set include: features of each target segment in the target segment set, wherein the features of the target segment include: a start position identifier of each character, a character content identifier, a character end position identifier, a non-character identifier, a text length, and a total number of target segments; A feature training module is used to input the features of the target segment set into a text semantic recognition model to obtain text semantic recognition results corresponding to the features of the target segment set, and loss values ​​between the text semantic recognition results corresponding to the features of the target segment set and the true labels of the target text data, wherein the loss values ​​are used to update the parameter values ​​of the text semantic recognition model.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the text semantic model training method described in any one of claims 1 to 5, or the text semantic recognition method described in claim 6, by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the text semantic model training method described in any one of claims 1 to 5, or the text semantic recognition method described in claim 6.

10. A computer program product, characterized in that It includes computer instructions, which are used to enable a computer to execute the text semantic model training method described in any one of claims 1 to 5, or the text semantic recognition method described in claim 6.

Images