Model training method, contract text classification method and device

Abstract

The application relates to the technical field of image processing, and provides a model training method and a contract text classification method and device. The model training method comprises the following steps: pre-training a text extraction model and an image extraction model of a feature extraction model according to each contract training sample, until a loss function of the feature extraction model converges, and a target feature extraction model is obtained; training the target feature extraction model according to a contract training sample set obtained after data enhancement of each contract training sample, and obtaining a trained target feature extraction model. The model training method provided in the application can more comprehensively extract contract features of a contract text, and improves the classification efficiency of the contract text.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a model training method, a method and apparatus for classifying contract texts. Background Technology

[0002] With economic development, more and more users are choosing financial services provided by banks. These financial services require the signing of relevant contracts, resulting in a large accumulation of such contracts for banks. To facilitate contract management, feature extraction can be performed on the contract texts to classify and archive them based on these extracted features. However, how to effectively extract contract features to improve the efficiency of contract classification is a pressing issue that needs to be addressed. Summary of the Invention

[0003] This application aims to address at least one of the technical problems existing in the related art. To this end, this application proposes a model training method, a contract text classification method, and an apparatus, which can more comprehensively extract the contract features of contract texts and improve the classification efficiency of contract texts.

[0004] The model training method according to the first aspect of this application includes:

[0005] Based on the training samples of each contract, the text extraction model and image extraction model of the feature extraction model are pre-trained until the loss function of the feature extraction model converges, thus obtaining the target feature extraction model.

[0006] The target feature extraction model is trained using the contract training sample set obtained after data augmentation of each contract training sample, to obtain a trained target feature extraction model.

[0007] The loss function of the feature extraction model is determined based on the cosine similarity between the text features obtained by inputting the contract training sample into the text extraction model and the image features obtained by inputting the contract training sample into the image extraction model, as well as the contract category to which the contract training sample belongs.

[0008] The trained target feature extraction model is used to extract contract features from the contract text.

[0009] By pre-training the text extraction model and image extraction model of the feature extraction model using various contract training samples, the target feature extraction model is obtained. Then, based on the contract training sample set obtained after data augmentation of each contract training sample, the target feature extraction model is trained to obtain a well-trained target feature extraction model. This well-trained target feature extraction model can extract image features, including text features such as keywords or text sentences, from the contract text, making the contract features extracted from the contract text more comprehensive and thus improving the classification efficiency of the contract text.

[0010] According to one embodiment of this application, based on training samples from various contracts, the text extraction model and image extraction model of the feature extraction model are pre-trained until the loss function of the feature extraction model converges, thereby obtaining the target feature extraction model, including:

[0011] Based on the training samples of each contract, the text extraction model and the image extraction model of the feature extraction model are trained until the loss function of the feature extraction model converges. The text extraction model is then discarded, and the pre-trained image extraction model is determined as the target feature extraction model.

[0012] According to one embodiment of this application, the target feature extraction model is trained using a contract training sample set obtained after data augmentation of each of the contract training samples, to obtain a trained target feature extraction model, including:

[0013] By swapping the image regions of any two contract training samples belonging to the same contract category, multiple modified training samples can be obtained.

[0014] The target feature extraction model is trained based on the contract training sample set composed of the contract training samples and the modified training samples, to obtain the trained target feature extraction model.

[0015] According to one embodiment of this application, the image regions of the two contract training samples that are being swapped do not intersect in the same coordinate system.

[0016] According to one embodiment of this application, the target feature extraction model is trained based on a contract training sample set composed of the contract training samples and the modified training samples to obtain a trained target feature extraction model, including:

[0017] The modified training samples are input into the target feature extraction model to obtain the sample image features output by the target feature extraction model; and,

[0018] Input the contract training sample corresponding to the modified training sample into the image feature extraction model to obtain the target image features output by the image feature extraction model;

[0019] Based on the similarity between the features of the sample image and the features of the target image, the target feature extraction model is adjusted until the loss function of the target feature extraction model converges, thus obtaining a trained target feature extraction model.

[0020] The image feature extraction model is a teacher model of the target feature extraction model, and the number of parameters in the image feature extraction model is greater than that in the target feature extraction model.

[0021] The method for classifying contract texts according to the second aspect of this application includes:

[0022] The contract text is input into the trained target feature extraction model to obtain the contract features of the contract text;

[0023] Based on the contract characteristics of the contract text and the comparison results with a preset feature set of any contract category, the contract category to which the contract text belongs is determined;

[0024] The target feature extraction model is trained according to the model training method described in any of the above embodiments;

[0025] A preset feature set for any of the contract categories is obtained by inputting training samples of each contract belonging to the contract category into a trained target feature extraction model.

[0026] According to one embodiment of this application, determining the contract category to which the contract text belongs based on a comparison of the contract features of the contract text with a preset feature set of any contract category includes:

[0027] If the contract characteristics of the contract text do not match the preset feature sets of each contract category, a new contract category is added based on the contract characteristics.

[0028] By inputting the contract text into a pre-trained target feature extraction model, the contract features of the contract text are obtained. Based on the comparison results of the contract features of the contract text with the preset feature set of any contract category, the contract category to which the contract text belongs is determined. Thus, the pre-trained target feature extraction model can extract image features, including text features such as keywords or text sentences, from the contract text as contract features, making the extracted contract features more comprehensive and thereby improving the classification efficiency of contract text.

[0029] The model training apparatus according to a third aspect embodiment of this application includes:

[0030] The first training module is used to pre-train the text extraction model and image extraction model of the feature extraction model based on the training samples of each contract until the loss function of the feature extraction model converges, thereby obtaining the target feature extraction model.

[0031] The second training module is used to train the target feature extraction model based on the contract training sample set obtained after data augmentation of each contract training sample, so as to obtain the trained target feature extraction model.

[0032] The loss function of the feature extraction model is determined based on the cosine similarity between the text features obtained by inputting the contract training sample into the text extraction model and the image features obtained by inputting the contract training sample into the image extraction model, as well as the contract category to which the contract training sample belongs.

[0033] The trained target feature extraction model is used to extract contract features from the contract text.

[0034] A contract text classification apparatus according to an embodiment of the fourth aspect of this application includes:

[0035] The feature extraction module is used to input the contract text into the trained target feature extraction model to obtain the contract features of the contract text;

[0036] The contract classification module is used to determine the contract category to which the contract text belongs based on the comparison results between the contract characteristics of the contract text and a preset feature set of any contract category.

[0037] The target feature extraction model is trained according to the model training method described in any of the above embodiments;

[0038] A preset feature set for any of the contract categories is obtained by inputting training samples of each contract belonging to the contract category into a trained target feature extraction model.

[0039] An electronic device according to a fifth aspect of this application includes a processor and a memory storing a computer program, wherein the processor executes the computer program to implement the model training method or the contract text classification method described in any of the above embodiments.

[0040] A computer-readable storage medium according to a sixth aspect of this application stores a computer program thereon, which, when executed by a processor, implements the model training method or the contract text classification method described in any of the above embodiments. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram of the first process of the model training method provided in the embodiments of this application;

[0043] Figure 2 This is a schematic diagram of the second process of the model training method provided in the embodiments of this application;

[0044] Figure 3 This is a schematic diagram of the third process of the model training method provided in the embodiments of this application;

[0045] Figure 4 This is a first flowchart illustrating the contract text classification method provided in this application embodiment;

[0046] Figure 5 This is a schematic diagram of the structure of the model training device provided in the embodiments of this application;

[0047] Figure 6 This is a schematic diagram of the structure of the contract text classification device provided in the embodiments of this application;

[0048] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0049] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0051] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0052] In the description of the embodiments of this application, the terms "multiple" or "each" refer to two or more (including two).

[0053] The model training method and contract text classification method disclosed in this application can be applied to a server to extract contract features from the contract text. The server can be a standalone server or a server cluster consisting of multiple servers. It can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence sampling point devices.

[0054] A model training method is provided according to some embodiments of this application, which can be applied to the aforementioned server. For example... Figure 1 As shown, the model training method includes:

[0055] S101, based on the training samples of each contract, pre-train the text extraction model and image extraction model of the feature extraction model until the loss function of the feature extraction model converges, and obtain the target feature extraction model;

[0056] S102, the target feature extraction model is trained using the contract training sample set obtained after data augmentation of each contract training sample to obtain a trained target feature extraction model.

[0057] The loss function of the feature extraction model is determined based on the cosine similarity between the text features obtained by inputting the contract training sample into the text extraction model and the image features obtained by inputting the contract training sample into the image extraction model, as well as the contract category to which the contract training sample belongs.

[0058] The trained target feature extraction model is used to extract contract features from the contract text.

[0059] In some embodiments, the feature extraction model can be a CLIP (Contrastive Language-Image Pre-Training) model, which includes a text extraction model and an image extraction model. Both the text extraction model and the image extraction model can be models built based on neural networks. To improve the accuracy of features extracted by the image extraction model, a mapping layer constructed from fully connected layers, pooling layers, and activation layers can be added to the output layer of the image extraction model. The contract training samples can be any contract text. Since there are too many contract training samples for model training, only a portion of the contract training samples can be pre-labeled with contract categories to obtain a portion of the labeled contract training samples for feature extraction model training.

[0060] After collecting the training samples of each contract, the samples labeled with contract categories can be preprocessed, such as by clearing and filtering, to ensure high quality and accuracy. After preprocessing, the preprocessed samples are input into the text extraction and image extraction models of the feature extraction model. The text extraction model extracts the text features of the sample, and the image extraction model extracts the image features. The cosine similarity between the text and image features is calculated. Based on the cosine similarity and the contract category, the loss function CELouss is calculated. Gradient descent is then used to adjust the network parameters of the text and image extraction models, such as through backpropagation. New training samples are then input for the next training iteration until the loss function of the feature extraction model converges, indicating that the pre-training of the feature extraction model is complete, thus obtaining the target feature extraction model. Text features include keywords or text phrases.

[0061] The learning rate of the feature extraction model can be set to 0.001 during pre-training.

[0062] After obtaining the target feature extraction model, the model can be retrained using a set of data-augmented contract training samples obtained from the original contract training samples. Data augmentation of the contract training samples can involve modifying them according to color, size, or certain local regions to obtain modified training samples. For example, two contract training samples belonging to the same contract category can be mixed proportionally to obtain modified training samples; alternatively, a portion of a contract training sample can be randomly cropped and filled with zero pixels. After obtaining the modified training samples, the original contract training samples and the data-augmented modified training samples can be combined to form the contract training sample set.

[0063] After obtaining the contract training sample set, each training sample in the set can be input into the target feature extraction model for training. Each training iteration involves checking whether the output contract features match the pre-labeled features of the training samples. If they do not match, a gradient descent algorithm is used to adjust the network parameters of the target feature extraction model, such as through error backpropagation. New training samples are then input for the next training iteration until the loss function of the target feature extraction model converges. This means that the contract features (including image features) obtained from any training sample entering the target feature extraction model match the pre-labeled features of that training sample. This completes the training of the target feature extraction model, resulting in a well-trained model.

[0064] By pre-training the text extraction model and image extraction model of the feature extraction model using various contract training samples, the target feature extraction model is obtained. Then, based on the contract training sample set obtained after data augmentation of each contract training sample, the target feature extraction model is trained to obtain a well-trained target feature extraction model. This well-trained target feature extraction model can extract image features, including text features such as keywords or text sentences, from the contract text, making the contract features extracted from the contract text more comprehensive and thus improving the classification efficiency of the contract text.

[0065] To improve the training efficiency of the target feature extraction model, in some embodiments, the text extraction model and image extraction model of the feature extraction model are pre-trained based on each contract training sample until the loss function of the feature extraction model converges, thereby obtaining the target feature extraction model, including:

[0066] Based on the training samples of each contract, the text extraction model and the image extraction model of the feature extraction model are trained until the loss function of the feature extraction model converges. The text extraction model is then discarded, and the pre-trained image extraction model is determined as the target feature extraction model.

[0067] In some embodiments, after pre-training the feature extraction model using training samples of each contract, the image features extracted by the image extraction model already include the text features extracted by the text extraction model. That is, the image extraction model simultaneously possesses the ability to extract text features. Therefore, after pre-training, the text extraction model can be removed from the feature extraction model, and the pre-trained image extraction model can be retained as the target feature extraction model. This eliminates the need to train the text extraction model during subsequent secondary training, saving training time and improving training efficiency. Furthermore, since the pre-trained image extraction model already possesses the ability to extract text features, removing the text extraction model has minimal impact on the accuracy of feature extraction. In addition, when subsequently using the trained target feature extraction model to extract contract features from the contract text, only the image extraction model needs to be used for contract feature extraction, improving the efficiency of contract feature extraction and thus increasing the efficiency of contract classification using contract features.

[0068] To further improve the accuracy of contract feature extraction, in some embodiments, such as Figure 2 As shown, the target feature extraction model is trained using the contract training sample set obtained after data augmentation of each contract training sample, resulting in a trained target feature extraction model, including:

[0069] S201, swap the image regions of any two contract training samples belonging to the same contract category in the contract training samples to obtain multiple modified training samples;

[0070] S202, the target feature extraction model is trained based on the contract training sample set composed of each contract training sample and each modified training sample to obtain the trained target feature extraction model.

[0071] In some embodiments, the training samples can first be paired based on the contract category to which each training sample belongs. Assuming the training samples in any pair are sample x and sample y, an image region a1 can be randomly selected from sample x, and an image region a2 can be randomly selected from sample y. The region a1 from sample x is then pasted onto sample y, making the coordinates of a1 in sample y the same as the coordinates of a1 in sample x, thus obtaining the modified sample, i.e., the modified training sample x' corresponding to sample x. Similarly, the region a2 from sample y is pasted onto sample x, making the coordinates of a2 in sample x the same as the coordinates of a2 in sample y, thus obtaining the modified training sample y' corresponding to sample y.

[0072] Alternatively, a region a1 can be randomly selected from sample x, and a region a2 of the same size as a1 can be randomly selected from sample y. The region a1 of sample x can be pasted onto the region a2 of sample y to obtain the modified training sample x' corresponding to sample x. The region a2 of sample y can be pasted onto the region a1 of sample y to obtain the modified training sample y' corresponding to sample y.

[0073] Specifically, the image regions of the two contract training samples undergoing region swapping do not intersect in the same coordinate system. That is, image region a1 of sample x and image region a2 of sample y do not intersect in the same coordinate system.

[0074] After obtaining the training samples and modified training samples of each contract, the training samples and modified training samples of each contract can be used as the contract training sample set and input into the target feature extraction model for training to obtain the trained target feature extraction model.

[0075] By swapping image regions of training samples of contracts of the same contract category, data augmentation is performed on the sample data to obtain multiple modified training samples. Then, the target feature extraction model is trained using the contract training sample set composed of each contract training sample and each modified training sample. This reduces the sensitivity of the target feature extraction model to images, improves the generalization ability and robustness of the target feature extraction model, and further improves the accuracy of contract feature extraction.

[0076] To improve both the training efficiency and computational speed of the target feature extraction model, in some embodiments, such as... Figure 3 As shown, the target feature extraction model is trained using a contract training sample set composed of the contract training samples and the modified training samples, resulting in a trained target feature extraction model, including:

[0077] S301, input the modified training samples into the target feature extraction model to obtain the sample image features output by the target feature extraction model; and,

[0078] S302, input the contract training sample corresponding to the modified training sample into the trained image feature extraction model to obtain the target image features output by the image feature extraction model;

[0079] S303, Based on the similarity between the sample image features and the target image features, adjust the target feature extraction model until the loss function of the target feature extraction model converges, and obtain the trained target feature extraction model;

[0080] The image feature extraction model is a teacher model of the target feature extraction model, and the number of parameters in the image feature extraction model is greater than that in the target feature extraction model.

[0081] In some embodiments, when pre-training the image extraction model using training samples from each contract, the image extraction model can be used as a student model. A moving average is applied to the student model during the iteration process, so that when the target feature extraction model is obtained, its corresponding teacher model, i.e., the image feature extraction model, can be derived. For example, the teacher model can be calculated as follows:

[0082]

[0083] Among them, t i For the current teacher model, t i-1 s represents the teacher model from the previous training round, and s represents the student model whose parameters have been updated after this training round.

[0084] After obtaining the image feature extraction model, the contract training sample m is first input into the teacher model (i.e., the image feature extraction model) for training to obtain the target image features output by the image feature extraction model. Simultaneously, the modified training sample m' corresponding to the contract training sample m is input into the student model (i.e., the target feature extraction model) for training to obtain the sample image features output by the target feature extraction model. Then, the similarity between the target image features and the sample image features is calculated, and based on this similarity, the loss function is calculated according to InfoNce. During training, the teacher model is fixed with its learning rate set to 0.0001 and the optimizer set to the Adam optimizer. Simultaneously, the parameters of the student model (i.e., the target feature extraction model) are adjusted according to the gradient descent algorithm until the loss function converges, thus obtaining the trained target feature extraction model.

[0085] By inputting modified training samples into the target feature extraction model, the sample image features output by the target feature extraction model are obtained. Then, the corresponding training samples, which are the teacher model (the target feature extraction model), are input into the image feature extraction model, resulting in the target image features output by the teacher model. The parameters of the target feature extraction model are adjusted based on the similarity between the sample image features and the target image features until the loss function of the target feature extraction model converges, resulting in a trained target feature extraction model. This allows the information learned by the teacher model (the image feature extraction model) to be transferred to the student model (the target feature extraction model). Target feature extraction models with smaller parameter sizes are well-suited for scenarios with extremely high time requirements or sensitive hardware resources, offering faster computation speed and thus improving both training efficiency and computational speed.

[0086] After training the target feature extraction model, each contract training sample can be sequentially input into the trained model to obtain the contract features of each sample. To make the contract features for each contract category more comprehensive, the contract training samples can be data augmented by adjusting color, size, random perturbation, etc. All the data-augmented contract training samples are then sequentially input into the trained target feature extraction model to obtain the contract features of each sample. After obtaining the contract features of any given contract training sample, its features can be assigned to the contract category to which it belongs, thus obtaining a preset feature set for each contract category.

[0087] A method for classifying contract texts is provided according to some embodiments of this application, and this method can be applied to the aforementioned server. For example... Figure 4 As shown, the classification methods for this contract text include:

[0088] S401, Input the contract text into the trained target feature extraction model to obtain the contract features of the contract text;

[0089] S402, Based on the comparison results between the contract features of the contract text and a preset feature set of any contract category, determine the contract category to which the contract text belongs;

[0090] The target feature extraction model is trained according to the model training method described in any of the above embodiments;

[0091] A preset feature set for any of the contract categories is obtained by inputting training samples of each contract belonging to the contract category into a trained target feature extraction model.

[0092] In some embodiments, after obtaining a trained target feature extraction model through any of the above embodiments, the contract text to be classified can be input into the trained target feature extraction model to obtain the contract features of the contract text. These contract features are image features that include text features.

[0093] To improve the accuracy of contract text classification, one possible implementation is to augment the contract text with data based on factors such as color, size, and random perturbation, resulting in multiple augmented texts. Then, this contract text and its multiple augmented texts are used as a contract text set and input into a trained target feature extraction model to obtain the contract features for each text in the set. Finally, the geometric mean of the contract features for each text in the set is calculated as the final contract feature. This approach makes the obtained contract features more accurate, thereby improving the accuracy of subsequent contract text classification.

[0094] After obtaining the contract features of the contract text, these features can be compared with a preset feature set for any contract category. If the contract features of the contract text match those in a preset feature set—for example, if the similarity between the contract features of the contract text and those in the preset feature set is greater than a preset similarity, or the distance between them is less than a threshold—then it can be determined that the contract features of the contract text match those in the preset feature set. This allows the determination of the contract category corresponding to the preset feature set, which is the contract category to which the contract text belongs.

[0095] By inputting the contract text into a pre-trained target feature extraction model, the contract features of the contract text are obtained. Based on the comparison results of the contract features of the contract text with the preset feature set of any contract category, the contract category to which the contract text belongs is determined. Thus, the pre-trained target feature extraction model can extract image features, including text features such as keywords or text sentences, from the contract text as contract features, making the extracted contract features more comprehensive and thereby improving the classification efficiency of contract text.

[0096] To enable each contract text to be categorized, in some embodiments, the contract category to which the contract text belongs is determined based on a comparison of the contract characteristics of the contract text with a preset feature set of any contract category, including:

[0097] If the contract characteristics of the contract text do not match the preset feature sets of each contract category, a new contract category is added based on the contract characteristics.

[0098] In some embodiments, after obtaining the contract features of the contract text, the contract features of the contract text can be compared with a preset feature set of any contract category. If the contract features of the contract text do not match any of the contract features in the preset feature set, it indicates that the contract features of the contract text are new contract features. At this time, a contract category can be created for the contract features, such as creating a new other contract category, and the contract features can be assigned to the other contract category.

[0099] The model training apparatus provided in this application is described below. The model training apparatus described below can be referred to in correspondence with the model training method described above.

[0100] In one embodiment, such as Figure 5 As shown, a model training device is provided, comprising:

[0101] The first training module 210 is used to pre-train the text extraction model and image extraction model of the feature extraction model according to the training samples of each contract, until the loss function of the feature extraction model converges, and thus obtain the target feature extraction model.

[0102] The second training module 220 is used to train the target feature extraction model based on the contract training sample set obtained after data augmentation of each contract training sample, so as to obtain a trained target feature extraction model.

[0103] The loss function of the feature extraction model is determined based on the cosine similarity between the text features obtained by inputting the contract training sample into the text extraction model and the image features obtained by inputting the contract training sample into the image extraction model, as well as the contract category to which the contract training sample belongs.

[0104] The trained target feature extraction model is used to extract contract features from the contract text.

[0105] By pre-training the text extraction model and image extraction model of the feature extraction model using various contract training samples, the target feature extraction model is obtained. Then, based on the contract training sample set obtained after data augmentation of each contract training sample, the target feature extraction model is trained to obtain a well-trained target feature extraction model. This well-trained target feature extraction model can extract image features, including text features such as keywords or text sentences, from the contract text, making the contract features extracted from the contract text more comprehensive and thus improving the classification efficiency of the contract text.

[0106] In one embodiment, the first training module 210 is specifically used for:

[0107] Based on the training samples of each contract, the text extraction model and the image extraction model of the feature extraction model are trained until the loss function of the feature extraction model converges. The text extraction model is then discarded, and the pre-trained image extraction model is determined as the target feature extraction model.

[0108] In one embodiment, the second training module 220 is specifically used for:

[0109] By swapping the image regions of any two contract training samples belonging to the same contract category, multiple modified training samples can be obtained.

[0110] The target feature extraction model is trained based on the contract training sample set composed of the contract training samples and the modified training samples, to obtain the trained target feature extraction model.

[0111] In one embodiment, the image regions of the two contract training samples that are being swapped do not intersect in the same coordinate system.

[0112] In some embodiments, the second training module 220 is specifically used for:

[0113] The modified training samples are input into the target feature extraction model to obtain the sample image features output by the target feature extraction model; and,

[0114] Input the contract training sample corresponding to the modified training sample into the image feature extraction model to obtain the target image features output by the image feature extraction model;

[0115] Based on the similarity between the features of the sample image and the features of the target image, the target feature extraction model is adjusted until the loss function of the target feature extraction model converges, thus obtaining a trained target feature extraction model.

[0116] The image feature extraction model is a teacher model of the target feature extraction model, and the number of parameters in the image feature extraction model is greater than that in the target feature extraction model.

[0117] The following describes the contract text classification device provided in this application. The contract text classification device described below can be referred to in correspondence with the contract text classification method described above.

[0118] In one embodiment, such as Figure 6 As shown, a contract text classification device is provided, comprising:

[0119] The feature extraction module 310 is used to input the contract text into the trained target feature extraction model to obtain the contract features of the contract text;

[0120] The contract classification module 320 is used to determine the contract category to which the contract text belongs based on the comparison result between the contract characteristics of the contract text and a preset feature set of any contract category;

[0121] The target feature extraction model is trained according to the model training method described in any of the above embodiments;

[0122] A preset feature set for any of the contract categories is obtained by inputting training samples of each contract belonging to the contract category into a trained target feature extraction model.

[0123] By inputting the contract text into a pre-trained target feature extraction model, the contract features of the contract text are obtained. Based on the comparison results of the contract features of the contract text with the preset feature set of any contract category, the contract category to which the contract text belongs is determined. Thus, the pre-trained target feature extraction model can extract image features, including text features such as keywords or text sentences, from the contract text as contract features, making the extracted contract features more comprehensive and thereby improving the classification efficiency of contract text.

[0124] In one embodiment, the contract classification module 320 is specifically used for:

[0125] If the contract characteristics of the contract text do not match the preset feature sets of each contract category, a new contract category is added based on the contract characteristics.

[0126] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call computer programs stored in the memory 830 to execute model training methods, such as:

[0127] Based on the training samples of each contract, the text extraction model and image extraction model of the feature extraction model are pre-trained until the loss function of the feature extraction model converges, thus obtaining the target feature extraction model.

[0128] The target feature extraction model is trained using the contract training sample set obtained after data augmentation of each contract training sample, to obtain a trained target feature extraction model.

[0129] The loss function of the feature extraction model is determined based on the cosine similarity between the text features obtained by inputting the contract training sample into the text extraction model and the image features obtained by inputting the contract training sample into the image extraction model, as well as the contract category to which the contract training sample belongs.

[0130] The trained target feature extraction model is used to extract contract features from the contract text.

[0131] Alternatively, processor 810 may invoke a computer program in memory 830 to execute a method for classifying contract texts, such as including:

[0132] The contract text is input into the trained target feature extraction model to obtain the contract features of the contract text;

[0133] Based on the contract characteristics of the contract text and the comparison results with a preset feature set of any contract category, the contract category to which the contract text belongs is determined;

[0134] The target feature extraction model is trained according to the model training method described in any of the above embodiments;

[0135] A preset feature set for any of the contract categories is obtained by inputting training samples of each contract belonging to the contract category into a trained target feature extraction model.

[0136] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0137] On the other hand, embodiments of this application also provide a processor-readable storage medium storing a computer program for causing the processor to execute the model training method or contract text classification method provided in the above embodiments.

[0138] Processor-readable storage media can be any available medium or model training device that the processor can access, including but not limited to magnetic memory (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical memory (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor memory (such as ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0139] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0140] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A model training method, characterized in that, include: Based on the training samples of each contract, the text extraction model and image extraction model of the feature extraction model are pre-trained until the loss function of the feature extraction model converges, thus obtaining the target feature extraction model. The target feature extraction model is trained using the contract training sample set obtained after data augmentation of each contract training sample, to obtain a trained target feature extraction model. The loss function of the feature extraction model is determined based on the cosine similarity between the text features obtained by inputting the contract training sample into the text extraction model and the image features obtained by inputting the contract training sample into the image extraction model, as well as the contract category to which the contract training sample belongs. The trained target feature extraction model is used to extract contract features from the contract text. The image features extracted by the image extraction model in the trained target feature extraction model include the text features extracted by the text extraction model. Based on the training samples of each contract, the text extraction model and image extraction model of the feature extraction model are pre-trained, including inputting the training samples of each contract into the text extraction model and image extraction model of the feature extraction model for training; Based on the training samples of each contract, the text extraction model and image extraction model of the feature extraction model are pre-trained until the loss function of the feature extraction model converges, thus obtaining the target feature extraction model, including: Based on the training samples of each contract, the text extraction model and the image extraction model of the feature extraction model are trained until the loss function of the feature extraction model converges. The text extraction model is then discarded, and the pre-trained image extraction model is determined as the target feature extraction model.

2. The model training method according to claim 1, characterized in that, Based on the contract training sample set obtained after data augmentation of each contract training sample, the target feature extraction model is trained to obtain a trained target feature extraction model, including: By swapping the image regions of any two contract training samples belonging to the same contract category, multiple modified training samples can be obtained. The target feature extraction model is trained based on the contract training sample set composed of the contract training samples and the modified training samples, to obtain the trained target feature extraction model.

3. The model training method according to claim 2, characterized in that, The image regions of the two contract training samples that are swapped do not intersect in the same coordinate system.

4. The model training method according to claim 2, characterized in that, The target feature extraction model is trained based on the contract training sample set composed of the contract training samples and the modified training samples, to obtain a trained target feature extraction model, including: The modified training samples are input into the target feature extraction model to obtain the sample image features output by the target feature extraction model; and, Input the contract training sample corresponding to the modified training sample into the image feature extraction model to obtain the target image features output by the image feature extraction model; Based on the similarity between the features of the sample image and the features of the target image, the target feature extraction model is adjusted until the loss function of the target feature extraction model converges, thus obtaining a trained target feature extraction model. The image feature extraction model is a teacher model of the target feature extraction model, and the number of parameters in the image feature extraction model is greater than that in the target feature extraction model.

5. A method for classifying contract texts, characterized in that, include: The contract text is input into the trained target feature extraction model to obtain the contract features of the contract text; Based on the contract characteristics of the contract text and the comparison results with a preset feature set of any contract category, the contract category to which the contract text belongs is determined; The target feature extraction model is trained using the model training method according to any one of claims 1-4; A preset feature set for any of the contract categories is obtained by inputting training samples of each contract belonging to the contract category into a trained target feature extraction model.

6. The method for classifying contract texts according to claim 5, characterized in that, Based on the contract characteristics of the contract text and the comparison results with a preset feature set of any contract category, the contract category to which the contract text belongs is determined, including: If the contract characteristics of the contract text do not match the preset feature sets of each contract category, a new contract category is added based on the contract characteristics.

7. A model training device, characterized in that, include: The first training module is used to pre-train the text extraction model and image extraction model of the feature extraction model based on the training samples of each contract until the loss function of the feature extraction model converges, thereby obtaining the target feature extraction model. The second training module is used to train the target feature extraction model based on the contract training sample set obtained after data augmentation of each contract training sample, so as to obtain the trained target feature extraction model. The loss function of the feature extraction model is determined based on the cosine similarity between the text features obtained by inputting the contract training sample into the text extraction model and the image features obtained by inputting the contract training sample into the image extraction model, as well as the contract category to which the contract training sample belongs. The trained target feature extraction model is used to extract contract features from the contract text. The image features extracted by the image extraction model in the trained target feature extraction model include the text features extracted by the text extraction model. Based on the training samples of each contract, the text extraction model and image extraction model of the feature extraction model are pre-trained, including inputting the training samples of each contract into the text extraction model and image extraction model of the feature extraction model for training; The second training module is specifically used for: Based on the training samples of each contract, the text extraction model and the image extraction model of the feature extraction model are trained until the loss function of the feature extraction model converges. The text extraction model is then discarded, and the pre-trained image extraction model is determined as the target feature extraction model.

8. A contract text classification device, characterized in that, include: The feature extraction module is used to input the contract text into the trained target feature extraction model to obtain the contract features of the contract text; The contract classification module is used to determine the contract category to which the contract text belongs based on the comparison results between the contract characteristics of the contract text and a preset feature set of any contract category. The target feature extraction model is trained using the model training method according to any one of claims 1-4; A preset feature set for any of the contract categories is obtained by inputting training samples of each contract belonging to the contract category into a trained target feature extraction model.

9. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the model training method according to any one of claims 1 to 4, or the contract text classification method according to any one of claims 5 to 6.

Images