Model training detection method and device, storage medium and program product
By detecting the marked and unlabeled sample data and iteratively updating the model parameters, the problem of poor performance of the layout detection model in the prior art in complex research and report scenarios is solved, and an efficient and robust object detection model is achieved.
Patent Information
- Application Number
- CN202411939977.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-26
- Publication Date
- 2025-04-22
AI Technical Summary
The layout detection models in the prior art perform poorly in complex research and reporting scenarios, have high training costs and poor detection performance.
By using the first initial detection model to detect the marked and unlabeled sample data, iteratively updates iteratively with large models with the same structure but different parameters as it is formed to form an object detection model to deal with complex scenarios.
It reduces the amount of labeled data required for model training, reduces the training cost, and improves detection performance and generalization capabilities, enhancing the robustness of the model.
Smart Images
Figure CN119863799A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of target detection technology, and in particular to a model training detection method, device, storage medium and program product. Background Art
[0002] In Optical Character Recognition (OCR) technology, the layout detection model can automatically identify different areas in a document, such as titles, paragraphs, lists, etc., so as to perform more accurate information extraction and content analysis, and improve the accuracy and efficiency of OCR.
[0003] However, due to the diversity of research report layouts and the complexity of typesetting, the layout detection model training process requires a large amount of data support. This results in the layout detection model in the existing technology being unable to cope with more difficult and complex research report scenarios, lacking the ability to adapt to complex scenarios, and having problems with high training costs and poor detection performance. Summary of the invention
[0004] The purpose of this application is to provide a model training detection method, device, storage medium and program product to address the deficiencies in the above-mentioned prior art, so as to solve the problems of high training cost and poor detection performance in the prior art.
[0005] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:
[0006] In a first aspect, an embodiment of the present application provides a model training detection method, the method comprising:
[0007] Inputting the labeled first sample data and the unlabeled second sample data into a first initial detection model to obtain a first detection result corresponding to the first sample data and a second detection result corresponding to the second sample data;
[0008] Inputting the third sample data into the second initial detection model to obtain a third detection result corresponding to the third sample data, wherein the third sample data and the second sample data are obtained by performing different interference processing based on the same original data, the second initial detection model and the first initial detection model have the same network structure, and the value of at least one model parameter of the second initial detection model is different from the value of the corresponding model parameter in the first initial detection model;
[0009] Determine a first loss according to the first sample data and the first detection result;
[0010] Determine a second loss according to the second detection result and the third detection result, where the second loss is used to characterize the difference between the results predicted by the first initial detection model and the second initial detection model for data with the same content;
[0011] According to the first loss and the second loss, the first initial detection model is iteratively updated, and the second initial detection model is updated according to the updated first initial detection model, and the first initial detection model at the end of the iteration is used as the target detection model, and detection processing is performed based on the target detection model.
[0012] In a second aspect, another embodiment of the present application provides a model training detection device, the device comprising:
[0013] A first detection module, used to input the labeled first sample data and the unlabeled second sample data into a first initial detection model to obtain a first detection result corresponding to the first sample data and a second detection result corresponding to the second sample data;
[0014] a second detection module, configured to input third sample data into a second initial detection model to obtain a third detection result corresponding to the third sample data, wherein the third sample data and the second sample data are obtained by performing different interference processing based on the same original data, the second initial detection model and the first initial detection model have the same network structure, and a value of at least one model parameter of the second initial detection model is different from a value of a corresponding model parameter in the first initial detection model;
[0015] A first loss determination module, configured to determine a first loss according to the first sample data and the first detection result;
[0016] a second loss determination module, configured to determine a second loss according to the second detection result and the third detection result, wherein the second loss is used to characterize the difference between the results predicted by the first initial detection model and the second initial detection model for data with the same content;
[0017] An iterative update module is used to iteratively update the first initial detection model according to the first loss and the second loss, and to update the second initial detection model according to the updated first initial detection model, and to use the first initial detection model at the end of the iteration as the target detection model, and to perform detection processing based on the target detection model.
[0018] In the third aspect, another embodiment of the present application provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of any method described in the first aspect above.
[0019] In a fourth aspect, another embodiment of the present application provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any method described in the first aspect are executed.
[0020] In the fifth aspect, another embodiment of the present application provides a computer program product, which includes a computer program code. When the computer program code is executed by a computing device, the computing device executes the steps of any method described in the first aspect above.
[0021] The beneficial effects of the present application are: the first initial detection model is used to detect the labeled first sample data and the unlabeled second sample data to obtain the first detection result and the second detection result, and the second initial detection model with the same network structure and different model parameter values as the first initial detection model is used to detect the unlabeled second sample data to obtain the third detection result, so that the first loss can be determined according to the first sample data and the first detection result, and the second loss can be determined according to the second detection result and the third detection result, so that the first initial detection model is iteratively updated according to the first loss and the second loss, and the second initial detection model is updated according to the updated first initial detection model, and the first initial detection model at the end of the iteration is used as the target detection model, and the detection processing is performed based on the target detection model, so that the target detection model obtained can cope with complex target detection scenarios, and the detection performance is guaranteed. At the same time, a large amount of labeled data is not required in the model training process, and the training cost is reduced. In addition, by updating the first initial detection model through the first loss and the second loss, the first initial detection model can be prompted to learn richer feature representations and more complex decision boundaries during the training process, thereby improving the generalization ability of the target detection model obtained and enhancing the robustness of the target detection model obtained. In addition, updating the second initial detection model according to the updated first initial detection model can also reduce the fluctuation of parameter updates, making the second initial detection model more stable during the training process, thereby ensuring the performance of the obtained target detection model. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0023] Figure 1 A schematic diagram of a flow chart of a model training detection method provided in an embodiment of the present application;
[0024] Figure 2 A schematic diagram of a model training detection method provided in an embodiment of the present application;
[0025] Figure 3 A schematic diagram of a process for obtaining second sample data and third sample data in the model training detection method provided in an embodiment of the present application;
[0026] Figure 4 A schematic diagram of the structure of a first initial detection model provided in an embodiment of the present application;
[0027] Figure 5 A schematic diagram of a process for obtaining a first detection result corresponding to first sample data and a second detection result corresponding to second sample data in the model training detection method provided in an embodiment of the present application;
[0028] Figure 6 A schematic diagram of the structure of a first position encoding module in a first initial detection model provided in an embodiment of the present application;
[0029] Figure 7 A schematic diagram of a process for generating first feature data in the model training detection method provided in an embodiment of the present application;
[0030] Figure 8 A structural schematic diagram of a first height processing module in a first initial detection model provided in an embodiment of the present application;
[0031] Fig. 9 A schematic diagram of a flow chart for obtaining height feature information corresponding to first feature image data in the model training detection method provided in an embodiment of the present application;
[0032] Fig.10 A structural schematic diagram of a first width processing module in a first initial detection model provided in an embodiment of the present application;
[0033] Fig.11 A schematic diagram of a flow chart for obtaining width feature information corresponding to first feature image data in the model training detection method provided in an embodiment of the present application;
[0034] Fig.12 A schematic diagram of a process for determining the second loss in the model training detection method provided in an embodiment of the present application;
[0035] Fig.13 A schematic diagram of a model training detection device provided in an embodiment of the present application;
[0036] Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0037] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.
[0038] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.
[0039] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.
[0040] In Optical Character Recognition (OCR) technology, the layout detection model can automatically identify different areas in a document, such as titles, paragraphs, lists, etc., so as to perform more accurate information extraction and content analysis, and improve the accuracy and efficiency of OCR.
[0041] However, due to the diversity of research report layouts and the complexity of typesetting, the layout detection model training process requires a large amount of data support. This results in the layout detection model in the existing technology being unable to cope with more difficult and complex research report scenarios, lacking the ability to adapt to complex scenarios, and having problems with high training costs and poor detection performance.
[0042] Based on the above problems, the embodiment of the present application proposes a model training detection method, which detects the labeled first sample data and the unlabeled second sample data through the first initial detection model to obtain the first detection result and the second detection result, and detects the unlabeled second sample data through the second initial detection model with the same network structure and different model parameter values as the first initial detection model to obtain the third detection result, so that the first loss can be determined according to the first sample data and the first detection result, and the second loss can be determined according to the second detection result and the third detection result, so that the first initial detection model is iteratively updated according to the first loss and the second loss, and the second initial detection model is updated according to the updated first initial detection model, and the first initial detection model at the end of the iteration is used as the target detection model, and the detection processing is performed based on the target detection model. The target detection model obtained can cope with complex target detection scenarios, and the detection performance is guaranteed. At the same time, a large amount of labeled data is not required during the model training process, which reduces the training cost.
[0043] It can be understood that the model training detection method provided in the embodiments of the present application can be applied to any scenario that requires target detection, such as layout detection, document detection, and image detection. By executing the model training detection method provided in the embodiments of the present application, target detection results can be obtained in complex target detection scenarios.
[0044] The model training detection method provided in the embodiments of the present application is described in detail below in combination with multiple embodiments.
[0045] Figure 1 A flow chart of a model training detection method provided in an embodiment of the present application, referring to Figure 1 As shown, the execution subject of the method can be any electronic device with processing capability, and the method includes:
[0046] S101: Input labeled first sample data and unlabeled second sample data into a first initial detection model to obtain a first detection result corresponding to the first sample data and a second detection result corresponding to the second sample data.
[0047] Optionally, Figure 2 A schematic diagram of the model training detection method provided in the embodiment of the present application, referring to Figure 2As shown, the labeled first sample data and the unlabeled second sample data are input into the first initial detection model, so that the first initial detection model can detect the labeled first sample data and the unlabeled second sample data respectively, thereby obtaining a first detection result corresponding to the first sample data and a second detection result corresponding to the second sample data.
[0048] Among them, the labeled first sample data can be understood as supervised data, that is, a set consisting of multiple labeled first sample images and labels corresponding to each first sample image. Specifically, taking layout detection as an example, the first sample image can be an image containing layout structure and content information, and the label corresponding to the first sample image can include a description or classification of the target object in the first sample image. For example, a first sample image can be a page of a portable document format (PDF), a page of a presentation (Microsoft Office PowerPoint, PPT) or a page of a newspaper, and the label corresponding to the first sample image can include text, graphics, and tables, etc.
[0049] The unlabeled second sample data can be understood as unsupervised data, that is, a set of multiple unlabeled second sample images. Specifically, taking layout detection as an example, the second sample image can be an image containing layout structure and content information, for example: a second sample image can be a page of Portable Document Format (PDF), a page of Microsoft Office PowerPoint (PPT), or a page of a newspaper. The first sample image and the second sample image can be the same sample image or different sample images.
[0050] Among them, the first detection result corresponding to the first sample data includes the label, position coordinates and other related information of each target object in each first sample image, and the second detection result corresponding to the second sample data includes the label, position coordinates and other related information of each target object in each second sample image.
[0051] Exemplarily, taking layout detection as an example, the first sample data may include each title and the position coordinates of each title, each graphic and the position coordinates of each graphic, each table and the position coordinates of each table, etc. in each first sample image. The second sample data may include each title and the position coordinates of each title, each graphic and the position coordinates of each graphic, each table and the position coordinates of each table, etc. in each second sample image.
[0052] Among them, the first initial detection model can be a detection model based on a neural network model, wherein the neural network model can be: YOLO model, NanoDet model, CNN model and Transformer model, etc.
[0053] Exemplarily, the first initial detection model may be an extended model based on Faster R-CNN or an extended model based on YOLO. For example, an extended model based on YOLO is obtained by adding a convolutional layer, a pooling layer, or an activation function to the YOLO model.
[0054] S102: Input the third sample data into the second initial detection model to obtain a third detection result corresponding to the third sample data.
[0055] Optionally, continue to refer to Figure 2 As shown, the unlabeled third sample data is input into the second initial detection model, so that the second initial detection model can detect the third sample data respectively, thereby obtaining a third detection result corresponding to the third sample data.
[0056] The third sample data can be understood as unsupervised data, that is, a set of multiple unlabeled third sample images. Specifically, taking layout detection as an example, the third sample image can be an image containing layout structure and content information, for example, a third sample image can be a page of a portable document format (PDF), a page of a presentation (Microsoft Office PowerPoint, PPT) or a page of a newspaper. The third sample image and the second sample image are obtained by performing different interference processing based on the same original image data.
[0057] Among them, the third sample data and the second sample data are obtained by performing different interference processing on the same original data, the second initial detection model and the first initial detection model have the same network structure, and the value of at least one model parameter of the second initial detection model is different from the value of the corresponding model parameter in the first initial detection model.
[0058] The third detection result corresponding to the third sample data includes the label, position coordinates and other relevant information of each target object in each third sample image.
[0059] The second initial detection model has the same network structure as the first initial detection model, and the value of at least one model parameter of the second initial detection model is different from the value of the corresponding model parameter in the first initial detection model. The second initial detection model can be a detection model based on a neural network model, wherein the neural network model can be: a YOLO model, a NanoDet model, a CNN model, a Transformer model, etc.
[0060] Exemplarily, the second initial detection model may be an extended model based on Faster R-CNN or an extended model based on YOLO. For example, an extended model based on YOLO is obtained by adding a convolutional layer, a pooling layer, or an activation function to the YOLO model.
[0061] S103: Determine a first loss according to the first sample data and the first detection result.
[0062] Optionally, continue to refer to Figure 2 As shown, after obtaining the first detection result, the first loss can be calculated based on the first sample data, the first detection result and a preset loss function.
[0063] Exemplarily, the first detection result and the true result corresponding to the first sample data may be input into a preset first loss function to calculate the first loss.
[0064] The first loss is used to characterize the difference between the first detection result and the true result corresponding to the first sample data. The preset first loss function can be a cross-entropy loss, a Hamming loss, a zero-one loss, etc.
[0065] S104: Determine a second loss according to the second detection result and the third detection result.
[0066] Optionally, continue to refer to Figure 2 As shown, the second detection result and the third detection result can be input into a preset second loss function to calculate the second loss.
[0067] The second loss is used to characterize the difference between the results predicted by the first initial detection model and the second initial detection model for data with the same content.
[0068] The second loss can help the model learn richer feature representations and more complex decision boundaries during training, which helps the target detection model to show better generalization ability when processing unseen data, thereby improving the practicality and reliability of the target detection model. At the same time, it can also help the model learn more stable and more generalized feature representations during training, thereby enhancing the robustness of the target detection model.
[0069] S105. Iteratively update the first initial detection model according to the first loss and the second loss, and update the second initial detection model according to the updated first initial detection model, and use the first initial detection model at the end of the iteration as the target detection model, and perform detection processing based on the target detection model.
[0070] Optionally, continue to refer to Figure 2 As shown, after obtaining the first loss and the second loss, the first loss and the second loss can be used as the difference obtained in this training, so that the first initial detection model can be iteratively updated according to the first loss and the second loss during each training process.
[0071] Exemplarily, the sum of the first loss and the second loss can be taken as the total loss, and the gradient of the total loss with respect to the first initial detection model is calculated, and the parameters of the first initial detection model are updated.
[0072] Optionally, continue to refer to Figure 2 As shown, after the first initial detection model is updated, the second initial detection model can be updated according to the updated first initial detection model. For example, part of the model parameters of the updated first initial detection model can be copied to the second initial detection model, or the updated first initial detection model and the second initial detection model can be transferred for learning, or the model parameters of the updated first initial detection model and the model parameters of the second initial detection model can be weighted averaged and weighted fused to achieve the update of the second initial detection model.
[0073] Optionally, after the first initial detection model is updated, the preset training end condition can be used to determine whether the training is completed. If so, the training is completed and the updated first initial detection model is used as the target detection model. If not, the second initial detection model is updated and training continues to iteratively update the first initial detection model and the second initial detection model.
[0074] Among them, the preset training end conditions may include: one or more of a preset number of iterations, a preset error change condition, a preset loss value, and a preset learning rate threshold, which can be adjusted according to actual conditions.
[0075] Optionally, after the training is completed, the updated first initial detection model is used as the target detection model, and the target to be detected is detected based on the target detection model, so as to obtain the target detection result.
[0076] In this embodiment, the first initial detection model is used to detect the labeled first sample data and the unlabeled second sample data to obtain the first detection result and the second detection result, and the second initial detection model having the same network structure and different model parameter values as the first initial detection model is used to detect the unlabeled second sample data to obtain the third detection result, so that the first loss can be determined according to the first sample data and the first detection result, and the second loss can be determined according to the second detection result and the third detection result, so that the first initial detection model is iteratively updated according to the first loss and the second loss, and the second initial detection model is updated according to the updated first initial detection model, and the first initial detection model at the end of the iteration is used as the target detection model, and the detection processing is performed based on the target detection model, so that the target detection model obtained can cope with complex target detection scenarios, and the detection performance is guaranteed. At the same time, a large amount of labeled data is not required in the model training process, and the training cost is reduced. In addition, by updating the first initial detection model through the first loss and the second loss, the first initial detection model can be prompted to learn richer feature representations and more complex decision boundaries during the training process, thereby improving the generalization ability of the target detection model obtained and enhancing the robustness of the target detection model obtained. In addition, updating the second initial detection model according to the updated first initial detection model can also reduce the fluctuation of parameter updates, making the second initial detection model more stable during the training process, thereby ensuring the performance of the obtained target detection model.
[0077] In one possible implementation, Figure 3 A schematic diagram of a flow chart for obtaining the second sample data and the third sample data in the model training detection method provided in the embodiment of the present application, referring to Figure 3 As shown, the process of obtaining the second sample data and the third sample data includes:
[0078] S301: Obtain original unlabeled sample data.
[0079] Optionally, original unlabeled sample data is obtained. The original unlabeled sample data refers to observation data that is not manually labeled or classified and directly comes from the real world. Taking layout detection as an example, the original unlabeled sample data can be multiple images containing layout structure and content information.
[0080] S302 , performing random noise processing and weak random noise processing on the original unlabeled sample data respectively to obtain second sample data and third sample data.
[0081] Optionally, random noise processing may be performed on the original unlabeled sample data to obtain second sample data, wherein the random noise processing refers to adding random noise to the original unlabeled sample data according to a preset noise type and a first intensity.
[0082] Optionally, the original unlabeled sample data may be subjected to weak random noise processing to obtain third sample data. The weak random noise processing refers to adding weak random noise to the original unlabeled sample data according to a preset noise type and a second intensity. The first intensity is greater than the second intensity.
[0083] Exemplarily, random noise processing can be understood as introducing noise into the original unlabeled data to generate new second sample data that is slightly different from the original unlabeled data, and weak random noise processing can be understood as introducing weak noise into the original unlabeled data to generate third sample data that maintains a high degree of similarity with the original unlabeled data.
[0084] By obtaining original unlabeled sample data and performing random noise processing and weak random noise processing on the original unlabeled sample data respectively, the second sample data and the third sample data are obtained. This allows the second sample data and the third sample data to retain the main features of the original unlabeled sample data. At the same time, the diversity and robustness of the training data can be increased to improve the generalization ability of the obtained target detection model and its robustness to noise.
[0085] The above is an illustrative description of the process of obtaining the second sample data and the third sample data. It can be understood that after obtaining the first sample data, the second sample data and the third sample data, the first initial detection model and the second initial detection model can be trained. Since the first initial detection model and the second initial detection model are the same in network structure, the network structure of the initial detection model is described below using the first initial detection model as an example.
[0086] In one possible implementation, Figure 4 A schematic diagram of the structure of a first initial detection model provided in an embodiment of the present application, Figure 5 A flow chart of obtaining a first detection result corresponding to the first sample data and a second detection result corresponding to the second sample data in the model training detection method provided in the embodiment of the present application, with reference to Figure 4 as well as Figure 5 As shown, the first initial detection model includes a first position encoding module and a first target detection module. When the above S101 inputs the labeled first sample data and the unlabeled second sample data into the first initial detection model to obtain a first detection result corresponding to the first sample data and a second detection result corresponding to the second sample data, it includes:
[0087] S501: Input the labeled first sample data and the unlabeled second sample data into a first position encoding module to generate first feature data.
[0088] Optionally, the labeled first sample data and the unlabeled second sample data are input into the first position encoding module to perform feature processing on each first sample image in the labeled first sample data and each second sample image in the unlabeled second sample data, and add position information, thereby helping the first initial detection model to understand the position of each target object in each first sample image and each second sample image, so that the first initial detection model can better process each first sample image and each second sample image. The position information may include height position information and width features, that is, height features and width features of each target object in each image.
[0089] The first feature data includes the height features and width features in the labeled first sample data and the unlabeled second sample data, that is, the first feature data includes the height features and width features in each first sample image and the height features and width features in each second sample image.
[0090] It should be understood that in the actual training process of the model, the first sample data and the second sample data can be input into the first initial detection model as training data of the same batch, so that the first initial detection model can process each sample image in the first sample data and the second sample data respectively to obtain the first feature data.
[0091] S502: Input the first feature data into a first target detection module to obtain a first detection result corresponding to the first sample data and a second detection result corresponding to the second sample data.
[0092] Optionally, after obtaining the first feature data, the first feature data is input into a first target detection module to obtain a first detection result corresponding to the first sample data and a second detection result corresponding to the second sample data.
[0093] Among them, the first target detection module includes a neural network model, wherein the neural network model can be: YOLO model, NanoDet model, CNN model and Transformer model, etc.
[0094] Exemplarily, the first target detection module may be a YOLO model or a Transformer model.
[0095] By inputting the labeled first sample data and the unlabeled second sample data into the first position encoding module to generate the first feature data, the labeled data and the unlabeled data can be processed simultaneously, thereby improving the utilization efficiency of the data, enhancing the generalization ability of the obtained target detection model, and more accurately reflecting the internal structure and relationship of the data. By inputting the first feature data into the first target detection module, a first detection result corresponding to the first sample data and a second detection result corresponding to the second sample data are obtained. Through a semi-supervised approach, the performance of the obtained target detection model is jointly improved, so that the obtained target detection model can better adapt to different application scenarios and needs.
[0096] In one possible implementation, Figure 6 A schematic diagram of the structure of a first position encoding module in a first initial detection model provided in an embodiment of the present application, Figure 7 A schematic diagram of a flow chart for generating first feature data in the model training detection method provided in the embodiment of the present application, referring to Figure 6 as well as Figure 7 As shown, the first position encoding module includes: a first feature extraction layer, a first height processing module, a first width processing module and a first fusion module. The above S501 inputs the labeled first sample data and the unlabeled second sample data into the first position encoding module to generate the first feature data, including:
[0097] S701 , inputting the labeled first sample data and the unlabeled second sample data into a first feature extraction layer for feature extraction to obtain first feature image data.
[0098] Optionally, the first sample data and the second sample data are input into the first feature extraction layer for feature convolution extraction to obtain first feature image data, which can efficiently capture local features in the first sample data and the second sample data and extract useful feature information, and can also reduce the amount of calculation and improve the model training speed.
[0099] S702: Input the first feature image data into a first height processing module for height encoding to obtain height feature information corresponding to the first feature image data.
[0100] Optionally, the first feature data is input into the first height processing module for height encoding, so that different horizontal areas in each sample image can be identified, thereby obtaining height feature information corresponding to the first feature image data.
[0101] Exemplarily, taking layout detection as an example, the first feature data is input into the first height processing module for height encoding to obtain height feature information corresponding to the first feature image data, which helps the model to more accurately locate text lines and detect the inclination angle of the text, as well as remove background interference, thereby improving the accuracy of text recognition.
[0102] S703: Input the first feature image data into a first width processing module for width encoding to obtain width feature information corresponding to the first feature image data.
[0103] Optionally, the first characteristic image data is input into the first width processing module for width encoding, so that different longitudinal regions in each sample image can be identified, thereby obtaining width characteristic information corresponding to the first characteristic image data.
[0104] Exemplarily, taking layout detection as an example, the first feature data is input into the first width processing module for width encoding to obtain width feature information corresponding to the first feature image data, which helps the model to more accurately locate each column and understand the layout and structure of the layout, as well as remove background interference, thereby improving the accuracy of text recognition.
[0105] S704: Input the first feature image data, the height feature information, and the width feature information into a first fusion module to obtain first feature data.
[0106] Optionally, after obtaining the height feature information and the width feature information, the first feature image data, the height feature information and the width feature information are input into the first fusion module and fused by pixel multiplication to obtain the first feature data. This can integrate feature information from different sources to form a more accurate and comprehensive feature representation, thereby improving the robustness and accuracy of the obtained target detection model.
[0107] In one possible implementation, Figure 8 A structural schematic diagram of a first height processing module in a first initial detection model provided in an embodiment of the present application, Fig. 9 A schematic diagram of a flow chart for obtaining the height feature information corresponding to the first feature image data in the model training detection method provided in the embodiment of the present application, referring to Figure 8 as well as Fig. 9 As shown, the first height processing module includes: a first height pooling layer, a first height encoding layer and a first height normalization layer. The above S702 inputs the first feature image data into the first height processing module for height encoding to obtain the height feature information corresponding to the first feature image data, including:
[0108] S901: Input the first feature image into the first height pooling layer for average pooling processing to obtain a height pooling image.
[0109] Optionally, the first feature image is input into the first height pooling layer, and after width-based 2d-w average pooling, the dimension w length is compressed to 1 to obtain a height pooled image, which helps to reduce the amount of computation and memory usage in subsequent processing steps, while retaining key feature information related to height, and can also reduce the complexity of the model and reduce the risk of overfitting.
[0110] S902: Input the height pooled image into the first height coding layer for sinusoidal height coding to obtain coded height feature information.
[0111] Optionally, the height pooled image is input into the first height coding layer for sinusoidal height position coding to obtain the encoded height feature information, which can capture the position information along the height direction in the height pooled image, so that the model can distinguish the features at different height positions, which helps to understand the vertical structure in the image, enhances the representation ability of the height feature information, and helps to improve the recognition accuracy and robustness of the model.
[0112] S903 , input the encoded height feature information into a first height normalization layer for normalization processing to obtain height feature information corresponding to the first feature image data.
[0113] Optionally, the encoded height feature information is input into the first height normalization layer for sigmoid normalization processing to obtain the height feature information corresponding to the first feature image data, so that the values of different features are comparable, avoiding the influence of extreme values on model training.
[0114] In one possible implementation, Fig.10 A structural schematic diagram of a first width processing module in a first initial detection model provided in an embodiment of the present application, Fig.11 A schematic diagram of a flow chart for obtaining width feature information corresponding to first feature image data in the model training detection method provided in an embodiment of the present application, referring to Fig.10 as well as Fig.11 As shown, the first width processing module includes: a first width pooling layer, a first width encoding layer and a first width normalization layer. The above S703 inputs the first feature image data into the first width processing module for width encoding to obtain width feature information corresponding to the first feature image data, including:
[0115] S1101, input the first feature image into the first width pooling layer for average pooling processing to obtain a width pooling image.
[0116] Optionally, the first feature image is input into the first width pooling layer, and after height-based 2d-h average pooling, the dimension h is compressed to 1 in length to obtain a width pooled image, which helps to reduce the amount of computation and memory usage in subsequent processing steps while retaining key feature information related to the width, and can also reduce the complexity of the model and reduce the risk of overfitting.
[0117] S1102: Input the width pooling image into the first width coding layer for sinusoidal width coding to obtain encoded width feature information.
[0118] Optionally, the width pooled image is input into the first width coding layer for sinusoidal width position encoding to obtain the encoded width feature information, which can capture the position information along the width direction in the width pooled image, so that the model can distinguish the features of different width positions, which helps to understand the horizontal structure in the image, enhances the representation ability of the width feature information, and helps to improve the recognition accuracy and robustness of the model.
[0119] S1103 , input the encoded width feature information into a first width normalization layer for normalization processing to obtain width feature information corresponding to the first feature image data.
[0120] Optionally, the encoded width feature information is input into the first width normalization layer for sigmoid normalization processing to obtain width feature information corresponding to the first feature image data, so that the values of different features are comparable, avoiding the influence of extreme values on model training.
[0121] The above uses the first initial detection model as an example to illustrate the network structure of the initial detection model. When the initial detection model is the second initial detection model, the network structure of the second initial detection model can be implemented with reference to the network structure of the first initial detection model. The third sample data can be input into the second initial detection model, so as to obtain the third detection result based on the second position encoding module and the second target detection module in the second initial detection model. This application does not go into details about this.
[0122] The following is an exemplary description of the process of updating the first initial detection model and the second initial detection model after obtaining the first detection result, the second detection result and the third detection result.
[0123] In a possible implementation, the above S103 determines the first loss according to the first sample data and the first detection result, including:
[0124] The cross entropy information loss between the true result corresponding to the first sample data and the first detection result is calculated as the first loss.
[0125] Exemplarily, the model's predicted probability distribution p for the first sample data can be extracted from the first detection result, and the true result corresponding to the first sample data can be obtained, and the true result can be converted into a "one-hot" encoding form to obtain the probability distribution q of the true label, thereby inputting the predicted probability distribution p and the probability distribution q into the cross-entropy loss calculation formula for calculation to obtain the first loss.
[0126] By calculating the cross entropy information loss as the first loss, the prediction probabilities of all categories can be considered to smooth the loss value, avoiding excessive punishment caused by extreme predictions of a single category and improving the robustness of the model.
[0127] In one possible implementation, Fig.12 A schematic diagram of a flow chart for determining the second loss in the model training detection method provided in the embodiment of the present application, referring to Fig.12 As shown, the above S104 determines the second loss according to the second detection result and the third detection result, including:
[0128] S1201. Filter the third detection result by using an adaptive threshold to obtain a filtered third detection result.
[0129] Optionally, the third detection result is screened by an adaptive threshold to obtain a screened third detection result. The threshold, as a network parameter, can be obtained through network autonomous learning. This enables more accurate screening of effective detection results, can cope with different scenarios and conditions, and improves the robustness of the model.
[0130] S1202. Calculate the consistency loss between the second test result and the screened third test result as the second loss.
[0131] For example, the consistency loss can be designed based on a variety of metrics, such as IoU (Intersection over Union), L1 / L2 loss, or other custom similarity metrics.
[0132] By filtering the third detection result through an adaptive threshold to obtain the filtered third detection result, and by calculating the consistency loss between the second detection result and the filtered third detection result as the second loss, the model can be guided to learn more accurate detection results, so that the model can be better generalized to unseen data sets and scenarios, which helps the obtained target detection model to perform more stably and reliably in practical applications.
[0133] In a possible implementation, the above S105 updates the second initial detection model according to the updated first initial detection model, including:
[0134] The weight parameters in the updated first initial detection model are updated to the second initial detection model through a preset exponential moving average algorithm.
[0135] Exemplarily, the decay rate of the exponential moving average algorithm (EMA) can be initialized, and after each update of the first initial detection model, the EMA weight is calculated, and the calculated EMA weight is assigned to the second initial detection model, thereby improving the stability and generalization ability of the model, thereby obtaining better performance in the reasoning or evaluation stage.
[0136] Based on the same inventive concept, a model training detection device corresponding to the model training detection method is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned model training detection method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0137] Fig.13 A schematic diagram of a model training detection device provided in an embodiment of the present application, referring to Fig.13 As shown, the apparatus includes: a first detection module 1301, a second detection module 1302, a first loss determination module 1303, a second loss determination module 1304 and an iterative update module 1305;
[0138] The first detection module 1301 is used to input the labeled first sample data and the unlabeled second sample data into a first initial detection model to obtain a first detection result corresponding to the first sample data and a second detection result corresponding to the second sample data;
[0139] The second detection module 1302 is used to input the third sample data into the second initial detection model to obtain a third detection result corresponding to the third sample data, where the third sample data and the second sample data are obtained by performing different interference processing based on the same original data, the second initial detection model and the first initial detection model have the same network structure, and the value of at least one model parameter of the second initial detection model is different from the value of the corresponding model parameter in the first initial detection model;
[0140] A first loss determination module 1303, configured to determine a first loss according to the first sample data and the first detection result;
[0141] A second loss determination module 1304 is used to determine a second loss according to the second detection result and the third detection result, where the second loss is used to characterize the difference between the results predicted by the first initial detection model and the second initial detection model for data with the same content;
[0142] The iterative update module 1305 is used to iteratively update the first initial detection model according to the first loss and the second loss, and to update the second initial detection model according to the updated first initial detection model, and to use the first initial detection model at the end of the iteration as the target detection model, and to perform detection processing based on the target detection model.
[0143] Optionally, an acquisition module is further included, and the acquisition module is used to:
[0144] Obtain original unlabeled sample data;
[0145] The original unlabeled sample data are subjected to random noise processing and weak random noise processing respectively to obtain second sample data and third sample data.
[0146] Optionally, the first initial detection model includes a first position encoding module and a first target detection module; the first detection module 1301 is specifically used to:
[0147] Inputting the labeled first sample data and the unlabeled second sample data into a first position encoding module to generate first feature data, where the first feature data includes height features and width features in the labeled first sample data and the unlabeled second sample data;
[0148] The first feature data is input into a first target detection module to obtain a first detection result corresponding to the first sample data and a second detection result corresponding to the second sample data.
[0149] Optionally, the first position encoding module includes: a first feature extraction layer, a first height processing module, a first width processing module and a first fusion module; the first detection module 1301 is specifically used to:
[0150] Inputting the labeled first sample data and the unlabeled second sample data into the first feature extraction layer for feature extraction to obtain first feature image data;
[0151] Inputting the first feature image data into a first height processing module for height encoding to obtain height feature information corresponding to the first feature image data;
[0152] Inputting the first feature image data into a first width processing module for width encoding to obtain width feature information corresponding to the first feature image data;
[0153] The first feature image data, the height feature information and the width feature information are input into a first fusion module to obtain the first feature data.
[0154] Optionally, the first height processing module includes: a first height pooling layer, a first height encoding layer and a first height normalization layer; the first detection module 1301 is specifically used to:
[0155] Inputting the first feature image into the first height pooling layer for average pooling processing to obtain a height pooling image;
[0156] Input the height pooling image into the first height coding layer for sinusoidal height coding to obtain the encoded height feature information;
[0157] The encoded height feature information is input into the first height normalization layer for normalization processing to obtain the height feature information corresponding to the first feature image data.
[0158] Optionally, the first width processing module includes: a first width pooling layer, a first width encoding layer and a first width normalization layer; the first detection module 1301 is specifically used to:
[0159] Inputting the first feature image into the first width pooling layer for average pooling processing to obtain a width pooling image;
[0160] Input the width pooled image into the first width coding layer for sinusoidal width coding to obtain encoded width feature information;
[0161] The encoded width feature information is input into the first width normalization layer for normalization processing to obtain the width feature information corresponding to the first feature image data.
[0162] Optionally, the first loss determination module 1303 is specifically configured to:
[0163] The cross entropy information loss between the first sample data and the first detection result is calculated as the first loss.
[0164] Optionally, the second loss determination module 1304 is configured to:
[0165] Filtering the third detection result by using an adaptive threshold to obtain a filtered third detection result;
[0166] The consistency loss between the second test result and the screened third test result is calculated as the second loss.
[0167] Optionally, the iterative update module 1305 is specifically configured to:
[0168] The weight parameters in the updated first initial detection model are updated to the second initial detection model through a preset exponential moving average algorithm.
[0169] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.
[0170] The present application also provides an electronic device, such as Fig.14As shown, Fig.14 The structural diagram of the electronic device provided in the embodiment of the present application includes: a processor 1401, a memory 1402, and optionally, a bus 1403. The memory 1402 stores machine-readable instructions executable by the processor 1401 (for example, Fig.13 The device includes the first detection module 1301, the second detection module 1302, the first loss determination module 1303, the second loss determination module 1304 and the execution instructions corresponding to the iterative update module 1305, etc.), when the electronic device is running, the processor 1401 communicates with the memory 1402 through the bus 1403, and when the machine-readable instructions are executed by the processor 1401, the steps of the above-mentioned model training detection method are performed.
[0171] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned model training detection method are executed.
[0172] An embodiment of the present application also provides a computer program product, which includes a computer program code. When the computer program code is executed by a computing device, the computing device performs the steps of the above-mentioned model training detection method.
[0173] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.
[0174] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or part of the technical solution that contributes to the prior art or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk and other media that can store program code.
[0175] The above are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.
Claims
1. A model training detection method, characterized in that: include: Inputting the labeled first sample data and the unlabeled second sample data into a first initial detection model to obtain a first detection result corresponding to the first sample data and a second detection result corresponding to the second sample data; Inputting the third sample data into the second initial detection model to obtain a third detection result corresponding to the third sample data, wherein the third sample data and the second sample data are obtained by performing different interference processing based on the same original data, the second initial detection model and the first initial detection model have the same network structure, and the value of at least one model parameter of the second initial detection model is different from the value of the corresponding model parameter in the first initial detection model; Determine a first loss according to the first sample data and the first detection result; Determine a second loss according to the second detection result and the third detection result, where the second loss is used to characterize the difference between the results predicted by the first initial detection model and the second initial detection model for data with the same content; According to the first loss and the second loss, the first initial detection model is iteratively updated, and the second initial detection model is updated according to the updated first initial detection model, and the first initial detection model at the end of the iteration is used as the target detection model, and detection processing is performed based on the target detection model.
2. The model training detection method according to claim 1, characterized in that: The process of acquiring the second sample data and the third sample data includes: Obtain original unlabeled sample data; The original unlabeled sample data is subjected to random noise processing and weak random noise processing respectively to obtain second sample data and third sample data.
3. The model training detection method according to claim 1, characterized in that: The first initial detection model includes a first position encoding module and a first target detection module; The step of inputting the labeled first sample data and the unlabeled second sample data into a first initial detection model to obtain a first detection result corresponding to the first sample data and a second detection result corresponding to the second sample data includes: Inputting the labeled first sample data and the unlabeled second sample data into the first position encoding module to generate first feature data, where the first feature data includes height features and width features in the labeled first sample data and the unlabeled second sample data; The first feature data is input into the first target detection module to obtain a first detection result corresponding to the first sample data and a second detection result corresponding to the second sample data.
4. The model training detection method according to claim 3, characterized in that: The first position encoding module includes: a first feature extraction layer, a first height processing module, a first width processing module and a first fusion module; The step of inputting the labeled first sample data and the unlabeled second sample data into the first position encoding module to generate the first feature data includes: Inputting the labeled first sample data and the unlabeled second sample data into the first feature extraction layer for feature extraction to obtain first feature image data; Inputting the first feature image data into the first height processing module for height encoding to obtain height feature information corresponding to the first feature image data; Inputting the first feature image data into the first width processing module for width encoding to obtain width feature information corresponding to the first feature image data; The first feature image data, the height feature information, and the width feature information are input into the first fusion module to obtain the first feature data.
5. The model training detection method according to claim 4, characterized in that: The first height processing module includes: a first height pooling layer, a first height encoding layer and a first height normalization layer; The step of inputting the first feature image data into the first height processing module for height encoding to obtain height feature information corresponding to the first feature image data includes: Inputting the first feature image into the first height pooling layer for average pooling processing to obtain a height pooling image; Inputting the height pooling image into the first height coding layer for sinusoidal height coding to obtain coded height feature information; The encoded height feature information is input into the first height normalization layer for normalization processing to obtain the height feature information corresponding to the first feature image data.
6. The model training detection method according to claim 4, characterized in that: The first width processing module includes: a first width pooling layer, a first width encoding layer and a first width normalization layer; The step of inputting the first feature image data into the first width processing module for width encoding to obtain width feature information corresponding to the first feature image data includes: Inputting the first feature image into the first width pooling layer for average pooling processing to obtain a width pooling image; Inputting the width pooled image into the first width coding layer for sinusoidal width coding to obtain encoded width feature information; The encoded width feature information is input into the first width normalization layer for normalization processing to obtain width feature information corresponding to the first feature image data.
7. The model training detection method according to claim 1, characterized in that: The determining the first loss according to the first sample data and the first detection result includes: A cross entropy information loss between the first sample data and the first detection result is calculated as a first loss.
8. The model training detection method according to claim 1, characterized in that: The determining the second loss according to the second detection result and the third detection result includes: Filtering the third detection result by using an adaptive threshold to obtain a filtered third detection result; The consistency loss between the second test result and the screened third test result is calculated as the second loss.
9. The model training detection method according to claim 1, characterized in that: The updating of the second initial detection model according to the updated first initial detection model comprises: The weight parameters in the updated first initial detection model are updated to the second initial detection model through a preset exponential moving average algorithm.
10. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor executes the machine-readable instructions to perform the steps of the model training detection method as described in any one of claims 1 to 9.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the model training detection method as described in any one of claims 1 to 9.
12. A computer program product, characterized in that The computer program product includes computer program code. When the computer program code is executed by a computing device, the computing device executes the steps of the model training detection method described in any one of claims 1 to 9.