Method, device, storage medium and program product for visual processing and model training

By constructing an image classification network with the same backbone as the visual model and using self-supervised training with untagged real data, followed by refinement with tagged data, the accuracy of visual models is enhanced, addressing the limitations of synthetic data reliance and overfitting.

JP7579965B2Active Publication Date: 2024-11-08BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2023512344
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2022-02-25
Filing Date
2022-06-22
Publication Date
2024-11-08
Estimated Expiration
2042-06-22

AI Technical Summary

Technical Problem

Existing visual model training methods face low accuracy due to reliance on synthetic data, leading to overfitting, especially when real data is scarce and costly to tag.

Method used

Construct an image classification network with the same backbone as the visual model, perform self-supervised training using untagged real data, initialize the visual model's weights with the trained backbone network, and refine using tagged data to enhance model accuracy.

Benefits of technology

Improves visual model accuracy by leveraging untagged real data for self-supervised training and optimizing backbone network weights, enabling high-accuracy models with minimal tagged data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007579965000001
    Figure 0007579965000001
  • Figure 0007579965000002
    Figure 0007579965000002
  • Figure 0007579965000003
    Figure 0007579965000003
Patent Text Reader

Abstract

The present disclosure provides a method, an apparatus, a storage medium, and a program product for visual processing and model training. ,Tool A concrete implementation scheme is as follows: build an image classification network with the same backbone network as the visual model, use an untagged first data set to perform self-supervised training on the image classification network, initialize the weights of the backbone network of the visual model according to the weights of the backbone network of the trained image classification network to obtain a pre-trained model that is consistent with the structure of the visual model, and use a real data set in the current computer vision task scene to optimize the weights of the backbone network to be more suitable for the current computer vision task, and then use a tagged second data set to train the pre-trained model to obtain a trained visual model, which can effectively improve the accuracy of the visual model.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to fields such as computer vision and deep learning in computer technology, and in particular to a method, an apparatus, a storage medium and a program product for vision processing and model training. [Background technology]

[0002] OCR (Optical Character Recognition) is a technology that converts photo information into text information that is easier to edit and remember. At present, it is widely applied in various scenarios such as handprint recognition, bank card information recognition, mathematical formula recognition, etc. OCR also supports many downstream tasks such as subtitle translation, security monitoring, and can also be applied to other computer vision tasks such as video search, etc.

[0003] In various computer vision tasks such as target detection, image segmentation, text recognition, etc., it is generally necessary to train a visual model for the computer vision task. In a real scene, there is a large amount of real image data, but the tagging cost for the real image data is high. Currently, model training is often performed based on a large amount of synthetic data and a small amount of real data with tagged information by using the synthesis of simple data with known tagged information and a background map, and there are mainly two methods as follows. One is to synthesize a large amount of tagged data, train a visual model based on the tagged synthetic data, and then train the visual model using the real data and the tagged information to obtain a final visual model. One is to synthesize a large amount of tagged data, mix the tagged synthetic data and real data as training data, train a visual model, and obtain a final visual model. However, when model training is performed based on synthetic data, the synthetic data is too simple, which causes overfitting, and the accuracy of the model obtained by training is often low. Summary of the Invention [Problem to be solved by the invention]

[0004] A method, apparatus, storage medium and program product for visual processing and model training are provided. [Means for solving the problem]

[0005] According to a first aspect of the present disclosure, there is provided a method for visual model training, comprising: constructing an image classification network that has the same backbone network as the visual model; performing self-supervised training on the image classification network using an untagged first dataset to obtain a trained image classification network; initializing weights of a backbone network of the visual model according to weights of a backbone network of a trained image classification network to obtain a pre-trained model; and training the pre-training model using a tagged second dataset to obtain a trained visual model.

[0006] According to a second aspect of the present disclosure, there is provided a visual processing method, comprising: obtaining an image to be processed; inputting the image to be processed into a visual model, and performing computer visual processing on the image to be processed using the visual model obtained by training using the method according to the first aspect above to obtain a processing result; and outputting the processing result.

[0007] According to a third aspect of the present disclosure, there is provided an apparatus for visual model training, comprising: a network construction module for constructing an image classification network having the same backbone network as the visual model; a first training module for performing self-supervised training on the image classification network using an untagged first data set to obtain a trained image classification network; a pre-training model determination module for initializing weights of a backbone network of the visual model according to weights of a backbone network of a trained image classification network to obtain a pre-training model; and a second training module for training the pre-training model using a tagged second dataset to obtain a trained visual model.

[0008] According to a fourth aspect of the present disclosure, there is provided a visual processing device, comprising: an image acquisition module for acquiring an image to be processed; a visual processing module for inputting the image to be processed into a visual model and performing computer visual processing on the image to be processed using the visual model generated by the device according to the third aspect of the present invention to obtain a processing result; and a result output module for outputting the processing result.

[0009] According to a fifth aspect of the present disclosure, there is provided an electronic device, comprising: At least one processor; and a memory communicatively coupled to the at least one processor, wherein: The memory stores instructions executable by the at least one processor, which when executed by the at least one processor can cause the at least one processor to perform a method according to the first or second aspect.

[0010] According to a sixth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions are used to cause the computer to perform a method according to the first or second aspect.

[0011] According to a seventh aspect of the present disclosure, there is provided a computer program product, the computer program product including a computer program stored in a readable storage medium, wherein at least one processor of an electronic device reads the computer program from the readable storage medium, and when the at least one processor executes the computer program, causes the electronic device to perform a method according to the first or second aspect.

[0012] The techniques disclosed herein improve the accuracy of the visual model.

[0013] It should be understood that the contents of this specification are not intended to identify key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure can be readily understood from the following specification. [Brief description of the drawings]

[0014] The drawings are used for a better understanding of the present invention and are not intended to limit the present disclosure. [Figure 1] 2 is a flowchart of a method for training a visual model according to a first embodiment of the present disclosure. [Diagram 2] 11 is a flowchart of a method for training a visual model according to a second embodiment of the present disclosure. [Diagram 3] 13 is a flowchart of a visual processing method according to a third embodiment of the present disclosure. [Figure 4] FIG. 11 is a schematic structural diagram of an apparatus for training a visual model according to a fourth embodiment of the present disclosure; [Diagram 5] FIG. 13 is a schematic structural diagram of an apparatus for training a visual model according to a fifth embodiment of the present disclosure; [Figure 6] FIG. 13 is a schematic structural diagram of a visual processing device according to a sixth embodiment of the present disclosure. [Figure 7] FIG. 1 is a block diagram of an electronic device for implementing a visual model training method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings. For ease of understanding, various details of the embodiments of the present disclosure are included in the description, which should be regarded as merely exemplary. Therefore, those skilled in the art will understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, the following description will omit the description of well-known functions and structures.

[0016] The terms "first", "second", and "third" in the present disclosure are merely for descriptive purposes and should not be understood to express or imply relative importance or the number of technical features shown. In the description of each embodiment below, "plurality" means two or more unless otherwise clearly defined and limited.

[0017] OCR (Optical Character Recognition) is a technology that converts photo information into text information that is easier to edit and remember. At present, it is widely applied in various scenarios such as handprint recognition, bank card information recognition, mathematical formula recognition, etc. OCR also supports many downstream tasks such as subtitle translation, security monitoring, and can also be applied to other computer vision tasks such as video search, etc.

[0018] In various computer vision tasks, such as target detection, image segmentation, text recognition, etc., it is generally necessary to train a vision model for the computer vision task.

[0019] In real scenes, there is a large amount of real image data, but the tagging cost for real image data is high. In order to improve the training effect of the model, a pre-training model in image classification can be used to accelerate the convergence process of the model and improve the accuracy of the model. However, in some computer vision tasks, such as text recognition tasks based on OCR, the image data of the text recognition task scene is very different from the dataset used for the pre-training model of image classification, so the structure difference between the text recognition model and the pre-training model is large, and if the pre-training model based on image classification is directly loaded and used to train the text recognition model, it cannot bring about the accuracy benefit of the text recognition model. Although untagged text recognition data is very common and easy to obtain, untagged data cannot be directly used to train the text recognition model.

[0020] Currently, the training of visual models is often based on a large amount of synthetic data and a small amount of real data with tagged information, using the synthesis of simple data with known tagged information and background map. There are mainly two methods: one is to synthesize a large amount of tagged data, train a visual model based on the tagged synthetic data, and then train the visual model using real data and tagged information to obtain the final visual model; one is to synthesize a large amount of tagged data, mix the tagged synthetic data and real data as training data, train a visual model, and obtain the final visual model.

[0021] However, when training a model based on synthetic data, the synthetic data is often too simple, leading to overfitting and resulting in low accuracy of the trained model.

[0022] The present disclosure relates to a visual model that is applied to computer vision, deep learning, and the like in computer technology to improve the accuracy of the visual model. Processing and A method, apparatus, storage medium and program product for model training are provided.

[0023] The method for training a visual model provided in the present disclosure includes constructing an image classification network based on a visual model, making the image classification network have the same backbone network as the visual model, performing self-supervised training on the image classification network using an untagged first dataset to optimize the backbone network of the image classification network, initializing weights of the backbone network of the visual model according to weights of the backbone network of the trained image classification network, to obtain a pre-trained model having a backbone network consistent with that of the visual model, and training the pre-trained model based on an actual untagged dataset, since it is easy to obtain an untagged dataset in a current computer vision task scene, makes the pre-trained model more suitable for training a visual model in a current computer vision task, and improves the effectiveness of the pre-trained model, and then training the pre-trained model using a tagged second dataset to obtain a trained visual model, thereby improving the accuracy of the visual model.

[0024] It is also possible to train highly accurate visual models using a relatively small amount of tagged data.

[0025] The visual model obtained by training can be applied to various computer vision task scenes to perform corresponding computer vision processing, for example, the visual model can be an OCR text recognition model to perform text recognition processing on an input image to obtain text information contained in the input image.

[0026] The following specific examples will be used to describe in detail the technical solution of the present application and how it solves the above technical problems. Some of the following specific examples can be combined with each other, and in some examples, detailed descriptions of the same or similar concepts or processes will be omitted. The following examples will be described with reference to the drawings.

[0027] 1 is a flowchart of a visual model training method according to a first embodiment of the present disclosure. The visual model training method provided in this embodiment is specifically applicable to an electronic device for realizing training of a visual model.

[0028] As shown in FIG. 1, the method specifically includes steps S101 to S104.

[0029] In step S101, an image classification network is constructed that has the same backbone network as the visual model.

[0030] Here, the image classification network includes a backbone network and a decoding module, and the backbone network is the encoding module in the network and also the module for feature extraction.

[0031] In this embodiment, an image classification network is constructed based on a visual model, and the image classification network has the same backbone network as the visual model, that is, the image classification network corresponds to the encoding module for feature extraction in the visual model, but is different from the decoding module.

[0032] In step S102, the image classification network is subjected to self-supervised training using the untagged first data set to obtain a trained image classification network.

[0033] Here, the first dataset is an untagged dataset that includes real image data of a current computer vision task scene, and may further include synthetic data.

[0034] In this embodiment, self-supervised training is performed on the image classification network based on a first untagged dataset in the current computer vision task scene, and the weights of the backbone network of the image classification network are optimized to make the backbone network more suitable for the current computer vision task, so that when applied to the computer vision task, feature extraction can be better performed, and there is no need to use a tagged dataset.

[0035] In step S103, the weights of the backbone network of the visual model are initialized according to the weights of the backbone network of the trained image classification network to obtain a pre-trained model.

[0036] untagged first data After completing the self-supervised training of the image classification network using the set, the weights of the backbone network of the trained image classification network are used as the initial weights of the backbone network of the visual model to initialize the visual model, thereby obtaining a pre-trained model that is consistent with the structure of the visual model, and then using the real dataset of the current computer vision task scene, In the preliminary training model Optimize the weights of the backbone network.

[0037] In step S104, the pre-training model is trained using the tagged second dataset to obtain a trained visual model.

[0038] Here, the second dataset is a tagged dataset, which contains real image data and tagged information on the current computer vision task scene.

[0039] The pre-training model obtained in step S103 is consistent with the structure of the vision model and is trained using a real dataset of the current computer vision task scene. In the preliminary training modelBased on the pre-training model, a second dataset tagged with the current computer vision task scene is used to train the model to optimize the weights of the backbone network, and the final visual model can be obtained, which effectively improves the accuracy of the visual model. Even if a relatively small amount of tagged data is used, a highly accurate visual model can be obtained through training.

[0040] In this embodiment, an image classification network with the same backbone network as the visual model is constructed, and a first untagged dataset is used to perform self-supervised training on the image classification network to obtain a trained image classification network, and the weights of the backbone network of the visual model are initialized according to the weights of the backbone network of the trained image classification network to obtain a pre-trained model that is consistent with the structure of the visual model, and a real dataset in the current computer vision task scene is used to In the preliminary training model The weights of the backbone network can be optimized to make it more suitable for the current computer vision task, and then the pre-training model can be trained using the tagged second dataset to obtain a trained vision model, which can effectively improve the accuracy of the vision model.

[0041] 2 is a flowchart of a visual model training method according to a second embodiment of the present disclosure. In addition to the above first embodiment, in this embodiment, the step of performing self-supervised training on an image classification network using an untagged first dataset to obtain a trained image classification network includes the steps of: obtaining an untagged first dataset including a plurality of sets of sample images and orientation information of each sample image, where each set of sample images includes a first sample image and a second sample image obtained by rotating the first sample image by a preset angle; extracting image features of each sample image in the first dataset by the image classification network, and determining an orientation prediction result of each sample image according to the image features; calculating a first loss according to image features of two sample images in the same set that have orientation information different by 180 degrees; and calculating the actual orientation of each sample image. information and calculating a second loss according to the direction prediction result; and adjusting weights of a backbone network of the image classification network according to the first loss and the second loss to realize self-supervised training of the image classification network using an untagged dataset; In the preliminary training model By optimizing the weights of the backbone network, the backbone network can be made more suitable for feature extraction in computer vision tasks, thereby improving the accuracy of the final visual model.

[0042] As shown in FIG. 2, the method specifically includes steps S201 to S209.

[0043] In step S201, an image classification network is constructed that has the same backbone network as the visual model.

[0044] Here, the image classification network includes a backbone network and a decoding module, and the backbone network is the encoding module in the network and also the module for feature extraction.

[0045] In this embodiment, an image classification network is constructed based on a visual model, and the image classification network has the same backbone network as the visual model, that is, the image classification network is consistent with the encoding module for feature extraction in the visual model, but the decoding module is different.

[0046] Illustratively, take the visual model as an example of a text recognition model, which can be realized using CRNN (Convolutional Recurrent Neural Network), RARE (Robust text recognizer with Automatic Rectification), NRTR, etc. The backbone network of the image classification network is used to realize the process of feature extraction in CRNN, RARE, and NRTR. Here, the general term CRNN is mainly used to recognize end-to-end indefinite-length text sequences, so it does not require cutting individual characters first, and converts text recognition into a time-series-dependent sequence learning problem, that is, image-based sequence recognition. CRNN mainly includes a convolution layer, a circulation layer, and a transcription layer, and finally realizes predicting an indefinite-length sequence using a fixed-length input.

[0047] The image classification network decoding module is used to obtain the decoding and classification results according to the features extracted by the backbone network.

[0048] In step S202, an untagged first dataset is obtained, which includes a plurality of sets of sample images and orientation information for each sample image, where each set of sample images includes a first sample image and a second sample image obtained by rotating the first sample image by a preset angle.

[0049] Here, the first dataset is an untagged dataset that includes real image data of a current computer vision task scene, and may further include synthetic data.

[0050] In real computer vision task scenes, it is easy to obtain a large amount of real image data, reducing the use of synthetic data and eliminating the dependency on synthetic data, so synthetic data may not be used.

[0051] In this embodiment, there is no need to acquire a large amount of real image data in the current computer vision task scene to generate a first dataset and perform tagging on the first dataset.

[0052] Specifically, this step includes the steps of: This can be achieved by obtaining a first sample image without a tag, determining the orientation information of the first sample image to be 0 degrees, rotating the first sample image by a preset angle to obtain a second sample image, and determining the orientation information of the second sample image to be the preset angle.

[0053] Here, the preset angle includes one or more different angles, and can be set according to the actual application scene, and is not particularly limited here.

[0054] Optionally, the preset angles may include only 180 degrees.

[0055] Optionally, the preset angles may include 180 degrees and at least one angle other than 180 degrees. For example, the preset angles may include 90 degrees and 180 degrees.

[0056] Optionally, the preset angles may include a first angle and a second angle, where the difference between the first angle and the second angle is 180 degrees. For example, the preset angles may include 90 degrees and 270 degrees.

[0057] Selectably, the preset angles may include a first angle, a second angle, and at least one third angle, where the difference between the first angle and the second angle is 180 degrees. For example, the preset angles may include 90 degrees (first angle), 270 degrees (second angle), and 180 degrees (third angle).

[0058] In this embodiment, the orientation information of the first sample image is set to 0 degree, and the orientation information of the second sample image obtained by rotating at a preset angle is set to the corresponding rotation angle.

[0059] As an example, the first sample image is rotated by 90 degrees, 180 degrees, and 270 degrees, respectively, and the orientation information of the three obtained second sample images is 90 degrees, 180 degrees, and 270 degrees, respectively, and the orientation information of the first sample image is 0 degrees.

[0060] A first sample image and a second sample image obtained by rotating the first sample image are taken as a set of sample images, a first loss is determined based on two sample images in the same set whose directional information difference is 180 degrees, and a second loss is determined based on the directional prediction result and actual directional information of the same sample image, so as to enrich the loss function for self-supervised training and improve the training effect of the pre-training model.

[0061] In addition, by setting multiple preset angles, in addition to the two sample images with a directional information difference of 180 degrees in the same set of sample images, sample images with other directional information may be further included, which can further enrich the sample data and improve the training effect of the pre-training model.

[0062] Furthermore, the step of acquiring the tagless first sample image specifically includes: This can be achieved by obtaining an original image including at least one of a synthetic image and a real image, performing pre-processing on the original image to obtain a sample image that meets model training requirements, and performing random data augmentation on the sample image to obtain a first sample image.

[0063] Here, each original image is a real or synthetic image of the current computer vision task scene.

[0064] During the model training process, there is a requirement for the dimensions of the sample images of the input model, whether they contain information such as face, text, and other specific content, etc. By performing preprocessing on each original image, the processed sample images can meet the training requirements of the model, so that the sample images can be used to train the image classification model to improve the effect of training the image classification model.

[0065] Optionally, applying a visual model to a text recognition scene and pre-processing the original image to obtain a sample image that satisfies the model training requirements, The method includes performing text detection on the original image to extract an image of an area containing text information, and performing image correction on the image of the area containing text information to obtain a sample image that meets the model training requirements, so as to ensure that the sample image is used to train the image classification model to improve the effect of training the image classification model.

[0066] For example, a text detection model is used to perform text detection on an original image, and an image of an area containing text information is extracted to obtain a text detection result. The text detection result may be a polygon, and an affine transformation method is used to perform image correction on the text detection result to obtain a sample image that meets the model training requirements, and generally requires that the frame of the sample image is a rectangular frame.

[0067] In addition, by performing random data augmentation on the sample images to obtain the first sample images, the number of sample images can be significantly increased to make the first dataset richer.

[0068] Furthermore, step S202 may be executed after step S201, or step S202 may be executed before step S201, and the order of execution of S202 and S201 is not specifically limited here.

[0069] The number of types of image classification models is set to be correlated with the number of preset angles, and the number of types of image classification models is equal to the number of preset angles plus 1. The image classification models are used to perform directional information classification on sample images and predict directional information corresponding to the sample images.

[0070] For example, the preset angles include 90 degrees, 180 degrees, and 270 degrees, and the number of types of the image classification model is set to 4. The image classification model is used to predict whether the directional information of the input sample image is 0 degrees, 90 degrees, 180 degrees, or 270 degrees.

[0071] After obtaining the first untagged data set, steps S203 to S207 are cyclically executed to perform self-supervised iterative training on the image classification model based on the first untagged data set, and the iteration is stopped when an iteration stopping condition is met.

[0072] In step S203, the image features of each sample image in the first data set are extracted by an image classification network, and a direction prediction result of each sample image is determined according to the image features.

[0073] In this step, the sample images in the first dataset are input into an image classification network, and the image features of the sample images are extracted by the backbone network of the image classification network. The decoding module of the image classification network predicts the orientation information of the sample images according to the image features of the sample images to obtain the orientation prediction result of the sample images.

[0074] Optionally, a normalization process is performed on the sample images before they are input into the image classification network.

[0075] In this embodiment, two loss functions are used in the self-supervised training of the image classification network. One loss function is used to reduce the image feature of two sample images in the same set that have 180 degrees different orientation information. Based on The first loss is used to calculate the real orientation of each sample image. information and direction prediction results Based on It is used to calculate the second loss, which can enrich the loss function for supervised training and improve the effectiveness of model training.

[0076] In step S204, a first loss is calculated according to image features of two sample images in the same set whose orientation information differs by 180 degrees.

[0077] In an alternative embodiment, the preset angle includes at least 180 degrees. For each set of sample images, the image features of the first sample image are obtained after rotating them by 180 degrees. image This may be achieved by a scheme of calculating a first loss depending on the difference between the feature and the image feature of a second sample image obtained after rotating the first sample image by 180 degrees.

[0078] If the first loss calculated in this manner can accurately reflect the difference between the image features of two sample images in the same set whose orientation information differs by 180 degrees, then self-supervised training is performed on the image classification model based on the first loss, so that the backbone network of the image classification model can learn the image rotation angle, so that the feature extraction module can better extract image features, further improve the robustness of the image classification model, and ultimately improve the accuracy of the obtained visual model.

[0079] In another optional embodiment, the preset angles include at least a first angle and a second angle, where the second angle is equal to the first angle plus 180 degrees, and the first angle is not 0 degrees, e.g., the first angle is 90 degrees and the second angle is 270 degrees.

[0080] Specifically, this step includes the following steps: The image features of the sample image in the same set whose orientation information is a first angle are obtained after rotating the image features by 180 degrees. image This may be achieved by a manner of calculating a first loss according to a difference between the feature and an image feature of a sample image whose directional information is a second angle.

[0081] If the first loss calculated by this method can also accurately reflect the difference between image features of two sample images in the same set whose orientation information differs by 180 degrees, then self-supervised training is performed on the image classification model based on the first loss, so that the backbone network of the image classification model can learn the image rotation angle, and the feature extraction module can better extract image features, thereby further improving the robustness of the image classification model, and ultimately improving the accuracy of the obtained visual model.

[0082] Illustratively, according to the image features of the sample images after rotating the same sample image by 90 degrees and 270 degrees, respectively, the image feature of the sample image after rotating by 90 degrees is a first feature, the image feature of the sample image after rotating by 270 degrees is a second feature, and the image feature obtained after rotating the first feature by 180 degrees is a second feature. image A first loss is calculated according to the difference between the feature and the second feature.

[0083] Optionally, if the preset angle includes not only 180 degrees but also a first angle and a second angle, in this step, the first loss can be calculated by combining the above two optional embodiments.

[0084] Optionally, the first loss may be an L2 loss. Alternatively, the first loss may be an L1 loss. Or, other similar The loss may be calculated using a loss function having the effect of the above, and the present embodiment is not specifically limited thereto.

[0085] In step S205, the actual orientation of each sample image information and a second loss is calculated according to the directional prediction result.

[0086] In this step, the actual orientation of each sample image information A second loss is calculated according to the difference between the first and second directional prediction results.

[0087] Optionally, the second loss may be calculated using a cross-entropy loss function, or other loss functions commonly used in classification models to measure the quality of the classification result, and this embodiment is not specifically limited thereto.

[0088] In this embodiment, steps S204 and S205 may be executed in parallel, or may be executed sequentially in any order.

[0089] In step S206, weights of a backbone network of the image classification network are adjusted according to the first loss and the second loss.

[0090] After calculating the first loss and the second loss, the weights of the backbone network of the image classification network are adjusted according to the first loss and the second loss to optimize the weights of the backbone network.

[0091] Optionally, the sum of the first loss and the second loss is calculated as the final loss, and the weights of the backbone network of the image classification network are adjusted according to the final loss, and the two different loss functions are combined to perform self-supervised training on the image classification network to improve the robustness and effectiveness of the image classification network.

[0092] In step S207, it is determined whether or not the iteration end condition is satisfied.

[0093] After completing one iterative training of the image classification network through the above steps S203 to S206, it is determined whether or not the iteration end condition is satisfied.

[0094] When the iteration end condition is met, the iteration is stopped, and the image classification model with the highest orientation classification accuracy is determined as the trained image classification model, and step S208 is then executed.

[0095] If the iteration end condition is not met, step S203 is continued to be executed to start the next iteration.

[0096] Here, the iteration stopping conditions may include at least one of the following: the number of iterations reaches a preset number threshold, the final loss is less than a loss threshold, and the change in weight value between two iterations is already very small.

[0097] In step S208, the weights of the backbone network of the visual model are initialized according to the weights of the backbone network of the trained image classification network to obtain a pre-trained model.

[0098] untagged first data After completing the self-supervised training of the image classification network using the set, the weights of the backbone network of the trained image classification network are used as the initial weights of the backbone network of the visual model to initialize the visual model, thereby obtaining a pre-trained model that is consistent with the structure of the visual model, and then using the real dataset of the current computer vision task scene, In the preliminary training model Optimize the weights of the backbone network.

[0099] In step S209, the pre-training model is trained using the tagged second dataset to obtain a trained visual model.

[0100] Here, the second dataset is a tagged dataset, which contains real image data and tagged information on the current computer vision task scene.

[0101] The pre-trained model is consistent with the structure of the visual model, and the weights of the backbone network of the pre-trained model are optimized using a real dataset of the current computer vision task scene. A second dataset tagged with the current computer vision task scene is used to re-train the pre-trained model to obtain a final visual model, which can effectively improve the accuracy of the visual model. Even if a relatively small amount of tagged data is used, a highly accurate visual model can be obtained through training.

[0102] In this embodiment, an image classification network having the same backbone network as the visual model is constructed, and a first untagged data set is used to perform self-supervised training on the image classification network to obtain a trained image classification network; the untagged data is fully utilized to train the backbone network for feature extraction; and two loss functions are used in the self-supervised training of the image classification network to obtain image features of two sample images in the same set that have 180 degree different orientation information. Depending on First, calculate the loss and the actual orientation of each sample image. information and calculate the second loss of the direction prediction result, and optimize the weights of the backbone network based on the first loss and the second loss, while considering the classification result loss and the loss due to image feature alignment to enrich the loss function for supervised training, so as to improve the robustness of the pre-trained model and the accuracy of feature extraction. On this basis, the weights of the backbone network of the visual model are initialized according to the weights of the backbone network of the trained image classification network to obtain a pre-trained model that is consistent with the structure of the visual model, and a real dataset in the current computer vision task scene is used to In the preliminary training model The weights of the backbone network can be optimized to make it more suitable for the current computer vision task, and then the pre-training model can be trained using the tagged second dataset to obtain a trained vision model, which can effectively improve the accuracy of the vision model.

[0103] 3 is a flowchart of a visual processing method according to a third embodiment of the present disclosure. The visual processing method provided in this embodiment can be specifically applied to an electronic device for implementing visual processing on an image using a trained visual model.

[0104] As shown in FIG. 3, the method specifically includes steps S301 to S303.

[0105] In step S301, an image to be processed is acquired.

[0106] In step S302, the image to be processed is input to a visual model, and computer visual processing is performed on the image to be processed by the visual model to obtain a processing result.

[0107] Here, the visual model is obtained by training using the method of the first or second embodiment described above, and a detailed description of the specific training process is omitted here.

[0108] Exemplarily, the visual model may be a text recognition model, in which the image to be processed is input to the trained text recognition model, and the text recognition model performs text recognition processing on the input image, and the obtained processing result is the text information contained in the input image.

[0109] In step S303, the processing result is output.

[0110] In this embodiment, visual processing is performed on an input image using a trained visual model, and visual processing results are obtained and output.

[0111] 4 is a schematic structural diagram of a visual model training apparatus according to a fourth embodiment of the present disclosure. The visual model training apparatus provided in the embodiment of the present disclosure can execute the process flow provided in the embodiment of the visual model training method. As shown in FIG. 4, the visual model training apparatus 40 specifically includes: a network construction module 41 for constructing an image classification network having the same backbone network as the visual model; a first training module 42 for performing self-supervised training on the image classification network using the first untagged data set to obtain a trained image classification network; a pre-training model determination module 43 for initializing weights of the backbone network of the visual model according to weights of the backbone network of the trained image classification network to obtain a pre-training model; and a second training module 44 for training the pre-training model using the tagged second dataset to obtain a trained visual model.

[0112] The apparatus provided in the embodiments of the present disclosure can be specifically used to execute the embodiments of the method provided in the first embodiment above, and detailed descriptions of the specific functions realized will be omitted here.

[0113] In this embodiment, an image classification network with the same backbone network as the visual model is constructed, and a first untagged dataset is used to perform self-supervised training on the image classification network to obtain a trained image classification network, and the weights of the backbone network of the visual model are initialized according to the weights of the backbone network of the trained image classification network to obtain a pre-trained model that is consistent with the structure of the visual model, and a real dataset in the current computer vision task scene is used to In the preliminary training model The weights of the backbone network can be optimized to make it more suitable for the current computer vision task, and then the pre-training model can be trained using the tagged second dataset to obtain a trained vision model, which can effectively improve the accuracy of the vision model.

[0114] 5 is a schematic structural diagram of a visual model training apparatus according to a fifth embodiment of the present disclosure. The visual model training apparatus provided in the embodiment of the present disclosure can execute the process flow provided in the embodiment of the visual model training method. As shown in FIG. 5, the visual model training apparatus 50 specifically includes: a network construction module 51 for constructing an image classification network having the same backbone network as the visual model; a first training module 52 for performing self-supervised training on the image classification network using the first untagged data set to obtain a trained image classification network; a pre-training model determination module 53 for initializing weights of the backbone network of the visual model according to weights of the backbone network of the trained image classification network to obtain a pre-training model; and a second training module 54 for training the pre-training model using the tagged second dataset to obtain a trained visual model.

[0115] Optionally, as shown in FIG. 5, a first training module 52 includes: an untagged dataset acquiring unit 521, used for acquiring an untagged first dataset including a plurality of sets of sample images and orientation information of each sample image, where each set of sample images includes a first sample image and a second sample image obtained by rotating the first sample image by a preset angle; A classification prediction unit 522 for extracting image features of each sample image in the first dataset by an image classification network, and determining a direction prediction result of each sample image according to the image features; a second loss computing unit 523 for calculating a first loss according to image features of two sample images in the same set of sample images whose orientation information differs by 180 degrees, and calculating a second loss according to the actual orientation information and the orientation prediction result of each sample image; and a weight optimization unit 524 for adjusting weights of a backbone network of the image classification network according to the first loss and the second loss.

[0116] Selectively, untagged dataset acquisition units can be a sample image acquiring subunit for acquiring a first sample image without a tag and determining orientation information of the first sample image to be 0 degrees; and an image rotation subunit for rotating the first sample image by a preset angle to obtain a second sample image, and determining orientation information of the second sample image to the preset angle.

[0117] Optionally, the preset angles include at least 180 degrees. The loss computing unit comprises: For each set of sample images, the image features of the first sample image are obtained after rotating them by 180 degrees. image a first loss calculation subunit for calculating a first loss according to a difference between the feature and an image feature of a second sample image obtained after rotating the first sample image by 180 degrees.

[0118] Optionally, the preset angles include at least a first angle and a second angle, the second angle being equal to the first angle plus 180 degrees, and the first angle being not equal to 0 degrees. The image features of the sample image in the same set whose orientation information is a first angle are obtained after rotating the image features by 180 degrees. image and a second loss calculation subunit for calculating a first loss according to a difference between the feature and an image feature of the sample image, the orientation information of which is a second angle.

[0119] Optionally, the sample image acquisition subunit further comprises: An original image, including at least one of a synthetic image and a real image, is obtained, pre-processing is performed on the original image to obtain a sample image that meets a model training requirement, and random data augmentation is performed on the sample image to obtain a first sample image.

[0120] Optionally, the sample image acquisition subunit further comprises: It is used to perform text detection on the original image to extract images of regions with text information, and perform image correction on the images of regions with text information to obtain sample images that meet the model training requirements.

[0121] Optionally, the weight optimization unit further comprises: The sum of the first loss and the second loss is calculated as the final loss, which is used to adjust the weights of the backbone network of the image classification network according to the final loss.

[0122] The apparatus provided in the embodiment of the present disclosure can be specifically used to execute the embodiment of the method provided in the second embodiment above, and detailed descriptions of the specific functions realized will be omitted here.

[0123] In this embodiment, an image classification network having the same backbone network as the visual model is constructed, and a first untagged data set is used to perform self-supervised training on the image classification network to obtain a trained image classification network; the untagged data is fully utilized to train the backbone network for feature extraction; and two loss functions are used in the self-supervised training of the image classification network to obtain image features of two sample images in the same set that have 180 degree different orientation information. Depending on First, calculate the loss and the actual orientation of each sample image. information and calculate the second loss of the direction prediction result, and optimize the weights of the backbone network based on the first loss and the second loss, and consider the classification result loss and the loss due to the alignment of the image features to enrich the loss function for supervised training, thereby improving the robustness of the pre-trained model and the accuracy of feature extraction. On this basis, the weights of the backbone network of the visual model are initialized according to the weights of the backbone network of the trained image classification network to obtain a pre-trained model that is consistent with the structure of the visual model, and the weights of the backbone network are optimized using a real dataset in the current computer vision task scene to make it more suitable for the current computer vision task, and then the pre-trained model is trained using a tagged second dataset to obtain a trained visual model, which effectively improves the accuracy of the visual model.

[0124] 6 is a schematic structural diagram of a visual processing device according to a sixth embodiment of the present disclosure. The visual processing device provided in the embodiment of the present disclosure can execute the processing flow provided in the embodiment of the visual processing method. As shown in FIG. 6, the visual processing device 60 includes an image acquisition module 61, a visual processing module 62 and a result output module 63.

[0125] Specifically, the image acquisition module 61 is used to acquire an image to be processed.

[0126] The visual processing module 62 is used to input the image to be processed into a visual model generated by the device of the fourth or fifth embodiment described above, and perform computer visual processing on the image to be processed using the visual model to obtain a processing result.

[0127] The result output module 63 is used to output the processing result.

[0128] The apparatus provided in the embodiment of the present disclosure can specifically execute an embodiment of the method provided in the third embodiment above, and detailed descriptions of the specific functions realized will be omitted here.

[0129] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure and other processing of users' personal information shall all comply with the provisions of relevant laws and regulations and shall not violate public order or morals.

[0130] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0131] According to an embodiment of the present disclosure, the present disclosure further provides a computer program product, the computer program product including a computer program stored in a readable storage medium, at least one processor of the electronic device is capable of reading the computer program from the readable storage medium, and when the at least one processor executes the computer program, the electronic device performs the means provided in any of the above embodiments.

[0132] 7 illustrates a schematic block diagram of an example electronic device 700 in which embodiments of the present disclosure may be implemented. The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may be, for example, a personal digital assistant (PDAS), a personal digital assistant (DMA), a personal digital assistant (DMB), a personal digital assistant (DMD ... assistant The components shown herein, their connections and relationships, and their functions, are merely examples and are not intended to limit the practice of the present disclosure as described and / or claimed herein.

[0133] 7, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 can further store various programs and data necessary for the operation of the device 700. The computing unit 701, the ROM 702 and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0134] Several components of the device 700 are connected to the I / O interface 705, including an input unit 706 such as a keyboard, a mouse, etc., an output unit 707 such as various types of displays, speakers, etc., a storage unit 708 such as a magnetic disk, an optical disk, etc., and a communication unit 709 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 enables the device 700 to exchange information / data with other devices, such as via a computer network such as the Internet and / or various telecommunication networks.

[0135] The computing unit 701 may be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various application-specific artificial intelligence (AI) computing chips, various computing units that execute machine learning model algorithms, and a digital signal processor (DSP), and any suitable processor, controller, microcomputer, etc. The computing unit 701 executes each of the above methods and processes, such as the visual model training method or the visual processing method. For example, in some embodiments, the visual model training method or the visual processing method may be implemented as a computer software program tangibly included in a machine-readable medium, such as the storage unit 708. In some embodiments, some or all of the computer program may be loaded and / or installed in the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the visual model training method or the visual processing method described above may be executed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the method of visual model training in any other suitable manner (eg, via firmware).

[0136] Various embodiments of the systems and techniques described herein may be implemented in digital electronic circuitry systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments include being embodied in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which may be an application specific or general purpose programmable processor, and that can receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device.

[0137] The program codes for implementing the methods of the present disclosure can be written using any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable data processing apparatus, so that when the program codes are executed by the processor or controller, the functions / operations defined in the flowcharts and / or block diagrams are performed. The program codes may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0138] In the context of the present disclosure, a machine-readable medium may be a tangible medium, and may include or store a program provided for use with or used in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium includes, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. Further specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), optical fibers, portable compact disk read-only memories (CDs), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0139] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) that allow the user to provide input to the computer. Other types of devices can provide interaction with a user, for example, feedback provided to the user can be any form of sensing feedback (e.g., visual feedback, auditory feedback, or haptic feedback) and can receive input from the user in any form (including acoustic, speech, or haptic input).

[0140] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such back-end, middleware, or front-end components. The components of the system can be connected to each other via any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0141] The computer system may include a client and a server. The client and the server are generally remote from each other and usually interact with each other via a communication network. The client-server relationship is generated by a computer program that runs on a corresponding computer and has a client-server relationship with each other. The server may be a cloud server, also called a cloud computing server or a cloud host, which is one of the host products in a cloud computing service system to solve the shortcomings of traditional physical hosts and VPS services ("Virtual Private Server", abbreviated as "VPS"), which are difficult to manage and have weak service scalability. The server may be a server of a distributed system or a server combined with a blockchain.

[0142] It should be understood that steps can be rearranged, added, or removed using the various forms of flow shown above. For example, each step described in the present disclosure can be performed in parallel, sequentially, or in a different order, and the present specification is not limited thereto, as long as the desired results of the technical solution disclosed in the present disclosure can be achieved.

[0143] The above specific embodiments do not limit the scope of protection of the present disclosure. Those skilled in the art should understand that various modifications, combinations, subcombinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

[0144] This disclosure claims priority to a Chinese patent application filed with the China Patent Office on February 25, 2022, bearing application number 202210181868.0 and entitled "Method, apparatus, storage medium and program product for visual processing and model training," the entire contents of which are incorporated herein by reference.

Claims

1. 1. A method for visual model training, comprising: Building an image classification network based on a visual model, the image classification network including a backbone network and a decoding module, the backbone network of the image classification network is the same as the backbone network for feature extraction in the visual model, and the decoding module is different; performing self-supervised training on the image classification network using an untagged first dataset to obtain a trained image classification network; initializing weights of a backbone network of the visual model according to weights of a backbone network of a trained image classification network to obtain a pre-trained model; training the pre-training model using a tagged second dataset to obtain a trained visual model for performing computer vision processing to obtain a processing result, the visual model being an optical character recognition (OCR) text recognition model for performing text recognition processing on an input image to obtain text information contained in the input image, the second dataset including real image data and tagged information in a current computer vision task scene; performing self-supervised training on the image classification network using an untagged first data set to obtain a trained image classification network; acquiring the untagged first data set including a plurality of sets of sample images and orientation information for each sample image, where each set of sample images includes a first sample image and a second sample image obtained by rotating the first sample image by a preset angle; extracting image features of each sample image in the first dataset using the image classification network, and determining a direction prediction result for each sample image according to the image features; Calculating a first loss according to image features of two sample images in the same set of sample images that have directional information different by 180 degrees, and calculating a second loss according to actual directional information and directional prediction results of each sample image; adjusting weights of a backbone network of the image classification network in response to the first loss and the second loss. A method for training visual models.

2. The step of obtaining the untagged first data set comprises: Obtaining an untagged first sample image and determining orientation information of the first sample image as 0 degrees; rotating the first sample image by the preset angle to obtain a second sample image, and determining orientation information of the second sample image at the preset angle. The method of claim 1.

3. the preset angles include at least 180 degrees; The step of calculating a first loss according to image features of two sample images in the same set, the two sample images having orientation information different by 180 degrees, includes: calculating a first loss for each set of sample images according to a difference between an image feature obtained after rotating an image feature of a first sample image by 180 degrees and an image feature of a second sample image obtained after rotating the first sample image by 180 degrees; The method of claim 2.

4. The preset angles include at least a first angle and a second angle, the second angle being equal to the first angle plus 180 degrees, and the first angle being not equal to 0 degrees; The step of calculating a first loss according to image features of two sample images in the same set, the two sample images having orientation information different by 180 degrees, includes: calculating a first loss according to a difference between an image feature obtained after rotating an image feature of a sample image in the same set, the sample image having orientation information of the first angle by 180 degrees, and an image feature of a sample image in the same set, the sample image having orientation information of the second angle; The method of claim 2.

5. The step of obtaining an untagged first sample image includes: acquiring an original image, the original image including at least one of a synthetic image and a real image; performing pre-processing on the original images to obtain sample images that meet model training requirements; and performing random data augmentation on the sample image to obtain the first sample image. The method of claim 2.

6. When the visual model is applied to a text recognition scene, the step of pre-processing the original image to obtain a sample image that meets a model training requirement includes: performing text detection on the original image to extract an image of an area containing text information; and performing image correction on the image of the region having text information to obtain a sample image that satisfies a model training requirement. The method according to claim 5.

7. adjusting weights of a backbone network of the image classification network in response to the first loss and the second loss, calculating a sum of the first loss and the second loss as a final loss; and adjusting weights of a backbone network of the image classification network in response to the final loss. The method of claim 1.

8. obtaining an image to be processed; inputting the image to be processed into a visual model obtained by training using the method according to claim 1, and performing computer vision processing on the image to be processed using the visual model to obtain a processing result; and outputting the processing result. Visual processing methods.

9. An apparatus for visual model training, comprising: A network construction module for constructing an image classification network based on a visual model, the network construction module including a backbone network and a decoding module, the backbone network of the image classification network being the same as a backbone network for performing feature extraction in the visual model, and the decoding module being different; a first training module for performing self-supervised training on the image classification network using an untagged first data set to obtain a trained image classification network; a pre-training model determination module for initializing weights of a backbone network of the visual model according to weights of a backbone network of a trained image classification network to obtain a pre-training model; a second training module for training the pre-training model using a tagged second dataset to obtain a trained vision model, the vision model being an optical character recognition (OCR) text recognition model for performing a text recognition process on an input image to obtain text information contained in the input image, the second dataset including real image data and tagged information in a current computer vision task scene; The first training module includes: an untagged dataset acquisition unit used for acquiring the untagged first dataset including a plurality of sets of sample images and orientation information of each sample image, wherein each set of sample images includes a first sample image and a second sample image obtained by rotating the first sample image by a preset angle; a classification prediction unit for extracting image features of each sample image in the first dataset by the image classification network, and determining a direction prediction result of each sample image according to the image features; a loss computing unit for calculating a first loss according to image features of two sample images in the same set of sample images that have directional information different by 180 degrees, and calculating a second loss according to actual directional information and directional prediction results of each sample image; a weight optimization unit for adjusting weights of a backbone network of the image classification network according to the first loss and the second loss. A device for visual model training.

10. The untagged data set acquisition unit comprises: a sample image acquisition subunit for acquiring a first sample image without a tag and determining orientation information of the first sample image to be 0 degrees; an image rotation subunit for rotating the first sample image by the preset angle to obtain a second sample image, and determining orientation information of the second sample image to the preset angle.

10. The apparatus of claim 9.

11. The preset angle includes at least 180 degrees, and the loss computing unit is a first loss calculation subunit for calculating a first loss for each set of sample images according to a difference between an image feature of a first sample image obtained after rotating the image feature of the first sample image by 180 degrees and an image feature of a second sample image obtained after rotating the first sample image by 180 degrees; 11. The apparatus of claim 10.

12. The preset angles include at least a first angle and a second angle, the second angle being equal to the first angle plus 180 degrees, and the first angle is not 0 degrees, and the loss computing unit: a second loss calculation subunit for calculating a first loss according to a difference between an image feature obtained after rotating an image feature of a sample image, the sample image having orientation information of the first angle, by 180 degrees among the sample images in the same set, and an image feature of a sample image, the sample image having orientation information of the second angle; 11. The apparatus of claim 10.

13. The sample image acquisition subunit further comprises: acquiring an original image, the original image including at least one of a synthetic image and a real image; Pre-processing the original image to obtain a sample image that meets model training requirements; a random data augmentation step is used to obtain the first sample image.

11. The apparatus of claim 10.

14. The sample image acquisition subunit further comprises: performing text detection on the original image to extract an image of an area containing text information; performing image correction on the image of the region having the text information to obtain a sample image that satisfies the model training requirements; 14. The apparatus of claim 13.

15. The weight optimization unit further comprises: Calculating the sum of the first loss and the second loss to obtain a final loss; and adjusting weights of a backbone network of the image classification network according to the final loss.

10. The apparatus of claim 9.

16. an image acquisition module for acquiring an image to be processed; a visual processing module for inputting the image to be processed into a visual model generated by the device according to any one of claims 9 to 15, and performing computer visual processing on the image to be processed using the visual model to obtain a processing result; a result output module for outputting a processing result; Visual processing device.

17. At least one processor; a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, the instructions, when executed by the at least one processor, enabling the at least one processor to perform a method according to any one of claims 1 to 8. electronic equipment.

18. A non-transitory computer readable storage medium having stored thereon computer instructions, the computer instructions being used to cause a computer to carry out the method of any one of claims 1 to 8. A non-transitory computer-readable storage medium.

19. A computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8. Computer program.

Citation Information

Patent Citations

  • A domain adaptive deep learning method and a readable storage medium

    CN109919209A

  • Ship target identification method and device

    CN113139623A

  • Microstructure image recognition method and device based on self-supervised neural network

    CN113486745A

  • Intelligent detection method and device based on self-supervised pre-training and robot

    CN113780342A

  • Target detection model training method, target detection method and related devices

    CN113963186A