An image processing method, a related model training method, and related devices

Abstract

This application discloses an image processing method, a related model training method, and related devices. The image processing method includes: obtaining an original image of a target object; using the original image to obtain multiple target sub-images, where each target sub-image contains a part of the target object, and the combination of the multiple target sub-images contains the target object; using an image reconstruction model to perform reconstruction processing on the multiple target sub-images to obtain multiple reconstructed sub-images corresponding to the multiple target sub-images respectively, where the image reconstruction model is trained using positive sample images of sample objects in a defect-free state, and the combination of the multiple reconstructed sub-images is used to represent the target object in a defect-free state. Through the above method, it is possible to reconstruct the object image in a defect-free state and reduce the reconstruction difficulty.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and in particular, to an image processing method, a related model training method, and related devices. Background Art

[0002] Currently, with the increasing development of computer graphics and machine vision, people apply machine vision to daily life and various industries, especially for the anomaly detection of products obtained from production. For example, in the production process of various commodities, printing product labels are prone to various types of defects, such as flying ink, spots, wrinkles, foreign objects, missing printing, scratches, knife wires, etc. Therefore, it is necessary to use machine vision to detect these defect situations of printed commodities.

[0003] The existing defect detection method usually is to take an image of the object to be detected, and compare the image of the object in the normal state with the captured image to determine whether there are defect situations for the object. However, since there are differences in the objects produced each time, the image of the object in the normal state used for reference also needs to be changed accordingly. Therefore, how to obtain the image of the object in the normal state is of great significance for defect detection. Summary of the Invention

[0004] The main technical problem to be solved by this application is to provide an image processing method, a related model training method, and related devices, which can realize the reconstruction of the object image in a defect-free state and reduce the reconstruction difficulty.

[0005] To solve the above technical problem, a technical solution adopted by this application is: to provide an image processing method, the method includes: obtaining an original image of a target object; obtaining multiple target sub-images from the original image, where each target sub-image includes a part of the target object, and the combination of the multiple target sub-images includes the target object; using an image reconstruction model to perform reconstruction processing on the multiple target sub-images to obtain multiple reconstructed sub-images corresponding to the multiple target sub-images respectively, where the image reconstruction model is trained using positive sample images of sample objects in a defect-free state, and the combination of the multiple reconstructed sub-images is used to represent the target object in a defect-free state.

[0006] Among them, obtaining multiple target sub-images from the original image includes: splitting the original image into regions to obtain multiple first image regions, and the first image regions include different parts of the target object; for each first image region, using the pixel values of the second image regions other than the first image region in the original image to obtain a target sub-image corresponding to the first image region.

[0007] Among them, splitting the original image into multiple first image regions includes: dividing the pixel points in the original image into multiple parts by means of average division or random division, where each part of pixel points forms a first image region; among them, the original image and the target sub-image have the same size, and in the target sub-image corresponding to the first image region, the pixel values corresponding to the first image region are preset pixel values, and the pixel values corresponding to the second image region remain unchanged.

[0008] Among them, multiple reconstructed sub-images respectively include different parts of the target object; and / or, the reconstructed sub-image and the corresponding target sub-image respectively include different parts of the target object, and the combination of each group of reconstructed sub-images and the corresponding target sub-image includes the target object.

[0009] Among them, the image reconstruction model is a convolutional autoencoder, and the convolutional autoencoder includes an encoder module and a decoder module;

[0010] Among them, before using the image reconstruction model to perform reconstruction processing on multiple target sub-images to obtain multiple reconstructed sub-images respectively corresponding to the multiple target sub-images, the image processing method further includes the following training steps of the image reconstruction model: obtaining multiple first sample sub-images from a positive sample image, where each first sample sub-image includes a sample object in a partially defect-free state, and the combination of the multiple first sample sub-images includes a sample object in a defect-free state; using the image reconstruction model to perform reconstruction processing on the multiple first sample sub-images to obtain multiple sample reconstructed sub-images respectively corresponding to the multiple first sample sub-images; using the difference between each sample reconstructed sub-image and the corresponding second sample sub-image to adjust the network parameters of the image reconstruction model, where the second sample sub-image corresponding to the sample reconstructed sub-image and the sample reconstructed sub-image both include the same part of the target object.

[0011] Among them, obtaining multiple first sample sub-images from a positive sample image includes: splitting the positive sample image into regions to obtain multiple first sample regions, where the first sample regions include different parts of the sample object; for each first sample region, using the pixel values of the first sample region in the positive sample image to obtain a second sample sub-image corresponding to the first sample region, and using the pixel values of the second image region in the positive sample image to obtain a first sample sub-image corresponding to the first sample region.

[0012] Among them, using an image reconstruction model to perform reconstruction processing on multiple target sub-images to obtain multiple reconstructed sub-images corresponding to the multiple target sub-images respectively, or using the image reconstruction model to perform reconstruction processing on multiple first sample sub-images to obtain multiple sample reconstructed sub-images corresponding to the multiple first sample sub-images respectively, includes: taking the multiple target sub-images / multiple first sample sub-images as multiple input images, and the multiple reconstructed sub-images / multiple sample reconstructed sub-images as multiple output images; using the encoder module to perform: performing first convolution processing on the input image to obtain the convolution features of the input image, and based on the convolution features, obtaining the image features of the input image; using the decoder module to perform decoding processing on the image features of the input image to obtain the output image corresponding to the input image.

[0013] Among them, performing first convolution processing on the input image to obtain the convolution features of the input image includes: using a preset convolution kernel to perform first convolution processing on the input image to obtain the convolution features of the input image, where the preset convolution kernel is determined using the features in at least one positive sample image for training the image reconstruction model;

[0014] Among them, based on the convolution features, obtaining the image features of the input image includes: fusing the convolution features and the input image to obtain a fused feature; performing second convolution processing on the fused feature to obtain the image features of the input image.

[0015] Among them, before using the preset convolution kernel to perform first convolution processing on the input image to obtain the convolution features of the input image, the image processing method further includes: for each positive sample image, using the pixel values of the positive sample image to obtain the feature vector of the positive sample image; selecting the feature vectors of at least one positive sample image that meet the preset feature conditions; using the selected feature vectors to form the preset convolution kernel.

[0016] Among them, using the pixel values of the positive sample image to obtain the feature vector of the positive sample image includes: performing a sliding window process on the positive sample image to obtain image regions corresponding to a number of sliding windows; counting the pixel values of the corresponding pixel points in each image region to obtain the feature values of the corresponding pixel points, and using the statistically obtained feature values of the corresponding pixel points to form the feature vector of the positive sample; among them, selecting the feature vectors of at least one positive sample image that meet the preset feature conditions includes: sorting the vector feature values of the feature vectors of each positive sample image from high to low, and selecting the first preset number of feature vectors.

[0017] After reconstructing multiple target sub-images using an image reconstruction model to obtain multiple reconstructed sub-images corresponding to the multiple target sub-images respectively, the image processing method further includes: forming a reference image using the multiple reconstructed sub-images; performing defect detection on the original image with respect to the target object using the reference image to obtain a defect detection result of the target object in the original image.

[0018] Wherein, the target object is a printed matter and the defect is a printing defect.

[0019] Wherein, performing defect detection on the original image with respect to the target object using the reference image to obtain a defect detection result of the target object in the original image includes: taking the difference between the pixel values of the corresponding pixel points of the original image and the reference image to obtain a reconstruction error; determining whether the reconstruction error is less than a preset threshold; if so, determining that the target object in the original image has no defect, and if not, determining that the target object in the original image has a defect.

[0020] To solve the above technical problems, another technical solution adopted by this application is: providing a training method for an image reconstruction model, which includes: obtaining a positive sample image including a sample object in a defect-free state; obtaining multiple first sample sub-images using the positive sample image, wherein each first sample sub-image includes a part of the sample object in a defect-free state, and the combination of the multiple first sample sub-images includes the target object in a defect-free state; reconstructing the multiple first sample sub-images using the image reconstruction model to obtain multiple sample reconstructed sub-images corresponding to the multiple first sample sub-images respectively; adjusting the network parameters of the image reconstruction model using the difference between each sample reconstructed sub-image and the corresponding second sample sub-image, wherein the second sample sub-image corresponding to the sample reconstructed sub-image and the sample reconstructed sub-image both include the same part of the target object.

[0021] Wherein, reconstructing the multiple first sample sub-images using the image reconstruction model to obtain multiple sample reconstructed sub-images corresponding to the multiple first sample sub-images respectively includes: performing a first convolution process on the first sample sub-image using the encoder module of the image reconstruction model to obtain a convolution feature of the first sample sub-image, and obtaining an image feature of the first sample sub-image based on the convolution feature; decoding the image feature of the first sample sub-image using the decoder module of the image reconstruction model to obtain a sample reconstructed sub-image corresponding to the first sample sub-image.

[0022] Wherein, performing a first convolution process on the first sample sub-image to obtain a convolution feature of the first sample sub-image includes: performing a first convolution process on the first sample sub-image using a preset convolution kernel to obtain a convolution feature of the first sample sub-image, wherein the preset convolution kernel is determined using the features in at least one positive sample image for training the image reconstruction model;

[0023] Among them, obtaining the image features of the first sample sub-image based on the convolutional features includes: fusing the convolutional features and the first sample sub-image to obtain a fused feature; performing a second convolution process on the fused feature to obtain the image features of the first sample sub-image.

[0024] Among them, before performing a first convolution process on the first sample sub-image using a preset convolution kernel to obtain the convolutional features of the first sample sub-image, the method further includes: for each positive sample image, obtaining a feature vector of the positive sample image using the pixel values of the positive sample image; selecting at least one feature vector of the positive sample images that meets the preset feature conditions; using the selected feature vectors to form the preset convolution kernel.

[0025] To solve the above technical problems, another technical solution adopted by this application is: providing an image processing device, the device includes: an acquisition module, configured to acquire an original image of a target object; a composition module, configured to obtain multiple target sub-images using the original image, where each target sub-image includes a part of the target object, and the combination of the multiple target sub-images includes the target object; a processing module, configured to perform a reconstruction process on the multiple target sub-images using an image reconstruction model to obtain multiple reconstructed sub-images respectively corresponding to the multiple target sub-images, where the image reconstruction model is trained using positive sample images of a sample object in a defect-free state, and the combination of the multiple reconstructed sub-images is used to represent the target object in a defect-free state.

[0026] To solve the above technical problems, another technical solution adopted by this application is: providing a training device for an image processing model, the device includes: a sample acquisition module, configured to acquire positive sample images of a sample object in a defect-free state; a sample composition module, configured to obtain multiple first sample sub-images using the positive sample images, where each first sample sub-image includes a part of the sample object in a defect-free state, and the combination of the multiple first sample sub-images includes the target object in a defect-free state; a sample processing module, configured to perform a reconstruction process on the multiple first sample sub-images using an image reconstruction model to obtain multiple sample reconstructed sub-images respectively corresponding to the multiple first sample sub-images; an adjustment module, configured to adjust the network parameters of the image reconstruction model using the difference between each sample reconstructed sub-image and the corresponding second sample sub-image, where the second sample sub-image corresponding to the sample reconstructed sub-image and the sample reconstructed sub-image both include the same part of the target object.

[0027] To solve the above technical problems, another technical solution adopted by this application is: providing an image processing device, the device includes a memory and a processor coupled to each other, the memory stores program instructions; the processor is configured to execute the program instructions stored in the memory to implement the above image processing method, and / or, implement the training method of the image reconstruction model.

[0028] To solve the above technical problems, another technical solution adopted in this application is: to provide a computer-readable storage medium for storing program instructions that can be executed to implement the above image processing method, and / or, to implement the training method of the image processing model.

[0029] In the above solution, multiple target sub-images are obtained from the original image of the target object, and then the multiple target sub-images are reconstructed by using the image reconstruction model to obtain multiple reconstructed sub-images corresponding to the multiple target sub-images. Among them, the image reconstruction model is trained by using the positive sample images of the sample object in the defect-free state, and the combination of the multiple reconstructed sub-images is used to represent the target object in the defect-free state. Since the solution of this application uses multiple target sub-images to obtain multiple reconstructed sub-images, that is, both the input image and the output image of the image reconstruction model are images containing partial image regions of the target object, it is possible to reconstruct the object image in the defect-free state. Moreover, the number of image features of the input image and the output image of the image reconstruction model in this application solution is greatly reduced compared with the original image, so the processing amount of the image reconstruction model for a single input image is reduced, and since the size of the reconstruction area is reduced from the entire original image to a local image, the training difficulty of the image reconstruction model is reduced, so the difficulty of realizing the reconstruction of the object image in the defect-free state is reduced.

[0030] Furthermore, during the training process, the image reconstruction model is trained by using multiple sub-images obtained by dividing the positive sample images. Compared with training by using the entire positive sample image, the training method using sub-images can amplify the training samples by several times, which is more conducive to the training of the image reconstruction model.

[0031] Furthermore, the encoder module in the image reconstruction model of this application can perform convolution processing on the input image by using a preset convolution kernel, and the preset convolution kernel is determined in advance based on the features of the positive sample images during the training process. The purpose of training the image reconstruction model is to obtain a convolution kernel that can achieve the image reconstruction effect. Therefore, the pre-determination of the preset convolution kernel can greatly reduce the training difficulty of the image reconstruction model.

[0032] Further, a preset convolution kernel obtained from positive sample images is used to perform a first convolution process on the input image to obtain convolution features of the input image. Then, the obtained convolution features of the input image and the input image are fused to obtain fused features, and the fused features are subjected to a second convolution process to obtain image features of the input image. Since the first convolution process on the input image is a dimensionality reduction process, this process will retain larger features in the input image, while some small features are likely to be missing. Therefore, fusing the input image and the convolution features obtained by the first convolution process on the input image, and performing a second convolution process on the fused features can retain the small features in the input image and the image output by the model. Therefore, the image reconstruction model trained through this process can more accurately reconstruct the small features of the input image, facilitating more accurate subsequent image detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1a is a schematic flowchart of an embodiment of an image processing method provided by the present application;

[0034] Figure 1b is Figure 1a a schematic diagram of the formation of each image in an embodiment of the image processing method;

[0035] Figure 2 is a schematic flowchart of an embodiment of an image reconstruction model training method provided by the present application.

[0036] Figure 3 is a schematic flowchart of an embodiment of image reconstruction processing in the image processing method provided by the present application.

[0037] Figure 4a is Figure 2 a schematic flowchart of an embodiment of determining a preset convolution kernel in the reconstruction model training method provided herein;

[0038] Figure 4b is Figure 4a a schematic diagram of the formation of a preset convolution kernel in an embodiment;

[0039] Figure 5a is a schematic flowchart of an embodiment of an image processing method provided by the present application;

[0040] Figure 5b is a schematic diagram of the formation of each image in an embodiment of the image processing method provided by the present application;

[0041] Figure 6 is a schematic framework diagram of an embodiment of an image processing apparatus provided by the present application;

[0042] Figure 7 is a schematic framework diagram of an embodiment of a training apparatus for an image reconstruction model provided by the present application;

[0043] Figure 8 It is a schematic framework diagram of an embodiment of the image processing device provided by this application;

[0044] Figure 9 It is a schematic framework diagram of the computer-readable storage medium provided by this application. Specific embodiments

[0045] To make the objectives, technical solutions and effects of this application clearer and more definite, the following further describes this application in detail with reference to the accompanying drawings and by way of examples.

[0046] It should be noted that if the descriptions in the embodiments of this application involve "first", "second", etc., such descriptions of "first", "second", etc. are only for descriptive purposes and cannot be construed as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. Additionally, the technical solutions between various embodiments may be combined with each other, but it must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0047] Refer to Figure 1a , Figure 1a It is a schematic flowchart of an embodiment of the image processing method provided by this application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 1a the process sequence shown. The method of this embodiment is used to obtain multiple target sub-images from the original image of the target object, and then use an image reconstruction model to perform reconstruction processing on the multiple target sub-images to obtain multiple reconstructed sub-images corresponding to the multiple target sub-images respectively.

[0048] As Figure 1a shown, the method of this embodiment includes:

[0049] Step S110: Obtain the original image of the target object.

[0050] The original image of the target object described in this article is an imaging image that reflects the true state of the target object. Here, the target object can be any physical object, such as a cup, an antique, a packaging bag, etc.; it can also be an image of any physical object, such as a printed label of any commodity, a handmade painting, etc. The original image of the target object can be an imaging image of the target physical object in any direction, or an imaging image of an existing image of the target physical object. The original image of the target object can be obtained by means such as camera shooting or scanning. The obtained original image of the target object can be an image in a defect-free state or an image with certain defects. The specific image acquisition method and image state are not specifically defined here. It can be understood that the target object described in this article can be understood as the part of the object that is photographed or scanned. That is, if a printed label of a commodity is photographed and only half of the printed label is photographed, then the photographed half label is used as the target object.

[0051] Step S120: Obtain multiple target sub-images by using the original image.

[0052] In this embodiment, multiple target sub-images are obtained by splitting the original image into regions.

[0053] Specifically, first split the original image into regions to obtain a plurality of first image regions, where each first image region contains different parts of the target object. In some embodiments, the pixel points in the original image can be divided into multiple parts by using an average division or a random division method. Each part of the pixel points forms a first image region. For example, as Figure 1b shown, the original image 100 is a 4×4 image, containing 16 pixel points numbered from 1 to 16. The pixel points of this original image are evenly divided into 4 parts, then 4 first image regions 11 can be evenly split out from this original image (by Figure 1bThe shaded part in (is shown), where each first image region 11 contains a different part of the original image 100. Each combined set of the split first image regions forms the complete original image 100, and the number of pixel points in each first image region is one-fourth of the total number of pixel points in the original image, that is, the number of pixel points in each first image region is 4. For another example, if the original image 100 is a 4×4 image containing a total of 16 pixel points numbered from 1 to 16, and the pixel points of the original image 100 are randomly divided into 4 parts, the number of pixel points in each part can be the same or different, and the sum of the number of pixel points in the 4 parts is the total number of pixel points in the original image. That is, the 4 split first image regions 11 can be combined to form the complete original image 100. For example, the number of pixel points in the 4 randomly divided parts can be 2, 4, 6, 4 corresponding to the original pixel points respectively, or 1, 3, 5, 7. Moreover, the split pixel points can be adjacent or separated. The specific splitting method and the position state of the split pixel points are not specifically defined here.

[0054] Then, taking the above example of evenly splitting the original image into 4 first image regions 11 (shown by the Figure 1b shaded part in), for each obtained first image region 11, using the pixel values of the second image region 12 (shown by the Figure 1b shaded part in) in the original image 100 except for the first image region, the target sub-image 13 corresponding to the first image region is obtained. And each obtained target sub-image 13 contains part of the target object. The combination of multiple target sub-images 13 contains the target object (if the original image is a defect-free image, the combination of multiple target sub-images forms the target object; if the original image is an image containing defects, the combination of multiple target sub-images contains the target object). That is, each obtained target sub-image 13 contains the pixel values of the second image region 12 of the original image, and the combination of multiple target sub-images 13 can form the complete original image 100. And the original image 100 and the multiple target sub-images 13 obtained from the original image have the same size. Among them, in the target sub-image 13 corresponding to the first image region 11, the pixel values corresponding to the first image region 11 are preset pixel values, and the pixel values corresponding to the second image region 12 remain unchanged. In other words, the image region of the target sub-image 13 includes two parts, corresponding to the first image region 11 and the second image region 12 of the original image 100 respectively. Among them, the pixel values corresponding to the first image region 11 of the original image 100 are all filled with preset pixel values, and the preset pixel value is any fixed pixel value. The preset pixel value can be, but is not limited to, 0 or 255. And the pixel values in the target sub-image 13 corresponding to the second image region 12 of the original image remain unchanged, that is, the pixel values of the second image region 12 in the original image.

[0055] Step S130: Use the image reconstruction model to perform reconstruction processing on multiple target sub-images to obtain multiple reconstructed sub-images respectively corresponding to the multiple target sub-images.

[0056] In this embodiment, use the image reconstruction model to perform reconstruction processing on multiple target sub-images to obtain multiple reconstructed sub-images respectively corresponding to the multiple target sub-images. Among them, the image reconstruction model is trained using positive sample images of sample objects in a defect-free state. Therefore, regardless of whether the input image contains defects, the multiple reconstructed sub-images reconstructed using this image reconstruction model are all defect-free images. More specifically, through this reconstruction processing, the image of the first image region in the defect-free state that the target sub-image lacks relative to the original image can be reconstructed, that is, the multiple reconstructed sub-images have the same size as the original image, and the multiple reconstructed sub-images all include two parts. One part is the image whose position and pixel values are in one-to-one correspondence with the position and pixel values of the first image region in the original image in the defect-free state, and the position of the other part of the image corresponds to the position of the second image region in the original image, but the pixel value is the preset pixel value. Therefore, the multiple reconstructed sub-images and the corresponding target sub-images respectively contain different parts of the target object, and the combination of each group of reconstructed sub-images and the corresponding target sub-images contains the target object. Because the multiple reconstructed sub-images reconstructed are images containing the first image region in the defect-free state, and as can be seen from step S120, the multiple first image regions are obtained by splitting the original image, and the pixel points of the multiple first image regions are combined correspondingly to form the original image. Therefore, the combination of the multiple reconstructed sub-images reconstructed can represent the complete target object, and since the image reconstruction model is trained using positive sample images, the output reconstructed sub-images can represent the part of the target object in the defect-free state, that is, the combination of the multiple reconstructed sub-images reconstructed can be used to represent the target object in the defect-free state. Continuing with the above example of evenly splitting the original image 100 into 4 first image regions 11 (by Figure 1bTaking the shaded part in [as an example], the multiple reconstructed sub-images 14 obtained by the image reconstruction model are respectively corresponding to multiple target sub-images 13. The reconstructed sub-image 14 and the corresponding target sub-image 13 respectively include different parts of the target object, and the parts of the target object included in the reconstructed sub-image 14 and the corresponding target sub-image 13 can form the target object. It can be understood that the parts of the target object included in the reconstructed sub-image 14 and the corresponding target sub-image 13 respectively correspond to different states of the target object. For example, the target sub-image 13 includes the part of the target object corresponding to pixel points 3, 5, 6, 8-16, and this part is a part of the target object in the original state (the original state described in this article is the state of the target object when the original image is captured, and at this time the target object may be defective or non-defective); the reconstructed sub-image 14 includes the part of the target object corresponding to pixel points 1, 2, 4, 7, and this part is a part of the target object in the non-defective state.

[0057] In this embodiment, multiple target sub-images are obtained from the original image of the target object, and then the image reconstruction model is used to perform reconstruction processing on the multiple target sub-images to obtain multiple reconstructed sub-images respectively corresponding to the multiple target sub-images. Among them, the image reconstruction model is trained by using the positive sample image of the sample object including the non-defective state, and the combination of the multiple reconstructed sub-images is used to represent the target object in the non-defective state. Since the solution of this application uses multiple target sub-images to obtain multiple reconstructed sub-images, that is, both the input image and the output image of the image reconstruction model are images including partial image regions of the target object, therefore, the reconstruction of the object image in the non-defective state can be realized. Moreover, the number of image features of the input image and the output image of the image reconstruction model in this application solution is greatly reduced compared with the original image, so the processing amount of the image reconstruction model for a single input image is reduced. And since the size of the reconstruction area is reduced from the entire original image to a local image, the training difficulty of the image reconstruction model is reduced, so the difficulty of realizing the reconstruction of the object image in the non-defective state is reduced.

[0058] In addition, the image reconstruction model mentioned in this article is a model that can automatically perform image reconstruction processing on the target sub-image. In some embodiments, the image reconstruction model is a convolutional autoencoder, which is a self-supervised neural network model and includes an encoder module and a decoder module. The encoder module is used to obtain the feature information and approximate position information of the image, and the decoder module is used to map the obtained information to specific pixel points to obtain the pixel points corresponding to the corresponding image. It can be understood that the image reconstruction model can also be other network structures capable of reconstructing images, and the network structure of the image reconstruction model is not specifically limited here.

[0059] Among them, the training process of the image reconstruction model can refer to Figure 2Related descriptions. In some embodiments, before step S130 (including before step S110), the image processing method of the present application further includes Figure 2 Steps in related embodiments.

[0060] Please refer to Figure 2 , Figure 2 FIG. is a schematic flowchart of an embodiment of the model training process in the image processing method provided by the present application. In this embodiment, the training process of the image reconstruction model includes:

[0061] Step S210: Obtain a positive sample image of a sample object in a defect-free state.

[0062] The defect-free state means that there are no defects in the positive sample image of the sample object. The positive sample image is the imaging image of the sample object. The positive sample image in the defect-free state represents the true appearance or state of the sample object. The sample object can be any object or the image of any object. The obtaining method of the positive sample image can refer to the obtaining method of the original image of the target object described in step S110, and will not be elaborated here.

[0063] Step S220: Obtain multiple first sample sub-images using the positive sample image.

[0064] In this embodiment, by splitting the positive sample image into regions, multiple first sample sub-images are obtained.

[0065] Specifically, first split the positive sample image into regions to obtain multiple first sample regions, where the first sample regions contain different parts of the sample object; among them, the splitting method of the multiple first sample regions can refer to the splitting method of using the original image to split into multiple first image regions in step S120, and will not be elaborated here.

[0066] Then, for each first sample region, use the pixel values of the first sample region in the positive sample image to obtain a second sample sub-image corresponding to the first sample region. Specifically, set the pixel values of the second sample region other than the first sample region in the positive sample image to a preset pixel value, which can be any fixed pixel value, and at the same time retain the pixel values of the first sample region in the positive sample image to obtain a second sample sub-image corresponding to the first sample region. The second sample sub-image obtained in this way contains two parts of image regions, where one part of the image region has the same position and pixel values as the first sample region in the positive sample image, and the position of the other part of the image is the same as the position of the second sample region in the positive sample image, but the pixel values of this part of the image are the set preset pixel values.

[0067] Meanwhile, the pixel values of the second image region in the positive sample image are used to obtain a first sample sub-image corresponding to the first sample region. Specifically, the positive sample image and the first sample sub-image have the same size. In the first sample sub-image corresponding to the first sample region, the pixel values corresponding to the first sample region are preset pixel values, and the pixel values corresponding to the second sample region remain unchanged. The first sample sub-image obtained in this way also includes other partial image regions. Among them, the position and pixel values of one part of the image region are the same as those of the second sample region in the positive sample image, and the position of the other part of the image is the same as that of the first sample region in the positive sample image, but the pixel values of this part of the image are the set preset pixel values. It can be seen that in this embodiment, the method of splitting the positive sample image into multiple first sample sub-images is to divide the positive sample image into multiple first image regions and the second image regions corresponding to the multiple first image regions respectively. Among them, there is no regional overlap between the first image region and the corresponding second image region, and the multiple first image regions and the multiple second image regions are combined in one-to-one correspondence to form a complete positive sample image. Therefore, each group of the first image region and the corresponding second image region of the positive sample image are used to form the above-mentioned second sample sub-image and the first sample sub-image with the same size as the positive sample image. Among them, the second sample sub-image is an image that retains the position and pixel points of the first image region of the positive sample image, and the remaining second image region different from the positive sample image is filled with preset pixel points; on the contrary, the first sample sub-image is an image that retains the position and pixel points of the second image region of the positive sample image, and the remaining first image region different from the positive sample image is filled with preset pixel points.

[0068] Among them, the obtaining method of the first sample sub-image is not limited to the above method. Further, the preset pixel value of the second image region of the second sample sub-image can be set to 0, that is, by subtracting the positive sample image and multiple second sample sub-images one by one, the first sample sub-image corresponding to the second sample sub-image is obtained. Among them, because the positive sample image is an image in a defect-free state, each first sample sub-image obtained from the positive sample image contains a partial sample object in a defect-free state, and the combination of multiple first sample sub-images contains a sample object in a defect-free state.

[0069] Step S230: Use the image reconstruction model to perform reconstruction processing on multiple first sample sub-images to obtain multiple sample reconstructed sub-images corresponding to the multiple first sample sub-images respectively.

[0070] Among them, for the description of the image reconstruction model performing reconstruction processing on multiple first sample sub-images to perform duplicate removal processing to obtain multiple sample reconstructed sub-images corresponding to the multiple first sample sub-images respectively and the introduction of the related network structure, reference can be made to the description of step S130 above, and details will not be repeated here.

[0071] In some embodiments, the positive sample image, multiple first sample sub-images, and multiple sample reconstruction sub-images are all of the same size.

[0072] Step S240: Adjust the network parameters of the image reconstruction model by using the difference between each sample reconstruction sub-image and the corresponding second sample sub-image.

[0073] Among them, the second sample sub-image corresponding to the sample reconstruction sub-image and the sample reconstruction sub-image both contain the same part of the target object. And the second sample sub-image is the reconstruction standard of the first sample sub-image. During the training process of the reconstruction model, the loss of the current image reconstruction model can be obtained according to the difference between the sample reconstruction sub-image and the second sample sub-image, and then the network parameters of the image reconstruction model can be adjusted by using the obtained loss. When the difference between the sample reconstruction sub-image and the second sample sub-image reaches the preset requirement, and / or the number of training times reaches the preset requirement, it means that the training of the image reconstruction model is completed. This preset requirement can be determined according to the actual situation. In some specific applications, this preset requirement can be based on not affecting subsequent image comparison, and no specific limitation is made here. Please refer to Figure 1b , such as Figure 1b shown: The positive sample image corresponds to Figure 1b the original image 100 in the above-mentioned embodiment, the first image region corresponds to Figure 1b the first image region 11 in Figure 1b the second image region corresponds to Figure 1b the image containing the first image region 11 in Figure 1b the first sample image corresponds to Figure 1b the target sub-image 13 in

[0074] Furthermore, during the training process, the image reconstruction model is trained by using multiple sub-images obtained by dividing the positive sample image. Compared with training using the entire positive sample image, the method of training with sub-images can amplify the training samples by several times, which is more conducive to the training of the image reconstruction model.

[0075] During the training process of the image reconstruction model, multiple sub-images obtained by dividing the positive sample image are used to train the image reconstruction model. Compared with training using the entire positive sample image, the method of using sample sub-images for training can amplify the training samples by several times, which is more conducive to the training of the image reconstruction model. Moreover, during the training process of the model, the network model is adjusted using the difference between the sample reconstructed sub-image and the second sample sub-image. Therefore, using the trained model to reconstruct the image can achieve a certain image reconstruction effect and quality. Further, during the training process of the image reconstruction model, it automatically learns using image data, which means that the trained image reconstruction model can achieve automatic image reconstruction with high efficiency. With the support of a large number of samples, the model does not learn the sample images themselves, but predicts unknown pixels using known pixels and learns the function of image reconstruction. Therefore, other types of untrained images can also be reconstructed, thereby improving the image application range of the image reconstruction model.

[0076] In this article, whether in step S130, multiple target sub-images are reconstructed to obtain multiple reconstructed sub-images corresponding to the multiple target sub-images respectively, or in step S230, multiple first sample sub-images are reconstructed to obtain multiple sample reconstructed sub-images corresponding to the multiple first sample sub-images respectively, the above image reconstruction model is used. For the convenience of subsequent explanation, the multiple target sub-images / multiple first sample sub-images are uniformly referred to as multiple input images, and the multiple reconstructed sub-images / multiple sample reconstructed sub-images are uniformly referred to as multiple output images.

[0077] Please refer to Figure 3 , Figure 3 is a schematic flowchart of an embodiment of image reconstruction processing in the image processing method of this application. The specific steps are as follows:

[0078] Step S310: Perform a first convolution process on the input image using the encoder module to obtain the convolution features of the input image.

[0079] Specifically, a preset convolution kernel in the encoder module is used to perform a first convolution process on the input image to obtain the convolution features of the input image. Among them, the preset convolution kernel is determined using the features in at least one positive sample image used to train the image reconstruction model. The determined preset convolution kernel is used to process the input image to extract the effective feature information of the input image. Since the purpose of training the encoder using the image is to obtain a convolution kernel that can extract the effective information of the image, and the preset convolution kernel is a specific convolution kernel determined using the image features of the positive sample image used to train the image reconstruction model, the determination of the preset convolution kernel can greatly reduce the training difficulty of the subsequent encoder.

[0080] Step S320: Based on the convolutional features, obtain the image features of the input image.

[0081] Since the process of using the encoder module to perform the first convolutional processing on the input image to obtain the convolutional features of the input image is a dimensionality reduction process, for an image, some obvious large features are easily retained after dimensionality reduction, but small features are easily lost during the convolution process. Therefore, to retain the small features of the image and make the subsequent image comparison results more accurate, the convolutional features of the input image obtained by performing the first convolutional processing on the input image with the above-determined preset convolution kernel and the image features of the input image can be fused to obtain the fused features of the input image convolutional features and the input image features, so as to retain the small features of the input image. Then, perform the second convolutional processing on the obtained fused features of the input image to obtain the image features of the input image that retain the small features of the input image. Among them, since the first convolutional processing on the input image is a dimensionality reduction process, and the size of the original input image becomes smaller during this dimensionality reduction process, before image fusion, the original image and the image obtained by performing the first convolutional processing on the original image need to be processed. For example, the resize code can be used to set the sizes of the original image and the image obtained by performing the first convolutional processing on the original image to be the same.

[0082] In this embodiment, first use the preset convolution kernel in the encoder module to perform the first convolutional processing on the input image, and then fuse it with the input image and perform the second convolutional processing to obtain the image features of the input image that retain the small features of the input image. In some embodiments, with the support of a large number of samples, it is also possible not to use the preset convolution kernel in the encoder module to perform the first convolutional processing on the input image, but directly perform the second convolutional processing on the input image as described above.

[0083] Step S330: Use the decoder module to perform decoding processing on the image features of the input image to obtain the output image corresponding to the input image.

[0084] In this article, the effective information of the input image is extracted through the encoder, and then the decoder is used to reconstruct the input image to output the reconstructed image corresponding to the input image.

[0085] During the training process of the image reconstruction model, a preset convolution kernel obtained from a positive sample image is used to perform a first convolution process on the input image. First, the convolution features of the input image are obtained, and then the obtained convolution features of the input image and the input image are fused to obtain fused features. Then, a second convolution process is performed on the fused features to obtain the image features of the input image. Since this process fuses the input image and the convolution features obtained by the first convolution process on the input image, the image reconstruction model trained through this process can more accurately reconstruct the minute features of the input image, retain the minute features of the input image, and obtain a reconstructed image with minute features retained, which is convenient for making the subsequent image detection results more accurate.

[0086] Please refer to Figure 4a , Figure 4a which is a schematic flowchart of an embodiment for determining a preset convolution kernel in the image reconstruction model of the present application. The steps for determining the preset convolution kernel include:

[0087] Step S410: For each positive sample image, use the pixel values of the positive sample image to obtain the feature vector of the positive sample image.

[0088] First, please refer to Figure 4b , as Figure 4b shown. Select N positive sample images (N≥1), perform a sliding window process on the selected positive sample images to obtain a number of image regions corresponding to the sliding windows. Specifically, use the method of obtaining multiple target sub-images from the original image in step S120 to evenly divide each selected positive sample image into K single images with a dimension of m*n, where m and n can be the same or different. For each single image with a dimension of m*n in these K images, starting from the first pixel point, select a sliding window with a dimension of k*k, where k is much smaller than m and n. Through this k*k sliding window, traverse all pixel points of each image with a dimension of m*n to obtain K sets of m*n k*k sliding window corresponding image regions. Since N positive samples (N≥1) are selected, N*K sets of m*n k*k sliding window corresponding image regions can be obtained. Among them, the value of k can be any natural number greater than 1, such as 3, 5, 7, etc. The specific value of k can be determined according to the image processing effect of the image reconstruction model and is not specifically limited here.

[0089] Then, count the pixel values of the corresponding pixel points in each image region to obtain the feature values of the corresponding pixel points. Specifically, for example, Figure 4bFor each of the obtained N*K image regions, the pixel values of the corresponding pixel points in the image regions corresponding to each of the m*n k*k sliding windows are added and then averaged. Finally, the eigenvalue of the corresponding pixel point in the image region corresponding to the N*K sliding windows of dimension k*k is obtained. Then, using the eigenvalues of the corresponding pixel points obtained by statistics, the feature vector of the selected positive samples is formed. Specifically, each of the N*K sliding windows of dimension k*k corresponding to the image regions is flattened to obtain N*K column vectors containing k 2 elements, and then these N*K column vectors containing k 2 elements are concatenated to obtain a feature vector with k 2 rows and N*K columns. Among them, the flattening method for each of the N*K sliding windows of dimension k*k should be the same, but the selected flattening method is not unique. For example, each row of the k*k sliding window can be arranged in order, or each column of the k*k sliding window can be arranged in order. Specifically, which method to choose is not specifically limited here.

[0090] Step S420: Select the feature vectors of at least one positive sample image that meet the preset feature conditions.

[0091] The feature vectors that meet the preset feature conditions mean that, from the above-obtained feature vectors with k 2 rows and N*K columns, the vector eigenvalues of the feature vectors of the selected positive sample images are sorted from high to low, and the first preset number of feature vectors corresponding to the larger eigenvalues are selected, so that the first preset number of feature vectors corresponding to the selected larger eigenvalues can represent the image information of the selected positive sample images to a great extent. Among them, the specific value of this preset number can be determined according to the reconstruction effect during the model reconstruction process. For the convenience of understanding the subsequent preset convolution kernels according to the diagram, this preset number can be expressed as, for example, Figure 4b shown as L1.

[0092] Step S430: Use the selected feature vectors to form a preset convolution kernel.

[0093] The selected preset number of feature vectors are concatenated to form a preset convolution kernel. Specifically, the selected preset number (L1) of feature vectors are first split into L1 column vectors containing k 2 elements, and then these L1 column vectors containing k 2 elements are recombined to obtain L1 feature vectors of k*k, and then these L1 feature vectors of k*k are formed into, as shown in Figure 4bThe convolution kernel with a kernel dimension of k and a number of channels of L1 shown is used as the preset convolution kernel to perform a convolution operation on the input image subsequently. In a specific application, the input image is a color image with three channels and a size of m*n. Therefore, the convolution kernel with a kernel dimension of k and a number of channels of L1 is used to perform a convolution operation on the color image with an input matrix dimension of m*n*3.

[0094] Please refer to Figure 5a , Figure 5a which is a schematic flowchart of an embodiment of the image processing method provided by this application. This embodiment includes step S110, step S120, and step S130. After step S130, it also includes using multiple reconstructed sub-images to form a reference image to implement defect detection of the target object. Specifically, as Figure 5a shown, this embodiment includes the following steps:

[0095] Step S510: Obtain the original image of the target object.

[0096] The relevant description of the target object and the acquisition method of the original image of the target object can be referred to the description in step S110, and will not be elaborated here. Among them, the obtained original image of the target object can be an image in a defect-free state as Figure 1b shown; it can also be an image with certain defects, such as Figure 5b shown, where there is a defect on the original image 50, and the defect is a small black dot.

[0097] Step S520: Obtain multiple target sub-images from the original image.

[0098] In this embodiment, the method of obtaining multiple target sub-images from the original image can be referred to the description in step S120, and will not be elaborated here. Among them, as Figure 5b shown, the multiple target sub-images obtained are 53, and the multiple target sub-images 53 are images where the pixel values of the first image area 51 corresponding to the original image 50 are preset pixel values, while the pixel values of the second image area 52 corresponding to the original image 50 remain unchanged.

[0099] Step S530: Use an image reconstruction model to perform reconstruction processing on the multiple target sub-images to obtain multiple reconstructed sub-images corresponding to the multiple target sub-images respectively.

[0100] In this embodiment, an image reconstruction model is used to perform reconstruction processing on the multiple target sub-images 53 to obtain multiple reconstructed sub-images 54 corresponding to the multiple target sub-images 53 respectively. As Figure 5bAs shown, the image reconstruction model is a convolutional autoencoder and includes an encoder module and a decoder module. When using the image reconstruction model to reconstruct multiple target sub-images 53 to obtain multiple reconstructed sub-images 54 corresponding to the multiple target sub-images 53 respectively, the process is the same as steps S310 and S320 above. First, perform a first convolution process on the multiple target sub-images 53 to obtain the convolution features of the target sub-images 53, and then fuse the convolution features and the image features of the multiple target sub-images 53 to obtain the fused features of the two, and then perform a second convolution process on the fused features to obtain the image features of the multiple target sub-images 53.

[0101] Step S540: Use multiple reconstructed sub-images to form a reference image.

[0102] In this embodiment, in order to facilitate subsequent defect detection of the target object, multiple reconstructed sub-images 54 can be combined to form a reference image 55. Specifically, for example, retain the image regions in the multiple reconstructed sub-images that contain the original image information, and record the position information of the image regions with the original image information in the original image, and splice the positions of the image regions in a one-to-one correspondence with their positions in the original image to form a reference image. It can be understood that the formed reference image is a combined image of multiple reconstructed sub-images in a defect-free state. Therefore, the reference image composed of multiple reconstructed sub-images is also a combined image without defects and can reflect the image corresponding to the original image of the target object in a defect-free state.

[0103] Step S550: Use the reference image to perform defect detection on the original image for the target object to obtain the defect detection result of the target object in the original image.

[0104] In some embodiments, the target object can be, but is not limited to, printed matter, scrolls, etc., and the target object can be defective or defect-free. The defects can be printing defects of printed matter, flaws in scrolls, etc. The defect detection result includes whether the target object has defects and the position information of the existing defects, etc. In a specific embodiment, as Figure 5b shown, the target object is a printed commodity label, and the defect is small ink dots existing after the label is printed. In some embodiments, the defect can also be spots, wrinkles, foreign objects, missing printing, scratches, knife wires, etc. existing after the label is printed.

[0105] Among them, defect detection is performed on the target object and a detection result is obtained, including: first, the pixel values of the corresponding pixel points of the original image and the reference image are subtracted one by one to obtain a reconstruction error, and then it is determined whether the reconstruction error is less than a preset threshold. If the reconstruction error is less than the preset threshold, it is determined that the target object in the original image has no defect. If the reconstruction error is greater than the preset threshold, it is determined that the target object in the original image has a defect. Among them, the determination of the preset threshold can be determined according to the comparison effect between the original image and the reference image, and no specific limitation is made here.

[0106] In this embodiment, a reference image is formed by combining multiple reconstructed sub-images, and the reference image and the original image are compared to determine the defect detection result. In some embodiments, the pixel points of the first image region in multiple reconstructed sub-images and the original image can also be directly compared one by one to determine the defect detection result.

[0107] Please refer to Figure 6 , Figure 6 is a schematic framework diagram of an embodiment of an image processing device provided by the present application. In this embodiment, the image processing device 60 includes: an acquisition module 61, a composition module 62, and a processing module 63. The acquisition module 61 is used to acquire the original image of the target object; the composition module 62 is used to obtain multiple target sub-images by using the original image, where each target sub-image contains part of the target object, and the combination of multiple target sub-images contains the target object; the processing module 63 is used to perform reconstruction processing on multiple target sub-images by using an image reconstruction model to obtain multiple reconstructed sub-images corresponding to the multiple target sub-images respectively, where the image reconstruction model is trained by using positive sample images of sample objects in a defect-free state, and the combination of multiple reconstructed sub-images is used to represent the target object in a defect-free state.

[0108] In some embodiments, the above composition module 62 is used to split the original image into regions to obtain multiple first image regions, and use the pixel values of the second image region in the original image except for the first image region to obtain multiple target sub-images.

[0109] In some embodiments, the above composition module 62 is used to divide the pixel points in the original image into multiple parts in a manner of average division or random division.

[0110] In some embodiments, the above composition module 62 is used to form a reference image by using multiple reconstructed sub-images.

[0111] In some embodiments, the above processing module 63 is used to perform defect detection on the original image for the target object by using the reference image to obtain the defect detection result of the target object in the original image.

[0112] In some embodiments, the above-mentioned processing module 63 is configured to calculate the difference between the pixel values of corresponding pixel points of the original image and the reference image to obtain a reconstruction error; and is configured to determine whether the reconstruction error is less than a preset threshold and determine that there is a defect in the target object in the original image.

[0113] It should be noted that the device in this embodiment can execute the steps in the above method. For the detailed description of related content, please refer to the above method part, and details will not be repeated here.

[0114] Please refer to Figure 7 , Figure 7 FIG. is a schematic framework diagram of an embodiment of a training device for an image reconstruction model provided by the present application. In this embodiment, the image processing device 70 includes: a sample acquisition module 71, a sample composition module 72, a sample processing module 73, and an adjustment module 74, which are used to train the image model. The sample acquisition module 71 is configured to acquire a positive sample image including a sample object in a defect-free state; the sample composition module 72 is configured to obtain multiple first sample sub-images by using the positive sample image, where each first sample sub-image includes a part of the sample object in a defect-free state, and the combination of the multiple first sample sub-images includes a target object in a defect-free state; the sample processing module 73 is configured to perform a reconstruction process on the multiple first sample sub-images by using the image reconstruction model to obtain multiple sample reconstruction sub-images corresponding to the multiple first sample sub-images; the adjustment module 74 is configured to adjust the network parameters of the image reconstruction model by using the difference between each sample reconstruction sub-image and the corresponding second sample sub-image, where the second sample sub-image corresponding to the sample reconstruction sub-image and the sample reconstruction sub-image both include the same part of the target object.

[0115] In some embodiments, the above-mentioned sample composition module 72 is configured to perform region splitting on the positive sample image to obtain multiple first sample regions, use the pixel values of the first sample regions in the positive sample image to obtain second sample sub-images corresponding to the first sample regions, and use the pixel values of the second image regions in the positive sample image to obtain first sample sub-images corresponding to the first sample regions.

[0116] In some embodiments, the above-mentioned sample processing module 73 is configured to execute by using an encoder module: perform a first convolution process on the input image to obtain a convolution feature of the input image, and based on the convolution feature, obtain an image feature of the input image, and use a decoder module to perform a decoding process on the image feature of the input image to obtain an output image corresponding to the input image.

[0117] In some embodiments, the above-mentioned sample processing module 73 is configured to perform a first convolution process on the input image by using a preset convolution kernel to obtain a convolution feature of the input image, and / or, based on the convolution feature, obtain an image feature of the input image.

[0118] In some embodiments, the above-mentioned sample processing module 73 is used to obtain the feature vector of each positive sample image by using the pixel values of the positive sample image; select the feature vectors of at least one positive sample image that meet the preset feature conditions; and use the selected feature vectors to form a preset convolution kernel.

[0119] In some embodiments, the above-mentioned sample processing module 73 is used to perform a sliding window process on the positive sample image to obtain several image regions corresponding to the sliding windows; count the pixel values of the corresponding pixel points in each image region to obtain the feature values of the corresponding pixel points, and use the feature values of the corresponding pixel points obtained by the statistics to form the feature vector of the positive sample; select the feature vectors of at least one positive sample image that meet the preset feature conditions, including: sorting the vector feature values of the feature vectors of each positive sample image from high to low, and selecting the first preset number of feature vectors.

[0120] It should be noted that the device of this embodiment can execute the steps in the above method. For the detailed description of the relevant content, please refer to the above method part, and it will not be repeated here.

[0121] Please refer to Figure 8 , Figure 8 is a schematic framework diagram of an embodiment of the image processing device provided in the present application. In this embodiment, the image processing device 80 includes a memory 81 and a processor 82.

[0122] The processor 82 can also be called a CPU (Central Processing Unit). The processor 82 may be an integrated circuit chip with signal processing capabilities. The processor 82 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor 82 can also be any conventional processor 82, etc.

[0123] The memory 81 in the image processing device 80 is used to store the program instructions required for the operation of the processor 82.

[0124] The processor 82 is used to execute program instructions to implement the methods provided by any embodiment and any non-conflicting combination of the above-mentioned image processing method and image reconstruction model training method in the present application.

[0125] Please refer to Figure 9 , Figure 9It is a schematic framework diagram of the computer-readable storage medium provided by this application. The computer-readable storage medium 90 of the embodiments of this application stores program instructions 91, and when the program instructions 91 are executed, they implement the methods provided by any one of the image processing method and the image reconstruction model training method of this application and any non-conflicting combination. Among them, the program instructions 91 can form a program file and be stored in the above-mentioned computer-readable storage medium 90 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) can execute all or part of the steps of the methods of various embodiments of this application. And the aforementioned computer-readable storage medium 90 includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0126] In the above solution, the image reconstruction model is trained using positive sample images of sample objects in a defect-free state. If the defects in the input original defective image cannot be reconstructed by the image reconstruction model trained with the positive sample images of the sample objects in the defect-free state, then the combination of multiple reconstructed sub-images output by the image reconstruction model is used to represent the target object in the defect-free state. And during the training process, the convolutional features of the input image and the input image are fused, and then the fused features are subjected to convolutional processing to obtain the image features of the input image. This fusion process preserves the tiny features of the input image to obtain a reconstructed image that retains tiny features, facilitating more accurate subsequent image detection results.

[0127] In addition, during the training process of the image reconstruction model, it automatically learns using image data, which means that the trained image reconstruction model can achieve automatic reconstruction of images, with high reconstruction efficiency. And with the support of a large number of samples, the model does not learn the sample images themselves, but uses known pixels to predict unknown pixels and learns the function of image reconstruction. Therefore, it can also reconstruct other types of images that have not been trained, thus improving the image applicability range of the image reconstruction model.

[0128] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. Its specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0129] The descriptions of the above embodiments tend to emphasize the differences between the embodiments. Their similarities or similarities can be referred to each other. For the sake of brevity, they will not be repeated in this article.

[0130] In several embodiments provided in the present application, it should be understood that the disclosed methods and apparatuses can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical or other forms.

[0131] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0132] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0133] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0134] The above is only the embodiment of the present application, and does not limit the patent scope of the present application. All equivalent structural or equivalent process transformations made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are equally included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that, The method includes: Obtaining an original image of a target object; Using a plurality of first image regions obtained by splitting the original image into regions, to obtain a plurality of target sub-images, wherein each of the first image regions contains different parts of the target object; the target sub-image corresponding to the first image region is obtained by setting the pixel values of the first image region in the original image to a preset pixel value and retaining the pixel values of a second image region other than the first image region in the original image; each of the target sub-images contains part of the target object, and the combination of the plurality of target sub-images contains the target object; Using an image reconstruction model to perform reconstruction processing on the plurality of target sub-images, to obtain a plurality of reconstructed sub-images respectively corresponding to the plurality of target sub-images, wherein the image reconstruction model is trained using positive sample images of a sample object in a defect-free state, and the combination of the plurality of reconstructed sub-images is used to represent the target object in a defect-free state; the reconstruction processing is used to reconstruct the image of the first image region with the preset pixel value in the target sub-image, and the pixel values of the second image region corresponding to the target sub-image in the reconstructed sub-image are the preset pixel values.

2. The method according to claim 1, wherein The splitting of the original image into a plurality of first image regions includes: Dividing the pixel points in the original image into multiple parts by means of average division or random division, wherein each part of the pixel points forms a first image region.

3. The method according to claim 1, characterized in that The plurality of reconstructed sub-images respectively contain different parts of the target object; And / or, the reconstructed sub-image and the corresponding target sub-image respectively contain different parts of the target object, and the combination of each group of the reconstructed sub-image and the corresponding target sub-image contains the target object.

4. The method according to claim 1, characterized in that The image reconstruction model is a convolutional autoencoder, and the convolutional autoencoder includes an encoder module and a decoder module; And / or, before using the image reconstruction model to perform reconstruction processing on the plurality of target sub-images to obtain a plurality of reconstructed sub-images respectively corresponding to the plurality of target sub-images, the method further includes the following training steps of the image reconstruction model: Using the positive sample images to obtain a plurality of first sample sub-images, wherein each of the first sample sub-images contains part of the sample object in the defect-free state, and the combination of the plurality of first sample sub-images contains the sample object in the defect-free state; Using the image reconstruction model to perform reconstruction processing on the plurality of first sample sub-images to obtain a plurality of sample reconstructed sub-images respectively corresponding to the plurality of first sample sub-images; Adjusting the network parameters of the image reconstruction model by using the difference between each sample reconstructed sub-image and the corresponding second sample sub-image, wherein the second sample sub-image corresponding to the sample reconstructed sub-image and the sample reconstructed sub-image both contain the same part of the target object.

5. The method according to claim 4, wherein The obtaining of the plurality of first sample sub-images by using the positive sample images includes: Perform regional splitting on the positive sample image to obtain multiple first sample regions, where each of the first sample regions contains different parts of the sample object; For each of the first sample regions, use the pixel values of the first sample region in the positive sample image to obtain the second sample sub-image corresponding to the first sample region, and use the pixel values of the second image region in the positive sample image to obtain the first sample sub-image corresponding to the first sample region.

6. The method according to claim 4, wherein The step of using the image reconstruction model to perform reconstruction processing on the multiple target sub-images to obtain multiple reconstructed sub-images corresponding to the multiple target sub-images respectively, or the step of using the image reconstruction model to perform reconstruction processing on the multiple first sample sub-images to obtain multiple sample reconstructed sub-images corresponding to the multiple first sample sub-images respectively, includes: Use the multiple target sub-images / the multiple first sample sub-images as multiple input images, and the multiple reconstructed sub-images / the multiple sample reconstructed sub-images as multiple output images; Use the encoder module to perform: perform first convolution processing on the input image to obtain the convolution features of the input image, and based on the convolution features, obtain the image features of the input image; Use the decoder module to perform decoding processing on the image features of the input image to obtain the output image corresponding to the input image.

7. The method according to claim 6, characterized in that, The step of performing first convolution processing on the input image to obtain the convolution features of the input image includes: Use a preset convolution kernel to perform first convolution processing on the input image to obtain the convolution features of the input image, where the preset convolution kernel is determined using the features in at least one positive sample image used for training the image reconstruction model; And / or, the step of obtaining the image features of the input image based on the convolution features includes: Fuse the convolution features and the input image to obtain fused features; Perform second convolution processing on the fused features to obtain the image features of the input image.

8. The method according to claim 7, wherein Before using the preset convolution kernel to perform first convolution processing on the input image to obtain the convolution features of the input image, the method further includes: For each positive sample image, use the pixel values of the positive sample image to obtain the feature vector of the positive sample image; Select at least one feature vector of the positive sample image that satisfies the preset feature condition; Use the selected feature vectors to form the preset convolution kernel.

9. The method according to claim 8, wherein The step of using the pixel values of the positive sample image to obtain the feature vector of the positive sample image includes: Perform a sliding window process on the positive sample image to obtain image regions corresponding to a number of sliding windows; Statistically calculate the pixel values of the corresponding pixel points in each image region to obtain the feature values of the corresponding pixel points, and use the statistically obtained feature values of the corresponding pixel points to form the feature vector of the positive sample; The step of selecting at least one feature vector of the positive sample image that satisfies the preset feature condition includes: Sort the vector feature values of the feature vectors of each positive sample image from high to low, and select the first preset number of feature vectors.

10. The method according to claim 1, characterized in that, After reconstructing the multiple target sub-images using the image reconstruction model to obtain multiple reconstructed sub-images respectively corresponding to the multiple target sub-images, the method further includes: Composing a reference image using the multiple reconstructed sub-images; Performing defect detection on the original image with respect to the target object using the reference image to obtain a defect detection result of the target object in the original image.

11. The method according to claim 10, wherein The target object is a printed matter, and the defect is a printing defect; And / or, the performing defect detection on the original image with respect to the target object using the reference image to obtain a defect detection result of the target object in the original image includes: Calculating the difference between the pixel values of the corresponding pixel points of the original image and the reference image to obtain a reconstruction error; Judging whether the reconstruction error is less than a preset threshold; If so, determining that the target object in the original image has no defect, and if not, determining that the target object in the original image has a defect.

12. A training method for an image reconstruction model, characterized in that, Includes: Obtaining a positive sample image of a sample object in a defect-free state; Using multiple first sample regions obtained by splitting the positive sample image by region to obtain multiple first sample sub-images, where each of the first sample regions contains different parts of the sample object, and the first sample sub-image corresponding to the first sample region is obtained by setting the pixel values of the first sample region in the positive sample image to a preset pixel value and retaining the pixel values of the second sample region other than the first sample region in the positive sample image; each of the first sample sub-images contains a part of the sample object in the defect-free state, and the combination of the multiple first sample sub-images contains the target object in the defect-free state; Reconstructing the multiple first sample sub-images using the image reconstruction model to obtain multiple sample reconstructed sub-images respectively corresponding to the multiple first sample sub-images; the reconstruction process is used to reconstruct the image of the first sample region that the first sample sub-image lacks relative to the positive sample image in the defect-free state, and the pixel values of the second sample region corresponding to the first sample sub-image in the sample reconstructed sub-image are preset pixel values; Adjusting the network parameters of the image reconstruction model using the difference between each sample reconstructed sub-image and the corresponding second sample sub-image, where the second sample sub-image corresponding to the sample reconstructed sub-image and the sample reconstructed sub-image both contain the same part of the target object.

13. The method according to claim 12, wherein The reconstructing the multiple first sample sub-images using the image reconstruction model to obtain multiple sample reconstructed sub-images respectively corresponding to the multiple first sample sub-images includes: Using the encoder module of the image reconstruction model to perform: performing a first convolution process on the first sample sub-image to obtain the convolution features of the first sample sub-image, and based on the convolution features, obtaining the image features of the first sample sub-image; Using the decoder module of the image reconstruction model to perform a decoding process on the image features of the first sample sub-image to obtain the sample reconstructed sub-image corresponding to the first sample sub-image; Among them, the first convolution process on the first sample sub-image to obtain the convolution features of the first sample sub-image includes: using a preset convolution kernel to perform the first convolution process on the first sample sub-image to obtain the convolution features of the first sample sub-image, where the preset convolution kernel is determined using the features in at least one positive sample image for training the image reconstruction model; and / or, based on the convolution features, obtaining the image features of the first sample sub-image includes: fusing the convolution features and the first sample sub-image to obtain a fused feature; performing a second convolution process on the fused feature to obtain the image features of the first sample sub-image.

14. The method according to claim 13, wherein Before performing the first convolution process on the first sample sub-image using the preset convolution kernel to obtain the convolution features of the first sample sub-image, the method further includes: For each of the positive sample images, using the pixel values of the positive sample image to obtain the feature vector of the positive sample image; Selecting the feature vectors of at least one positive sample image that meet the preset feature conditions; Using the selected feature vectors to form the preset convolution kernel.

15. An image processing apparatus, characterized in that, The device includes: An acquisition module, configured to acquire the original image of the target object; A composition module, using a plurality of first image regions obtained by splitting the original image into regions to obtain a plurality of target sub-images, where each of the first image regions contains different parts of the target object; the target sub-image corresponding to the first image region is obtained by setting the pixel values of the first image region in the original image to preset pixel values and retaining the pixel values of the second image region other than the first image region in the original image; each of the target sub-images contains a part of the target object, and the combination of the plurality of target sub-images contains the target object; A processing module, using an image reconstruction model to perform a reconstruction process on the plurality of target sub-images to obtain a plurality of reconstructed sub-images corresponding to the plurality of target sub-images respectively, where the image reconstruction model is trained using positive sample images of a sample object in a defect-free state, and the combination of the plurality of reconstructed sub-images is used to represent the target object in a defect-free state; the reconstruction process is used to reconstruct the image of the first image region that is missing in the target sub-image relative to the original image in a defect-free state, and the pixel values of the second image region corresponding to the target sub-image in the reconstructed sub-image are preset pixel values.

16. A training device for an image reconstruction model, characterized in that, The device includes: A sample acquisition module, configured to acquire positive sample images of a sample object in a defect-free state; A sample composition module that obtains multiple first sample sub-images by using multiple first sample regions obtained by splitting the positive sample image into regions, where each of the first sample regions contains different parts of the sample object, and the first sample sub-image corresponding to the first sample region is obtained by setting the pixel values of the first sample region in the positive sample image to a preset pixel value and retaining the pixel values of the second sample region other than the first sample region in the positive sample image; each of the first sample sub-images contains a part of the sample object in the defect-free state, and the combination of the multiple first sample sub-images contains the target object in the defect-free state; A sample processing module that performs reconstruction processing on the multiple first sample sub-images by using an image reconstruction model to obtain multiple sample reconstruction sub-images corresponding to the multiple first sample sub-images respectively; the reconstruction processing is used to reconstruct the image of the first sample region that is missing from the first sample sub-image relative to the positive sample image in the defect-free state, and the pixel values of the second sample region corresponding to the first sample sub-image in the sample reconstruction sub-image are preset pixel values; An adjustment module that adjusts the network parameters of the image reconstruction model by using the difference between each sample reconstruction sub-image and the corresponding second sample sub-image, where the second sample sub-image corresponding to the sample reconstruction sub-image and the sample reconstruction sub-image both contain the same part of the target object.

17. An image processing apparatus, characterized in that, Comprising a memory and a processor that are coupled to each other, The memory stores program instructions; The processor is configured to execute the program instructions stored in the memory to implement the method according to any one of claims 1-11, and / or, to implement the method according to any one of claims 12-14.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program instructions that can be executed to implement the method according to any one of claims 1-11, and / or, to implement the method according to any one of claims 12-14.

Images