Image Processing Method, Apparatus, Device, and Storage Medium

Abstract

An embodiment of the present invention discloses an image processing method, apparatus, device, and storage medium. The method includes: respectively extracting M feature data from N image regions of an image to be processed, where both M and N are integers greater than or equal to 1, and M is greater than or equal to N, and one image region corresponds to one or more feature data; performing defect detection on the N image regions according to the M feature data to obtain a predicted probability of the presence of a defect in each of the N image regions; obtaining the attention degree of each image region, and adjusting the predicted probability of the presence of a defect in each image region according to the attention degree of each image region; and generating a prediction result of the image to be processed and a confidence level of the prediction result according to the adjusted predicted probability of each image region, which can improve the accuracy of the recognition result of identifying whether an image is a defective image.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to an image processing method, apparatus, device, and storage medium. Background Art

[0002] In order to detect defects in an image (such as defects that make the image unclear, like a screen freeze), existing defect detection methods usually first extract features from the image to be processed, and then identify the extracted features. If it is determined that there are defect features among the extracted features, then the image to be processed is determined to be a defective image. Thus, when using the current image detection method to detect defects in an image, only based on the recognition of feature data, the result of whether the image is a defective image is output in a binary classification. However, since many defective images are only slightly defective, or locally defective, or have small-area defects, and due to the limitations of the features extracted from the image, the result of determining whether the image is a defective image based on the current recognition result of the feature data is unreliable. Therefore, how to improve the accuracy of identifying whether an image is a defective image has become a current research hotspot. Summary of the Invention

[0003] Embodiments of the present invention provide an image processing method, apparatus, device, and storage medium, which can improve the accuracy of the recognition result of identifying whether an image is a defective image.

[0004] On the one hand, an embodiment of the present invention provides an image processing method, including:

[0005] Extract M feature data from N image regions of the image to be processed, where both M and N are integers greater than or equal to 1, and M is greater than or equal to N, and one image region corresponds to one or more feature data;

[0006] Perform defect detection on the N image regions according to the M feature data to obtain the predicted probability that each image region in the N image regions has a defect;

[0007] Obtain the attention degree of each image region, and adjust the predicted probability that each image region has a defect according to the attention degree of each image region;

[0008] Generate a prediction result of the image to be processed and the confidence level of the prediction result according to the adjusted predicted probability of each image region.

[0009] On the other hand, an embodiment of the present invention provides an image processing apparatus, including:

[0010] An extraction unit, configured to extract M feature data from N image regions of a to-be-processed image respectively, where both M and N are integers greater than or equal to 1, and M is greater than or equal to N, and one image region corresponds to one or more feature data;

[0011] A detection unit, configured to perform defect detection on the N image regions according to the M feature data, and obtain a predicted probability of defect existence for each of the N image regions;

[0012] An acquisition unit, configured to acquire the attention degree of each image region;

[0013] An adjustment unit, configured to adjust the predicted probability of defect existence for each image region according to the attention degree of each image region;

[0014] A generation unit, configured to generate a prediction result of the to-be-processed image and a confidence level of the prediction result according to the adjusted predicted probability of each image region.

[0015] In another aspect, an embodiment of the present invention provides an image processing device, including a processor, an input device, an output device, and a memory, where the processor, the input device, the output device, and the memory are interconnected. Among them, the memory is used to store a computer program for supporting the image processing device to execute the above method, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the following steps:

[0016] Extract M feature data from N image regions of a to-be-processed image respectively, where both M and N are integers greater than or equal to 1, and M is greater than or equal to N, and one image region corresponds to one or more feature data;

[0017] Perform defect detection on the N image regions according to the M feature data, and obtain a predicted probability of defect existence for each of the N image regions;

[0018] Acquire the attention degree of each image region, and adjust the predicted probability of defect existence for each image region according to the attention degree of each image region;

[0019] Generate a prediction result of the to-be-processed image and a confidence level of the prediction result according to the adjusted predicted probability of each image region.

[0020] In another aspect, an embodiment of the present invention provides a computer-readable storage medium, where program instructions are stored in the computer-readable storage medium. When the program instructions are executed by a processor, the program instructions are used to execute the image processing method as described in the first aspect.

[0021] In an embodiment of the present invention, after the image processing device obtains the image to be processed, one or more feature data can be extracted from each image region of the image to be processed. Furthermore, the image to be processed can perform defect detection on each image region of the image to be processed according to the extracted feature data, and obtain the prediction probability of whether there is a defect in each image region. In addition, the image processing device will also obtain the attention degree for each image region, and based on the attention degree of each image region, adjust the prediction probability of whether there is a defect in each determined image region, so as to generate the prediction result of the image to be processed and the confidence level of the prediction result by using the adjusted prediction probability. Since the feature data extracted by the image processing device is extracted based on different image regions, the image processing device can fully consider the local features in the image to be processed. Moreover, by adjusting the prediction probability of whether there is a defect in different image regions based on the attention degree of each image region, the prediction result obtained by the image processing device can be adaptively integrated according to the different attention degrees of different image regions, making the obtained prediction result smoother, and further improving the accuracy of defect prediction for the image to be processed. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 It is a schematic diagram of an image processing method provided by an embodiment of the present invention;

[0024] Figure 2 It is a schematic flowchart of an image processing method provided by an embodiment of the present invention;

[0025] Figure 3a It is a schematic diagram of segmenting an image to be processed provided by an embodiment of the present invention;

[0026] Figure 3b It is a schematic diagram of segmenting an image to be processed provided by an embodiment of the present invention;

[0027] Figure 4 It is a schematic flowchart of an image processing method provided by an embodiment of the present invention;

[0028] Figure 5a It is a schematic diagram of performing defect prediction provided by an embodiment of the present invention;

[0029] Figure 5bIt is a schematic diagram for determining the attention degree of an image area provided by an embodiment of the present invention;

[0030] Figure 5c It is a schematic diagram of images with different defect degrees provided by an embodiment of the present invention;

[0031] Figure 5d It is a schematic diagram for performing recognition processing on different images provided by an embodiment of the present invention;

[0032] Figure 6 It is a schematic block diagram of an image processing device provided by an embodiment of the present invention;

[0033] Figure 7 It is a schematic block diagram of an image processing device provided by an embodiment of the present invention. Detailed implementation manners

[0034] An embodiment of the present invention proposes an image processing method, enabling an image processing device to fully consider the attention degrees of different image areas in an image to be processed. Thus, according to the attention degrees of each image area in the image to be processed, an adaptive integration of the prediction probabilities obtained after defect detection using different image areas can be performed, and then a prediction result regarding whether the image to be processed has a defect can be generated. Based on the consideration of the attention degrees of different image areas in the image to be processed during the process of obtaining the prediction result regarding whether the image to be processed has a defect, the image processing device can obtain a more accurate prediction result, achieving an improvement in the prediction accuracy of the image processing device. Among them, a defective image is an image in which the image information is incomplete due to the existence of a defect. The defective image includes a screen-frozen image or a blurred image, etc. The existence of the defective image will cause the original image to be unclear, thus affecting the user's viewing of the original image. The defective image can be, for example, a screen-frozen image or a blurred image. A screen-frozen image refers to an image in which the original image is covered with dot-like, patch-like, or mosaic interference. A blurred image refers to an image with a low resolution. And defect detection is used to determine whether there is the above-mentioned defect in the image. In one embodiment, after performing defect detection on the image, the probability that the detected image (such as the above-mentioned image to be processed) is a defective image will be output. Then, based on this probability, a binary detection result regarding whether the detected image is a defective image can be determined. Then, according to the binary detection result of the detected image, the image processing device can determine the subsequent processing rule for the detected image. In one embodiment, if the binary detection result of the detected image is that the detected image is a defective image, then the image processing device will first perform image restoration processing on the detected image to eliminate the defect in the detected image, and then output and display the restored image; or, if the image processing device determines that the binary detection result of the detected image is that the detected image is a normal image, the detected image can be directly output and displayed.

[0035] In one embodiment, the image processing method may be executed by invoking a trained recognition model in an image processing device. When executing the image processing method, the trained recognition model may be as follows Figure 1 As shown, the trained recognition model includes a feature extraction network, a prediction network, and a classifier. Among them, the feature extraction network is used to extract image features, and the image features include gradient features, texture features, etc. The feature extraction network may be a Convolutional Neural Networks (CNN) structure, and the convolutional neural network may be any convolutional neural network, and a neural network with a larger or smaller computational amount may be used based on actual application requirements. Thus, when the trained recognition model uses the feature extraction network to extract image features, it can not only obtain a good extraction effect but also ensure the efficiency of feature extraction. In addition, the prediction network is used to perform defect prediction on the image to be processed according to the feature data extracted by the feature extraction network, and determine the prediction result (or prediction results) of the image to be processed. The classifier is used to perform binary classification on whether the image to be processed is a defective image according to the prediction result obtained by the prediction network. The result of the binary classification indicates that the image to be processed is determined to be a defective image or the image to be processed is a non-defective image (i.e., a normal image). In one embodiment, the classifier may be a classifier of a Support Vector Machine (SVM), or it may also be a tree-based classifier. In the embodiments of the present invention, the specific network structures in the trained recognition model are not limited. When the image processing device invokes the trained recognition model to process the image to be processed, it may be as follows Figure 1 As shown, and the following steps will be specifically executed:

[0036] ① After determining the image to be processed, the image processing device may input the image to be processed into the trained recognition model. Then, after the trained recognition model obtains the image to be processed, it may first invoke the feature extraction network to perform image segmentation processing on the image to be processed, and obtain one or more image regions of the image to be processed. Further, the feature extraction network may separately extract features from each image region to extract one or more feature data from each image region. After extracting the features of each image region, the feature extraction network will send the extracted feature data to the prediction network.

[0037] ② After receiving the feature data sent by the feature extraction network, the prediction network can predict the prediction probability of defects in each image region based on the obtained feature data. After predicting the prediction probability of defects in each image region, the attention of each image region can be further obtained. Thus, according to the attention of each image region, the prediction probability of defects in the corresponding image region can be adjusted, and the adjusted prediction probability is sent to the classifier.

[0038] ③ After obtaining the adjusted prediction probability from the prediction network, the classifier can determine the prediction result of the image to be processed according to the defect classification result. And the classifier can also add a classification label to the image to be processed according to the obtained prediction result. In one embodiment, the classification labels added by the classifier to the image to be processed include a label for indicating that the image to be processed is a defective image and a label for indicating that the image to be processed is a normal image. Among them, the label added to indicate that the image to be processed is a defective image can be 1, and correspondingly, the label added to indicate that the image to be processed is a normal image can be 0. In the embodiments of the present invention, the form of the classification labels added by the classifier to the image to be processed is not limited.

[0039] Please refer to Figure 2 , which is a schematic flowchart of an image processing method proposed in the embodiments of the present invention. This image processing method can be executed by the above-mentioned image processing device, such as Figure 2 shown. The method may include:

[0040] S201, extract M feature data from N image regions of the image to be processed respectively, and one image region corresponds to one or more feature data.

[0041] In one embodiment, the image to be processed can be any picture image, or it can also be any image frame in a video. After obtaining the image to be processed, the image processing device can first perform image segmentation processing on the image to be processed to obtain N image regions of the image to be processed, where N is an integer greater than or equal to 1. Among them, when the image processing device performs image segmentation on the image to be processed, the image processing device can adopt, for example, Figure 3aThe image to be processed is segmented using the regularized segmentation method (or sampling method) shown, so as to ensure that the image regions obtained by segmentation can cover the entire region of the image to be processed. Moreover, since the image processing device performs feature extraction based on each image region obtained by segmentation during subsequent feature extraction, it can enable the image processing device to ensure that all features of the image to be processed are obtained subsequently. Then, it can avoid the problem of missing feature extraction of local defects in the image to be processed due to only local or minor defects existing in the image to be processed, thereby ensuring the reliability of the feature data extracted by the image processing device. Among them, as Figure 3a shown, if N = 5, the image to be processed can be the image marked by 301 in Figure 3a . Then, the five image regions obtained by the image processing device based on the image segmentation processing of the image to be processed can be the image regions marked by each dotted box in Figure 3a . It should be noted that in the embodiments of the present invention, a defective image is taken as an example of a mosaic image for detailed description. When the defective image is other defective images, reference can be made to the embodiments of the present invention. In addition, in the embodiments of the present invention, gray scale is used to occlude the original image as a schematic illustration of the mosaic in the image, that is, the mosaic in the image can be, for example, Figure 3a the gray scale region marked by 301 in

[0042] In one embodiment, when the image processing device performs segmentation processing on the image to be processed and then obtains N image regions of the image to be processed, the image processing device can also randomly obtain N image regions from the image to be processed based on the image content in the image to be processed, as Figure 3b shown, where, as Figure 3b the defect in the image described, can also be caused by, for example, Figure 3bFor the gray area marked by 302 in the figure, when the image processing device randomly obtains N image regions from the image to be processed based on the image content, it can first determine the image elements in the image content that have a greater impact on the user's viewing experience when defects occur. Thus, the image processing device can select the region where the image elements that have a greater impact on the user's viewing experience are located as an image region of the image to be processed. For example, if the image content of the image to be processed includes a human image, it can be considered that the human image in the image to be processed will greatly affect the user's viewing experience when there are defects. Therefore, the image processing device can select the regions including human features, such as facial features, hand features, etc. as the image regions of the image to be processed. Or, if the image processing device believes that if there are defects in the central part of the image to be processed, it will affect the user's viewing experience, then after obtaining the image to be processed, the image processing device can select the central region of the image to be processed as an image region of the image to be processed. It can be understood that other ways of segmenting the image to be processed to obtain one or more image regions of the image to be processed can also be applied to the embodiments of the present invention.

[0043] After the image processing device obtains N image regions from the image to be processed, it can perform feature extraction on each image region respectively to extract M feature data from the N images of the image to be processed. M is an integer greater than or equal to 1, and M is greater than or equal to N. After the image processing device obtains the M feature data, it can perform defect detection on the corresponding image region according to the M feature data to obtain the predicted probability that each image region has a defect, that is, then execute step S202. In one embodiment, when the image processing device performs feature extraction on each image region, it can extract one or more feature data from each image region. In one embodiment, when the image processing device extracts multiple feature data from each image region, it can also directly perform defect detection on the image region based on the multiple feature data of the image region to obtain the predicted probability that the image region has a defect; or, when the image processing device extracts multiple feature data from each image region, it can also first perform feature fusion on the extracted multiple feature data to obtain a target feature data corresponding to the image region, and then perform defect detection on the image region based on the obtained target feature data to determine the predicted probability that the image region has a defect.

[0044] S202. Perform defect detection on the N image regions according to the M feature data to obtain the predicted probability that each image region in the N image regions has a defect.

[0045] After an image processing device extracts M feature data from N image regions of an image to be processed respectively, the image processing device can perform defect detection on each image region of the image to be processed based on the extracted M feature data. It can be understood that the defect detection performed by the image processing device on each image region is used to determine whether there is a defect in the image to be processed and the probability of the existence of a defect in this region. Then, based on the M feature data extracted by the image processing device, defect detection is performed on each image region to obtain the predicted probability of the existence of a defect in each image region. The magnitude of the corresponding probability value of the predicted probability of the existence of a defect can be used to reflect the possibility of the existence of a defect (or a defective image) in the corresponding image region. Generally speaking, if, after the image processing device performs defect detection on an image region, the obtained predicted probability of the existence of a defect in this image region is relatively high, it indicates that the possibility of the existence of a defect in this image region is relatively large. If the predicted probability of the existence of a defect in an image region is 0.9, the image processing device can consider that there is a great possibility that this image region has a defect.

[0046] In one embodiment, when the image processing device performs defect detection on the N image regions based on M feature data, the image processing device will perform defect detection on each of the N image regions. It should be noted that when the image processing device performs defect detection on each image region based on the M feature data, the image processing device can perform defect detection on the corresponding image region according to one or more feature data corresponding to each image region among the M feature data. For example, if the image region currently being defect-detected by the image processing device is image region A in the image to be processed, and the feature data extracted from image region A includes feature data 1 and feature data 2, then when the image processing device performs defect detection on image region A, the image processing device selects feature data 1 and feature data 2 from the obtained M feature data, and uses feature data 1 and feature data 2 to perform defect detection on image region A. In another embodiment, when the image processing device performs defect detection on each image region, in addition to referring to the feature data extracted from the image region being detected, it can also refer to the feature data extracted from other image regions adjacent to the image region being detected. If the image region currently being defect-detected is image region A in the image to be processed, and the feature data extracted from image region A includes feature data 1 and feature data 2, and the image regions adjacent to image region A include image region B, then when the image processing device performs defect detection on image region A, it will select feature data 1 and feature data 2 from the obtained M feature data, and will also obtain the feature data (assumed to be feature data 3) extracted from image region B. Then, when the image processing device performs defect detection on image region A, it will use feature data 1, feature data 2, and feature data 3 to perform defect detection on image region A.

[0047] It can be understood that the process of the image processing device performing defect detection on each image region based on the feature data is a process of feature recognition for each piece of feature data. The image processing device can be a terminal device, such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc.; or, the image processing device can also be a server, which can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. In addition, the image processing device can also be a processing module in the terminal device or the server. After the image processing device obtains the prediction probability that each image region has a defect, it can generate the prediction result of the image to be processed and the confidence level of the prediction result based on the attention degree of each image region and in combination with the prediction probability that each image region has a defect, that is, the image processing device can then execute step S203.

[0048] S203. Obtain the attention degree of each image region, and adjust the prediction probability that each image region has a defect according to the attention degree of each image region.

[0049] S204. Generate the prediction result of the image to be processed and the confidence level of the prediction result according to the adjusted prediction probability of each image region.

[0050] In steps S203 and S204, after the image processing device determines the predicted probability of defects in each image area of the image to be processed, it can further obtain the attention degree of each image area. In one embodiment, the attention degree for each image area can be preset into the image processing device by a technician based on the experience value of whether the image has defects. For example, according to the experience value, it is considered that the probability of defects in images related to people is relatively high. Then, when the image processing device obtains the attention degree corresponding to the image area in the image to be processed, the attention degree of the image area including content related to people obtained by the image processing device is relatively high. For example, if the image processing device determines that the image to be processed includes image area A and image area B respectively, and image area A includes content related to people (such as a face image), while image area B does not include content related to people, then the attention degree of the image processing device for image area A must be greater than the attention degree for image area B obtained. In another implementation manner, the image processing device can also determine the attention degree of each image area according to the influence degree of each image on the user's viewing of the image after the defect. For example, when the central area of an image has a higher influence degree on the user's viewing of the image after the defect, and the edge position of an image has a lower influence degree on the user's viewing of the image after the defect. Then, if the image processing device determines that the obtained image area A included in the image to be processed is the central area image of the image to be processed, and the obtained image area B is the edge area image of the image to be processed, then the attention degree of the image processing device for image area A will be greater than the attention degree for image area B obtained.

[0051] In addition, in another embodiment, the image processing device may also use an attention mechanism to determine the attention degree of each image. The attention mechanism is similar to the attention mechanism of human vision. It uses artificial intelligence (AI) technology to enable a machine (such as an image processing device) to focus its attention on important points among numerous pieces of information, select key information, and ignore other unimportant information. Then, based on the attention mechanism, the image processing device can determine the image regions that need to be focused on from the to-be-processed image. Furthermore, the attention degree of the obtained image regions that need to be focused on will be greater than that of the image regions that do not need to be focused on. Among them, artificial intelligence technology refers to the theory, method, technology, and application system that use a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines to enable the machines to have the functions of perception, reasoning, and decision-making.

[0052] In one embodiment, artificial intelligence technology is an interdisciplinary subject with a wide range of fields involved, including both hardware-level technologies and software-level technologies. The basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, large image processing technology, operation / interaction systems, and mechatronics. The artificial intelligence software technologies mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning. The embodiments of the present invention mainly relate to the field of computer vision (CV) technology in artificial intelligence technology. Computer vision is a science that studies how to enable machines to "see". Further, it refers to using cameras and computers to replace human eyes to perform machine vision such as target recognition, tracking, and measurement, and further performing graphic processing to make the images processed by the computer more suitable for human eyes to observe or be transmitted to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes technologies such as image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, three-dimensional technology, virtual reality, augmented reality, simultaneous localization and mapping, and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0053] After the image processing device obtains the attention degree for each image region, the prediction probability of defects in the corresponding image region can be adjusted according to the attention degree of each image region. In a specific implementation, when the image processing device adjusts the prediction probability of defects in the corresponding image region based on the attention degree corresponding to the image region, the prediction probability of defects in each corresponding image region can be adjusted according to a certain ratio, and this adjustment ratio is positively correlated with the attention degree of each image region. That is to say, if the attention degree of an image region obtained by the image processing device is greater than the attention degree of another image region obtained, then the adjustment ratio of the prediction probability of defects in this one image region by the image processing device is greater than the adjustment ratio of the prediction probability of defects in the other image region. For example, if the attention degree of the image processing device for image region A is greater than the attention degree for image region B, then the adjustment ratio of the prediction probability of defects in image region A by the image processing device is a, and the adjustment ratio of the prediction probability of defects in image region B is b, and a is greater than b.

[0054] After the image processing device adjusts the prediction probability of defects in the corresponding image region based on the attention degree of each image region, the adjusted prediction probability of defects corresponding to each image region can be obtained. Then, after the image processing device obtains the adjusted prediction probability of each image region, further, the image processing device can generate the prediction result of the image to be processed and the confidence level of this prediction result based on the adjusted prediction probability of each image region. When the image processing device generates the prediction result of the image to be processed based on the adjusted prediction probability of each image region, if the adjusted prediction probability of an image region is greater than or equal to the prediction threshold, it can be considered that the prediction result of the image to be processed is: the image to be processed is a defective image; and if the adjusted prediction probability of an image region is less than the prediction threshold, it can be considered that the prediction result of the image to be processed is that the image to be processed is a normal image. To ensure the prediction accuracy of the image processing device, when the image processing device determines whether the image to be processed is a defective image based on the prediction probability of whether there are defects, the prediction threshold can be set to a small value, such as 0 or 0.1, etc., so that the image processing device can effectively avoid missing the detection of defective images. In addition, the image processing device can also determine the confidence level of the prediction result for the image to be processed according to the adjusted prediction probability. Specifically, the image processing device can directly use the adjusted prediction probability as the confidence level of the prediction result corresponding to the image to be processed, or the image processing device can also use the normalized prediction result as the confidence level of the prediction result of the image to be processed after normalizing the prediction result.

[0055] In an embodiment of the present invention, after the image processing device obtains the image to be processed, one or more feature data can be respectively extracted from each image region of the image to be processed. Further, the image to be processed can perform defect detection on each image region of the image to be processed according to the extracted feature data, and obtain the prediction probability of the existence of defects in each image region. In addition, the image processing device will also obtain the attention degree for each image region, and can further adjust the prediction probability of the existence of defects in each image region based on the attention degree of each image region, so as to generate the prediction result of the image to be processed and the confidence level of the prediction result by using the adjusted prediction probability. Since the feature data extracted by the image processing device is extracted based on different image regions, the image processing device can fully consider the local features in the image to be processed. Moreover, by adjusting the prediction probability of whether there are defects in different image regions based on the attention degree of each image region, the prediction result obtained by the image processing device can be adaptively integrated according to the different attention degrees of different image regions, making the prediction result smoother, and further improving the accuracy of defect prediction for the image to be processed.

[0056] Please refer to Figure 4 , which is a schematic flowchart of an image processing method provided by an embodiment of the present invention. This image processing method can also be executed by the above-mentioned image processing device, such as Figure 4 shown, the method may include:

[0057] S401. Respectively extract M feature data from N image regions of the image to be processed, where both M and N are integers greater than or equal to 1, and M is greater than or equal to N. One image region corresponds to one or more feature data.

[0058] S402. Perform defect detection on the N image regions according to the M feature data, and obtain the prediction probability of the existence of defects in each of the N image regions.

[0059] In step S401 and step S402, after the image processing device obtains the image to be processed, it can first divide the image to be processed, and then obtain one or more image regions obtained by the division. After the image processing device obtains one or more image regions obtained by the division, it can call the trained recognition model to perform feature extraction processing on any image region to obtain one or more feature data corresponding to each image region. In one embodiment, when the image processing device obtains one or more image regions obtained by the division, it can first perform plane interpolation processing on each image region obtained by the division to adjust the region size of each image region, where the adjusted region size of each image region is the same. Further, the image processing device can use the adjusted one image region as one image region obtained by the division. Such asFigure 5a As shown, after the image processing device obtains the image to be processed, it can perform random image block cropping on the image to be processed. After the image processing device obtains the cropped image block (or image region), the image processing device can perform planar interpolation processing on the cropped image block, so that the cropped image block is of a fixed size, for example, it can be 224 millimeters × 224 millimeters. Further, the image processing device will perform feature extraction on each obtained image region through a convolutional neural network. It should be noted that the weights of the convolutional neural network here are shared, so there is no need to train five convolutional neural networks, which greatly saves parameters and the computing power of the Graphics Processing Unit (GPU). For the features extracted from each image region, the image processing device can keep the feature data extracted from each image region unchanged, so as to perform defect detection processing on the corresponding image region based on the feature data extracted from each image region. Or, the image processing device can also perform feature fusion on the feature data extracted from each image region, and then determine the prediction probability of the corresponding image region having a defect based on the fused feature data corresponding to each image region. And then the prediction result and the corresponding confidence level of the image to be processed can be obtained.

[0060] In one embodiment, when the image processing device performs segmentation on the image to be processed, it can be segmented in the manner as Figure 5a shown, or the image processing device can also perform segmentation on the image to be processed according to the regular sampling method as Figure 3a shown to achieve full coverage of the regions of the image to be processed. After the image processing device finishes segmenting the image to be processed and obtains multiple image regions, it can perform feature extraction on each image region to obtain the feature data of each image region, and perform defect detection processing on the corresponding image region according to the feature data of each image region to obtain the prediction probability of each image region having a defect. After the image processing device obtains the prediction probability of each image region having a defect, it can further obtain the attention degree of each image region, that is, turn to execute step S403.

[0061] S403. Obtain the attention degree of each image region, and the attention degree includes the prediction weight of the prediction probability.

[0062] S404. Use the prediction weight of each image region to perform weighted processing on the prediction probability of the corresponding image region having a defect, and use the prediction probability after weighted processing of each image region as the adjusted prediction probability of the corresponding image region.

[0063] In steps S403 and S404, the attention degree obtained by the image processing device includes the prediction weight for the prediction probability of defects. The way for the image processing device to obtain the attention degree of the target image area includes: using an attention mechanism to determine the attention area from the image to be processed, and obtaining the overlapping area between the target image area and the attention area, so that the image processing device can determine the attention degree of the target image area according to the area of the overlapping area; among them, the attention degree of the target image area is positively correlated with the size of the overlapping area. As Figure 5b shown, if the attention area determined by the image processing device from the image to be processed using the attention mechanism is the area marked by 50 as shown in Figure 5b , and the image areas obtained from the image to be processed include the area marked by 51 as shown in Figure 5b , and the area marked by 52 as shown in Figure 5b , then, since the overlapping area between the attention area 50 and the image area 51 is larger than the overlapping area between the attention area 50 and the image area 52, then, the image processing device determines that the attention degree of the image area 51 is greater than that of the image area 52.

[0064] In one embodiment, the attention mechanism is included in the trained recognition model. Among them, the training process of the recognition model includes: obtaining a sample image, and the sample image is added with an annotation label, and the annotation label is used to indicate whether the sample image is a defective image (such as the above-mentioned screen freeze image, etc.); among them, if the annotation label added to the sample image is 1, it can indicate that the sample image is a defective image, and if the annotation label added to the sample image is 0, it can indicate that the sample image is not a defective image. After the image processing device obtains the sample image, it can call the initial recognition model to determine the prediction label of the sample image, and then adjust the model parameters of the initial recognition model according to the difference between the prediction label and the annotation label to obtain the trained recognition model. Since the annotation label added to the sample image is a binary label (such as 0 or 1 as mentioned above), and the binary label is only used to indicate whether the image is defective, but when the defective image is a local defect, such an annotation label is a weak label with a relatively high uncertainty. Therefore, when the image processing device trains the recognition model, an attention mechanism will be introduced to reduce the impact of regional defects on model training. Specifically, if the sample image is also divided into N sample areas, then the predicted value of the prediction result of the i-th sample area obtained by using the recognition model may be p i , when training the recognition model, due to the introduction of the attention mechanism, therefore, the image processing device will train the attention parameter W i , and this parameter will be gradually updated as the network training of the recognition model progresses, and the trained attention parameter W iAs the attention degree of each sample region, it enables the image processing device to integrate the prediction probabilities of defects existing in each sample region. Therefore, the attention degree W of each image region is adopted. i The prediction probability p of defects existing in each image region i After weighted processing, the adjusted prediction probability obtained can be p, and there is an expression as shown in Equation 1:

[0065]

[0066] where W i is the attention degree of each image region (i.e., the prediction weight of the prediction probability of defects existing in each image region), and p i is determined by using the feature data of each image region for defect detection, and is the prediction probability of defects existing in each image region. In an embodiment, the significance of the attention weight W i is to adaptively evaluate the contribution degree of the current image region to the defect classification judgment. When the network training gradually converges, for the image blocks in the above local defect images that do not have defect regions, the weight value of W i is relatively low, while for the local image blocks with defects, W i will be relatively high. In this way, the result after fusion can fully consider the importance of each image block and obtain a stable output result. That is to say, when the image processing device adjusts the model parameters of the initial recognition model to obtain a trained recognition model, it is to adjust the attention parameters to reduce the difference between the predicted label and the annotated label, and when the difference between the predicted label and the annotated label reaches the minimum value, a trained recognition model is obtained. In an embodiment, the softmax cross-entropy loss function (a kind of logistic regression loss function) can be used to indicate the difference between the predicted label and the annotated label, and the softmax cross-entropy loss function can be as shown in Equation 2:

[0067] Loss = -[ylogp+(1 - y)log(1 - p)] Equation 2

[0068] where y represents the annotated information on whether the sample image is defective (such as the identification label of the sample image), p represents the probability value output by the model after integrating multiple image regions, and p can be calculated by the above Equation 1. The loss obtained between the two is used to drive the weight update of the model, thereby optimizing the entire recognition model, and thus completing the optimization of the recognition model to obtain a trained recognition model.

[0069] After the image processing device obtains the attention degree of each image region, the image processing device can then use the prediction weight of each image region to perform a weighted processing on the prediction probability of the corresponding image region having a defect. That is, the image processing device can sum up the prediction probabilities after the weighted processing of each image region to obtain the prediction probability after the weighted summation. Among them, the prediction probability after the weighted summation of each image region is the prediction probability of the corresponding image region having a defect after adjustment. After the image processing device obtains the prediction probability of each image region having a defect after adjustment, it can then proceed to step S405.

[0070] S405. Generate the prediction result of the image to be processed and the confidence level of the prediction result according to the prediction probability after adjustment of each image region.

[0071] In one embodiment, the prediction probability after adjustment of the image processing device is the prediction probability after performing a weighted processing on the prediction probability of the corresponding image region having a defect. As a result, the image processing device can generate the prediction result of the image to be processed and the confidence level of the prediction result based on the prediction probability after the weighted processing. In a specific implementation, when the prediction probability after the weighted processing is greater than the probability threshold, the image processing device can determine that the image to be generated is a defective image, and the confidence level that the image to be generated is a defective image is the weight value of the prediction probability after the weighted summation. After the image processing device determines the confidence level of the prediction result of the image to be processed, since the confidence level can be used to indicate the accuracy of the prediction result, the image processing device can add a defect label to the image to be processed if the confidence level is greater than or equal to the first threshold; otherwise, add a normal label to the image to be processed. Among them, the first threshold can be, for example, values such as 0.9 or 0.86. In addition, the confidence level can also be used to reflect the defect degree of the image to be processed. It can be understood that the greater the confidence level, the higher the corresponding defect degree. Among them, the defect degree can be determined based on the defect area, that is, the larger the defect area, the higher the defect degree. As Figure 5c shown, the image processing device can determine different defect degrees based on the area with defects; in addition, the defect degree can also be determined based on the magnitude of the pixel difference in the image. Therefore, after the image processing device determines the confidence level of the prediction result of the image to be processed, if the confidence level is greater than or equal to the second threshold, it indicates that the defect degree of the image to be processed is relatively high, and then the image to be processed can be restored to eliminate the defects in the image to be processed.

[0072] In one embodiment, the prediction result and the prediction result can be used to detect the image quality. If the image to be processed is an image frame in a video, the image processing device can use the video frame as the image to be processed, and then determine the data quality of the video data based on the prediction result of the video frame and the corresponding confidence level. As Figure 5dAs shown, if the image frame obtained by the image processing device from the video data for use as the image to be processed is as shown in the image marked by 501 in Figure 5d , then after the image processing device obtains the image frame of the image to be processed from the video data, it can call the trained recognition model to obtain the prediction result and the corresponding confidence level for the image frame 501. For example, the output prediction result is as shown in Figure 5d "defective image", and the confidence level is 0.95; and when the prediction result is "normal image", the confidence level is 0.05; then the image processing device can determine that the image frame 501 is a defective image and has a relatively high degree of defect. Or, if the image frame obtained by the image processing device from the video data for use as the image to be processed is as shown in the image marked by 502 in Figure 5d , then after the image processing device obtains the image frame of the image to be processed from the video data, it can call the trained recognition model to obtain the prediction result and the corresponding confidence level for the image frame 502. For example, the output prediction result is as shown in Figure 5d "normal image", and the confidence level is 0.99, and the confidence level for "defective image" is 0.01. Then, the image processing device can determine that the image frame 502 is a normal image.

[0073] In the embodiment of the present invention, after the image processing device obtains a plurality of image regions from the image to be processed and extracts a plurality of feature data from the image regions, the image processing device can perform defect detection on each image region according to the extracted feature data to obtain the prediction probability that each image region has a defect. Furthermore, after obtaining the attention degree of each image region, according to the prediction weight indicated by the attention degree of each image region, the prediction probability that the corresponding image region has a defect is weighted, and the weighted prediction probability is used as the adjusted prediction probability of the corresponding image region. Then, the image processing device can generate the prediction result of the image to be processed and the confidence level of the prediction result according to the adjusted prediction probability of each image region. Thus, the accuracy of the prediction result obtained by the image processing device for defect detection can be improved.

[0074] Based on the description of the above embodiment of the image processing method, an embodiment of the present invention also proposes an image processing device, and this image processing device can be a computer program (including program code) running in the above image processing device. This image processing device can be used to execute the image processing method as described in Figure 2 and Figure 4 . Please refer to Figure 6 . This image processing device includes: an extraction unit 601, a detection unit 602, an acquisition unit 603, an adjustment unit 604, and a generation unit 605.

[0075] An extraction unit 601 is configured to extract M feature data from N image regions of a to-be-processed image respectively, where both M and N are integers greater than or equal to 1, and M is greater than or equal to N, and one image region corresponds to one or more feature data.

[0076] A detection unit 602 is configured to perform defect detection on the N image regions according to the M feature data, and obtain a predicted probability of each of the N image regions having a defect.

[0077] An acquisition unit 603 is configured to acquire the attention degree of each of the image regions.

[0078] An adjustment unit 604 is configured to adjust the predicted probability of each of the image regions having a defect according to the attention degree of each of the image regions.

[0079] A generation unit 605 is configured to generate a prediction result of the to-be-processed image and a confidence level of the prediction result according to the adjusted predicted probability of each of the image regions.

[0080] In one embodiment, the to-be-processed image includes a target image region; the acquisition unit 603 is configured to:

[0081] Determine a focus region from the to-be-processed image by using an attention mechanism, and acquire an overlapping region between the target image region and the focus region.

[0082] Determine the attention degree of the target image region according to the area of the overlapping region; wherein, the attention degree of the target image region is positively correlated with the area size of the overlapping region.

[0083] In one embodiment, the attention mechanism is included in a trained recognition model, and the acquisition unit 603 is specifically configured to:

[0084] Acquire a sample image, where the sample image is added with an annotation label for indicating whether the sample image is a defective image.

[0085] Call an initial recognition model to determine a prediction label of the sample image.

[0086] Adjust model parameters of the initial recognition model according to a difference between the prediction label and the annotation label to obtain a trained recognition model.

[0087] In one embodiment, the adjustment unit 604 is specifically configured to:

[0088] Adjust the attention parameter to reduce the difference between the predicted label and the annotated label, and obtain a trained recognition model when the difference between the predicted label and the annotated label reaches the minimum value.

[0089] In one embodiment, the attention degree includes the prediction weight; the adjustment unit 604 is specifically configured to:

[0090] Use the prediction weight of each image region to weight the prediction probability of the corresponding image region having a defect, and use the weighted prediction probability of each image region as the adjusted prediction probability of the corresponding image region.

[0091] In one embodiment, the generation unit 605 is specifically configured to:

[0092] Sum up the weighted prediction probabilities of each image region to obtain the weighted sum of prediction probabilities;

[0093] If the weighted prediction probability is greater than the probability threshold, determine that the image to be generated is a defective image, and the confidence that the image to be generated is a defective image is the weight value of the weighted sum of prediction probabilities.

[0094] In one embodiment, the confidence is used to indicate the accuracy of the prediction result; the apparatus further includes: an adding unit 606.

[0095] The adding unit 606 is configured to add a defect label to the image to be processed if the confidence is greater than or equal to the first threshold; otherwise, add a normal label to the image to be processed.

[0096] In one embodiment, the confidence is used to reflect the defect degree of the image to be processed; the apparatus further includes: a restoring unit 607.

[0097] The restoring unit 607 is configured to perform a restoration process on the image to be processed if the confidence is greater than or equal to the second threshold to eliminate the defect in the image to be processed.

[0098] In one embodiment, the extraction unit 601 is specifically configured to:

[0099] Divide the image to be processed and obtain one or more image regions obtained by the division;

[0100] Call the trained recognition model to perform feature extraction processing on any image region to obtain one or more feature data corresponding to each image region.

[0101] In one embodiment, the extraction unit 601 is specifically configured to:

[0102] Perform planar interpolation processing on each of the divided image regions to adjust the region size of each image region, where the region sizes of each adjusted image region are the same;

[0103] Use one of the adjusted image regions as one of the divided image regions.

[0104] In an embodiment of the present invention, after obtaining the image to be processed, the extraction unit 601 can extract one or more feature data from each image region of the image to be processed. Furthermore, the detection unit 602 can perform defect detection on each image region of the image to be processed according to the extracted feature data to obtain the predicted probability of defects in each image region. In addition, the acquisition unit 601 will also obtain the attention degree for each image region, and the adjustment unit 601 can then adjust the predicted probability of defects in each image region based on the attention degree of each image region, so that the generation unit 605 can generate the prediction result of the image to be processed and the confidence level of the prediction result using the adjusted predicted probability. Since the extracted feature data is extracted based on different image regions, local features in the image to be processed can be fully considered. Moreover, by adjusting the predicted probability of defects in different image regions based on the attention degree of each image region, the obtained prediction result can be adaptively integrated according to the different attention degrees of different image regions, making the prediction result smoother, and further improving the accuracy of defect prediction for the image to be processed.

[0105] Please refer to Figure 7 , which is a schematic block diagram of the structure of an image processing device provided by an embodiment of the present invention. As Figure 7 shown, the image processing device in this embodiment may include: one or more processors 701; one or more input devices 702, one or more output devices 703, and a memory 704. The above-mentioned processor 701, input device 702, output device 703, and memory 704 are connected through a bus 705. The memory 704 is used to store computer programs, and the computer programs include program instructions. The processor 701 is used to execute the program instructions stored in the memory 704.

[0106] The memory 704 may include volatile memory, such as random-access memory (RAM); the memory 704 may also include non-volatile memory, such as flash memory, solid-state drive (SSD), etc.; the memory 704 may also include a combination of the above types of memory.

[0107] The processor 701 may be a central processing unit (CPU). The processor 701 may further include a hardware chip. The above-mentioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), etc. The PLD may be a field-programmable gate array (FPGA), a generic array logic (GAL), etc. The processor 701 may also be a combination of the above structures.

[0108] In the embodiment of the present invention, the memory 704 is used to store a computer program, the computer program includes program instructions, and the processor 701 is used to execute the program instructions stored in the memory 704 to implement the corresponding method steps as described above in Figure 2 and Figure 4 above.

[0109] In one embodiment, the processor 701 is configured to call the program instructions to perform:

[0110] Extract M feature data from N image regions of the image to be processed respectively, both M and N are integers greater than or equal to 1, and M is greater than or equal to N, and one image region corresponds to one or more feature data;

[0111] Perform defect detection on the N image regions according to the M feature data to obtain the predicted probability of each image region in the N image regions having a defect;

[0112] Obtain the attention degree of each image region, and adjust the predicted probability of each image region having a defect according to the attention degree of each image region;

[0113] Generate a prediction result of the image to be processed and a confidence level of the prediction result according to the adjusted predicted probability of each image region.

[0114] In one embodiment, the image to be processed includes a target image region, and the processor 701 is configured to call the program instructions to perform:

[0115] Determine a region of interest from the image to be processed by using an attention mechanism, and obtain an overlapping region between the target image region and the region of interest;

[0116] Determine the attention degree of the target image region according to the area of the overlapping region; wherein, the attention degree of the target image region is positively correlated with the size of the area of the overlapping region.

[0117] In one embodiment, the attention mechanism is included in the trained recognition model, and the processor 701 is configured to call the program instructions to perform:

[0118] Obtain a sample image, and the sample image is added with an annotation label, and the annotation label is used to indicate whether the sample image is a defective image;

[0119] Call the initial recognition model to determine the predicted label of the sample image;

[0120] Adjust the model parameters of the initial recognition model according to the difference between the predicted label and the annotation label to obtain a trained recognition model.

[0121] In one embodiment, the processor 701 is configured to call the program instructions to perform:

[0122] Adjust the attention parameters to reduce the difference between the predicted label and the annotation label, and obtain a trained recognition model when the difference between the predicted label and the annotation label reaches the minimum value.

[0123] In one embodiment, the attention degree includes the prediction weight of the prediction probability; the processor 701 is configured to call the program instructions to perform:

[0124] Use the prediction weight of each image region to weight the prediction probability of the presence of defects in the corresponding image region, and use the weighted prediction probability of each image region as the adjusted prediction probability of the corresponding image region.

[0125] In one embodiment, the processor 701 is configured to call the program instructions to perform:

[0126] Sum up the weighted prediction probabilities of each image region to obtain the weighted sum of the prediction probabilities;

[0127] If the weighted prediction probability is greater than the probability threshold, determine that the to-be-generated image is a defective image, and the confidence that the to-be-generated image is a defective image is the weight value of the weighted sum of the prediction probabilities.

[0128] In one embodiment, the confidence is used to indicate the accuracy of the prediction result; the processor 701 is configured to call the program instructions to perform:

[0129] If the confidence level is greater than or equal to the first threshold, add a defect label to the image to be processed; otherwise, add a normal label to the image to be processed.

[0130] In one embodiment, the confidence level is used to reflect the degree of defect of the image to be processed; the processor 701 is configured to call the program instructions to perform:

[0131] If the confidence level is greater than or equal to the second threshold, perform restoration processing on the image to be processed to eliminate the defect in the image to be processed.

[0132] In one embodiment, the processor 701 is configured to call the program instructions to perform:

[0133] Divide the image to be processed and obtain one or more image regions obtained by the division;

[0134] Call the trained recognition model to perform feature extraction processing on any image region to obtain one or more feature data corresponding to each image region.

[0135] In one embodiment, the processor 701 is configured to call the program instructions to perform:

[0136] Perform plane interpolation processing on each image region obtained by the division to adjust the region size of each image region, wherein the region sizes of each adjusted image region are the same;

[0137] Use one of the adjusted image regions as one of the image regions obtained by the division.

[0138] An embodiment of the present invention provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method embodiments as described above Figure 2 or Figure 4 shown. Among them, the computer-readable storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0139] The foregoing disclosures are only partial embodiments of the present invention. Of course, the scope of rights of the present invention cannot be limited thereby. Those of ordinary skill in the art can understand the entire or partial processes of implementing the above embodiments, and the equivalent changes made according to the claims of the present invention still fall within the scope covered by the present invention.

Claims

1. An image processing method, characterized in that, Including: Extract M feature data from N image regions of the image to be processed respectively, where both M and N are integers greater than or equal to 1, and M is greater than or equal to N, and one image region corresponds to one or more feature data; Each feature data is an image feature, and the image feature includes a gradient feature or a texture feature; Perform defect detection on the N image regions according to the M feature data to obtain the prediction probability of each image region in the N image regions having a defect; Obtain the attention degree of each image region, and adjust the prediction probability of each image region having a defect according to the attention degree of each image region; the attention degree of each image region is related to the viewing influence degree of the corresponding image region on the image to be processed when there is a defect; Generate the prediction result of the image to be processed and the confidence level of the prediction result according to the adjusted prediction probability of each image region.

2. The method according to claim 1, wherein The image to be processed includes a target image region, and the method for obtaining the attention degree of the target image region includes: Determine a focus region from the image to be processed by using an attention mechanism, and obtain an overlapping region between the target image region and the focus region; Determine the attention degree of the target image region according to the area of the overlapping region; wherein, the attention degree of the target image region is positively correlated with the size of the area of the overlapping region.

3. The method according to claim 2, wherein The attention mechanism is included in a trained recognition model, and the training process of the recognition model includes: Obtain a sample image, and the sample image is added with a labeled label, and the labeled label is used to indicate whether the sample image is a defective image; Call an initial recognition model to determine the prediction label of the sample image; Adjust the model parameters of the initial recognition model according to the difference between the prediction label and the labeled label to obtain a trained recognition model.

4. The method according to claim 3, wherein The model parameters include attention parameters, and adjusting the model parameters of the initial recognition model to obtain a trained recognition model includes: Adjust the attention parameters to reduce the difference between the prediction label and the labeled label, and obtain a trained recognition model when the difference between the prediction label and the labeled label reaches the minimum value.

5. The method according to claim 1, wherein The attention degree includes the prediction weight of the prediction probability; adjusting the prediction probability of each image region having a defect according to the attention degree of each image region includes: Perform weighted processing on the prediction probability of the corresponding image region having a defect by using the prediction weight of each image region, and use the weighted prediction probability of each image region as the adjusted prediction probability of the corresponding image region.

6. The method according to claim 5, wherein Generating the prediction result of the image to be processed and the confidence level of the prediction result according to the adjusted prediction probability of each image region includes: Sum the weighted prediction probabilities of each image region to obtain the weighted sum prediction probability; If the predicted probability after weighted summation is greater than the probability threshold, it is determined that the image to be processed is a defective image, and the confidence level that the image to be processed is a defective image is the weight value of the predicted probability after weighted summation.

7. The method according to claim 1, characterized in that The confidence level is used to indicate the accuracy of the prediction result; the method further includes: If the confidence level is greater than or equal to the first threshold, a defect label is added to the image to be processed; otherwise, a normal label is added to the image to be processed.

8. The method according to claim 1, wherein The confidence level is used to reflect the defect degree of the image to be processed; the method further includes: If the confidence level is greater than or equal to the second threshold, the image to be processed is restored to eliminate the defects in the image to be processed.

9. The method according to claim 1, characterized in that, The extracting M feature data from N image regions of the image to be processed respectively includes: Dividing the image to be processed and obtaining one or more image regions obtained by the division; Invoking the trained recognition model to perform feature extraction processing on any one of the image regions to obtain one or more feature data corresponding to each image region.

10. The method according to claim 9, characterized in that, The obtaining one or more image regions obtained by the division includes: Performing planar interpolation processing on each of the image regions obtained by the division to adjust the region size of each image region, wherein the region sizes of each of the adjusted image regions are the same; Taking one of the adjusted image regions as one of the image regions obtained by the division.

11. An image processing apparatus, characterized in that, Includes: An extraction unit, configured to extract M feature data from N image regions of the image to be processed respectively, both M and N are integers greater than or equal to 1, and M is greater than or equal to N, and one image region corresponds to one or more feature data; Each feature data is an image feature, and the image feature includes a gradient feature or a texture feature; A detection unit, configured to perform defect detection on the N image regions according to the M feature data to obtain the predicted probability that each of the N image regions has a defect; An acquisition unit, configured to acquire the attention degree of each image region; An adjustment unit, configured to adjust the predicted probability that each image region has a defect according to the attention degree of each image region; The attention degree of each image region is related to the viewing influence degree of the corresponding image region on the image to be processed when there is a defect; A generation unit, configured to generate a prediction result of the image to be processed and the confidence level of the prediction result according to the adjusted predicted probability of each image region.

12. An image processing apparatus, characterized in that, Includes a processor, an input device, an output device, and a memory, the processor, the input device, the output device, and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 10.

14. A computer program product, characterized in that, The computer program product includes a computer program, the computer program includes program instructions, and when the program instructions are called by a processor, the processor is caused to execute the method according to any one of claims 1 to 10.

Images