Defect Detection Method and Device in Image Pair, and Electronic Device

Abstract

The present invention provides a method and apparatus for defect detection in an image pair, and an electronic device. The method includes: obtaining a hub image to be detected; performing a centering operation on the hub image using a pre-constructed centering frame template to obtain a centering result, where the centering result is used to indicate whether the centering of the hub image is successful; inputting the hub image into a pre-trained hub defect detection network according to the centering result to obtain preselected candidate boxes, where the preselected candidate boxes include the positions, defect categories, and confidence levels of hub defects in the hub image; generating defect information in the hub image according to the preselected candidate boxes and a pre-constructed target attention matrix, where the attention matrix is used to represent the probability information of defects occurring at corresponding pixel positions. Through the embodiments of the present invention, the technical problem of low accuracy in detecting hub defects by machine vision in the related art is solved, the accuracy and speed of hub defect detection are improved, and the robustness and stability of the detection system are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of industrial control technology, and in particular, to a method and device for defect detection in image alignment, and an electronic device. Background Art

[0002] During the production process of automotive parts, the method of manually checking each wheel hub with the naked eye or the method of traditional machine vision template matching is used to detect external defects of the wheel hub.

[0003] In related technologies, the defect detection rate by manual means is low and the false detection rate is very high. Since the scale grasped by each person is different and there is no quantitative standard, the detection standards vary from person to person and are not unified. The manual method is also affected by energy and working methods, resulting in low detection efficiency. Using the traditional machine vision template matching method, it is greatly affected by factors such as lighting, angle, position, picture size, texture, and complex background during photographing. Frequent template replacement is required during use, which greatly limits the implementation and deployment of this method. Especially when the product model changes, the entire algorithm needs to be re-adapted, resulting in poor robustness of the algorithm.

[0004] In view of the above problems existing in related technologies, no efficient and accurate solution has been found yet. Summary of the Invention

[0005] The present invention provides a method and device for defect detection in image alignment, and an electronic device to solve the above technical problems existing in related technologies.

[0006] According to an embodiment of the present invention, a method for defect detection in image alignment is provided, including: obtaining a wheel hub image to be detected; performing an alignment operation on the wheel hub image using a pre-constructed alignment frame template to obtain an alignment result, where the alignment result is used to indicate whether the alignment of the wheel hub image is successful; inputting the wheel hub image into a pre-trained wheel hub defect detection network according to the alignment result to obtain a preselected candidate box, where the preselected candidate box includes the position, defect category, and confidence level of the wheel hub defect in the wheel hub image; generating defect information in the wheel hub image according to the preselected candidate box and a pre-constructed target attention matrix, where the attention matrix is used to represent the probability information of a defect occurring at a corresponding pixel position.

[0007] Optionally, performing an alignment operation on the wheel hub image using a pre-constructed alignment frame template to obtain an alignment result includes: rotating the alignment frame template at a preset step length to generate a template set, where the template set includes alignment frame templates corresponding to multiple rotation angles; finding a target alignment frame template that best matches the wheel hub image in the template set; generating an alignment result of the wheel hub image based on the target alignment frame template.

[0008] Optionally, finding the target pair of middle frame templates that best match the hub image in the template set includes: finding the best matching region in the hub image; calculating the similarity with the best matching region for each template in the template set; and determining the template with the highest similarity as the target pair of middle frame templates that best match the hub image.

[0009] Optionally, generating the centering result of the hub image based on the target pair of middle frame templates includes: calculating the horizontal offset and vertical offset between the target pair of middle frame templates and the best matching region in the hub image; determining whether both the horizontal offset and the vertical offset are less than a preset threshold; if both the horizontal offset and the vertical offset are less than the preset threshold, generating a first centering result; if the horizontal offset or the vertical offset is greater than or equal to the preset threshold, generating a second centering result, where the first centering result is used to indicate that the centering of the hub image is successful, and the second centering result is used to indicate that the centering of the hub image fails.

[0010] Optionally, inputting the hub image into a pre-trained hub defect detection network according to the centering result includes: if the centering of the hub image is successful, obtaining the centering rotation angle and centering translation amount of the hub image; rotating the hub image by the centering rotation angle to obtain a first intermediate image; translating the first intermediate image by the centering translation amount to obtain a second intermediate image; cutting the background pixels other than the hub in the second intermediate image to obtain a third intermediate image; inputting the third intermediate image into the pre-trained hub defect detection network; if the centering of the hub image fails, inputting the hub image into the pre-trained hub defect detection network.

[0011] Optionally, generating defect information in the hub image according to the preselected candidate box and the pre-constructed target attention matrix includes: calculating the center of the box of the preselected candidate box; calculating the distances between the center of the box and the cluster centers of each attention matrix in the pre-constructed attention matrix set, and selecting the target attention matrix with the closest distance; multiplying the preselected candidate box by the corresponding coordinates in the target attention matrix and then accumulating to obtain an initial candidate box set, where each candidate box corresponds to a confidence level; filtering the initial candidate boxes with a confidence level less than a first threshold in the initial candidate box set to obtain an intermediate candidate box set; calculating the intersection area and union area in the intermediate candidate box set, and calculating the intersection over union based on the intersection area and the union area; filtering the intermediate candidate boxes with an intersection over union less than a second threshold in the intermediate candidate box set to obtain a defect box set, where each defect box includes the hub defect position, defect category, and confidence level.

[0012] Optionally, before performing the centering operation on the hub image using the pre-built centering frame template, the method further includes: selecting multiple grayscale sample pictures; sequentially performing the following steps on each of the multiple grayscale sample pictures: starting from the initial position of the current grayscale sample picture, selecting a sliding window of a fixed size, and selecting the window picture within the sliding window as the sliding template; using the sliding template to match the grayscale sample picture to find the lowest similarity of the matching degree; selecting the window area where the sliding window with the smallest similarity is located in the grayscale sample picture; after the multiple grayscale sample pictures are processed, reading the multiple window areas corresponding to the multiple grayscale sample pictures; calculating the average position value of the multiple window areas to obtain the optimal centering frame position; based on the optimal centering frame position, respectively extracting an optimal window picture of a centering frame from the multiple grayscale sample pictures to obtain multiple optimal window pictures; calculating the pixel grayscale mean value of the multiple optimal window pictures to obtain the centering frame template.

[0013] Optionally, before generating the defect information in the hub image according to the preselected candidate box and the pre-built target attention matrix, the method further includes: obtaining multiple defect sample pictures; counting the calibration coordinates and defect sizes of the defect areas in each defect sample picture in the multiple defect sample pictures in a preset calibration direction; clustering the centers of the defect sizes into an initial number of categories; iteratively performing the following steps until an intersection with an area larger than a preset number of pixels appears: calculating the intersection of multiple defect masks in each cluster; determining whether the number of pixels within the intersection is greater than a preset number; if the number of pixels within the intersection is less than or equal to the preset number, increasing the number of categories of the cluster; obtaining multiple clusters after the iteration ends and configuring an initial attention matrix for each cluster, where the elements of the initial attention matrix are all 0; for each cluster, resetting the element values in the initial attention matrix according to the number of overlapping defect masks at each element position to obtain a set of attention matrices, where the element values are positively correlated with the number of overlapping defect masks.

[0014] According to another embodiment of the present invention, there is provided a defect detection device in an image pair, comprising: a first acquisition module for acquiring a hub image to be detected; an alignment module for performing an alignment operation on the hub image using a pre-constructed alignment frame template to obtain an alignment result, wherein the alignment result is used to characterize whether the alignment of the hub image is successful; a processing module for inputting the hub image into a pre-trained hub defect detection network according to the alignment result to obtain a preselected candidate box, wherein the preselected candidate box includes the position, defect category and confidence level of the hub defect in the hub image; a generation module for generating defect information in the hub image according to the preselected candidate box and a pre-constructed target attention matrix, wherein the attention matrix is used to characterize the probability information of defects occurring at corresponding pixel positions.

[0015] Optionally, the alignment module includes: a rotation unit for rotating the alignment frame template at a preset step length to generate a template set, wherein the template set includes alignment frame templates corresponding to multiple rotation angles; a search unit for searching for a target alignment frame template that best matches the hub image in the template set; a generation unit for generating an alignment result of the hub image based on the target alignment frame template.

[0016] Optionally, the search unit includes: a search subunit for searching for the best matching region in the hub image; a calculation subunit for calculating the similarity with the best matching region for each template in the template set respectively; a determination subunit for determining the template with the highest similarity as the target alignment frame template that best matches the hub image.

[0017] Optionally, the generation unit includes: a calculation subunit for calculating the horizontal offset and vertical offset between the target alignment frame template and the best matching region in the hub image; a judgment subunit for judging whether both the horizontal offset and the vertical offset are less than a preset threshold; a generation subunit for generating a first alignment result if both the horizontal offset and the vertical offset are less than the preset threshold; and generating a second alignment result if the horizontal offset or the vertical offset is greater than or equal to the preset threshold, wherein the first alignment result is used to characterize that the alignment of the hub image is successful, and the second alignment result is used to characterize that the alignment of the hub image fails.

[0018] Optionally, the processing module includes: a first processing unit, used to obtain the centering rotation angle and centering translation of the wheel hub image if the wheel hub image is successfully centered; rotate the wheel hub image by the centering rotation angle to obtain a first intermediate image; translate the first intermediate image by the centering translation to obtain a second intermediate image; cut background pixels other than the wheel hub in the second intermediate image to obtain a third intermediate image; input the third intermediate image into a pre-trained wheel hub defect detection network; a second processing unit, used to input the wheel hub image into a pre-trained wheel hub defect detection network if the wheel hub image fails to be centered.

[0019] Optionally, the generation module includes: a first calculation unit, used to calculate the box center of the pre-selected candidate box; a second calculation unit, used to calculate the distance between the box center and the cluster center of each attention matrix in the pre-constructed attention matrix set, and select the target attention matrix with the closest distance; a first operation unit, used to multiply the pre-selected candidate box with the corresponding coordinates in the target attention matrix and then accumulate them to obtain an initial candidate box set, wherein each candidate box corresponds to a confidence level; a first filtering unit, used to filter the initial candidate boxes whose confidence levels are less than a first threshold in the initial candidate box set to obtain an intermediate candidate box set; a second operation unit, used to calculate the intersection area and the union area in the intermediate candidate box set, and calculate the intersection-union ratio based on the intersection area and the union area; a second filtering unit, used to filter the intermediate candidate boxes whose intersection-union ratios are less than a second threshold in the intermediate candidate box set to obtain a defect box set, wherein each defect box includes a wheel hub defect position, a defect category, and a confidence level.

[0020] Optionally, the device further includes: a second acquisition module, configured to select multiple grayscale sample images before the centering module uses a pre-built centering frame template to perform a centering operation on the wheel hub image; a first iteration module, configured to sequentially perform the following steps on each of the multiple grayscale sample images: selecting a sliding window of a fixed size starting from an initial position for the current grayscale sample image, and selecting a window image within the sliding window as a sliding template; using the sliding template to match the grayscale sample images to find the similarity with the lowest matching degree; selecting a window area in the grayscale sample image where the sliding window with the smallest similarity is located; a reading module, configured to read multiple window areas corresponding to the multiple grayscale sample images after the execution of the multiple grayscale sample images is completed; a first calculation module, configured to calculate the average position value of the multiple window areas to obtain an optimal centering frame position; an extraction module, configured to extract an optimal window image of a centering frame from each of the multiple grayscale sample images based on the optimal centering frame position to obtain multiple optimal window images; and a second calculation module, configured to calculate the pixel grayscale mean value of the multiple optimal window images to obtain a centering frame template.

[0021] Optionally, the device further includes: a third acquisition module, configured to acquire multiple defect sample pictures before the generation module generates defect information in the hub image according to the preselected candidate box and the pre-constructed target attention matrix; a statistics module, configured to count the calibration coordinates and defect sizes of the defect areas in each defect sample picture in the multiple defect sample pictures in a preset calibration direction; a clustering module, configured to perform clustering of an initial number of categories on the center points of the defect sizes; a second iteration module, configured to iteratively execute the following steps until an intersection with an area greater than a preset number of pixels appears: calculate the intersection of multiple defect masks in each cluster; determine whether the number of pixels in the intersection is greater than a preset number; if the number of pixels in the intersection is less than or equal to the preset number, increase the number of categories of the cluster; a configuration module, configured to obtain multiple clusters after the iteration ends and configure an initial attention matrix for each cluster, where all elements of the initial attention matrix are 0; a reset module, configured to, for each cluster, reset the element values in the initial attention matrix according to the number of defect masks overlapping at each element position, to obtain a set of attention matrices, where the element values are positively correlated with the number of overlapping defect masks.

[0022] According to another embodiment of the present invention, there is also provided a storage medium storing a computer program, where the computer program is configured to execute the steps in any one of the above device embodiments when running.

[0023] According to another embodiment of the present invention, there is also provided an electronic device including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above device embodiments.

[0024] Through the embodiments of the present invention, a hub image to be detected is obtained; a centering operation is performed on the hub image by using a pre-constructed centering frame template to obtain a centering result, where the centering result is used to represent whether the centering of the hub image is successful; according to the centering result, the hub image is input into a pre-trained hub defect detection network to obtain a preselected candidate box, where the preselected candidate box includes the position, defect category, and confidence of the hub defect in the hub image; defect information in the hub image is generated according to the preselected candidate box and the pre-constructed target attention matrix, where the attention matrix is used to represent the probability information of a defect appearing at a corresponding pixel position. By centering the hub image by using a centering frame template and optimizing the preselected candidate box output by the network by using a target attention matrix, the technical problem of low accuracy in detecting hub defects in the related art is solved, the accuracy and speed of hub defect detection are improved, and the robustness and stability of the detection system are improved. Description of the Drawings

[0025] The accompanying drawings described herein are used to provide a further understanding of the present invention and form a part of this application. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0026] Figure 1 is a block diagram of the hardware structure of a computer according to an embodiment of the present invention;

[0027] Figure 2 is a flowchart of a method for defect detection in an image pair according to an embodiment of the present invention;

[0028] Figure 3 is a schematic diagram of a wheel hub defect detection network in an embodiment of the present invention;

[0029] Figure 4 is an overall flowchart of an embodiment of the present invention;

[0030] Figure 5 is a block diagram of a device for defect detection in an image pair according to an embodiment of the present invention. Detailed Embodiments

[0031] In order to enable those skilled in the art of the present technology to better understand the solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application. It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned accompanying drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, product or device including a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0033] Embodiment 1

[0034] The method embodiment provided in the first embodiment of the present application can be executed in computing devices such as servers, computers, and cameras. Taking running on a computer as an example, Figure 1 is a hardware structure block diagram of a computer according to an embodiment of the present invention. As Figure 1 shown, the computer may include one or more ( Figure 1 only one is shown in the figure) processors 102 (the processor 102 may include, but is not limited to, processing devices such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the above computer may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above computer. For example, the computer may further include more or fewer components than Figure 1 shown in the figure, or have a different configuration from Figure 1 shown in the figure.

[0035] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the method for defect detection in an image pair in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories may be connected to the computer through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0036] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of the computer. In one instance, the transmission device 106 includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (abbreviated as RF) module, which is used to communicate with the Internet wirelessly.

[0037] In this embodiment, a method for defect detection in an image pair is provided. Figure 2 is a flowchart of the method for defect detection in an image pair according to an embodiment of the present invention. As Figure 2 shown, the process includes the following steps:

[0038] Step S202, obtain the wheel hub image to be detected;

[0039] Optionally, the wheel hub image may be the wheel hub of workpieces such as automotive parts, such as automotive wheel hubs.

[0040] Step S204, perform a centering operation on the wheel hub image using a pre-constructed centering frame template to obtain a centering result, where the centering result is used to characterize whether the centering of the wheel hub image is successful;

[0041] Step S206, input the wheel hub image into a pre-trained wheel hub defect detection network according to the centering result to obtain preselected candidate boxes, where the preselected candidate boxes include the positions, defect categories, and their confidence levels of wheel hub defects in the wheel hub image;

[0042] The defect data corresponding to the preselected candidate box is the initial defect data.

[0043] Step S208, generate defect information in the wheel hub image according to the preselected candidate box and a pre-constructed target attention matrix, where the attention matrix is used to characterize the probability information of defects occurring at corresponding pixel positions.

[0044] Through the above steps, obtain the wheel hub image to be detected; perform a centering operation on the wheel hub image using a pre-constructed centering frame template to obtain a centering result, where the centering result is used to characterize whether the centering of the wheel hub image is successful; input the wheel hub image into a pre-trained wheel hub defect detection network according to the centering result to obtain preselected candidate boxes, where the preselected candidate boxes include the positions, defect categories, and their confidence levels of wheel hub defects in the wheel hub image; generate defect information in the wheel hub image according to the preselected candidate box and a pre-constructed target attention matrix, where the attention matrix is used to characterize the probability information of defects occurring at corresponding pixel positions. By centering the wheel hub image using a centering frame template and optimizing the preselected candidate boxes output by the network using a target attention matrix, the technical problem of low accuracy in detecting wheel hub defects in the related art is solved, the accuracy and speed of wheel hub defect detection are improved, and the robustness and stability of the detection system are improved.

[0045] In an implementation manner of this embodiment, performing a centering operation on the wheel hub image using a pre-constructed centering frame template to obtain a centering result includes: rotating the centering frame template according to a preset step length to generate a template set, where the template set includes centering frame templates corresponding to multiple rotation angles; searching for the target centering frame template that best matches the wheel hub image in the template set; generating a centering result of the wheel hub image based on the target centering frame template.

[0046] In this embodiment, before performing the centering operation on the hub image using the pre-constructed centering frame template, the following steps are further included: selecting multiple grayscale sample pictures; sequentially performing the following steps on each of the multiple grayscale sample pictures: starting from the initial position of the current grayscale sample picture, selecting a sliding window of a fixed size, and taking the window picture within the sliding window as the sliding template; using the sliding template to match the grayscale sample picture to find the lowest similarity degree of matching; selecting the window area where the sliding window with the smallest similarity degree is located in the grayscale sample picture; after the multiple grayscale sample pictures are processed, reading the multiple window areas corresponding to the multiple grayscale sample pictures; calculating the average position value of the multiple window areas to obtain the optimal centering frame position; based on the optimal centering frame position, respectively extracting a best window picture of a centering frame from the multiple grayscale sample pictures to obtain multiple best window pictures; calculating the pixel grayscale average value of the multiple best window pictures to obtain the centering frame template.

[0047] Optionally, rotate the centering frame template in steps of 1 degree, rotating from -10 degrees counterclockwise to 10 degrees clockwise to generate 20 templates.

[0048] In one embodiment, the process of constructing the centering frame template includes:

[0049] a1: Randomly select 10 pictures from the input single-channel grayscale picture of 2048 * 2048 as the grayscale sample pictures;

[0050] a2: For each of the 10 grayscale sample pictures of the hub, select a 200*200 sliding window starting from the upper left corner, and take the small picture within the sliding window as the template;

[0051] a3: Use the template in the previous step to match the entire picture and find the similarity value of the least matching point;

[0052] a4: Slide the sliding window in step a2 in steps of 50 pixels from left to right and from top to bottom, and perform the matching in step a3;

[0053] a5: Compare the similarity values of the least matching points of each sliding window, and find the coordinates of the sliding window with the smallest similarity value in the entire picture;

[0054] a6: Repeat steps a2 to a5 for each of the 10 pictures, calculate the average values of the coordinates, width, and height of the sliding windows with the smallest similarity values in the 10 pictures to obtain the window position where the optimal centering frame is located;

[0055] a7: According to the position where the optimal centering frame is located, extract the small pictures of the centering frame from the 10 pictures, and take the average of the pixel grayscale values to obtain the picture of the centering frame template.

[0056] In one example, finding the target pair of middle frame templates that best matches the hub image in the template set includes: finding the best matching region in the hub image; calculating the similarity with the best matching region for each template in the template set; and determining the template with the highest similarity as the target pair of middle frame templates that best matches the hub image.

[0057] Find the best matching region (from the hub image) with the pair of middle frame template in the currently input hub image, and calculate the similarity between the best matching region and the pair of middle frame template.

[0058] In one example, generating the alignment result of the hub image based on the target pair of middle frame templates includes: calculating the horizontal offset and vertical offset between the target pair of middle frame templates and the best matching region in the hub image; determining whether both the horizontal offset and the vertical offset are less than a preset threshold; if both the horizontal offset and the vertical offset are less than the preset threshold, generating a first alignment result; if the horizontal offset or the vertical offset is greater than or equal to the preset threshold, generating a second alignment result, where the first alignment result is used to represent that the alignment of the hub image is successful, and the second alignment result is used to represent that the alignment of the hub image fails.

[0059] Obtain the rotation angle angle of the template with the highest similarity match and the offset (coordinates) of the best matching region. If the offset in the x direction and the offset in the y direction are both less than or equal to 100, the alignment is successful; otherwise, the alignment fails.

[0060] In an implementation scenario of this embodiment, inputting the hub image into a pre-trained hub defect detection network according to the alignment result includes: if the alignment of the hub image is successful, obtaining the alignment rotation angle and alignment translation amount of the hub image; rotating the hub image by the alignment rotation angle to obtain a first intermediate image; translating the first intermediate image by the alignment translation amount to obtain a second intermediate image; cutting the background pixels other than the hub in the second intermediate image to obtain a third intermediate image; inputting the third intermediate image into the pre-trained hub defect detection network; if the alignment of the hub image fails, inputting the hub image into the pre-trained hub defect detection network.

[0061] In a scenario, if the centering is successful, the translation offset offset of the current hub image is obtained according to the calculated horizontal and vertical offsets between the target centering frame template and the best matching area. If the centering fails, set angle to 0 and offset to (0, 0), and directly input the original hub image into the pre-trained hub defect detection network. If the centering is successful, for the current hub image to be inspected, after rotating by the angle angle and then translating by offset, the centered image is obtained, and the pixels outside the hub in the image are cut off and supplemented according to the edge pixels.

[0062] Figure 3 It is the schematic diagram of the hub defect detection network in the embodiment of the present invention. The entire detection algorithm process is a closed-loop feedback cycle. After the algorithm network is deployed on the hub appearance defect detection device, it performs real-time detection on the input batch of hub images, outputs the detected defect categories and the pixel positions in the images, and at the same time collects the images with poor detection effects for retraining and redeployment, and performs cycling to gradually improve the detection effect. The hub defect detection network includes multiple convolutional layers and multiple fully connected layers. The first 5 convolutional layers and the last 3 fully connected layers need to be initialized, and their weights weights are initialized to a random normal distribution of 0 - 10^-2, bias = 0. Finally, the features of the hub are extracted into a 1*1000 feature vector. The network structure uses 1*1 convolution, and 1*1 convolution first appeared in NIN (network in network). 1*1 convolution greatly reduces the number of parameters and reduces the dimension, which can speed up the model training speed.

[0063] In an implementation manner of this embodiment, generating the defect information in the hub image according to the preselected candidate box and the pre-constructed target attention matrix includes: calculating the center of the preselected candidate box; calculating the distances between the center of the box and the clustering centers of each attention matrix in the pre-constructed attention matrix set, and selecting the target attention matrix with the closest distance; multiplying the corresponding coordinates of the preselected candidate box and the target attention matrix and then accumulating to obtain an initial candidate box set, where each candidate box corresponds to a confidence level; filtering the initial candidate boxes with a confidence level less than the first threshold in the initial candidate box set to obtain an intermediate candidate box set; calculating the intersection area and union area in the intermediate candidate box set, and calculating the intersection over union based on the intersection area and the union area; filtering the intermediate candidate boxes with an intersection over union less than the second threshold in the intermediate candidate box set to obtain a defect box set, where each defect box includes the hub defect position, defect category, and confidence level.

[0064] In this embodiment, before generating the defect information in the hub image based on the preselected candidate boxes and the pre-constructed target attention matrix, the following steps are further included: obtaining multiple defect sample pictures; counting the calibration coordinates and defect sizes of the defect areas in each defect sample picture in a preset calibration direction; performing clustering of an initial number of categories on the center points of the defect sizes; iteratively executing the following steps until an intersection with an area larger than a preset number of pixels appears: calculating the intersection of multiple defect masks in each cluster; determining whether the number of pixels in the intersection is greater than a preset number; if the number of pixels in the intersection is less than or equal to the preset number, increasing the number of categories of the cluster; obtaining multiple clusters after the iteration ends, and configuring an initial attention matrix for each cluster, where the elements of the initial attention matrix are all 0; for each cluster, resetting the element values in the initial attention matrix according to the number of overlapping defect masks at each element position to obtain a set of attention matrices, where the element values are positively correlated with the number of overlapping defect masks.

[0065] After rotation, translation, and cutting, the centered picture is input into the hub defect detection network for inference to generate preselected candidate boxes, including information such as position, category, and corresponding confidence. Calculate the distance between the attention matrix and the clustering center of the preselected candidate boxes, select the attention matrix with the closest distance, multiply the elements (confidence and probability) corresponding to the coordinates in the preselected candidate box and the attention matrix and then sum them. After filtering by the candidate box confidence and IOU (Intersection over Union) filtering of NMS (Non-Maximum Suppression), the position, category, and corresponding confidence of the defects in the hub product are obtained.

[0066] Optionally, the value range of the element values of the reset attention matrix is [0, 1]. If all defect rectangles overlap at the position of pixel A, the value of the element of the attention matrix at pixel A is 1. 1 represents a 100% probability of a defect occurring at the corresponding pixel position, 0 represents a 0% probability of a defect occurring at the corresponding pixel position. The higher the value, the higher the probability of a defect occurring, and vice versa.

[0067] In one embodiment, the process of constructing the attention matrix of the defect includes:

[0068] b1: Count the upper left coordinates and the length and width of the defects in all hub pictures in the image data of the labeled defect sample pictures to obtain defect rectangles, that is, defect masks mask;

[0069] b2: Perform clustering of 5 categories on the upper left coordinates and the length and width of the defects;

[0070] Optionally, performing initial number of categories of clustering on the center point of the defect size includes: minimizing the clustering using the following objective function J:

[0071] ;

[0072] is the defect center point of the i-th defect sample image, is the probability that point i belongs to cluster j, is the center of cluster j, = 1, is the center of cluster l, n is the number of defect sample images, k is the number of initial categories, i.e., the number of clusters, here it is taken as 5, m is the fuzzy index, m = 2. and are updated according to the following formula:

[0073] ;

[0074] ;

[0075] b3: Calculate the intersection of the defect masks in each clustering in the previous step. If there is no intersection with an area greater than 25 pixels, increase the number of categories of clustering until there is an intersection with an area greater than 25 pixels;

[0076] b4: Initialize an attention matrix with the same size as the hub image; the element values are set to all 0, and each clustering corresponds to the attention matrix;

[0077] b5: Set the matrix elements in the intersection of each completely overlapping defect mask to 1, set the elements outside the rectangular box to 0, and generate the other elements according to the number of overlapping rectangular boxes by linear interpolation;

[0078] b6: All elements in the generated attention matrix are between [0, 1].

[0079] During the wheel hub production process, various types of wheel hub defects are generated due to factors such as the temperature and humidity of the production environment, process flow, raw material factors, equipment or human factors, such as scratches, black lines, streaks, bubbles, inclusions, extrusion rings, mold scratches, scrapes, film loss, rust spots, oil stains, color difference, aluminum chips, film explosion, burrs, mixed materials, etc. This embodiment designs an algorithm network based on image centering templates and defect attention matrices based on the image features collected from the wheel hub and the location, size, length, density and other characteristics of the defects in the images. This aims to solve the problem of inconsistent wheel hub locations on each photo when extracting image features. The defect attention matrix is constructed based on the statistical regularity of the locations where defects appear on the wheel hub. This embodiment customizes the design of a wheel image centering and attention matrix algorithm network based on the characteristics of the wheel image. It can solve the speed and accuracy issues of inputting multiple wheel images into the defect detection network for parallel training and prediction. Using this network training model does not require high computing power, CUDA (Compute Unified Device Architecture) cores and memory and other computing resources. After training and verification on the training data set, the model is exported and applied to the wheel defect detection equipment to achieve real-time detection of wheel defect categories and positions and output of confidence levels.

[0080] This paper proposes a new method for constructing a centering frame template and defect attention matrix. Based on the wheel hub big data collected by the wheel hub appearance defect detection equipment, after the wheel hub image is centered, a deep learning defect feature extraction network model based on image centering is trained, and then the attention matrix is superimposed. Using this algorithm model, a wheel hub defect detection system can be constructed. Figure 4 It is an overall flow chart of an embodiment of the present invention.

[0081] By adopting the solution of this embodiment, defects on a wheel hub image with a resolution of 2048*2048 can be predicted in real time in an average of 11 ms, with up to 18 types of defects. The accuracy and speed of the system meet the needs of actual applications. Compared with existing systems based on traditional machine vision or manual methods, the accuracy and detection speed are improved.

[0082] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0083] Embodiment 2

[0084] In this embodiment, a defect detection device for an image pair is further provided. This device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated. As used hereinafter, the term "module" can be a combination of software and hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware can also be conceived.

[0085] Figure 5 is a structural block diagram of a defect detection device for an image pair according to an embodiment of the present invention. As Figure 5 shown, the device includes:

[0086] A first acquisition module 50, configured to acquire a hub image to be detected;

[0087] An alignment module 52, configured to perform an alignment operation on the hub image by using a pre-constructed alignment frame template to obtain an alignment result, where the alignment result is used to characterize whether the alignment of the hub image is successful;

[0088] A processing module 54, configured to input the hub image into a pre-trained hub defect detection network according to the alignment result to obtain a preselected candidate box, where the preselected candidate box includes the hub defect position, defect category, and its confidence level in the hub image;

[0089] A generation module 56, configured to generate defect information in the hub image according to the preselected candidate box and a pre-constructed target attention matrix, where the attention matrix is used to characterize the probability information of a defect occurring at a corresponding pixel position.

[0090] Optionally, the centering module includes: a rotating unit configured to rotate the centering frame template by a preset step length to generate a template set, where the template set includes centering frame templates corresponding to multiple rotation angles; a searching unit configured to search for a target centering frame template that best matches the hub image in the template set; and a generating unit configured to generate a centering result of the hub image based on the target centering frame template.

[0091] Optionally, the searching unit includes: a searching subunit configured to search for the best matching region in the hub image; a calculating subunit configured to calculate the similarity with the best matching region for each template in the template set; and a determining subunit configured to determine the template with the highest similarity as the target centering frame template that best matches the hub image.

[0092] Optionally, the generating unit includes: a calculating subunit configured to calculate the horizontal offset and the vertical offset between the target centering frame template and the best matching region in the hub image; a judging subunit configured to judge whether both the horizontal offset and the vertical offset are less than a preset threshold; and a generating subunit configured to generate a first centering result if both the horizontal offset and the vertical offset are less than the preset threshold, and generate a second centering result if the horizontal offset or the vertical offset is greater than or equal to the preset threshold, where the first centering result is used to indicate that the centering of the hub image is successful, and the second centering result is used to indicate that the centering of the hub image fails.

[0093] Optionally, the processing module includes: a first processing unit configured to, if the centering of the hub image is successful, obtain the centering rotation angle and the centering translation amount of the hub image; rotate the hub image by the centering rotation angle to obtain a first intermediate image; translate the first intermediate image by the centering translation amount to obtain a second intermediate image; cut the background pixels other than the hub in the second intermediate image to obtain a third intermediate image; and input the third intermediate image into a pre-trained hub defect detection network; and a second processing unit configured to, if the centering of the hub image fails, input the hub image into the pre-trained hub defect detection network.

[0094] Optionally, the generating module includes: a first calculation unit for calculating the center of the preselected candidate box; a second calculation unit for calculating the distances between the center of the box and the cluster centers of each attention matrix in a pre-constructed set of attention matrices, and selecting the target attention matrix with the shortest distance; a first operation unit for multiplying the corresponding coordinates in the preselected candidate box and the target attention matrix and then accumulating to obtain an initial set of candidate boxes, where each candidate box corresponds to a confidence level; a first filtering unit for filtering out the initial candidate boxes with a confidence level less than a first threshold in the initial set of candidate boxes to obtain an intermediate set of candidate boxes; a second operation unit for calculating the intersection area and union area in the intermediate set of candidate boxes and calculating the intersection over union based on the intersection area and the union area; a second filtering unit for filtering out the intermediate candidate boxes with an intersection over union less than a second threshold in the intermediate set of candidate boxes to obtain a set of defect boxes, where each defect box includes the position of the wheel hub defect, the defect category, and the confidence level.

[0095] Optionally, the apparatus further includes: a second acquisition module for selecting multiple grayscale sample pictures before the centering module performs centering operation on the wheel hub image using a pre-constructed centering box template; a first iteration module for sequentially performing the following steps on each of the multiple grayscale sample pictures: starting from an initial position, selecting a sliding window of a fixed size in the current grayscale sample picture, and selecting the window picture within the sliding window as a sliding template; using the sliding template to match the grayscale sample picture to find the lowest similarity; selecting the window area where the sliding window with the smallest similarity is located in the grayscale sample picture; a reading module for reading the multiple window areas corresponding to the multiple grayscale sample pictures after the multiple grayscale sample pictures are processed; a first calculation module for calculating the average position value of the multiple window areas to obtain the best centering box position; an extraction module for respectively extracting a best window picture of a centering box from the multiple grayscale sample pictures based on the best centering box position to obtain multiple best window pictures; a second calculation module for calculating the average pixel grayscale value of the multiple best window pictures to obtain the centering box template.

[0096] Optionally, the device further includes: a third acquisition module, configured to acquire multiple defect sample pictures before the generation module generates defect information in the hub image according to the preselected candidate box and the pre-constructed target attention matrix; a statistics module, configured to count the calibration coordinates and defect sizes of the defect areas in each defect sample picture in the multiple defect sample pictures in a preset calibration direction; a clustering module, configured to perform clustering of an initial number of categories on the center points of the defect sizes; a second iteration module, configured to iteratively execute the following steps until an intersection with an area greater than a preset number of pixels appears: calculate the intersection of multiple defect masks in each cluster; determine whether the number of pixels in the intersection is greater than a preset number; if the number of pixels in the intersection is less than or equal to the preset number, increase the number of categories of the cluster; a configuration module, configured to acquire multiple clusters after the iteration ends and configure an initial attention matrix for each cluster, where the elements of the initial attention matrix are all 0; a reset module, configured to, for each cluster, reset the element values in the initial attention matrix according to the number of overlapping defect masks at each element position to obtain a set of attention matrices, where the element values are positively correlated with the number of overlapping defect masks.

[0097] It should be noted that the above-mentioned respective modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited thereto: the above-mentioned modules are all located in the same processor; or, the above-mentioned respective modules are located in different processors in any combination form.

[0098] Embodiment 3

[0099] An embodiment of the present invention further provides a storage medium, in which a computer program is stored, where the computer program is set to execute the steps in any one of the above method embodiments when running.

[0100] Optionally, in this embodiment, the above storage medium can be set to store a computer program for execution:

[0101] S1, acquire a hub image to be detected;

[0102] S2, perform a centering operation on the hub image by using a pre-constructed centering box template to obtain a centering result, where the centering result is used to represent whether the centering of the hub image is successful;

[0103] S3, input the hub image into a pre-trained hub defect detection network according to the centering result to obtain a preselected candidate box, where the preselected candidate box includes the hub defect position, defect category, and its confidence level in the hub image;

[0104] S4. Generate defect information in the hub image based on the preselected candidate boxes and a pre-constructed target attention matrix, where the attention matrix is used to represent the probability information of defects occurring at corresponding pixel positions.

[0105] Optionally, in this embodiment, the above storage medium may include, but is not limited to: USB flash drives, read-only memories (ROM), random access memories (RAM), external hard drives, magnetic disks, or optical discs, etc., all kinds of media that can store computer programs.

[0106] An embodiment of the present invention also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0107] Optionally, the above electronic device may further include a transmission device and input / output devices. Among them, the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0108] Optionally, in this embodiment, the above processor may be configured to execute the following steps through a computer program:

[0109] S1. Obtain a hub image to be detected;

[0110] S2. Perform a centering operation on the hub image using a pre-constructed centering box template to obtain a centering result, where the centering result is used to represent whether the centering of the hub image is successful;

[0111] S3. Input the hub image into a pre-trained hub defect detection network according to the centering result to obtain preselected candidate boxes, where the preselected candidate boxes include the hub defect positions, defect categories, and their confidence levels in the hub image;

[0112] S4. Generate defect information in the hub image based on the preselected candidate boxes and a pre-constructed target attention matrix, where the attention matrix is used to represent the probability information of defects occurring at corresponding pixel positions.

[0113] Optionally, specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and will not be elaborated here.

[0114] The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0115] In the above embodiments of the present application, the descriptions of the various embodiments each have their own emphasis. For the parts not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0116] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of units or modules can be in electrical or other forms.

[0117] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0118] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0119] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of the present application. The foregoing storage medium includes: USB flash drive, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other various media that can store program codes.

[0120] The above is only the preferred embodiment of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A defect detection method in an image pair, characterized in that, Including: Obtain a hub image to be detected; Perform centering operation on the hub image by using a pre-constructed centering frame template to obtain a centering result, where the centering result is used to characterize whether the centering of the hub image is successful; Input the hub image into a pre-trained hub defect detection network according to the centering result to obtain preselected candidate boxes, where the preselected candidate boxes include the hub defect positions, defect categories, and their confidence levels in the hub image; Generate defect information in the hub image according to the preselected candidate boxes and a pre-constructed target attention matrix, where the attention matrix is used to characterize the probability information of defects appearing at corresponding pixel positions; Wherein, generating defect information in the hub image according to the preselected candidate boxes and a pre-constructed target attention matrix includes: calculating the center of the preselected candidate box; calculating the distances between the center of the box and the clustering centers of each attention matrix in the pre-constructed attention matrix set, and selecting the target attention matrix with the closest distance; multiplying the corresponding coordinates of the preselected candidate box and the target attention matrix and then accumulating to obtain an initial candidate box set, where each candidate box corresponds to a confidence level; filtering the initial candidate boxes with confidence levels less than a first threshold in the initial candidate box set to obtain an intermediate candidate box set; calculating the intersection area and union area in the intermediate candidate box set, and calculating the intersection over union based on the intersection area and the union area; filtering the intermediate candidate boxes with intersection over union less than a second threshold in the intermediate candidate box set to obtain a defect box set, where each defect box includes the hub defect position, defect category, and confidence level; Wherein, before generating defect information in the hub image according to the preselected candidate boxes and a pre-constructed target attention matrix, it further includes: obtaining multiple defect sample pictures; counting the calibration coordinates and defect sizes of the defect regions in each defect sample picture in a preset calibration direction in the multiple defect sample pictures; performing clustering of an initial number of categories on the center points of the defect sizes; iteratively executing the following steps until an intersection with an area larger than a preset number of pixels appears: calculating the intersection of multiple defect masks in each cluster; judging whether the number of pixels in the intersection is greater than a preset number; if the number of pixels in the intersection is less than or equal to the preset number, increasing the number of categories of the cluster; obtaining multiple clusters after the iteration ends, and configuring an initial attention matrix for each cluster, where the elements of the initial attention matrix are all 0; for each cluster, resetting the element values in the initial attention matrix according to the number of overlapping defect masks at each element position to obtain an attention matrix set, where the element values are positively correlated with the number of overlapping defect masks.

2. The method according to claim 1, characterized in that Performing centering operation on the hub image by using a pre-constructed centering frame template to obtain a centering result, including: Rotating the centering frame template at a preset step length to generate a template set, where the template set includes centering frame templates corresponding to multiple rotation angles; Searching for the target centering frame template that best matches the hub image in the template set; A centering result of the wheel hub image is generated based on the target centering frame template.

3. The method according to claim 2, wherein Searching the template set for the target centering frame template that best matches the wheel hub image includes: Finding the best matching area in the wheel hub image; For each template in the template set, respectively calculating the similarity with the best matching area; The template with the highest similarity is determined as the target centering frame template that best matches the wheel hub image.

4. The method according to claim 2, wherein Generating the centering result of the hub image based on the target centering frame template includes: Calculating horizontal and vertical offsets between the target centering frame template and the best matching area in the wheel hub image; Determining whether both the horizontal offset and the vertical offset are less than a preset threshold; If both the horizontal offset and the vertical offset are smaller than a preset threshold, a first centering result is generated; if the horizontal offset or the vertical offset is greater than or equal to the preset threshold, a second centering result is generated, wherein the first centering result is used to characterize the success of the wheel hub image centering, and the second centering result is used to characterize the failure of the wheel hub image centering.

5. The method according to claim 1, characterized in that, Inputting the wheel hub image into a pre-trained wheel hub defect detection network according to the alignment result comprises: If the hub image is successfully centered, obtain the hub image's centering rotation angle and centering translation; rotate the hub image by the centering rotation angle to obtain a first intermediate image; translate the first intermediate image by the centering translation to obtain a second intermediate image; cut out background pixels excluding the hub in the second intermediate image to obtain a third intermediate image; and input the third intermediate image into a pre-trained hub defect detection network. If the wheel hub image fails to be centered, the wheel hub image is input into a pre-trained wheel hub defect detection network.

6. The method according to claim 1, characterized in that Before performing a centering operation on the wheel hub image using a pre-built centering frame template, the method further includes: Select multiple grayscale sample images; The following steps are sequentially performed on each of the plurality of grayscale sample images: a sliding window of a fixed size is selected from an initial position of the current grayscale sample image, and a window image within the sliding window is selected as a sliding template; the sliding template is used to match the grayscale sample image to find the lowest similarity; and a window area of the grayscale sample image where the sliding window with the lowest similarity is located is selected; After the plurality of grayscale sample images are executed, a plurality of window areas corresponding to the plurality of grayscale sample images are read; Calculating the average position values of the multiple window areas to obtain the optimal centering frame position; Based on the optimal centering frame position, an optimal window image of the centering frame is respectively taken out from the multiple grayscale sample images to obtain multiple optimal window images; Calculate the average grayscale values of the pixels of the plurality of optimal window images to obtain a centering frame template.

7. A defect detection device in an image pair, characterized in that, include: A first acquisition module is used to acquire an image of a wheel hub to be detected; a centering module, configured to perform a centering operation on the wheel hub image using a pre-built centering frame template to obtain a centering result, wherein the centering result is used to indicate whether the centering of the wheel hub image is successful; a processing module, configured to input the wheel hub image into a pre-trained wheel hub defect detection network according to the alignment result to obtain a pre-selected candidate frame, wherein the pre-selected candidate frame includes the wheel hub defect position, defect category and confidence level in the wheel hub image; a generation module, configured to generate defect information in the wheel hub image based on the preselected candidate box and a pre-built target attention matrix, wherein the attention matrix is used to represent probability information of defects occurring at corresponding pixel positions; Wherein, the generation module includes: a first calculation unit, used to calculate the box center of the pre-selected candidate box; a second calculation unit, used to calculate the distance between the box center and the cluster center of each attention matrix in the pre-constructed attention matrix set, and select the target attention matrix with the closest distance; a first operation unit, used to multiply the pre-selected candidate box with the corresponding coordinates in the target attention matrix and then accumulate them to obtain an initial candidate box set, wherein each candidate box corresponds to a confidence level; a first filtering unit, used to filter the initial candidate boxes with a confidence level less than a first threshold in the initial candidate box set to obtain an intermediate candidate box set; a second operation unit, used to calculate the intersection area and the union area in the intermediate candidate box set, and calculate the intersection-union ratio based on the intersection area and the union area; a second filtering unit, used to filter the intermediate candidate boxes with an intersection-union ratio less than a second threshold in the intermediate candidate box set to obtain a defect box set, wherein each defect box includes a wheel hub defect position, a defect category, and a confidence level; The device further includes: a third acquisition module for acquiring a plurality of defect sample images before the generation module generates the defect information in the hub image according to the pre-selected candidate box and the pre-built target attention matrix; a statistical module for counting the calibration coordinates and defect sizes of the defect area in each of the plurality of defect sample images in a preset calibration direction; a clustering module for clustering the center points of the defect sizes into an initial number of categories; a second iteration module for iteratively executing the following steps until an intersection with an area greater than a preset number of pixels appears: calculating the number of pixels in each cluster; The intersection of multiple defect masks; determining whether the number of pixels in the intersection is greater than a preset number; if the number of pixels in the intersection is less than or equal to the preset number, increasing the number of cluster categories; a configuration module for obtaining multiple clusters after the iteration is completed, and configuring an initial attention matrix for each cluster, wherein the elements of the initial attention matrix are all 0; a reset module for resetting the element values in the initial attention matrix for each cluster according to the number of defect masks overlapping at each element position, to obtain an attention matrix set, wherein the element values are positively correlated with the number of overlapping defect masks.

8. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 6.

Images