Image defect detection method, image defect detection model generation method, device, equipment, readable storage medium and program product

Abstract

The invention relates to an image defect detection method, an image defect detection model generation method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring to-be-detected image data, wherein the to-be-detected image data comprises multiple types of to-be-detected defects; inputting the to-be-detected image data into the image defect identification model, and obtaining defect detection results corresponding to various types of to-be-detected defects output by the image defect identification model; the image defect recognition model is obtained by training image training sample data, and an image training sample is determined after historical initial image data are screened for multiple times. By adopting the method, the accuracy of detecting the tiny defects in the image can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image defect detection method, an image defect detection model generation method, an apparatus, a computer device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] Power line communication (PLC) chips serve as the core communication carrier for smart grid power distribution monitoring and industrial IoT device interconnection. The chips integrate precision interconnection structures such as aluminum wire bonding and gold wire bonding (the bonding point size is mostly at the micrometer level), and the surface is covered with a thin plastic encapsulation layer with dense circuit texture. At the same time, in order to adapt to the complex channel environment of power lines, the chips also need to have anti-interference and high stability communication capabilities.

[0003] However, the aforementioned structural characteristics and usage scenarios of chips make them prone to defects, such as broken aluminum wire bonds, chip scratches, and wrinkles on the adhesive surface. Although these defects are small in physical size, they can directly damage the chip's communication link, leading to an increase in the error rate of the PLC system, and even causing serious malfunctions such as industrial production line shutdowns and power distribution data transmission interruptions. Therefore, accurate detection of minute defects in PLC chips is particularly important.

[0004] The detection accuracy of related technologies for minute surface defects of chips is insufficient. Summary of the Invention

[0005] Therefore, it is necessary to provide an image defect detection method, image defect detection model generation method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can accurately identify image defects, in order to address the above-mentioned technical problems.

[0006] In a first aspect, this application provides an image defect detection method, comprising:

[0007] Acquire the image data to be detected, which includes various types of defects to be detected;

[0008] The image data to be detected is input into the image defect recognition model, and the defect detection results corresponding to various types of defects are obtained from the output of the image defect recognition model. The image defect recognition model is trained using image training sample data, which is determined after multiple screenings of historical initial image data.

[0009] In one embodiment, before inputting the image data to be detected into the image defect recognition model and obtaining the defect detection results corresponding to various types of defects output by the image defect recognition model, the image defect detection method includes: acquiring historical image data, each historical image data including various types of defects; identifying the defects and defect types in each historical image data, and labeling the defects using the defect types to generate image training sample data; and training using the image training sample data to generate the image defect recognition model.

[0010] In one embodiment, before acquiring historical image data, which includes various types of defects, the image defect detection method includes: acquiring historical initial image data; inputting the historical initial image data into a defect classification model, and acquiring the defects and their credit values ​​corresponding to each historical initial image data output by the defect classification model; if the credit value is greater than or equal to a preset threshold, then the historical initial image data is used as historical image data.

[0011] In an optional embodiment, after using historical initial image data as historical image data, the method includes: determining positioning markers in the historical image data; using the positioning markers as references, processing the historical image data using a perspective transformation matrix to obtain corrected historical image data; determining first defect region information and second defect region information of the corrected historical image data; and determining that the historical image data has a defect when the first defect region information matches the second defect region information.

[0012] Secondly, this application provides a method for generating an image defect detection model, the method comprising:

[0013] Acquire historical initial image data, input the historical initial image data into the defect classification model, and obtain the defects and their credit values ​​corresponding to each historical initial image data output by the defect classification model.

[0014] Historical initial image data with a defect credit value greater than a preset threshold is used as historical image data, and the location markers in the historical image data are determined.

[0015] Historical image data is processed using a perspective transformation matrix to obtain corrected historical image data.

[0016] Determine the first defect region information and the second defect region information of the corrected historical image data;

[0017] Historical image data from matching the information of the first defect region with the information of the second defect region is used to generate image training samples, and the image defect recognition model is trained using the image training samples.

[0018] Thirdly, this application provides an image defect detection device, the device comprising:

[0019] The image data acquisition module is used to acquire the image data to be detected, which includes various types of defects to be detected.

[0020] The defect result acquisition module is used to input the image data to be detected into the image defect recognition model and obtain the defect detection results corresponding to various types of defects output by the image defect recognition model. The image defect recognition model is trained using image training sample data, which is determined after multiple screenings of historical initial image data.

[0021] Fourthly, this application provides an image defect detection model generation apparatus, the apparatus comprising:

[0022] The historical data acquisition module is used to acquire historical initial image data, input the historical initial image data into the defect classification model, and obtain the defects and their credit values ​​corresponding to each historical initial image data output by the defect classification model.

[0023] The marker point determination module is used to take historical initial image data with a credit value greater than a preset threshold as historical image data and determine the location marker points in the historical image data.

[0024] The correction module is used to process historical image data using a perspective transformation matrix to obtain corrected historical image data.

[0025] The defect region determination module is used to determine the first defect region information of the corrected historical image data and the second defect region information of the historical image data.

[0026] The model generation module is used to generate image training samples by using historical image data when matching the information of the first defect region with the information of the second defect region, and to train the image defect recognition model using the image training samples.

[0027] Fifthly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method embodiments.

[0028] Sixthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method embodiments.

[0029] In a seventh aspect, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method embodiments.

[0030] The above-mentioned image defect detection method, image defect detection model generation method, apparatus, computer equipment, computer-readable storage medium, and computer program product, wherein the image defect detection method uses image data obtained after multiple screenings to determine image training samples, and uses the determined image training samples to realize the detection of the image defect recognition model, effectively ensuring the accuracy of the image defect recognition model detection, thereby ensuring the accuracy of the defects detected by the image defect recognition model. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is an application environment diagram of the image defect detection method and the image defect detection model generation method in one embodiment;

[0033] Figure 2 This is a flowchart illustrating an image defect detection method in one embodiment;

[0034] Figure 3 Here is a structural block diagram of the improved model in one embodiment;

[0035] Figure 4 This is a flowchart illustrating the image defect detection model generation method in another embodiment;

[0036] Figure 5 This is a flowchart illustrating the image defect detection method in another embodiment;

[0037] Figure 6 This is a structural block diagram of an image defect detection device in one embodiment;

[0038] Figure 7 This is a structural block diagram of an image defect detection model generation device in one embodiment;

[0039] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0041] The image defect detection method and image defect detection model generation method provided in this application can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Server 104 acquires the image data to be detected uploaded by terminal 102, which includes various types of defects to be detected; it inputs the image data to be detected into an image defect recognition model and obtains the defect detection results corresponding to various types of defects output by the image defect recognition model; the image defect recognition model is trained using image training sample data, which is determined after multiple screenings of historical initial image data. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. Server 104 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0042] In one exemplary embodiment, such as Figure 2 As shown, an image defect detection method is provided, which is applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 202 to 204. Wherein:

[0043] Step 202: Obtain the image data to be detected.

[0044] In one embodiment, the image data to be detected can be image data from a power line communication chip.

[0045] Among them, power line communication chips serve as the communication carrier for smart grid power distribution monitoring and industrial IoT device interconnection.

[0046] Alternatively, the image data to be detected can be image data generated by capturing images of the power line communication chip using a camera device.

[0047] In one embodiment, the image data to be detected includes multiple types of defects to be detected.

[0048] In one embodiment, defects may include, but are not limited to, aluminum wire bond fractures, shallow scratches on the chip, adhesive surface wrinkles, chip damage, and gold wire bond fractures. Optionally, aluminum wire bond fractures can be defects with dimensions on the micrometer scale. Optionally, gold wire bond fractures can cause defects with strong reflectivity and blurred edges. Optionally, shallow scratches on the chip can be less than 5 μm in depth and have low contrast. Optionally, adhesive surface wrinkles can be shapeless and have weak edge features. Optionally, chip damage can be localized micro-dimples with little difference from normal texture.

[0049] In an optional embodiment, the image data to be detected can be image data that has been pre-identified and screened as potentially defective, in order to exclude the situation where it is mistakenly identified as a defect due to distortion caused by issues such as the shooting angle of the camera equipment, thereby reducing the amount of data processing when using the image defect recognition model to identify defects in the image and improving the efficiency of defect recognition in the image.

[0050] For example, the initial image data to be detected can be obtained, and the positioning markers in the initial image data to be detected can be obtained. Using the positioning markers as a reference, the initial image data to be detected can be processed using a perspective transformation matrix to obtain the corrected initial image data to be detected. The defect area information of the corrected initial image data to be detected and the defect area information of the initial image data to be detected can be determined. If the defect area information of the corrected initial image data to be detected matches the defect area information of the initial image data to be detected, it is considered that there is indeed a defect in the initial image data to be detected, and it can be divided into image data to be detected so that the server can obtain the image data to be detected and identify the defects in the image data based on the image data to be detected.

[0051] In this context, positioning markers are used to locate target objects in image data. For example, if the target object in the image data is a chip, the chip's boundary baseline or circuit texture feature points can be used as positioning markers to locate the chip.

[0052] The defect region information is used to characterize the attribute information of the defect. Optionally, the defect region information may include, but is not limited to, crack length, stain area, and pixel dimension.

[0053] Optionally, the defect region information of the corrected initial image data to be detected is compared with the defect region information of the initial image data to be detected to determine whether the defect region information of the corrected initial image data to be detected matches the defect region information of the initial image data to be detected. This can be a comparison of crack length, stain area and pixel dimension, or a comparison of one of the parameter features among crack length, stain area and pixel dimension.

[0054] Step 204: Input the image data to be detected into the image defect recognition model, and obtain the defect detection results corresponding to various types of defects to be detected output by the image defect recognition model.

[0055] The image defect recognition model is used to identify defects in an image. Optionally, the image defect recognition model can also be used to identify the types of defects in an image and the corresponding credit values ​​for those types.

[0056] Optionally, the image defect recognition model is trained using image training sample data.

[0057] Among them, the image training sample data is used to train and generate an image defect recognition model.

[0058] In one embodiment, the image training samples can be determined by filtering historical initial image data multiple times.

[0059] In one embodiment, the image training sample generation process may include: acquiring historical image data, each historical image data including multiple types of defects; identifying defects and defect types in each historical image data, and labeling defects using defect types to generate image training sample data; and training using the image training sample data to generate an image defect recognition model.

[0060] In one embodiment, historical image data can be a single image containing multiple types of defects. Optionally, defects may include, but are not limited to, broken aluminum wire bonds, chip scratches, and wrinkles on the adhesive surface.

[0061] Alternatively, historical image data can be records retained during the production process.

[0062] In one embodiment, the identification of historical image data and the determination of defects and defect types in the historical image data can be done manually. For example, the manual determination of the presence of defects in the image data and the determination of defect types can be used to label the defects. The labeled image data can then be used as one of the image training samples.

[0063] In an optional embodiment, a defect recognition model can also be used to identify defects and defect types in historical image data. Alternatively, the identified defects and their types can be manually verified, and then labeled using the defect types. Once labeled, these can be used as image training samples.

[0064] In one exemplary embodiment, labeled image data can also be acquired, and the labeled image data can be used as a reference to identify defects and defect types in historical image data. The identified defects and defect types can then be used to label the historical image data to form image training samples.

[0065] In one embodiment, historical image data can also be identified to determine defective and normal regions; defect attribute information, such as defect type and defect size, can be determined; the defective region can be compared with the defect attribute information to determine the type of defect; and the image data can be labeled using the defect type to form image training samples.

[0066] In one embodiment, before acquiring historical image data and processing it to obtain an image defect recognition model, the process may further include processing historical initial image data to obtain historical image data. This may include: acquiring historical initial image data; inputting the historical initial image data into a defect classification model and acquiring the defects and their credit values ​​corresponding to each historical initial image data output by the defect classification model; if the credit value is greater than or equal to a preset threshold, then the historical initial image data is used as historical image data.

[0067] The credit score characterizes the credibility of defects identified in historical initial image data and can be used to filter the credibility of defects in historical initial image data. For example, if the credit score is less than a preset threshold, it can be considered that the possibility of defects in historical initial image data is low, and historical initial image data will not be used as historical image data.

[0068] In one exemplary embodiment, when the credit value is determined to be less than a preset threshold, the historical initial image data can be output for a second manual review to determine whether there are indeed defects in the historical initial image data.

[0069] In an alternative embodiment, the defect classification model can be the YOLOv5n-C2f-SSPA detection network.

[0070] Optionally, the preset threshold can be 50%.

[0071] In one embodiment, the historical initial image data can also be numbered to enable sequential processing of the historical initial image data.

[0072] In one embodiment, to ensure the accuracy of the generated image defect recognition, the historical image data can be further processed to ensure the accuracy of the generated image training samples. This can include: determining the location markers in the historical image data; processing the historical image data using a perspective transformation matrix to obtain corrected historical image data; determining the first defect region information and the second defect region information of the corrected historical image data; and determining that the historical image data has a defect when the first defect region information matches the second defect region information.

[0073] Among them, positioning markers are used to locate target objects in image data. For example, when the target object in the image data is a chip, the positioning markers can be corner reference lines, circuit texture feature points, etc.

[0074] Among them, the defect region information is used to characterize the attribute information of the defect. Optionally, the defect region information can be crack length, stain area, pixel dimension of stain area, etc.

[0075] In one embodiment, first defect region information and second defect region information are compared. If the first defect region information matches the second defect region information, then it is considered that a defect does exist in the historical image data. Optionally, if the first defect region information does not match the second defect region information, then it is considered that no defect exists in the historical image data.

[0076] Optionally, comparing the information of the first defect region and the information of the second defect region to determine whether the information of the first defect region matches the information of the second defect region can be comparing crack length, stain area and pixel dimension, or comparing one of the parameter features of crack length, stain area and pixel dimension.

[0077] In one embodiment, after generating image training samples, the YOLOv5n model can be trained using the image training sample data to generate an image defect recognition model.

[0078] In an optional embodiment, the YOLOv5n model can first be subjected to attention learning to obtain an improved YOLOv5n model, and the improved YOLOv5n model can be used to train and generate an image defect recognition model.

[0079] Optionally, a focus mechanism can be used to learn the focus of the YOLOv5n model. Optionally, the focus mechanism can be SSPA.

[0080] In one embodiment, the improved YOLOv5n model may include a backbone network and a detection head.

[0081] Optionally, such as Figure 3 As shown, the backbone network, from bottom to top, performs downsampling and feature extraction on the input image through alternating operations of multiple Conv (convolutional layers) and C3 modules: the Conv layer is responsible for basic feature mapping, while the C3 module strengthens feature expression with an efficient structure of residual connections. Finally, multi-scale features are aggregated through the SPPF spatial pyramid pooling fast version to provide feature inputs at different levels for the detection head.

[0082] Optionally, such as Figure 3As shown, the detection head employs a multi-scale feature fusion strategy, fusing deep and shallow features output from the backbone network across scales through Upsample and Concat operations. Optionally, the C2f-SSPA module is a module obtained by learning the focus of the YOLOv5n model using the SSPA mechanism. It is used to efficiently capture multi-scale features while accurately refining local details, such as capturing minute defects like broken aluminum wire bonds and broken gold wire bonds in communication chips.

[0083] In one embodiment, the process of using an improved YOLOv5n model to generate an image defect recognition model for recognizing image data may include: high-level features output by the backbone are processed by Conv and Upsample, and then Concatted with mid-level features, before being input into the first C2f-SSPA module for feature enhancement; subsequently, they are processed again by Conv, Upsample, and Concat, before entering the second C2f-SSPA module; low-level features are processed by Conv and Concat, before being input into the third C2f-SSPA module. After three feature enhancements, three detection heads perform classification and bounding box regression on "small-scale (e.g., tiny defects)," "medium-scale," and "large-scale" targets, respectively, and output the final detection result. For example, as shown... Figure 3 As shown, the image defect recognition model is used to identify the broken aluminum wire bond in the image and mark it.

[0084] In one embodiment, the C2f-SSPA module is a two-level attention mechanism module used to optimize feature extraction in stages. For example, the C2f-SSPA module may include a spatial attention module and a pixel attention module.

[0085] In one embodiment, to adapt to the random distribution and multi-scale characteristics of defects on the chip surface, the spatial attention module employs asymmetric depthwise separable convolutional combinations to capture spatial information.

[0086] For example, formula (1) specifically uses 5x5, 1x7, 7x1, 1x11, 11x1, 1x21, and 21x1 convolutional kernels to capture spatial information at different scales. Furthermore, it replaces large convolutional kernels with a series of asymmetric convolutional combinations, which can increase the receptive field while reducing the computational cost. Among them, the 5x5 convolutional kernel is used for basic local feature extraction, the receptive field of the 1x7 and 7x1 convolutional kernels is equivalent to that of the 7x7 convolutional kernel, and is used to capture mid-range horizontal and vertical spatial relationships, the receptive field of the 1x11 and 11x1 convolutional kernels is equivalent to that of the 11x11 convolutional kernel, and is used to capture long-range horizontal and vertical relationships, and the receptive field of the 1x21 and 21x1 convolutional kernels is equivalent to that of the 21x21 convolutional kernel. By using multiple types of convolutional kernels, the global spatial context can be captured, thereby adapting to targets of different sizes.

[0087] In one embodiment, spatial feature maps with differentiated characteristics are generated after each convolution operation. Then, these feature maps are summed and fused using formula (2) with the help of 1x1 convolution, unifying the multi-branch features to the same spatial dimension. The above method can capture spatial relationships at different scales and improve the accuracy of data acquisition.

[0088] (1)

[0089] (2)

[0090] In the above formula, (X) is the differential spatial feature map output after multi-scale asymmetric convolution processing. This represents a convolution operation with a kernel size of k×1, where X is the input feature map; It is a spatial attention map obtained by fusing multi-scale spatial features. It is a 1×1 convolution (used for channel fusion and dimension unification). This involves element-wise summation of the differential spatial feature maps corresponding to k∈{7,11,21}, where k represents the scale level of the equivalent receptive field of the convolution kernel, taking values ​​of 7, 11, and 21 to cover spatial relationships at different distances.

[0091] In one embodiment, the pixel attention module adjusts the criticality of each pixel location in the input feature map by learning a spatial weight map.

[0092] Optionally, the specific process is as follows: In the forward propagation process, two 1×1 convolutions are used. First, formula (3) is executed, which compresses the number of channels to 1 / 8 of the original number. This reduces the computational load, promotes cross-channel interaction, and introduces nonlinearity by activating ReLU. The second 1x1 convolution restores C / 8 to 1 channel, and the weight value of each pixel position is determined by all the compressed channels. The staged channel compression design reduces the number of parameters while avoiding information loss caused by single-step compression, and prevents the number of parameters from being too large due to the direct use of C→1 1x1 convolution.

[0093] For example, after completing the second 1x1 convolution to restore C / 8 to 1 channel, the Sigmoid activation function can be used to normalize the pixel weight values ​​to the [0, 1] interval. This interval is used to represent the criticality of each pixel, thereby ensuring that the weights are within a reasonable range and preventing gradient explosion or gradient vanishing.

[0094] For example, after normalizing the weight values, spatial weighting is used, and single-channel attention is repeatedly copied C times along the channel dimension. Then, it is multiplied element by element with the input feature map to improve the feature response of key regions, while suppressing irrelevant background parts and improving the accuracy of image data extraction.

[0095] In one embodiment, the pixel attention module can be a PixelAttention module. The PixelAttention module employs residual-free connections, which allows it to maintain its lightweight nature and is suitable for embedding into various layers of the network. Similar to mainstream attention modules such as SENet, CBAM, and ECA-Net, which also lack residual connections and directly scale the feature channels, this module is optimized for compatibility.

[0096] (3)

[0097] (4)

[0098] in, It is the input feature map. This represents the first 1×1 convolution operation, where x is the compressed intermediate feature map obtained. This is the output feature map after pixel weight adjustment. * represents the element-wise multiplication operation between feature maps, σ represents the Sigmoid activation function that normalizes the features to the [0,1] interval, the inner Conv represents the first convolution operation on the intermediate feature map x, and ReLU represents the introduction of a non-linear activation function; the outer Conv represents the second convolution operation used to restore the number of channels and generate the pixel weight map.

[0099] In one embodiment, an image defect recognition model is used to identify the image data to be processed, and the defect detection result output by the image defect recognition model can be image data. Optionally, the image data can include the defect label, the defect location, and the defect type.

[0100] Optionally, the defect detection results can also be a table file, in which the table file records the image number of the image to be processed containing defects, the defect in the image to be processed, the location of the defect, and the confidence level corresponding to the defect.

[0101] In the aforementioned image defect detection method, image training samples are determined using image data obtained after multiple screenings. These training samples are then used to implement the image defect recognition model, effectively ensuring the accuracy of the model's detection and thus guaranteeing the accuracy of the defects detected. Furthermore, the generation of the image defect recognition model utilizes an attention mechanism with YOLOv5n, effectively ensuring that the generated model possesses a two-level attention mechanism (spatial and pixel-level), thus guaranteeing the accuracy of data extraction and consequently, the accuracy of defect recognition.

[0102] In one exemplary embodiment, such as Figure 4 As shown, an image defect detection method is provided, which is applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 402 to 410. Wherein:

[0103] Step 402: Obtain historical initial image data, input the historical initial image data into the defect classification model, and obtain the defects and their credit values ​​corresponding to each historical initial image data output by the defect classification model.

[0104] The credit score characterizes the credibility of defects identified in historical initial image data and can be used to filter the credibility of defects in historical initial image data. For example, if the credit score is less than a preset threshold, it can be considered that the possibility of defects in historical initial image data is low, and historical initial image data will not be used as historical image data.

[0105] In an alternative embodiment, the defect classification model can be the YOLOv5n-C2f-SSPA detection network.

[0106] Step 404: Use historical initial image data with a defect credit value greater than a preset threshold as historical image data, and determine the location markers in the historical image data.

[0107] Optionally, the preset threshold can be 50%.

[0108] In one exemplary embodiment, when the credit value is determined to be less than a preset threshold, the historical initial image data can be output for a second manual review to determine whether there are indeed defects in the historical initial image data.

[0109] Step 406: Using the positioning markers as a reference, process the historical image data using the perspective transformation matrix to obtain the corrected historical image data.

[0110] Among them, positioning markers are used to locate target objects in image data. For example, when the target object in the image data is a chip, the positioning markers can be corner reference lines, circuit texture feature points, etc.

[0111] It is understandable that perspective transformation matrices are used to correct historical image data, reduce image distortion caused by camera angles, and ensure data accuracy.

[0112] Step 408: Determine the first defect region information of the corrected historical image data and the second defect region information of the historical image data.

[0113] Step 410: Use historical image data from when matching the information of the first defect region with the information of the second defect region to generate image training samples, and use the image training samples to train an image defect recognition model.

[0114] Among them, the defect region information is used to characterize the attribute information of the defect. Optionally, the defect region information can be crack length, stain area, pixel dimension of stain area, etc.

[0115] In one embodiment, first defect region information and second defect region information are compared. If the first defect region information matches the second defect region information, then the historical image data is considered to indeed contain a defect, and it is used as a training sample. Optionally, if the first defect region information does not match the second defect region information, then the historical image data is considered to contain no defect.

[0116] Optionally, comparing the information of the first defect region and the information of the second defect region to determine whether the information of the first defect region matches the information of the second defect region can be comparing crack length, stain area and pixel dimension, or comparing one of the parameter features of crack length, stain area and pixel dimension.

[0117] In one embodiment, after generating image training samples, the YOLOv5n model can be trained using the image training sample data to generate an image defect recognition model.

[0118] In this embodiment, image training samples are determined using image data obtained after multiple screenings, and the determined image training samples are used to realize the detection of the image defect recognition model, which effectively ensures the accuracy of the image defect recognition model detection, thereby ensuring the accuracy of the defects detected by the image defect recognition model.

[0119] In one exemplary embodiment, such as Figure 5 As shown, an image defect detection method is provided, which is applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 502 to 514. Wherein:

[0120] Step 502: Obtain historical initial image data, input the historical initial image data into the defect classification model, and obtain the defects and their credit values ​​corresponding to each historical initial image data output by the defect classification model.

[0121] Step 504: Use historical initial image data with a defect credit value greater than a preset threshold as historical image data, and determine the location markers in the historical image data.

[0122] Step 506: Using the positioning markers as a reference, process the historical image data using the perspective transformation matrix to obtain the corrected historical image data.

[0123] Step 508: Determine the first defect region information of the corrected historical image data and the second defect region information of the historical image data.

[0124] Step 510: Use historical image data from when matching the information of the first defect region with the information of the second defect region to generate image training samples, and use the image training samples to train an image defect recognition model.

[0125] Step 512: Obtain the image data to be detected, which includes various types of defects to be detected.

[0126] Step 514: Input the image data to be detected into the image defect recognition model, and obtain the defect detection results corresponding to various types of defects to be detected output by the image defect recognition model.

[0127] In the above embodiments, image training samples are determined using image data obtained after multiple screenings, and the determined image training samples are used to realize the detection of the image defect recognition model, which effectively ensures the accuracy of the image defect recognition model detection, thereby ensuring the accuracy of the defects detected by the image defect recognition model.

[0128] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0129] Based on the same inventive concept, this application also provides an image defect detection apparatus for implementing the image defect detection method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image defect detection apparatus embodiments provided below can be found in the limitations of the image defect detection method described above, and will not be repeated here.

[0130] In one exemplary embodiment, such as Figure 6 As shown, an image defect detection device 600 is provided, including: an image data acquisition module 602 and a defect result acquisition module 604, wherein:

[0131] The image data acquisition module 602 is used to acquire the image data to be detected, which includes various types of defects to be detected.

[0132] The defect result acquisition module 604 is used to input the image data to be detected into the image defect recognition model and obtain the defect detection results corresponding to various types of defects to be detected output by the image defect recognition model; the image defect recognition model is trained using image training sample data, and the image training samples are determined after multiple screenings of historical initial image data.

[0133] In one embodiment, the image defect detection device further includes a model generation module, which is used to acquire historical image data, each historical image data including multiple types of defects, before acquiring the defect detection results corresponding to multiple types of defects to be detected output by the image defect recognition model; identify the defects and defect types in each historical image data, and label the defects using the defect types to generate image training sample data; and train using the image training sample data to generate an image defect recognition model.

[0134] In one embodiment, the image defect detection device further includes acquiring historical image data, wherein each historical image data includes multiple types of defects, and acquiring historical initial image data; inputting the historical initial image data into a defect classification model, and acquiring the defects and credit values ​​of each historical initial image data output by the defect classification model; if the credit value is greater than or equal to a preset threshold, then the historical initial image data is used as historical image data.

[0135] In an optional embodiment, the model generation module is configured to: use historical initial image data as historical image data; determine positioning markers in the historical image data; use the positioning markers as references to process the historical image data using a perspective transformation matrix to obtain corrected historical image data; determine first defect region information and second defect region information of the corrected historical image data; and determine that the historical image data has defects when the first defect region information matches the second defect region information.

[0136] Based on the same inventive concept, this application also provides an image defect detection apparatus for implementing the image defect detection model generation method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the image defect detection model generation apparatus provided below can be found in the limitations of the image defect detection method described above, and will not be repeated here.

[0137] In one exemplary embodiment, such as Figure 7 As shown, an image defect detection model generation device 700 is provided, including: a historical data acquisition module 702, a marker point determination module 704, a correction module 707, a defect region determination module 708, and a model generation module 710, wherein:

[0138] The historical data acquisition module 702 is used to acquire historical initial image data, input the historical initial image data into the defect classification model, and obtain the defects and credit values ​​corresponding to each historical initial image data output by the defect classification model.

[0139] The marker point determination module 704 is used to take historical initial image data with a credit value of defects greater than a preset threshold as historical image data and determine the location marker points in the historical image data.

[0140] The correction module 706 is used to process historical image data using a perspective transformation matrix to obtain corrected historical image data.

[0141] The defect region determination module 708 is used to determine the first defect region information of the corrected historical image data and the second defect region information of the historical image data.

[0142] The model generation module 710 is used to generate image training samples by using historical image data when matching the information of the first defect region with the information of the second defect region, and to train an image defect recognition model using the image training samples.

[0143] Each module in the aforementioned image defect detection device and image defect detection model generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0144] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements an image defect detection method and an image defect detection model generation method.

[0145] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0146] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method embodiments.

[0147] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method embodiments.

[0148] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the above-described method embodiments.

[0149] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0150] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0151] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0152] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image defect detection method, characterized in that, The method includes: Acquire image data to be detected, which includes various types of defects to be detected; The image data to be detected is input into the image defect recognition model, and the defect detection results corresponding to the various types of defects to be detected are obtained from the output of the image defect recognition model; the image defect recognition model is trained using image training sample data, and the image training samples are determined after multiple screenings of historical initial image data.

2. The method according to claim 1, characterized in that, Before inputting the image data to be detected into the image defect recognition model and obtaining the defect detection results corresponding to the various types of defects output by the image defect recognition model, the method includes: Acquire historical image data, which includes various types of defects; Identify defects and defect types in each of the historical image data, and label the defects using the defect types to generate image training sample data; The image training sample data is used to train and generate the image defect recognition model.

3. The method according to claim 1, characterized in that, Before acquiring historical image data, where each historical image data includes multiple types of defects, the method includes: Obtain historical initial image data; The historical initial image data is input into the defect classification model, and the defects and their credit values ​​are obtained from the output of the defect classification model for each historical initial image data. If the credit value is greater than or equal to a preset threshold, then the historical initial image data will be used as historical image data.

4. The method according to claim 1, characterized in that, After using the historical initial image data as historical image data, the method includes: Identify the location markers in the historical image data; Using the positioning markers as a reference, the historical image data is processed using a perspective transformation matrix to obtain corrected historical image data. Determine the first defect region information and the second defect region information of the corrected historical image data; When the first defect area information matches the second defect area information, it is determined that the historical image data has a defect.

5. A method for generating an image defect detection model, characterized in that, The method includes: Acquire historical initial image data, input the historical initial image data into the defect classification model, and obtain the defects and credit values ​​corresponding to each historical initial image data output by the defect classification model. Historical initial image data in which the credit value of the defect is greater than a preset threshold is used as historical image data, and the location markers in the historical image data are determined. The historical image data is processed using a perspective transformation matrix to obtain corrected historical image data; Determine the first defect region information and the second defect region information of the corrected historical image data; Historical image data from when the first defect region information and the second defect region information are matched are used to generate image training samples, and an image defect recognition model is trained using the image training samples.

6. An image defect detection device, characterized in that, The device includes: The image data acquisition module is used to acquire image data to be detected, which includes various types of defects to be detected; The defect result acquisition module is used to input the image data to be detected into the image defect recognition model and obtain the defect detection results corresponding to the various types of defects to be detected output by the image defect recognition model; the image defect recognition model is trained using image training sample data, and the image training samples are determined after multiple screenings of historical initial image data.

7. An image defect detection model generation device, characterized in that, The device includes: The historical data acquisition module is used to acquire historical initial image data, input the historical initial image data into the defect classification model, and acquire the defects and credit values ​​of each historical initial image data output by the defect classification model. The marker point determination module is used to take historical initial image data in which the credit value of the defect is greater than a preset threshold as historical image data, and determine the location marker points in the historical image data; The correction module is used to process the historical image data using a perspective transformation matrix to obtain corrected historical image data. The defect region determination module is used to determine the first defect region information of the corrected historical image data and the second defect region information of the historical image data. The model generation module is used to generate image training samples using historical image data when the first defect region information and the second defect region information are matched, and to train an image defect recognition model using the image training samples.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images