Deep learning defect region precise labeling method and system for industrial quality inspection image

Abstract

This invention discloses a deep learning-based method and system for accurate labeling of defect regions in industrial quality inspection images, belonging to the field of image processing technology. The method includes: obtaining multimodal input data based on the industrial quality inspection image to be labeled and the corresponding production process parameters; obtaining preliminary labeled defect categories and boundaries based on the multimodal input data and a multimodal defect gene library; and obtaining the labeled defect region results based on the preliminary labeled defect categories and boundaries and Bayesian iterative update rules. This invention integrates multimodal data and matrix iterative algorithms, and solves the problems of overlapping defect confusion and missing labeling of minor defects in complex backgrounds through defect temporal tracing, physical constraint segmentation, and self-evolutionary closed loop, thereby improving labeling accuracy and generalization, adapting to the needs of real-time industrial quality inspection, and reducing labor costs.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method and system for accurate annotation of defect regions in industrial quality inspection images using deep learning. Background Technology

[0002] With the integration of industrial automation and deep learning technologies, industrial quality inspection is gradually developing towards high precision and automation. In scenarios such as automotive steel plate welds, PCB boards, and aerospace parts, defects often exist in the form of overlapping, small, and low-contrast areas. Accurate labeling of defect areas is a core prerequisite for subsequent defect detection and quality assessment. Currently, defect labeling in industrial quality inspection images relies on multi-source data support and complex algorithm adaptation. However, existing labeling methods do not fully combine the process and physical characteristics of industrial scenarios, lack sensitivity to small, low-contrast defects, and have prominent issues with missed labeling. At the same time, the models have poor generalization and lack a self-evolutionary iteration mechanism, making it impossible to balance the real-time quality inspection efficiency and labeling accuracy of the production line. They are also difficult to adapt to the industrial automation quality inspection needs of multiple materials and scenarios. Therefore, existing technologies have shortcomings. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for accurate annotation of defect regions in industrial quality inspection images using deep learning. This method achieves accurate annotation by combining core algorithms such as full-process time-series steps, matrix iteration, and multimodal fusion with the physical and technological characteristics of industrial scenarios.

[0004] To achieve the above objectives, the present invention provides the following technical solution: This invention provides a deep learning-based method for accurate annotation of defect regions in industrial quality inspection images, comprising: Based on the industrial quality inspection images to be labeled and the corresponding production process parameters, multimodal input data is obtained; Based on the multimodal input data and the multimodal defect gene library, the preliminary labeled defect categories and boundaries are obtained; Based on the initially labeled defect categories and boundaries and the Bayesian iterative update rule, the labeled defect regions are obtained.

[0005] As a further improvement of the present invention, the step of obtaining multimodal input data based on the industrial quality inspection image to be labeled and the corresponding production process parameters includes: Based on the industrial quality inspection image to be labeled and the preset texture template, a background mask matrix is ​​obtained; The coordinates of the candidate overlapping defect regions are obtained based on the background mask matrix and the defect density threshold. The multimodal input data is obtained based on the coordinates of the candidate overlapping defect regions and the corresponding production process parameters.

[0006] As a further improvement of the present invention, a deep learning-based method for accurate annotation of defect regions in industrial quality inspection images also includes: Based on the pixel connected components and grayscale difference threshold of the candidate overlapping defect regions, the defect coverage adjacency matrix is ​​obtained; Based on the defect coverage adjacency matrix and damping factor, the defect generation time series score vector is obtained; Based on the time-series score vector and the preset score sorting rules, the order in which defects are generated is obtained.

[0007] As a further improvement of the present invention, based on the multimodal input data and the multimodal defect gene library, preliminary labeled defect categories and boundaries are obtained, including: Based on the order in which the defects were generated and the multimodal defect gene code, the physical constraint Laplace matrix is ​​obtained; Based on the Laplacian matrix and the matrix-based energy function, the segmentation result matrix is ​​obtained by ADMM iteration. Based on the segmentation result matrix and the attention-weighted matching rule, the preliminary labeled defect categories and boundaries are obtained.

[0008] As a further improvement of the present invention, the step of obtaining the labeled defect region result based on the initially labeled defect category and boundary and the Bayesian iterative update rule includes: Based on the initially labeled defect categories and boundaries and the Bayesian iterative update rule, the calibration defect categories are obtained; Based on the calibration defect category and the segmentation result matrix, the labeled defect area results are obtained.

[0009] As a further improvement of the present invention, the step of obtaining the calibrated defect category based on the initially labeled defect category and boundary and the Bayesian iterative update rule includes: Based on the initially labeled defect categories and corresponding production process parameters, a causal probability matrix is ​​obtained; Based on the causal probability matrix and the Bayesian iterative update rule, the posterior probability distribution of the defect category is obtained; The calibration defect category is obtained based on the posterior probability distribution and the maximum probability determination rule.

[0010] As a further improvement of the present invention, the step of obtaining the labeled defect region result based on the calibration defect category and the segmentation result matrix includes: Based on the calibration defect category and the segmentation result matrix, the integer pixel coordinates of the defect boundary are obtained; Based on the integer pixel coordinates and the bilinear interpolation algorithm, sub-pixel level defect boundary coordinates are obtained; Based on the sub-pixel level defect boundary coordinates and deviation threshold verification rules, the labeled defect area results are obtained.

[0011] As a further improvement of the present invention, a deep learning-based method for accurate annotation of defect regions in industrial quality inspection images also includes: Based on the labeled defect region results and the multimodal defect gene library, the updated gene library features are obtained; Based on the updated gene pool features and cross-scene feature matrix, the shared feature base of NMF decomposition is obtained; Based on the shared feature base and self-supervised transfer rules, the model parameters labeled across scenes are obtained.

[0012] As a further improvement of the present invention, the step of obtaining cross-scene labeled model parameters based on the shared feature base and self-supervised transfer rules includes: Based on the shared feature base and the feature matrix of the target scene, the cross-scene feature mapping relationship is obtained; Based on the cross-scene feature mapping relationship and the incremental learning rate, the updated model parameters are obtained; Based on the updated model parameters and the newly added annotation data, the updated annotation model is obtained.

[0013] This invention provides a deep learning-based system for accurate annotation of defect regions in industrial quality inspection images, comprising: The data acquisition module is used to obtain multimodal input data based on the industrial quality inspection images to be labeled and the corresponding production process parameters; The first annotation module is used to obtain the preliminary annotation of defect categories and boundaries based on the multimodal input data and the multimodal defect gene library; The second annotation module is used to obtain the annotated defect region results based on the defect categories and boundaries of the initial annotation and the Bayesian iterative update rule.

[0014] This invention constructs a multimodal defect gene library based on multi-source data and combines iterative matching with attention-weighted matrices to effectively improve the accuracy of defect category matching. It traces the defect generation timeline using adjacency matrices and the PageRank iterative algorithm, and solves the matrix-based energy function using ADMM iterative solution to resolve the problem of overlapping defect boundary confusion in complex backgrounds, achieving accurate segmentation. It uses Bayesian matrix iterative inference to calibrate categories and bilinear interpolation for sub-pixel calibration, reducing mislabeling and omissions and improving boundary positioning accuracy. It achieves cross-scene feature transfer through NMF iterative decomposition and reduces the cost of labeling new scene samples through dynamic model iterative optimization, constructing a closed-loop self-evolving labeling system. Ultimately, it achieves accurate labeling of defect regions across all dimensions, balancing labeling accuracy and efficiency, adapting to the real-time quality inspection needs of industrial production lines, improving the automation and intelligence level of industrial quality inspection, forming a high technological barrier, and adapting to the industrial quality inspection needs of multiple materials and scenarios. Attached Figure Description

[0015] Figure 1 This is a schematic diagram illustrating the steps of a deep learning-based method for accurately annotating defect regions in industrial quality inspection images according to the present invention. Figure 2 A schematic diagram illustrating the steps to obtain the posterior probability distribution. Detailed Implementation

[0016] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof.

[0017] The term "and / or" in the following text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0018] like Figure 1 , Figure 2 As shown in the embodiment of this application, a method for accurate annotation of defect regions in industrial quality inspection images using deep learning is provided, including: Based on the industrial quality inspection images to be labeled and the corresponding production process parameters, multimodal input data is obtained; Based on the multimodal input data and the multimodal defect gene library, the preliminary labeled defect categories and boundaries are obtained; Based on the initially labeled defect categories and boundaries and the Bayesian iterative update rule, the labeled defect regions are obtained.

[0019] Specifically, the process begins by acquiring images of the weld seams of the automotive steel plates to be labeled using industrial quality inspection equipment. These images must clearly show the weld area and any potential defects, such as porosity and cracks. Simultaneously, the corresponding production process parameters for the steel plate weld seam are retrieved from the production management system, including key parameters such as welding current, welding speed, and welding voltage. Subsequently, the acquired images are preprocessed to remove irrelevant interference such as equipment shadows and ambient light reflections. The preprocessed images are then individually bound to the production process parameters using data identifiers, ensuring that each image corresponds to a unique set of production process parameters, ultimately forming multimodal input data.

[0020] Next, a pre-built multimodal defect gene library is invoked. This library stores multimodal defect gene codes for common defects in automotive steel sheet welds, such as porosity, cracks, and slag inclusions. It covers the visual features of various defects, such as the circular outline of porosity and the linear extension morphology of cracks; process features, such as the welding current fluctuation characteristics corresponding to cracks; and physical features, such as the material density anomalies corresponding to porosity. Then, the image features in the multimodal input data are compared one by one with the defect gene codes in the gene library. Through feature similarity analysis, candidate defect categories with high matching degrees to the defect features in the image are initially screened. Simultaneously, based on the boundary feature references of defects in the gene library, combined with the pixel distribution of defects in the image, the boundary range of the defects is initially delineated, ultimately obtaining the preliminary labeled defect categories and boundaries.

[0021] Next, for the initially labeled defect categories and boundaries, relevant auxiliary verification data is collected, including historical labeling correction records for this production batch and typical defect labeling results under similar process parameters. Then, following the Bayesian iterative update rule, the initial labeling results are combined with the auxiliary verification data. First, the initial probability of the initially labeled category is calculated, and then this probability value is iteratively corrected by incorporating effective information from the auxiliary data. For example, if a defect is initially labeled as a crack, but the auxiliary data shows that the probability of cracks occurring under this process parameter is extremely low, while the probability of porosity is relatively high, then the probability of the crack category is reduced and the probability of the porosity category is increased through iterative updates. Finally, based on the probability results after iterative updates, the final defect category is determined, and fine-tuned in conjunction with the initially defined boundaries, to obtain the final labeled defect area result, which includes a clear defect category and a precise boundary range.

[0022] This embodiment breaks through the limitations of traditional single visual data annotation by fusing image data with process parameters to form a multimodal input, making the annotation process more in line with actual industrial production. Preliminary annotation is achieved using a multimodal defect gene library, ensuring basic accuracy. Precise correction is performed through Bayesian iterative update rules, effectively reducing errors in the preliminary annotation and realizing a closed loop from data preparation to final result output. This provides support for the implementation of subsequent detailed steps, ensuring the systematic nature and integrity of the entire solution, and providing an efficient and accurate core approach for industrial quality inspection image defect annotation.

[0023] Furthermore, this embodiment provides a step for obtaining multimodal input data based on the industrial quality inspection image to be labeled and the corresponding production process parameters, including: Based on the industrial quality inspection image to be labeled and the preset texture template, a background mask matrix is ​​obtained; The coordinates of the candidate overlapping defect regions are obtained based on the background mask matrix and the defect density threshold. Multimodal input data is obtained based on the coordinates of the candidate overlapping defect regions and the corresponding production process parameters.

[0024] Specifically, firstly, a texture template corresponding to the image of the automotive steel sheet weld to be labeled is retrieved. This template is constructed based on the texture features of a defect-free automotive steel sheet weld, including typical features such as the normal texture distribution and material reflectivity of the weld area. Then, the image to be labeled is compared pixel-by-pixel with the texture template, calculating the texture similarity between each pixel in the image and the corresponding pixel in the template. Pixels with high similarity are identified as background pixels, i.e., areas with no possibility of defects; pixels with low similarity are identified as defect candidate pixels, i.e., areas where defects may exist. Finally, a background mask matrix is ​​constructed based on the determination results, setting the matrix positions corresponding to background pixels to 0 and the matrix positions corresponding to defect candidate pixels to 1. This matrix can quickly mask background interference in the image and focus on the defect candidate areas. For example, when constructing a template, it is necessary to statistically analyze the grayscale fluctuation range of the defect-free weld area and the texture gradient distribution of the local 3×3 neighborhood. For instance, the grayscale value of a normal weld area is concentrated between 180 and 220, and the gradient value is usually within ±10. Based on this, the standard for high similarity is set as follows: the grayscale difference between the pixel to be labeled and the corresponding pixel in the template is ≤10, and the deviation between the texture gradient of the 3×3 neighborhood where the pixel is located and the gradient of the template neighborhood is ≤15%.

[0025] Next, based on the background mask matrix, the defect distribution density of the regions with a pixel value of 1 in the matrix is ​​statistically analyzed, i.e., the defect candidate regions. Specifically, the image is divided into several fixed-size sub-regions, and the number of pixels with a pixel value of 1 in each sub-region is calculated. Combined with the area of ​​the sub-region, the defect distribution density of each sub-region is obtained. Then, a defect density threshold is set. This threshold is determined based on the common distribution patterns of defects in automotive steel plate welds and is used to determine whether there is a possibility of defect overlap in the sub-regions. For example, when the proportion of defect pixels in a sub-region exceeds 5%, there is a high probability that two or more defects overlap. In this case, the defect density threshold is set to ≥5% of the number of defect pixels in the sub-region. Sub-regions with a defect distribution density exceeding the threshold are identified as candidate regions with defect overlap. Finally, the pixel coordinates of the upper left and lower right corners of these candidate overlapping defect regions are recorded to clarify the specific location range of the candidate regions in the image.

[0026] Next, local image data corresponding to the candidate overlapping defect regions is extracted. This data focuses on the core areas where defects may overlap, significantly reducing the data volume compared to the complete image and improving subsequent processing efficiency. Then, production process parameters corresponding to the image to be labeled are retrieved from the production process parameter library, including key parameters such as welding current, welding speed, and welding voltage. These parameters reflect the process factors that may lead to defects during production. Finally, the local image data of the candidate overlapping defect regions is aligned with the corresponding production process parameters to ensure a one-to-one correspondence between image data and process parameters, forming multimodal input data. This data includes both the visual features of the defect candidate regions and related process features, providing rich multi-dimensional data support for subsequent accurate labeling.

[0027] This embodiment constructs a background mask matrix through texture template matching, effectively filtering out irrelevant background interference in the image, reducing the amount of data for subsequent processing, and improving annotation efficiency. By determining candidate overlapping defect regions through defect density threshold judgment, it achieves precise focusing on key annotation areas, avoiding over-processing of defect-free or single-defect regions, and concentrating subsequent annotation resources on complex overlapping defect regions. By fusing candidate region image data with process parameters to form multimodal input data, it provides a more comprehensive basis for subsequent defect category determination and boundary segmentation, making the annotation process more aligned with actual industrial production, improving the accuracy of initial annotation, and providing data support for the efficient implementation of the entire solution.

[0028] Furthermore, the aforementioned method for accurate annotation of defect regions in industrial quality inspection images using deep learning also includes: Based on the pixel connected components and grayscale difference threshold of the candidate overlapping defect regions, the defect coverage adjacency matrix is ​​obtained; Based on the defect coverage adjacency matrix and damping factor, the defect generation time series score vector is obtained; Based on the time-series score vector and the preset score sorting rules, the order in which the defects were generated is obtained.

[0029] Specifically, firstly, pixel connectivity analysis is performed on the images of candidate overlapping defect regions. By identifying regions with continuous pixel values ​​and similar features, independent candidate defect regions are initially divided, with each candidate region corresponding to a potential defect. Next, the grayscale distribution of each candidate defect region is calculated. By comparing the grayscale differences between different candidate defect regions, the coverage relationship between defects is determined. Specifically, if a portion of the pixel region of defect A covers the pixel region of defect B, and the grayscale value of the covered region is closer to that of defect A and significantly different from that of defect B, then defect A is determined to cover defect B. Finally, a defect coverage adjacency matrix is ​​constructed based on the above coverage relationships. The rows and columns of the matrix correspond to the divided defects. If the element in the i-th row and j-th column is 1, it indicates that defect i covers defect j; if the element is 0, it indicates that defect i does not cover defect j.

[0030] Next, a damping factor is determined. This factor is used to balance the impact of defect coverage relationship on the score with the impact of the initial uniform score, ensuring the rationality of the score calculation. The method for determining the damping factor is implemented by combining the defect characteristics, iterative stability and robustness tests of the industrial quality inspection scenario. For example, firstly, the value range is limited based on the defect statistical characteristics of the target scenario. Taking the quality inspection scenario of automotive steel plate welds as an example, 1000 sets of weld image samples containing overlapping defects are collected. Statistically, the effective defect coverage relationship, that is, the actual coverage caused by the order of defect generation, accounts for approximately 95%. In this embodiment, the order of defect generation refers to the physical time sequence in which different defects are actually formed during the weld welding process. Porosity defects are formed in the early stage of weld metal solidification and are considered first-generation defects, while crack defects are formed in the later stage of weld cooling and are considered second-generation defects. In the image, pixels with defects generated later will cover pixels with defects generated earlier. This coverage relationship provides a physical basis for the temporal tracing of defects. The proportion of misjudged coverage relationships caused by weld texture interference and image noise is about 5%. At the same time, the overlapping scene of industrial weld defects is mainly 2-3 defects. The number of defects is small and the complexity of the coverage relationship is low. Therefore, the initial value range of the damping factor is limited to 0.8-0.9. This range can suppress the influence of a small amount of misjudgment noise while retaining the weight of the true coverage relationship. Secondly, the optimal value was screened through multi-index testing. Several candidate values ​​were selected within the range of 0.8-0.9, such as 0.8, 0.82, 0.85, 0.88, and 0.9. These values ​​were tested on 100 sets of weld overlap defect samples with labeled actual generation time sequences. The test indicators included three aspects: first, the iteration convergence speed, recording the number of iterations required for the time sequence score vector to stabilize under each damping factor; second, the time sequence ranking accuracy, comparing the defect generation order obtained through iteration with the manually labeled actual order to calculate the accuracy; and third, noise robustness, artificially adding 5% false positive coverage relationships to the adjacency matrix and testing the decrease in time sequence ranking accuracy. The tests showed that when the damping factor was 0.85, convergence was achieved in only 12-18 iterations, with a time sequence ranking accuracy of 96%. The accuracy decrease after adding false positive noise did not exceed 2%, demonstrating the best overall performance. Therefore, this scheme determines the damping factor to be 0.85. Under this value, 85% of the weight during iterative updates comes from the coverage relationship of the defect coverage adjacency matrix, and 15% comes from the initial uniform score. This ensures that the actual coverage relationship dominates the time-series score, avoiding the weakening effect of the initial uniform score on the order of defect generation. Furthermore, the 15% initial score weight suppresses the scoring bias caused by a small number of misjudged coverage relationships, preventing the score from being overly biased towards a particular defect. Simultaneously, the iterative process is stable and converges quickly, adapting to the real-time processing requirements of industrial quality inspection. Then, based on the defect coverage adjacency matrix, a time-series score vector is initialized, with each element in the vector having the same initial value, representing the initial uniform probability of each defect generation time.Next, following the pre-defined calculation logic and considering the damping factor and the covering relationship in the adjacency matrix, the initial score vector is iteratively updated. For example, if defect i covers defect j, since the covered defect is usually generated earlier, the score of defect j will be appropriately increased during the iteration process. At the same time, the damping factor is used to prevent the score from being overly biased towards a particular defect. Through multiple iterations, the element values ​​in the score vector tend to stabilize and no longer change significantly, finally yielding the defect generation time-series score vector.

[0031] Next, the pre-defined scoring and sorting rules are established: the larger the element value in the time-series score vector, the earlier the corresponding defect was generated. Then, the elements in the time-series score vector are sorted in descending order of their numerical values, and the defect identifier corresponding to each element is recorded. For example, if the element value corresponding to defect B in the time-series score vector is the largest, followed by the element value corresponding to defect A, and the element value corresponding to defect C is the smallest, then the corresponding defect generation order is: defect B was generated first, followed by defect A, and finally defect C. Finally, this sorting result is output to clarify the generation order of each defect within the candidate overlapping defect region, providing a physical and logical basis for subsequent accurate segmentation.

[0032] This embodiment constructs a defect coverage adjacency matrix, quantifying the coverage relationships between defects into a computable matrix form, thus achieving a clear description of the relationships between overlapping defects. By calculating and sorting the temporal score vectors, the order in which defects are generated is clarified, enabling the subsequent segmentation process to follow the logic of "first generating defects and preserving their complete outlines, then generating defects and segmenting them based on coverage relationships," avoiding problems such as boundary confusion and outline distortion that may occur during segmentation. This step lays a solid logical foundation for the accurate segmentation of overlapping defects and effectively improves the accuracy of overlapping defect segmentation.

[0033] Furthermore, this embodiment provides a step for obtaining preliminary labeled defect categories and boundaries based on multimodal input data and a multimodal defect gene library, including: Based on the order in which defects are generated and the multimodal defect gene code, the physical constraint Laplace matrix is ​​obtained; Based on the Laplacian matrix and the matrix-based energy function, the segmentation result matrix is ​​obtained by ADMM iteration. Based on the segmentation result matrix and the attention-weighted matching rule, the preliminary labeled defect categories and boundaries are obtained.

[0034] Specifically, firstly, by combining the obtained defect generation sequence, the generation logic and extension trend of each defect are clarified. For example, the pore defects generated first are usually uniformly distributed in a circular shape, while the crack defects generated later extend linearly along the stress concentration direction of the weld. Then, the physical characteristics of various defects in the multimodal defect gene library are retrieved, such as the material stress concentration direction corresponding to cracks and the material density distribution characteristics corresponding to pores. Based on this information, a Laplacian matrix is ​​constructed. The element values ​​of the matrix are set according to the physical constraint characteristics of the defects. For example, for crack defects, the weight of adjacent pixels along the stress concentration direction is set to a higher value. For example, the weight of crack pixels and their adjacent pixels along the weld direction is set to 0.8, and the weight of adjacent pixels perpendicular to the weld direction is set to 0.1, thereby strengthening the constraint of crack extension along the stress direction and ensuring that the extension direction of cracks conforms to physical laws during the segmentation process. For pore defects, the weight of adjacent pixels in each direction is set to a uniform value. For example, the weight of pore pixels and their four adjacent pixels (up, down, left, and right) is set to 0.25 each, ensuring that the outline of the pores after segmentation is circular. This matrix transforms the physical formation law of defects into constraints in the segmentation process, resulting in the physical constraint Laplace matrix.

[0035] Next, a matrix-based energy function is constructed. This function comprehensively considers multiple factors such as pixel similarity, physical constraints, and sparsity. Pixel similarity ensures the consistency between the segmentation result and the image pixel features. Physical constraints ensure that the segmentation result conforms to the physical characteristics of the defect. Sparsity ensures the rationality of the defect region in the segmentation result. This function consists of three weighted terms. The weights of each term are preset according to the distribution characteristics of industrial weld defects and the segmentation requirements. The sum of the three weights is 1 to ensure that the constraints work synergistically. The first term is the pixel similarity term, with a weight of 0.6, which has the highest proportion. It is used to measure the degree of matching between the segmentation result and the pixel features of the original image, ensuring that the segmented defect region is consistent with the pixel distribution of the actual defect in the image. The second term is the physical constraint term, with a weight of 0.25, which has a moderate proportion. By introducing the constraints of the aforementioned physical constraint Laplacian matrix, it ensures that the segmentation result conforms to the physical formation law of the defect. The third term is the sparsity regularization term, with a weight of 0.15, which has a moderate proportion. It is used to constrain the sparsity of the segmentation result and avoid large-area missegmented background regions. Then, the ADMM iterative solution method is used to optimize the energy function, decomposing the complex energy function optimization problem into three simple sub-problems: pixel similarity optimization, physical constraint optimization, and sparsity optimization. These three sub-problems are solved iteratively, gradually adjusting the segmentation results. The rationality of the segmentation results is verified after each iteration until the segmentation results stabilize and no longer change significantly, ultimately yielding a segmentation result matrix. Each pixel in this matrix corresponds to a label value used to distinguish different defect regions from background regions. For example, firstly, the segmentation result matrix is ​​optimized. Under the premise of fixed physical constraints and sparsity regularization, the pixel values ​​of the segmentation matrix are adjusted to maximize their pixel similarity with the original image, initially obtaining the segmentation contour of the defect region. Secondly, the physical constraint auxiliary variable is optimized. Based on the constraints of the Laplacian matrix, the parts of the segmentation contour that do not conform to the physical characteristics of the defect are corrected. Finally, the dual variable is updated to balance the solution results of the first two sub-problems, ensuring that the overall energy function continues to decrease. When the change in the energy function between two consecutive iterations is less than a preset convergence threshold, such as 1×10⁻⁶, the segmentation function is optimized. -5 When the judgment result stabilizes, the iteration stops. For the segmentation scenario of weld defects in automotive steel plates, convergence is usually achieved after 20-40 iterations, resulting in the final segmentation matrix. Each pixel in this matrix corresponds to a label value, such as 1 representing pores, 2 representing cracks, and 0 representing background, used to distinguish different defect areas from background areas.

[0036] Based on the segmentation result matrix, features of each defect region are extracted, including visual features (such as contour shape and grayscale distribution) and pixel distribution features (such as region area and aspect ratio). Then, according to the attention-weighted matching rule, the extracted defect region features are matched with the defect gene codes in the multimodal defect gene library. During the matching process, core defect features (such as the linear extension features of cracks and the circular contour features of pores) are assigned higher weights, while secondary features (such as slight grayscale fluctuations) are assigned lower weights. For example, for crack defects, their linear extension features, such as contour aspect ratio and edge gradient changes, are key core features that distinguish cracks from other defects, and are assigned a weight of 0.7. Slight grayscale fluctuations within the crack region are easily affected by image noise and material reflection, and are considered secondary features, assigned a weight of 0.15. The remaining 0.15 weight is allocated to the auxiliary features of the crack, such as the region area. For pore defects, their circular outline features, such as outline curvature and roundness, are the key core features distinguishing pores from other defects, and are assigned a weight of 0.7. The minute grayscale differences inside the pore are considered secondary features, and are assigned a weight of 0.15. The remaining 0.15 weight is allocated to auxiliary features of the pore, such as the area of ​​the region. By calculating the weighted feature similarity, the defect gene code with the highest matching degree to the defect region features is selected. Finally, the defect category is determined based on the successfully matched defect gene code, and the defect boundary is delineated by combining the pixel distribution of the defect region in the segmentation result matrix. For example, for the defect region identified as pore in the segmentation result matrix, the set of defect pixels with a value of 1 is extracted. Statistically, the center coordinates of this pixel set are (320, 250), and the maximum distance from the outermost defect pixel to the center is 12 pixels. Therefore, a circular boundary is delineated with (320, 250) as the center and 12 pixels as the radius. For the defect region identified as crack, the set of defect pixels with a value of 2 is extracted. The starting coordinates of the pixel set are (200, 100), and the ending coordinates are (400, 300). Combining the distribution of the outermost pixels on both sides of the crack, the boundary points are delineated as (198, 101) and (398, 299) on the left and (202, 99) and (402, 301) on the right. Connecting these points forms a continuous linear boundary, ensuring that the boundary accurately delineates the defect outline, and the preliminary labeled defect category and boundary are obtained.

[0037] This embodiment incorporates the physical formation laws of industrial defects into the segmentation process by constructing a physical constraint Laplace matrix, thus avoiding the problem of segmentation results not matching actual defect characteristics. The ADMM iterative solution method is adopted to ensure the optimality of the segmentation results, effectively solving the segmentation problem of overlapping defects and enabling different defect regions to be clearly distinguished. The attention-weighted matching rule improves the accuracy of defect category matching and avoids misclassification caused by interference from secondary features.

[0038] Furthermore, this embodiment provides a step for obtaining the labeled defect region result based on the initially labeled defect categories and boundaries and the Bayesian iterative update rule, including: Based on the initially labeled defect categories and boundaries and the Bayesian iterative update rule, the calibration defect categories are obtained; Based on the calibration defect category and the segmentation result matrix, the labeled defect area results are obtained.

[0039] Specifically, the process begins by collecting auxiliary data for category calibration, including detailed records of welding process parameters for the production batch, historical amendments to the labeling of similar defects, and industry standards for judging defects in automotive steel sheet welds. Then, for the initially labeled defect categories, a rationality analysis is conducted based on the auxiliary data. For example, if a defect is initially labeled as a crack, but the auxiliary data shows that the probability of a crack occurring under the given welding process parameters is extremely low, and the characteristics of this defect are more similar to those of porosity, then the category calibration process is initiated. Following a Bayesian iterative update rule, the probability of the initially labeled category is combined with valid information from the auxiliary data to continuously iterate and correct the probability judgment of the defect category, gradually reducing the probability of misclassification and increasing the probability of correct classification. Finally, based on the iteratively updated probability results, the final calibrated defect category is determined.

[0040] Based on the calibrated defect category, boundary feature references for that type of defect are retrieved from the multimodal defect gene library, including the shape characteristics and pixel distribution patterns of the boundary. Then, combined with the segmentation result matrix, the initially defined defect boundary is fine-tuned. For example, if the calibrated defect category is stomata, the gene library shows that the boundary of stomata is usually a smooth circular outline, while the initially defined boundary has local irregular protrusions. Therefore, based on the boundary feature references in the gene library, these protrusions are corrected to make the boundary conform to the typical characteristics of stomata. Simultaneously, the actual pixel distribution of the defect in the image is considered to ensure that the boundary accurately encompasses the defect region, without omitting defect pixels or including excessive background pixels. Finally, the calibrated defect category and the fine-tuned boundary range are integrated to obtain the labeled defect region result.

[0041] This embodiment uses auxiliary data and Bayesian iterative update rules for category calibration, effectively eliminating possible category misjudgments in the initial annotation, making the defect category determination more in line with industrial production realities and industry standards; by combining the boundary feature reference in the defect gene library with the actual pixel distribution of the image, the defect boundary is fine-tuned, improving the accuracy of the boundary.

[0042] Furthermore, this embodiment provides a step for obtaining a calibrated defect category based on the initially labeled defect category and boundary and a Bayesian iterative update rule, including: Based on the initially labeled defect categories and corresponding production process parameters, a causal probability matrix is ​​obtained; Based on the causal probability matrix and the Bayesian iterative update rule, the posterior probability distribution of the defect category is obtained; The calibration defect category is obtained based on the posterior probability distribution and the maximum probability determination rule.

[0043] Specifically, the key factors influencing the categories of weld defects in automotive steel sheets are first identified. These mainly include production process parameters such as welding current, welding speed, and welding voltage, as well as image features such as defect outline shape, grayscale distribution, and area size. Then, a large amount of historical data is collected, including defect annotation results under different combinations of process parameters and image feature data of various defects. Based on this historical data, the frequency of each defect type under different values ​​of each influencing factor is statistically analyzed, and this frequency is used as a probability value to construct a causal probability matrix. For example, a row in the matrix corresponds to the factor of "high welding current," and a column corresponds to the defect category of "crack." The element value of that row and column represents the probability of crack occurrence when the welding current is high. This matrix clearly presents the probabilistic relationship between influencing factors and defect categories.

[0044] Then, based on the initially labeled defect categories, the corresponding initial probability values ​​are extracted from the causal probability matrix as the initial values ​​for the posterior probability distribution, i.e., initializing the prior probability. This probability is determined based on a combination of historical data statistical patterns and the current initial labeling results. Next, new auxiliary data is introduced, including the specific process parameter values ​​for the image to be labeled and detailed feature data of the defects in the image. Following the Bayesian iterative update rule, the initial probability values ​​are combined with the newly introduced auxiliary data to correct and update the probability values ​​for various defect types, calculating the updated probability values. For example, if the initial probability for the crack category is 0.6, and the newly introduced process parameters show normal welding current, and the probability of crack occurrence when the welding current is normal in the causal probability matrix is ​​0.1, then the probability of the crack category is reduced through iterative updates. Through multiple iterations, each iteration is based on the probability results updated in the previous round, introducing new auxiliary data, such as defect texture features, shape features, equipment operating parameters, etc., to repeatedly perform probability correction operations until the iteration termination conditions are met: including the difference between the probability results of two adjacent iterations not exceeding the industrial quality inspection preset threshold, such as 0.001, the probability of a certain defect category reaching the preset confidence level, such as 0.95, or the number of iterations reaching the preset upper limit, such as 10 times. Valid information from the auxiliary data is continuously absorbed until the probability distribution tends to stabilize, and the posterior probability distribution of the defect category is obtained.

[0045] Next, the probability values ​​of each category in the posterior probability distribution are sorted to clarify the probability of each category. Then, according to the maximum probability determination rule, the category with the highest probability value is selected as the final calibration defect category. For example, if the probability of the porosity category is 0.8, the probability of the crack category is 0.15, and the probability of the slag inclusion category is 0.05 in the posterior probability distribution, then the calibration defect category is determined to be porosity. At the same time, the rationality of the determination result is verified, combining industry standards for automotive steel plate weld defects with historical annotation cases to ensure that the calibrated category conforms to the actual situation. If the verification finds that the category with the highest probability is unreasonable, auxiliary data is retrieved again for iterative updates until a reasonable calibration defect category is obtained.

[0046] This embodiment quantifies the relationship between defect categories and influencing factors by constructing a causal probability matrix, providing a scientific quantitative basis for category calibration. By adopting a Bayesian iterative update rule, it can fully absorb new auxiliary data, dynamically correct the probability of defect categories, and ensure the accuracy of the probability distribution. Based on the maximum probability judgment rule, the final calibrated defect category is clarified, avoiding errors caused by subjective judgment, solving the problem of possible category misjudgment in the initial labeling, and greatly improving the reliability of defect category determination.

[0047] Furthermore, this embodiment provides a step for obtaining the labeled defect region result based on the calibration defect category and the segmentation result matrix, including: Based on the calibration defect category and the segmentation result matrix, the integer pixel coordinates of the defect boundary are obtained; Sub-pixel level defect boundary coordinates are obtained based on integer pixel coordinates and bilinear interpolation algorithm; Based on the sub-pixel level defect boundary coordinates and deviation threshold verification rules, the labeled defect area results are obtained.

[0048] Specifically, firstly, based on the calibrated defect category, the boundary features of that type of defect are defined. For example, the boundary of a pore is circular, while the boundary of a crack is a linearly extending irregular curve. Then, combined with the segmentation result matrix, the boundary pixels between the defect region and the background region are identified. These boundary pixels constitute the integer pixel set of the defect boundary. For regularly shaped defects, such as pores, the integer pixel coordinates of the circular boundary are obtained by fitting the distribution of the boundary pixels; for irregularly shaped defects, such as cracks, the integer coordinates of the boundary pixels are directly recorded to form the integer pixel coordinate set of the defect boundary, thus initially delineating the approximate range of the defect boundary.

[0049] For boundary pixels corresponding to integer pixel coordinates, the grayscale distribution and feature information of their neighboring pixels are analyzed. For example, for a certain integer pixel coordinate point on the crack boundary, the grayscale values ​​of pixels in the surrounding 3×3 or 5×5 neighborhood are extracted, and the gradient changes of the grayscale values ​​are analyzed. Then, a bilinear interpolation algorithm is used to refine and calibrate the integer pixel coordinates based on the grayscale gradient information of the neighboring pixels. Specifically, by calculating the positions of the extreme points of grayscale value changes, the precise position of the defect boundary within integer pixels is determined, and the integer pixel coordinates are corrected to sub-pixel level coordinates including the decimal part. For example, if a certain integer pixel coordinate is (x, y), and interpolation calculation shows that the actual position of the boundary is 0.3 pixels to the right and below this pixel, then the corrected sub-pixel coordinates are (x+0.3, y+0.3). By calibrating all integer boundary pixels one by one, the complete sub-pixel level defect boundary coordinates are obtained.

[0050] A deviation threshold verification rule is set, which is determined based on the accuracy requirements of industrial quality inspection and is used to judge the accuracy of sub-pixel boundary coordinates. In this embodiment, the deviation threshold is subdivided into two categories: distance deviation threshold and angle deviation threshold, corresponding to the two dimensions of positional consistency and contour smoothness of boundary verification, respectively. For example, the deviation threshold is set to 0.2 pixels and the angle deviation threshold is 5°. Then, the sub-pixel boundary coordinates are verified by calculating the distance and angle changes between adjacent boundary coordinate points. If the distance deviation of a coordinate point is greater than the distance deviation threshold or the angle deviation is greater than the angle deviation threshold, interpolation calibration is performed again to ensure the consistency and accuracy of all boundary coordinate points. Finally, based on the verified sub-pixel boundary coordinates and the calibrated defect category, a complete defect area annotation result is constructed. This result contains both a clear defect category and a boundary range with sub-pixel accuracy, which can accurately reflect the actual size and location of the defect.

[0051] This embodiment lays the foundation for boundary calibration by extracting integer pixel coordinates; it uses a bilinear interpolation algorithm to obtain sub-pixel level coordinates, overcoming the accuracy limitations of traditional pixel-level annotation and making the boundary positioning more closely match the actual contour of the defect; through deviation threshold verification rules, it ensures the accuracy and consistency of sub-pixel level coordinates. This step significantly improves the accuracy of defect boundary annotation, enabling it to more accurately reflect the actual size and location of defects, providing a reliable basis for the quantitative assessment of defects in industrial quality inspection, such as the accurate calculation of defect length and area.

[0052] Furthermore, the aforementioned method for accurate annotation of defect regions in industrial quality inspection images using deep learning also includes: Based on the labeled defect region results and the multimodal defect gene library, the updated gene library features are obtained; Based on the updated gene pool features and cross-scene feature matrix, the shared feature basis of NMF decomposition is obtained; Based on shared feature bases and self-supervised transfer rules, model parameters labeled across scenes are obtained.

[0053] Specifically, the process begins by collecting the final annotated defect regions obtained during this annotation process, including defect category information, boundary features, corresponding process parameters, and physical characteristics. Then, these new defect features are compared with existing features in the multimodal defect gene library. If new defect features are found, such as specific process-related features for a certain type of defect or supplementary information to existing features, they are added to the gene library. If discrepancies are found between some features in the gene library and the new annotation results, the existing features are corrected. For example, if the process feature description of a certain type of crack in the gene library is not comprehensive enough, and the new annotation results supplement the welding voltage fluctuation range corresponding to this type of crack, then the process feature of this defect in the gene library is updated. Finally, the newly added and corrected features are integrated to obtain the updated gene library features, enabling continuous iterative optimization of the gene library.

[0054] Next, defect feature data from source scenarios, such as automotive steel plate welds, and target scenarios, such as aerospace aluminum alloy welds, are collected to construct a cross-scenario feature matrix. This matrix contains multimodal features of various defects in both scenarios, including scenario-specific features, such as the material differences between automotive steel plates and aerospace aluminum alloys, as well as common features, such as the linear extension of cracks and the circular outline of pores. Then, the NMF decomposition algorithm is used to decompose the cross-scene feature matrix into a basis matrix and a coefficient matrix. First, the structure of the cross-scene feature matrix is ​​defined: each row corresponds to a multimodal feature vector of a defect sample. The first half of the rows corresponds to defect samples from the source scene, and the second half corresponds to defect samples from the target scene. Next, a shared feature basis is set, i.e., the dimension of the basis matrix, which is determined by the number of common features of the defects in the two scenarios. Then, the basis matrix and coefficient matrix are initialized, where the number of columns in the basis matrix equals the dimension of the shared feature basis, and the number of rows in the coefficient matrix corresponds one-to-one with the number of rows in the cross-scene feature matrix. The first half of the rows corresponds to the specific feature coefficients of the source scene samples, and the second half corresponds to the specific feature coefficients of the target scene samples. Then, according to the NMF iterative update rule, the basis matrix and coefficient matrix are alternately optimized. During the iteration process, the coefficient values ​​of the source and target scenes in the coefficient matrix are continuously adjusted so that the basis matrix can simultaneously fit the common features of the two scenarios, while the scenario-specific coefficients in the coefficient matrix reflect the feature differences between the two scenarios. The iteration terminates when the matrix error between two iterations is less than a preset threshold, such as 1×10⁻⁶. -4After decomposition, the basis matrix is ​​the shared feature basis, containing common features of defects such as cracks and porosity that are unaffected by material and process. The coefficient matrix consists of feature coefficients specific to the two scenarios. The first half of the coefficients corresponds to the feature weights specific to the source scenario, reflecting the influence of automotive steel material and welding process on defect features. The second half of the coefficients corresponds to the feature weights specific to the target scenario, reflecting the influence of aerospace aluminum alloy material and welding process on defect features. Through decomposition, common features of defects in the two scenarios are extracted. These common features are unaffected by differences in material and process details and can serve as the core basis for cross-scenario transfer, resulting in the shared feature basis of NMF decomposition.

[0055] Next, based on the shared feature base, a feature mapping relationship between the source and target scenes is established, clarifying the correspondence between the source scene's labeled model parameters and the shared feature base. Then, following the self-supervised transfer rule, the labeled model parameters of the source scene are mapped to the target scene through the shared feature base. During the mapping process, the model parameters are adaptively adjusted by incorporating the target scene's specific feature coefficients to adapt them to the defect features of the target scene. For example, the model parameters used to identify cracks in the source scene are adjusted by incorporating the material feature coefficients of aerospace aluminum alloy after mapping through the shared feature base, enabling them to accurately identify crack defects in aerospace aluminum alloy welds. Finally, labeled model parameters adapted to the target scene are obtained. These parameters can be obtained without a large number of labeled target scene samples, achieving rapid construction of cross-scene labeled models.

[0056] This embodiment updates the gene pool features, enabling the multimodal defect gene pool to continuously accumulate annotation experience and improve its support capabilities. By extracting shared feature bases through NMF decomposition, it effectively extracts common defect features across different scenarios, laying the foundation for cross-scenario transfer. Based on self-supervised transfer rules, it obtains cross-scenario annotation model parameters, avoiding the tedious work of annotating a large number of samples in new scenarios and reducing the cost and cycle of annotation in new scenarios. This step endows the annotation model with self-evolution and cross-scenario adaptability, allowing the solution to flexibly meet the needs of different industrial quality inspection scenarios, significantly improving the application scope and practical value of the solution.

[0057] Furthermore, this embodiment provides a step for obtaining cross-scene labeled model parameters based on shared feature bases and self-supervised transfer rules, including: Based on the shared feature base and the feature matrix of the target scene, the cross-scene feature mapping relationship is obtained; The updated model parameters are obtained based on the cross-scene feature mapping relationship and the incremental learning rate; Based on the updated model parameters and the newly added annotation data, the updated annotation model is obtained.

[0058] Specifically, first, the shared feature basis obtained from NMF decomposition, as well as the specific feature coefficients of the source and target scenes, are retrieved. Then, the correspondence between the source scene features and the shared feature basis is analyzed to clarify the expression form of various defect features in the source scene within the shared feature basis; simultaneously, the correspondence between the target scene features and the shared feature basis is analyzed to clarify the expression form of various defect features in the target scene within the shared feature basis. For example, assuming the shared feature basis obtained from NMF decomposition is 32-dimensional, where dimensions 1-8 correspond to the common features of crack-type defects, such as linear extension direction, edge gradient change, and aspect ratio distribution; dimensions 9-16 correspond to the common features of porosity-type defects, such as roundness, contour curvature, and uniformity of gray-level distribution; and the remaining dimensions correspond to the common features of other defect types. Taking a crack defect in the target scene as an example, its linear extension feature is expressed in the first three dimensions of the shared feature basis. The first dimension corresponds to the crack extension direction. In this dimension, the expression of the target scene and the source scene are consistent, both along the direction of weld stress concentration. The second dimension corresponds to the edge gradient. Because aerospace aluminum alloy has better thermal conductivity and a narrower weld heat-affected zone, the gray-scale change at the crack edge is steeper. Therefore, the expression of the target scene in this dimension is a linear distribution with a higher gradient value, which needs to be adjusted in conjunction with the specific feature coefficients of the target scene. The third dimension corresponds to the aspect ratio. The expression in this dimension is not significantly different from that of the source scene. Taking a porosity defect in the target scene as another example, its circular contour feature is expressed in the 9th to 11th dimensions of the shared feature basis. The 9th dimension corresponds to... Circularity is crucial because the pores in aerospace aluminum alloy welds are mostly small, highly circular micropores. In this dimension, the target scene's representation has a circularity coefficient closer to 1.0, which differs from the representation of pores in automotive steel welds in the source scene. Therefore, it needs to be adapted using the target scene's specific feature coefficients. The 10th dimension corresponds to contour curvature, and the 11th dimension corresponds to edge smoothness. These two dimensions are not significantly different from the source scene. Based on these two correspondences, mapping rules between source and target scene features are established. For example, the linear extension feature of automotive steel cracks in the source scene, after being mapped to the target scene through a shared feature base, corresponds to the linear extension feature of aerospace aluminum alloy cracks. This is further adjusted by incorporating the target scene's material feature coefficients. Finally, these mapping rules are integrated to obtain a cross-scene feature mapping relationship, which can guide the transfer of source scene annotation experience to the target scene.

[0059] Next, an incremental learning rate is set, determined based on the degree of difference between the target scene and the source scene. If the two scenes differ significantly, a smaller learning rate, such as 0.001, is set to ensure the model parameters adapt slowly; if the difference is small, the learning rate can be increased appropriately. Then, based on the cross-scene feature mapping relationship, the labeled model parameters of the source scene are used as initial parameters, and a small number of labeled samples from the target scene are input. A large number of samples are not required; only the adaptation effect needs to be verified. Through incremental learning, the model parameters are gradually fine-tuned using the feature information of the target scene samples, according to the set incremental learning rate. For example, if the weight of a certain feature in the source scene model is 0.8, and verification using target scene samples reveals that this feature has low importance in the target scene, its weight is gradually adjusted to 0.5 according to the incremental learning rate. Through multiple iterations of fine-tuning, updated model parameters adapted to the target scene are obtained.

[0060] Next, newly labeled data for the target scene is collected. This data can be accurate data verified manually after initial model labeling, or newly acquired defect image labeling data for the target scene. Then, the updated model parameters are input into the labeling model, and the model is trained using the newly labeled data to further optimize the model parameters, enabling it to more accurately capture the defect features of the target scene. During training, the model's labeling accuracy is continuously monitored. If the accuracy reaches a preset threshold, training stops; if the accuracy does not meet the threshold, the incremental learning rate and model parameters are adjusted until the model's labeling accuracy meets the quality inspection requirements of the target scene. Finally, the updated labeling model is obtained, which can be directly applied to the labeling of industrial quality inspection images in the target scene, achieving accurate and efficient labeling.

[0061] This embodiment fine-tunes model parameters by setting an incremental learning rate, ensuring the stability and rationality of model parameter adjustments and avoiding accuracy fluctuations during model adaptation. It also optimizes the model by combining newly added labeled data, enabling the model to continuously absorb new scene feature information and achieve dynamic evolution. This step allows the labeled model to quickly adapt to new industrial quality inspection scenarios, significantly reducing the construction cost and cycle of new scene labeled models while ensuring the accuracy of new scene labeling.

[0062] This application provides a deep learning-based system for accurate annotation of defect regions in industrial quality inspection images, including: The data acquisition module is used to obtain multimodal input data based on the industrial quality inspection images to be labeled and the corresponding production process parameters; The first annotation module is used to obtain the preliminary annotation of defect categories and boundaries based on multimodal input data and multimodal defect gene library; The second annotation module is used to obtain the annotated defect region results based on the initially annotated defect categories and boundaries and the Bayesian iterative update rules.

[0063] This application's embodiments effectively improve the accuracy of defect category matching by constructing a multimodal defect gene library based on multi-source data and combining iterative matching with attention-weighted matrices. It traces the defect generation sequence using adjacency matrices and the PageRank iterative algorithm, and solves the matrix-based energy function using ADMM iterative solution to resolve the problem of overlapping defect boundary confusion in complex backgrounds, achieving accurate segmentation. It uses Bayesian matrix iterative inference to calibrate categories and bilinear interpolation for sub-pixel calibration, reducing mislabeling and omissions and improving boundary positioning accuracy. It achieves cross-scene feature transfer through NMF iterative decomposition, and reduces the cost of labeling new scene samples by combining dynamic model iterative optimization, constructing a closed-loop self-evolving labeling system. Ultimately, it achieves full-dimensional accurate labeling of defect regions, balancing labeling accuracy and efficiency, adapting to the real-time quality inspection needs of industrial production lines, improving the automation and intelligence level of industrial quality inspection, forming a high technical barrier, and adapting to the industrial quality inspection needs of multiple materials and scenarios.

[0064] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An industrial quality inspection image deep learning defect region precise labeling method, characterized in that, include: Based on the industrial quality inspection images to be labeled and the corresponding production process parameters, multimodal input data is obtained; Based on the multimodal input data and the multimodal defect gene library, the preliminary labeled defect categories and boundaries are obtained; Based on the initially labeled defect categories and boundaries and the Bayesian iterative update rule, the labeled defect regions are obtained. The process of obtaining multimodal input data based on the industrial quality inspection image to be labeled and the corresponding production process parameters includes: Based on the industrial quality inspection image to be labeled and the preset texture template, a background mask matrix is ​​obtained; The coordinates of the candidate overlapping defect regions are obtained based on the background mask matrix and the defect density threshold. The multimodal input data is obtained based on the coordinates of the candidate overlapping defect regions and the corresponding production process parameters. Also includes: Based on the pixel connected components and grayscale difference threshold of the candidate overlapping defect regions, the defect coverage adjacency matrix is ​​obtained; Based on the defect coverage adjacency matrix and damping factor, the defect generation time series score vector is obtained; Based on the time-series score vector and the preset score sorting rules, the order in which defects are generated is obtained.

2. The method for accurate annotation of defect regions in industrial quality inspection images using deep learning, as described in claim 1, is characterized in that... Based on the multimodal input data and the multimodal defect gene library, preliminary labeled defect categories and boundaries are obtained, including: Based on the order in which the defects were generated and the multimodal defect gene code, the physical constraint Laplace matrix is ​​obtained; Based on the Laplacian matrix and the matrix-based energy function, the segmentation result matrix is ​​obtained by ADMM iteration. Based on the segmentation result matrix and the attention-weighted matching rule, the preliminary labeled defect categories and boundaries are obtained.

3. The method for accurate annotation of defect regions in industrial quality inspection images using deep learning, as described in claim 2, is characterized in that... The step of obtaining the labeled defect region results based on the initially labeled defect categories and boundaries and the Bayesian iterative update rule includes: Based on the initially labeled defect categories and boundaries and the Bayesian iterative update rule, the calibration defect categories are obtained; Based on the calibration defect category and the segmentation result matrix, the labeled defect area results are obtained.

4. The method for accurate annotation of defect regions in industrial quality inspection images using deep learning according to claim 3, characterized in that, The process of obtaining the calibrated defect category based on the initially labeled defect category and boundary and the Bayesian iterative update rule includes: Based on the initially labeled defect categories and corresponding production process parameters, a causal probability matrix is ​​obtained; Based on the causal probability matrix and the Bayesian iterative update rule, the posterior probability distribution of the defect category is obtained; The calibration defect category is obtained based on the posterior probability distribution and the maximum probability determination rule.

5. The method for accurate annotation of defect regions in industrial quality inspection images using deep learning, as described in claim 3, is characterized in that... The step of obtaining the labeled defect region results based on the calibration defect category and the segmentation result matrix includes: Based on the calibration defect category and the segmentation result matrix, the integer pixel coordinates of the defect boundary are obtained; Based on the integer pixel coordinates and the bilinear interpolation algorithm, sub-pixel level defect boundary coordinates are obtained; Based on the sub-pixel level defect boundary coordinates and deviation threshold verification rules, the labeled defect area results are obtained.

6. The method for accurate annotation of defect regions in industrial quality inspection images using deep learning according to claim 1, characterized in that, Also includes: Based on the labeled defect region results and the multimodal defect gene library, the updated gene library features are obtained; Based on the updated gene pool features and cross-scene feature matrix, the shared feature base of NMF decomposition is obtained; Based on the shared feature base and self-supervised transfer rules, the model parameters labeled across scenes are obtained.

7. The method for accurate annotation of defect regions in industrial quality inspection images using deep learning according to claim 6, characterized in that, The process of obtaining cross-scene labeled model parameters based on the shared feature base and self-supervised transfer rules includes: Based on the shared feature base and the feature matrix of the target scene, the cross-scene feature mapping relationship is obtained; Based on the cross-scene feature mapping relationship and the incremental learning rate, the updated model parameters are obtained; Based on the updated model parameters and the newly added annotation data, the updated annotation model is obtained.

8. A deep learning-based system for accurately annotating defect regions in industrial quality inspection images, used to implement the deep learning-based method for accurately annotating defect regions in industrial quality inspection images as described in any one of claims 1-7, characterized in that, include: The data acquisition module is used to obtain multimodal input data based on the industrial quality inspection images to be labeled and the corresponding production process parameters; The first annotation module is used to obtain the preliminary annotation of defect categories and boundaries based on the multimodal input data and the multimodal defect gene library; The second annotation module is used to obtain the annotated defect region results based on the defect categories and boundaries of the initial annotation and the Bayesian iterative update rule.

Images