An automobile key casting part defect automatic identification method and system

Abstract

The present application relates to the technical field of image analysis, in particular to a kind of automobile key casting parts defect automatic identification method and system.To avoid misjudgment or missed detection when NMS screening, the specific distribution of crack in the bounding box is analyzed, and in view of the extension characteristics of the crack, the crack prominence value is determined by comparing the position distribution and gray level change of edge pixel points, and the completeness value and missing degree value of the bounding box are quantified according to the integrity and missing degree of the edge line, combined with the crack prominence value, further, the relationship between these values and confidence is analyzed, the possibility of multiple crack defects existing in each bounding box is evaluated, and the adjusted weight is obtained accordingly, finally, in the NMS screening process, the preset initial threshold value is adaptively adjusted according to the adjusted weight to obtain the adjusted threshold value of each bounding box, so that the bounding box is more accurately screened, and the final crack bounding box is obtained.This method effectively reduces the probability of misjudgment or missed detection.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, specifically to an automatic identification method and system for defects in key cast automotive components. Background Technology

[0002] Key cast automotive components mainly include engine blocks, cylinder heads, crankshafts, gearbox housings, steering knuckles, wheel hubs, and suspension supports. These components are typically made of materials such as aluminum alloys, cast iron, or cast steel, and require high strength and precision. They are core components of vehicle powertrain, suspension, and braking systems. Cracks in these components can lead to reduced strength, shortened lifespan, and even serious safety accidents. Therefore, defect identification is necessary to ensure consistent and stable product quality.

[0003] For crack defects on the surface of key cast automotive components, a pre-trained neural network is typically used to identify them. Then, post-processing NMS (Non-maximum suppression) is used to compare the confidence and overlap of bounding boxes to remove overlapping ones. However, crack defects within the bounding boxes can exhibit various distribution characteristics, such as single or multiple cracks, and the cracks can be complete or incomplete. Existing NMS systems usually use a fixed threshold to filter bounding boxes, but this fixed threshold cannot adapt to the different distribution characteristics of cracks within the bounding boxes. This can lead to the incorrect rejection of bounding boxes containing complete crack defects, resulting in missed detections or misjudgments. Summary of the Invention

[0004] To address the technical problem that existing Non-Mechanical Systems (NMS) typically use a fixed threshold to filter bounding boxes, which cannot adapt to the different distribution characteristics of cracks within the bounding boxes, leading to the erroneous rejection of bounding boxes with complete crack defects and resulting in missed detections or misjudgments, the present invention aims to provide an automatic defect identification method and system for key automotive casting components. The specific technical solution adopted is as follows:

[0005] An automatic defect identification method for key cast automotive components includes:

[0006] Acquire component images, and obtain all bounding boxes in the component images and their corresponding confidence scores based on a pre-trained neural network;

[0007] Obtain the edge lines in the component image; within each bounding box, compare the positional distribution features and grayscale changes of the edge pixels between the edge lines to determine the crack protrusion value of each edge line;

[0008] Based on the crack protrusion value of the edge line of each bounding box, and combined with the distribution characteristics of the number of edge pixels on the edge line, the integrity value and missing value of each bounding box are determined respectively; by comprehensively analyzing the integrity value, missing value, and the correlation between integrity value and confidence, the adjustment weight of each bounding box is determined.

[0009] When using NMS and the confidence level of the bounding box to filter the bounding box, the preset initial threshold is adjusted based on the adjustment weight of the bounding box, and the adjusted threshold is used for filtering to obtain the crack bounding box.

[0010] Furthermore, the method for obtaining the crack protrusion degree value includes:

[0011] Within each bounding box, all endpoints of each edge line are used as target points;

[0012] Each edge line is segmented to obtain multiple line segments, and the average gray level gradient of all edge pixels on each line segment is used as the gray level change factor.

[0013] On each edge line, analyze the differences in grayscale variation factors between line segments to determine the degree of crack manifestation on each edge line;

[0014] Calculate the difference in crack manifestation value between each edge line and each of the remaining edge lines, and use it as a highlighting factor;

[0015] The sum of all protrusion factors corresponding to each edge line is normalized and used as the crack protrusion value for each edge line.

[0016] Furthermore, the process of segmenting each edge line to obtain multiple line segments includes:

[0017] On each edge line, the edge pixels are clustered based on the DBSCAN clustering algorithm to obtain multiple clusters, where the neighborhood radius and the minimum number of points are preset values.

[0018] The edge pixels in each cluster are grouped into a line segment.

[0019] Furthermore, the method for obtaining the crack manifestation degree value includes:

[0020] On each edge line, for any line segment, the line segments that the line segment passes through to each target point are taken as comparison segments. The difference between the gray scale change factor of the line segment and each comparison segment is calculated as the gray scale change difference. The mean of the gray scale change differences between the line segment and all comparison segments is taken as the crack trend factor.

[0021] The maximum value of all crack tendency factors corresponding to the line segment is taken as the crack tendency degree value of the line segment.

[0022] On each edge line, the crack trend degree value is weighted and fused using the gray-scale change factor of the line segment to obtain the crack performance degree value of each edge line.

[0023] Furthermore, the determination of the integrity and missingness values ​​of each bounding box based on the crack protrusion value of the edge line of each bounding box, combined with the distribution characteristics of the number of edge pixels on the edge line, includes:

[0024] In the component image, the number of edge pixels on each edge line is taken as the total number.

[0025] Within each bounding box, the ratio of the number of edge pixels within the bounding box to the total number of corresponding edge pixels is used as the integrity coefficient for each edge line.

[0026] In each bounding box, the normalized product of the integrity coefficient of each edge line and the crack protrusion value is used as the integrity factor of each edge line, and the maximum integrity factor of all edge lines in the bounding box is used as the integrity value of the bounding box.

[0027] In each bounding box, the integrity coefficient of each edge line is negatively correlated and normalized, and then multiplied by the crack protrusion value. The normalized value of the resulting product is used as the missing factor for each edge line. The maximum missing factor of all edge lines in the bounding box is used as the missing value of the bounding box.

[0028] Furthermore, the method for obtaining the adjustment weights includes:

[0029] In each bounding box, the edge line corresponding to the largest missing factor is used as the target edge line;

[0030] Group the bounding boxes whose target edge lines belong to the same edge lines in the component images into the same type of bounding boxes;

[0031] In each type of bounding box, a reference weight is determined based on the completeness value and the missingness value of each bounding box. The completeness value and the reference weight are positively correlated, and the missingness value and the reference weight are negatively correlated.

[0032] Choose any bounding box as the target bounding box, and use the remaining bounding boxes of the same type as the target bounding box as reference bounding boxes;

[0033] The difference in confidence between the target bounding box and each reference bounding box is used as the confidence bias factor, the difference in completeness value between the target bounding box and each reference bounding box is used as the completeness bias factor, and the product of the confidence bias factor and the completeness bias factor is used as the positive correlation factor.

[0034] The positive correlation factor is weighted, fused, and normalized using the reference weights of all reference bounding boxes corresponding to the target bounding box to obtain the positive correlation coefficient. The value of the positive correlation coefficient after negative correlation mapping is used as the adjustment weight of the target bounding box.

[0035] Furthermore, the method for obtaining the adjustment threshold includes:

[0036] The adjustment magnitude is obtained by multiplying the value of the preset initial threshold after negative correlation mapping with the adjustment weight of each bounding box;

[0037] The sum of the preset initial threshold and the corresponding adjustment range for each bounding box is used as the adjustment threshold for each bounding box.

[0038] Furthermore, the preset initial threshold is 0.5.

[0039] Furthermore, the method for obtaining the edge line includes:

[0040] The component image is processed using the Canny operator to obtain all edge lines.

[0041] An automatic defect identification system for key cast automotive components includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. When the processor loads and executes the at least one instruction, at least one program, code set, or instruction set, it implements the steps of the automatic defect identification method for key cast automotive components.

[0042] The present invention has the following beneficial effects:

[0043] The process involves acquiring component images and using a pre-trained neural network to obtain bounding boxes and their corresponding confidence scores. To avoid misjudgments or missed detections when using NMS to filter bounding boxes due to the potential for different crack distributions, it's necessary to determine the specific distribution of crack defects within each bounding box. Given that cracks exhibit a certain extension characteristic—that is, a clear gradient in grayscale changes—the distribution characteristics and grayscale changes of edge pixels within each bounding box are compared to determine the crack prominence value of the edge line, reflecting the degree to which the edge line represents a crack. The confidence score of the bounding box is usually related to its localization accuracy; therefore, the integrity of the edge lines representing cracks within the bounding box can be further assessed to quantify the integrity and missing value of the bounding box. Specifically, the distribution characteristics of the number of edge pixels can be analyzed and combined with the crack prominence value of the edge line for quantification. Then, based on the integrity and missing value of the bounding box, as well as the correlation between the integrity value and the confidence score, the probability of multiple crack defects existing in each bounding box is analyzed, resulting in an adjustment weight for each bounding box. Finally, in order to reduce the possibility that bounding boxes with multiple crack defects are simply judged as a single incomplete large defect and thus screened out, the preset initial threshold is adjusted according to the adjustment weight of each bounding box when using NMS to screen the bounding boxes. This results in an adaptive adjustment threshold for each bounding box, thereby enabling more accurate screening of the bounding boxes and obtaining the final crack bounding boxes, effectively reducing the probability of false positives or false negatives. Attached Figure Description

[0044] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A flowchart illustrating an automatic defect identification method for key cast automotive components, provided in one embodiment of the present invention.

[0046] Figure 2 This is a schematic diagram showing the result of a component image being recognized by a neural network according to an embodiment of the present invention.

[0047] Figure 3 A flowchart illustrating a method for obtaining a crack protrusion degree value according to an embodiment of the present invention;

[0048] Figure 4This is a system block diagram of an automatic defect identification system for key cast automotive components provided in one embodiment of the present invention;

[0049] Figure 5 This is a schematic diagram of the system structure of an automatic defect identification system for key cast automotive components provided in an embodiment of the present invention. Detailed Implementation

[0050] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an automatic defect identification method and system for key cast automotive components proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0052] The following description, in conjunction with the accompanying drawings, details the specific scheme of the automatic identification method and system for defects in key cast automotive components provided by this invention.

[0053] Please see Figure 1 The diagram illustrates a method flowchart for automatic defect identification of key automotive casting components according to an embodiment of the present invention. The method includes the following steps:

[0054] Step S1: Obtain the component image and obtain all bounding boxes in the component image and the confidence scores corresponding to the bounding boxes based on the pre-trained neural network.

[0055] In the automotive manufacturing industry, the surface quality of critical cast components is paramount, especially crack defects, which directly affect the strength and safety of these components. Currently, deep learning algorithms, such as YOLOv5, are widely used in the automatic identification of defects in critical cast automotive components due to their efficiency and accuracy. YOLOv5 demonstrates significant advantages in identifying crack defects through its powerful feature extraction capabilities and real-time detection speed.

[0056] However, the surfaces of cast parts are often complex and varied, including various textures, shadows, and reflections, which increases the difficulty of defect identification. When processing such complex surfaces, YOLOv5 may incorrectly identify the same defect as multiple independent defects, or mistakenly identify multiple adjacent defects as a single large defect. Therefore, to optimize the identification results of YOLOv5, post-processing NMS (Non-maximum suppression) technology is currently mainly used. NMS compares the confidence and overlap of bounding boxes, removes overlapping bounding boxes based on a fixed threshold, and retains the most likely detection result. This method is effective in most cases, but because there may be various crack distribution characteristics within the bounding box, such as a single intact crack, a single incomplete crack, multiple intact cracks, multiple incomplete cracks, and the coexistence of intact and incomplete cracks, when an intact crack and incomplete crack coexist within a bounding box, due to its low confidence, NMS may mistakenly identify them as the same large defect, causing the bounding box to be incorrectly discarded, resulting in missed detections and misjudgments.

[0057] In this embodiment of the invention, in order to avoid missing or misjudging bounding boxes with crack defects by screening them out, the distribution of cracks in the bounding boxes is analyzed, and then the preset initial threshold is adjusted to obtain an adaptive adjustment threshold for each bounding box for screening the final cracked bounding boxes.

[0058] First, in order to perform subsequent defect identification, high-resolution images of key automotive casting components are required. Industrial cameras can be placed above the production line, and appropriate light sources can be configured to reduce shadows and reflections based on the surface material of the components and the inspection requirements. The key automotive casting components are placed on the conveyor belt of the production line, and images of the components are captured by industrial cameras (after image preprocessing to grayscale; the preprocessing process is a well-known technique and will not be described in detail here).

[0059] Next, a pre-trained neural network is used to process the component images. This neural network is typically built on a deep learning framework and trained on a large amount of pre-labeled component image data with crack defects. During training, the neural network learns various features of the components, including shape, texture, and color, as well as the correlation between these features and crack defects. When the neural network receives a new component image, it iterates through each pixel of the image, classifying the pixels using the learned features to determine whether they belong to a crack defect. Furthermore, the neural network generates a series of bounding boxes, each surrounding a possible crack area; simultaneously, the neural network assigns a confidence score to each bounding box, representing the probability that the network believes a crack defect actually exists within that bounding box.

[0060] At this point, we can obtain the component image, as well as all bounding boxes in the component image and the confidence score for each bounding box. Please refer to [link / reference]. Figure 2 The diagram shows the result of a component image being recognized by a neural network in one embodiment of the present invention. The red boxes in the diagram are bounding boxes, and the confidence level of each bounding box is not shown.

[0061] It should be noted that the training process of neural networks is a well-known technique, and the specific process will not be described in detail here. In this embodiment of the invention, the backbone network of YOLOv5 is used to extract features for training the neural network. The bounding box regression loss function is GIoU Loss, the classification loss function is binary cross-entropy loss, and the confidence loss function is BCE Loss.

[0062] Step S2: Obtain the edge lines in the component image; in each bounding box, compare the positional distribution features and grayscale changes of the edge pixels between the edge lines to determine the crack protrusion value of each edge line.

[0063] When detecting crack defects on the surface of key cast automotive components, since cracks themselves exhibit a strong texture, the bounding box containing the crack will display crack-like textures. Therefore, the edge lines in the component image can be acquired first. Because the component surface also has texture, most of the acquired edge lines will be characterized by the surface texture. However, compared to the relatively regular texture distribution on the cast component surface, the edge lines formed by cracks have higher grayscale variations and unique extension characteristics. Therefore, this allows analysis of the positional distribution and grayscale variations of edge pixels within the bounding box to determine the degree of crack prominence on the edge lines, thus characterizing the likelihood that the edge lines are cracks.

[0064] First, the edge lines in the component image are obtained. Preferably, in one embodiment of the present invention, the method for obtaining the edge lines includes:

[0065] The Canny operator is used to process the component image to obtain all edge lines. It should be noted that edge line acquisition based on the Canny operator is a well-known technique, and the specific process will not be elaborated here.

[0066] Cracks have unique extension characteristics. Specifically, cracks often start from a certain point and spread outwards. As the crack's extent increases, the crack width at the starting point increases, becoming more noticeable in the image. Furthermore, the absorption and scattering of light are enhanced, leading to a decrease in grayscale value. Therefore, the grayscale change near the crack point is more pronounced. From the crack's starting point, the crack width gradually decreases as it spreads outwards. In other words, the less noticeable the crack is in the image, the less noticeable the grayscale change of the edge lines. Therefore, within each bounding box, the positional distribution characteristics and grayscale changes of edge pixels between edge lines are compared to determine the crack prominence value of each edge line, reflecting the probability that the edge line corresponds to a crack.

[0067] Preferably, in one embodiment of the present invention, the method for obtaining the crack protrusion degree value includes:

[0068] Please see Figure 3 The diagram illustrates a method flowchart for obtaining the crack protrusion degree value in one embodiment of the present invention, which includes the following steps:

[0069] Step S201: In each bounding box, take the endpoint of each edge line as the target point.

[0070] In each bounding box, all endpoints of each edge line are used as target points. By using the endpoints as target points, it is easier to trace the extension path of the crack.

[0071] Step S202: Divide each edge line into segments to obtain multiple line segments.

[0072] As a crack extends, its width and grayscale gradient may change. Segmented processing can capture these changes, providing a basis for subsequent feature extraction and crack severity assessment.

[0073] Therefore, each edge line is segmented to obtain multiple line segments: on each edge line, the edge pixels are clustered based on the DBSCAN clustering algorithm to obtain multiple clusters, where the neighborhood radius and the minimum number of points are preset values.

[0074] Finally, the edge pixels in each cluster are grouped into a line segment.

[0075] It should be noted that the clustering radius is set to 0.3, and the minimum number of points within the radius is set to 5. Both of these can be adjusted according to the implementation scenario and are not limited here. The DBSCAN clustering algorithm is a well-known technology, and the specific process will not be described in detail here.

[0076] Step S203: On each edge line, analyze the grayscale changes of edge pixels between segments to determine the crack performance value of each edge line.

[0077] The grayscale changes are more obvious at the crack, so the grayscale gradient is larger. Therefore, the grayscale gradient of the edge pixels is obtained based on the Canny operator (a well-known technique, which will not be elaborated here). On each line segment, the mean of the grayscale gradients of all edge pixels is used as the grayscale change factor. The larger the grayscale change factor, the more obvious the grayscale change of this line segment on the edge line.

[0078] Cracks in images typically appear as continuous areas with significant changes in grayscale values. The grayscale variation factor is larger near the crack initiation point, and as the crack spreads outward from the initiation point, the width of the crack gradually decreases, and the grayscale variation factor also decreases. However, the texture features of the component itself will have more regular grayscale variations.

[0079] Therefore, by comparing the differences in grayscale change factors between line segments on the edge line, the degree to which the edge line represents a crack can be assessed, and a crack representation value can be obtained: On each edge line, for any line segment, the line segments that the line segment passes through to each target point are taken as comparison segments. The difference in grayscale change factors between the line segment and each comparison segment is calculated as the grayscale change difference value. The larger the grayscale change difference value, the more the path between the line segment and the target point conforms to the extension characteristics of grayscale change exhibited by the crack. The mean of the grayscale change differences between the line segment and all comparison segments is taken as the crack trend factor. At this time, the larger the crack trend factor, the more likely the path between the line segment and the target point is to be the path corresponding to the real crack.

[0080] At this point, there is a crack trend factor between the line segment and each target point. The maximum value of all crack trend factors corresponding to the line segment is taken as the crack trend degree value of the line segment. Based on the above analysis, the maximum value can be regarded as the maximum probability that the extension path is a crack.

[0081] Finally, on each edge line, the crack tendency value is weighted and fused using the grayscale variation factor of the line segment to obtain the crack manifestation value for each edge line. Since the grayscale variation of edge lines belonging to cracks is more obvious, i.e., the grayscale variation factor is larger, the grayscale variation factor is normalized using the softmax() function to obtain grayscale weights. The product of the grayscale weight of each line segment on the edge line and the crack tendency value is used as the weighting factor. The larger the weighting factor, the more likely it is to be a real crack. The sum of the weighting factors of all line segments on each edge line is normalized and used as the crack manifestation value for each edge line. The larger this value, the higher the probability that the edge line is a crack. Normalization is a technique well-known to those skilled in the art. The choice of normalization function can be linear normalization or standard normalization, etc., and the specific normalization method is not limited here.

[0082] Step S204: In each bounding box, compare the differences in crack performance values ​​between different edge lines to determine the crack prominence value for each edge line.

[0083] Within a bounding box, there may be multiple edge lines, but not all edge lines are cracks. By comparing the crack manifestation values ​​between edge lines and quantifying the crack prominence value of each edge line, we can further reflect the likelihood that it is a crack.

[0084] Within each bounding box, the difference in crack manifestation value between each edge line and each of the remaining edge lines is calculated as a salience factor. A positive and larger salience factor indicates that the edge line is more likely to be a crack edge line.

[0085] Finally, the sum of all prominence factors corresponding to each edge line is normalized and used as the crack prominence value for each edge line. The larger the crack prominence value of an edge line, the more prominent the crack feature is compared with other edge lines in the same bounding box, and the more likely it is to be a true crack edge line. Since the sum of prominence factors can be positive or negative, the Sigmoid() function can be used for normalization.

[0086] Step S3: Based on the crack protrusion value of the edge line of each bounding box, and combined with the distribution characteristics of the number of edge pixels on the edge line, determine the integrity value and missing value of each bounding box respectively; comprehensively analyze the integrity value, missing value, and the correlation between integrity value and confidence level of the bounding box to determine the adjustment weight of each bounding box.

[0087] The confidence level of YOLOv5 is related to the accuracy of the bounding box localization. If the model correctly locates the bounding box of the defect and has a high degree of overlap with the actual defect area, the confidence level is high. If the bounding box has little overlap with the actual defect area or deviates significantly, the confidence level is low. Therefore, the confidence level of the bounding box can accurately reflect the accuracy of the bounding box localization. Therefore, analyzing the integrity of edge lines with high crack prominence values ​​within the bounding box can determine the degree of missing and integrity values ​​of the bounding box. Furthermore, given the normal correlation between the confidence level of the bounding box and the integrity value of the cracks within it—that is, when the crack in the bounding box is complete, the confidence level is high, and when the crack is incomplete, the confidence level is low—by comprehensively analyzing the integrity value, the missing value, and the aforementioned correlation between the integrity value and the confidence level of the bounding box, we can more accurately determine whether there are two or more crack defects within the bounding box and the integrity status of the crack defects. This allows us to determine the adjustment weight of each bounding box, which helps to more accurately evaluate the probability of each bounding box being a crack bounding box in the subsequent bounding box screening process, avoiding the screening of bounding boxes with complete cracks that could lead to misjudgment or missed detection.

[0088] First, based on the crack protrusion value of the edge line in each bounding box, and combined with the distribution characteristics of the number of edge pixels on the edge line, the integrity value and the missing value of each bounding box are determined respectively. Preferably, in one embodiment of the present invention, the method for obtaining the integrity value and the missing value of the bounding box includes:

[0089] In the component image, the number of edge pixels on each edge line is used as the total number value.

[0090] To measure the completeness of edge lines within a bounding box, the ratio of the number of edge pixels within the bounding box to the total number of corresponding edge pixels in the component image is used as the completeness coefficient for each edge line. The larger the completeness coefficient, the better the bounding box can frame the edge line, meaning the higher the completeness of the edge line within the bounding box.

[0091] Based on the analysis in step S2 above, it is known that within the bounding box, the larger the crack prominence value of a certain edge line, the more likely that edge line is a true crack edge line compared to other edge lines within the same bounding box. Therefore, the normalized value of the product of the integrity coefficient and the crack prominence value of each edge line in the bounding box is used as the integrity factor of each edge line. The integrity factor combines the probability that the edge line is a crack with the integrity of the edge line. Therefore, the larger the integrity factor, the more completely the bounding box can frame the edge line representing the real crack. Thus, the maximum integrity factor of all edge lines in the bounding box is used as the integrity value of the bounding box. Taking the maximum integrity factor as the integrity value, the larger the integrity value, the more complete the edge line representing the crack exists in the bounding box. Normalization is a technique well-known to those skilled in the art. The choice of normalization function can be linear normalization or standard normalization, etc., and the specific normalization method is not limited here.

[0092] Then, within each bounding box, the integrity coefficient of each edge line is negatively correlated and normalized, and multiplied by the crack prominence value. The normalized product is then used as the missing factor for each edge line. A larger product indicates a larger missing factor, meaning the edge line representing the real crack within the bounding box is less complete, and the bounding box cannot fully enclose the edge line representing the real crack. The maximum missing factor of all edge lines within the bounding box is used as the missing value of the bounding box. A larger missing value indicates the presence of edge lines with low integrity representing cracks within the bounding box. Negative correlation mapping and normalization can be performed using the formula exp(-x), where exp() represents an exponential function with the natural constant e as the base, and x represents the independent variable.

[0093] Under normal circumstances, there are five possible cases for a bounding box:

[0094] (1) If there is only one complete edge line corresponding to a real crack inside, the completeness value will be larger, the missing value will be smaller, and the confidence level will be higher. (2) If there is only one incomplete edge line corresponding to a real crack inside, the completeness value will be smaller, the missing value will be larger, and the confidence level will be lower. (3) If there are multiple edge lines corresponding to real cracks inside, and all of them are complete, the completeness value will be larger, the missing value will be smaller, and the confidence level will be higher. (4) If there are multiple edge lines corresponding to real cracks inside, and all of them are incomplete, the completeness value will be smaller, the missing value will be larger, and the confidence level will be lower. (5) If there are both complete and incomplete edge lines corresponding to real cracks inside the bounding box, the completeness value will be larger, the missing value will be larger, and the confidence level will be lower.

[0095] When filtering bounding boxes, NMS sorts them from highest to lowest confidence level and selects those with higher confidence as the retained objects. It then calculates the intersection-union ratio (IU) for subsequent bounding boxes and discards them. However, based on the previous analysis, the confidence level of bounding boxes with both intact and incomplete cracks is relatively low. Therefore, during NMS operation, they may be misjudged as redundant bounding boxes due to their large IU. But since they contain intact crack defects, it is necessary to increase the threshold to retain these bounding boxes and avoid filtering out bounding boxes with intact cracks, which could lead to misjudgments or missed detections.

[0096] By analyzing the aforementioned five cases, it is easy to understand that in the three cases with low confidence, when complete cracks and incomplete cracks coexist in the bounding box, the confidence and completeness values ​​will show a unique negative correlation. Therefore, the characteristics of this correlation can be used as an indicator to determine whether the crack distribution in the bounding box is the fifth case. Combined with the completeness value and the missing value of the bounding box, the adjustment weight of the bounding box can be determined, which will help to obtain an adaptive adjustment threshold in the future.

[0097] Preferably, in one embodiment of the present invention, the method for obtaining the adjusted weights includes:

[0098] This invention primarily addresses the coexistence of complete and incomplete cracks within a bounding box. Therefore, in each bounding box, the edge line corresponding to the largest missing factor is used as the target edge line. In this case, each bounding box with an incomplete crack corresponds to a target edge line. Bounding boxes whose target edge lines belong to the same edge line in the component image are grouped into the same category. The purpose of classifying the bounding boxes is mainly to filter them for reference; bounding boxes with the same edge line have higher reference value.

[0099] Then, for each type of bounding box, a reference weight is determined based on the completeness and missingness values ​​of each bounding box. The completeness value and the reference weight are positively correlated, while the missingness value and the reference weight are negatively correlated. The formula model for the reference weight includes:

[0100]

[0101] Where CQ represents the reference weight; QS represents the missing value; WZ represents the completeness value; and ε represents the preset first parameter.

[0102] In the formula model of reference weight, since the reference value of a bounding box with a more complete real crack should be higher, the completeness value is used as the numerator and the missing value is used as the denominator to obtain the reference weight of each bounding box. The larger the reference weight, the higher the completeness of the crack in the bounding box, the higher the probability that it contains only one crack, the lower the interference, and the greater the reference value.

[0103] It should be noted that the function of the preset first parameter ε is to prevent the denominator from being 0, and the value can be 0.001. The specific value can be adjusted according to the implementation scenario, and is not limited here.

[0104] Choose any bounding box as the target bounding box, and use the remaining bounding boxes of the same category as the target bounding box as reference bounding boxes, thereby analyzing the correlation between the confidence and completeness values ​​of the bounding boxes:

[0105] The difference in confidence level between the target bounding box and each reference bounding box is used as the confidence bias factor, and the difference in integrity level between the target bounding box and each reference bounding box is used as the integrity bias factor. When the confidence bias factor and the integrity bias factor have the same sign, it indicates that there is a positive correlation between the confidence level and the integrity level. Therefore, the product of the confidence bias factor and the integrity bias factor is used as the positive correlation factor. The larger the positive correlation factor, the lower the probability of the coexistence of complete cracks and incomplete cracks in the target bounding box.

[0106] The positive correlation factor is obtained by weighting and normalizing the positive correlation factors using the reference weights of all reference bounding boxes corresponding to the target bounding box. The normalized value of the reference weights of the reference bounding boxes is multiplied by the corresponding positive correlation factor using the softmax() function to obtain the weighted correlation factor. The sum of the weighted correlation factors of all reference bounding boxes is then normalized and used as the positive correlation coefficient. A larger weighted correlation factor and a larger positive correlation coefficient indicate higher reference value of the reference bounding boxes, thus lowering the probability of coexistence of complete and incomplete cracks in the target bounding box. Therefore, the positive correlation coefficient is negatively correlated to correct the logical relationship, resulting in an adjusted weight for the target bounding box. A larger adjusted weight indicates a higher probability of coexistence of complete and incomplete cracks in the target bounding box. This negative correlation mapping can be expressed using formula 1-x, where x represents the independent variable. To avoid incorrectly filtering out the target bounding box in the subsequent NMS process, the initial threshold can be adjusted based on the adjusted weight of this bounding box.

[0107] Step S4: When filtering bounding boxes using NMS and the confidence of the bounding boxes, the preset initial threshold is adjusted based on the adjustment weight of the bounding boxes to obtain the adjusted threshold for filtering, thereby obtaining the crack bounding boxes.

[0108] When using NMS (Non-Maximum Suppression) to filter bounding boxes, the intersection-union ratio (IUR) of the bounding box with the highest confidence score and any other bounding box is calculated. The IUR is the intersection of two bounding boxes divided by their union, and is used to quantify the degree of overlap between the bounding boxes. A preset initial threshold is used to filter out bounding boxes with high IUR. If the IUR exceeds the preset initial threshold, the bounding box is considered redundant and needs to be removed.

[0109] Based on the aforementioned steps, the adjustment weights of all bounding boxes can be obtained, and the adjustment weights reflect, to some extent, the possibility of the coexistence of complete cracks and incomplete cracks in the bounding boxes. Since the bounding boxes with the coexistence of complete cracks and incomplete cracks can also reflect the location of crack defects, in this step, the adjustment weights of the bounding boxes can be used to adjust the preset initial threshold in the NMS process, and the adjustment threshold corresponding to each bounding box can be used to determine whether it is a redundant bounding box, thereby filtering out the final crack bounding boxes.

[0110] Preferably, in one embodiment of the present invention, the method for obtaining the adjustment threshold includes:

[0111] The adjustment magnitude is obtained by multiplying the value of the preset initial threshold after negative correlation mapping with the adjustment weight of each bounding box. The sum of the preset initial threshold and the corresponding adjustment magnitude of each bounding box is used as the adjustment threshold for each bounding box. The formula model for the adjustment threshold includes:

[0112] TY = CY + (1-CY) × TQ

[0113] Where TY represents the adjustment threshold for each bounding box; CY represents the preset initial threshold; and TQ represents the adjustment weight.

[0114] In the formula model for adjusting the threshold, a negative correlation mapping is performed on the preset initial threshold to obtain (1-CY). The purpose of this process is to ensure that the final adjusted threshold is not greater than 1. Then, when the adjustment weight is larger, it means that the probability of complete cracks and incomplete cracks coexisting in the bounding box is higher. Therefore, in order to preserve the bounding box, the threshold of the bounding box in the NMS process should be increased. Thus, (1-CY) is multiplied by the adjustment weight to obtain the adjustment magnitude (1-CY)×TQ. The larger the adjustment magnitude, the greater the degree to which the threshold needs to be increased. Finally, the sum of the adjustment magnitude and the preset initial threshold is used as the adjustment threshold of the bounding box. The larger the adjustment threshold, the higher the probability of complete cracks and incomplete cracks coexisting in the bounding box.

[0115] It should be noted that the preset initial threshold is 0.5 in this embodiment of the invention. In other embodiments of the invention, the preset initial threshold can be adjusted according to the implementation scenario, and is not limited here.

[0116] Based on the aforementioned steps, the adjustment threshold corresponding to each bounding box can be obtained. Then, NMS can be used to filter the bounding boxes to obtain cracked bounding boxes: First, all bounding boxes are sorted in descending order of confidence to obtain a sorted sequence. In the sorted sequence, the bounding box with the highest confidence is selected as the retention object. Then, for the bounding boxes to be traversed after the retention object in the sorted sequence, the intersection-over-union ratio (IoU) between the currently traversed bounding box and the retention object is checked in turn. If the IoU is greater than the adjustment threshold of the bounding box to be traversed, the current bounding box to be traversed is considered redundant and is removed. The above sorting and suppression steps are repeated until all bounding boxes are processed. The final set of remaining bounding boxes is the result after NMS and is used as the cracked bounding boxes.

[0117] To illustrate this process, consider a set containing 5 bounding boxes, sorted in descending order of confidence as (A, B, C, D, E). The iterative calculation proceeds as follows: First round: A has the highest confidence and is retained. The remaining bounding boxes B, C, D, and E are selected for iteration. If the IoU between A and B is greater than the adjustment threshold for B, B is removed. The same applies to C, D, and E. If B and D are removed in the first round (indicating that A, B, and D detect the same target, retaining the bounding box A with the highest confidence; while CE might detect a different target), A is removed from the set and becomes a cracked bounding box. At this point, only bounding box CE remains in the set. The second round begins: C is retained, and E is selected for iteration. If the IoU between C and E is greater than the adjustment threshold for E, E is removed, and C is removed from the set and becomes a cracked bounding box. The process stops when the set is empty. Thus, two cracked bounding boxes, A and E, are obtained from the 5 bounding boxes.

[0118] It should be noted that the NMS process is a well-known technology, and its process is only briefly described in the embodiments of this invention for illustrative purposes.

[0119] In summary, the process involves acquiring component images and using a pre-trained neural network to obtain bounding boxes and their corresponding confidence scores. To avoid misjudgments or missed detections when using NMS to filter bounding boxes due to the potential for different crack distributions, it's necessary to determine the specific distribution of crack defects within each bounding box. Given that cracks exhibit a certain extension characteristic—that is, a clear gradient in grayscale changes—the distribution characteristics and grayscale changes of edge pixels within each bounding box are compared to determine the crack prominence value of the edge line, reflecting the degree to which the edge line represents a crack. The confidence score of the bounding box is usually related to its localization accuracy; therefore, the integrity of the edge lines representing cracks within the bounding box can be further assessed to quantify the integrity and missing value of the bounding box. Specifically, the distribution characteristics of the number of edge pixels can be analyzed and combined with the crack prominence value of the edge line for quantification. Then, based on the integrity and missing value of the bounding box, as well as the correlation between the integrity value and the confidence score, the probability of multiple crack defects existing in each bounding box is analyzed, resulting in an adjustment weight for each bounding box. Finally, in order to reduce the possibility that bounding boxes with multiple crack defects are simply judged as a single incomplete large defect and thus screened out, the preset initial threshold is adjusted according to the adjustment weight of each bounding box when using NMS to screen the bounding boxes. This results in an adaptive adjustment threshold for each bounding box, thereby enabling more accurate screening of the bounding boxes and obtaining the final crack bounding boxes, effectively reducing the probability of false positives or false negatives.

[0120] This invention also provides an automatic defect identification system for key automotive casting components. Please refer to [link / reference]. Figure 4 The diagram shows a system block diagram, including a data acquisition module 401 for implementing step S1 in the above method embodiment; a crack analysis module 402 for implementing step S2 in the above method embodiment; a weight adjustment module 403 for implementing step S3 in the above method embodiment; and a crack boundary box determination module 404 for implementing step S4 in the above method embodiment.

[0121] It should be noted that the system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the automatic identification system for defects in key cast automotive parts and the automatic identification method for defects in key cast automotive parts provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0122] Please see Figure 5This illustration shows a schematic diagram of the system structure of an automatic defect identification system for key cast automotive components according to an embodiment of the present invention. The system includes a processor 500, a memory 501, a bus 502, and a communication interface 503. The processor 500, communication interface 503, and memory 501 are connected via the bus 502. The memory 501 may contain a high-speed random access memory, and the bus 502 may be an ISA bus, PCI bus, or EISA bus, etc. The processor 500 may be an integrated circuit chip with signal processing capabilities. The memory 501 stores at least one instruction, at least one program, code set, or instruction set. When the processor loads and executes the at least one instruction, at least one program, code set, or instruction set, it implements the steps in an automatic defect identification method for key cast automotive components.

[0123] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0124] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for automatic identification of defects in key cast automotive components, characterized in that, The method includes: Acquire component images, and obtain all bounding boxes in the component images and their corresponding confidence scores based on a pre-trained neural network; Obtain the edge lines in the component image; within each bounding box, compare the positional distribution features and grayscale changes of the edge pixels between the edge lines to determine the crack protrusion value of each edge line; Based on the crack protrusion value of the edge line of each bounding box, and combined with the distribution characteristics of the number of edge pixels on the edge line, the integrity value and missing value of each bounding box are determined respectively; by comprehensively analyzing the integrity value, missing value, and the correlation between integrity value and confidence, the adjustment weight of each bounding box is determined. When using NMS and the confidence level of the bounding box to filter the bounding box, the preset initial threshold is adjusted based on the adjustment weight of the bounding box, and the adjusted threshold is used for filtering to obtain the crack bounding box.

2. The method for automatic identification of defects in key cast automotive components according to claim 1, characterized in that, The method for obtaining the crack protrusion degree value includes: Within each bounding box, all endpoints of each edge line are used as target points; Each edge line is segmented to obtain multiple line segments, and the average gray level gradient of all edge pixels on each line segment is used as the gray level change factor. On each edge line, analyze the differences in grayscale variation factors between line segments to determine the degree of crack manifestation on each edge line; Calculate the difference in crack manifestation value between each edge line and each of the remaining edge lines, and use it as a highlighting factor; The sum of all protrusion factors corresponding to each edge line is normalized and used as the crack protrusion value for each edge line.

3. The method for automatic identification of defects in key cast automotive components according to claim 2, characterized in that, The process of segmenting each edge line to obtain multiple line segments includes: On each edge line, the edge pixels are clustered based on the DBSCAN clustering algorithm to obtain multiple clusters, where the neighborhood radius and the minimum number of points are preset values. The edge pixels in each cluster are grouped into a line segment.

4. The method for automatic identification of defects in key cast automotive components according to claim 2, characterized in that, The method for obtaining the crack manifestation degree value includes: On each edge line, for any line segment, the line segments that the line segment passes through to each target point are taken as comparison segments. The difference between the gray scale change factor of the line segment and each comparison segment is calculated as the gray scale change difference. The mean of the gray scale change differences between the line segment and all comparison segments is taken as the crack trend factor. The maximum value of all crack tendency factors corresponding to the line segment is taken as the crack tendency degree value of the line segment. On each edge line, the crack trend degree value is weighted and fused using the gray-scale change factor of the line segment to obtain the crack performance degree value of each edge line.

5. The method for automatic identification of defects in key cast automotive components according to claim 1, characterized in that, The degree of crack protrusion based on the edge line of each bounding box, combined with the distribution characteristics of the number of edge pixels on the edge line, determines the integrity value and the degree of missingness of each bounding box, including: In the component image, the number of edge pixels on each edge line is taken as the total number. Within each bounding box, the ratio of the number of edge pixels within the bounding box to the total number of corresponding edge pixels is used as the integrity coefficient for each edge line. In each bounding box, the normalized product of the integrity coefficient of each edge line and the crack protrusion value is used as the integrity factor of each edge line, and the maximum integrity factor of all edge lines in the bounding box is used as the integrity value of the bounding box. In each bounding box, the integrity coefficient of each edge line is negatively correlated and normalized, and then multiplied by the crack protrusion value. The normalized value of the resulting product is used as the missing factor for each edge line. The maximum missing factor of all edge lines in the bounding box is used as the missing value of the bounding box.

6. The method for automatic identification of defects in key cast automotive components according to claim 5, characterized in that, The method for obtaining the adjustment weights includes: In each bounding box, the edge line corresponding to the largest missing factor is used as the target edge line; Group the bounding boxes whose target edge lines belong to the same edge lines in the component images into the same type of bounding boxes; In each type of bounding box, a reference weight is determined based on the completeness value and the missingness value of each bounding box. The completeness value and the reference weight are positively correlated, and the missingness value and the reference weight are negatively correlated. Choose any bounding box as the target bounding box, and use the remaining bounding boxes of the same type as the target bounding box as reference bounding boxes; The difference in confidence between the target bounding box and each reference bounding box is used as the confidence bias factor, the difference in completeness value between the target bounding box and each reference bounding box is used as the completeness bias factor, and the product of the confidence bias factor and the completeness bias factor is used as the positive correlation factor. The positive correlation factor is weighted, fused, and normalized using the reference weights of all reference bounding boxes corresponding to the target bounding box to obtain the positive correlation coefficient. The value of the positive correlation coefficient after negative correlation mapping is used as the adjustment weight of the target bounding box.

7. The method for automatic identification of defects in key cast automotive components according to claim 1, characterized in that, The method for obtaining the adjustment threshold includes: The adjustment magnitude is obtained by multiplying the value of the preset initial threshold after negative correlation mapping with the adjustment weight of each bounding box; The sum of the preset initial threshold and the corresponding adjustment range for each bounding box is used as the adjustment threshold for each bounding box.

8. The method for automatic identification of defects in key cast automotive components according to claim 1, characterized in that, The preset initial threshold is 0.

5.

9. The method for automatic identification of defects in key cast automotive components according to claim 1, characterized in that, The method for obtaining the edge line includes: The component image is processed using the Canny operator to obtain all edge lines.

10. An automatic defect identification system for key cast automotive components, characterized in that, The method includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set, or instruction set, and the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the steps of the automatic identification method for defects in key cast automotive components as described in any one of claims 1-9.

Images