Automobile key casting part defect automatic identification method and system

Abstract

The invention relates to the technical field of image analysis, in particular to a method and a system for automatically identifying defects of key casting parts of an automobile. In order to avoid misjudgment or missing detection during NMS screening, the specific distribution condition of cracks in a bounding box is analyzed, in view of the extension characteristics of the cracks, the crack protrusion degree value is determined by comparing the position distribution and gray level change of edge pixel points, and meanwhile, according to the integrity and the missing degree of edge lines and in combination with the crack protrusion degree value, the crack protrusion degree value is determined. According to the method, the integrity degree value and the missing degree value of the bounding box are quantified, further, the relationship between the values and the confidence coefficient is analyzed, the possibility that a plurality of crack defects exist in each bounding box is evaluated, the adjustment weight is obtained accordingly, and finally, in the NMS screening process, the preset initial threshold value is adjusted in a self-adaptive mode according to the adjustment weight. The adjustment threshold value of each bounding box is obtained, so that the bounding boxes are screened more accurately, and the final crack bounding box is obtained. According to the method, the occurrence probability of misjudgment or missing detection is effectively reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and particularly relates to a method and system for automatically identifying defects in key cast components of automobiles. Background Art

[0002] The key cast components of automobiles mainly include engine blocks, cylinder heads, crankshafts, transmission cases, steering knuckles, wheels, suspension brackets, etc. These components are usually made of materials such as aluminum alloy, cast iron or cast steel, and have high requirements for strength and precision. They are the core components of vehicle power transmission, suspension and braking systems. If there are cracks, it may lead to a decrease in component strength, shortening of service life, and even serious safety accidents. Therefore, it is necessary to identify their defects to ensure the consistency and stability of product quality.

[0003] For the crack defects on the surface of key cast components of automobiles, a pre-trained neural network is usually used to identify them, and then post-processing NMS (Non-Maximum Suppression) is used to compare the confidence levels and overlap degrees of bounding boxes to remove overlapping bounding boxes. However, there are various distribution characteristics of crack defects in the bounding boxes, such as single cracks or multiple cracks, and the cracks may also be complete or incomplete. The existing NMS usually uses a fixed threshold to screen the bounding boxes, and the fixed threshold cannot adapt to the different distribution characteristics of cracks in the bounding boxes, which may lead to the incorrect discarding of bounding boxes with complete crack defects, resulting in missed detections or misjudgments. Summary of the Invention

[0004] In order to solve the technical problem that the existing NMS usually uses a fixed threshold to screen the bounding boxes, and the fixed threshold cannot adapt to the different distribution characteristics of cracks in the bounding boxes, which may lead to the incorrect discarding of bounding boxes with complete crack defects, resulting in missed detections or misjudgments, the purpose of the present invention is to provide a method and system for automatically identifying defects in key cast components of automobiles, and the specific technical solutions adopted are as follows:

[0005] A method for automatically identifying defects in key cast components of automobiles includes:

[0006] Obtain a component image, and based on a pre-trained neural network, obtain all the bounding boxes in the component image and the confidence levels corresponding to the bounding boxes;

[0007] Obtain the edge lines in the component image; in each bounding box, compare the position distribution characteristics and gray-scale change conditions of the edge pixel points between the edge lines to determine the crack prominence value of each edge line;

[0008] Based on the crack prominence values of the edge lines in each bounding box and in combination with the quantity distribution characteristics of the edge pixel points on the edge lines, determine the integrity value and the missing degree value of each bounding box respectively; comprehensively analyze the integrity value, the missing degree value of the bounding box, and the correlation between the integrity value and the confidence level, so as to determine the adjustment weight of each bounding box;

[0009] When screening the bounding boxes by using NMS and the confidence level of the bounding boxes, adjust the preset initial threshold based on the adjustment weight of the bounding box to obtain an adjusted threshold for screening, so as to obtain the crack bounding boxes.

[0010] Furthermore, the method for obtaining the crack prominence value includes:

[0011] In each bounding box, take all the endpoints of each edge line as target points;

[0012] Segment each edge line to obtain a plurality of line segments, and take the mean value of the gray gradients of all the edge pixel points on each line segment as the gray change factor;

[0013] On each edge line, analyze the difference situation of the gray change factors between the line segments to determine the crack manifestation degree value of each edge line;

[0014] Calculate the difference between the crack manifestation degree value of each edge line and the crack manifestation degree value of each remaining edge line as the prominence factor;

[0015] Take the normalized value of the sum of all the prominence factors corresponding to each edge line as the crack prominence value of each edge line.

[0016] Furthermore, the segmenting each edge line to obtain a plurality of line segments includes:

[0017] On each edge line, perform clustering analysis on the edge pixel points based on the DBSCAN clustering algorithm to obtain a plurality of clustering clusters, where the neighborhood radius and the minimum number of points are both preset values;

[0018] Form a line segment with the edge pixel points in each clustering cluster.

[0019] Furthermore, the method for obtaining the crack manifestation degree value includes:

[0020] On each edge line, for any one line segment, take the line segments passed between this line segment and each target point as comparison segments, calculate the difference between the gray change factor of this line segment and the gray change factor of each comparison segment as the gray change difference, and take the mean value of the gray change differences between this line segment and all the comparison segments as the crack trend factor;

[0021] Take the maximum value of all crack trend factors corresponding to the line segment as the crack trend degree value of the line segment;

[0022] On each edge line, use the gray-scale change factor of the line segment to perform weighted fusion on the crack trend degree value to obtain the crack manifestation degree value of each edge line.

[0023] Further, based on the crack prominence degree value of the edge line in each bounding box and in combination with the quantity distribution characteristics of the edge pixel points on the edge line, determine the integrity degree value and the missing degree value of each bounding box respectively, including:

[0024] In the component image, take the number of edge pixel points on each edge line as the total quantity value;

[0025] In each bounding box, take the ratio of the number of edge pixel points of each edge line within the bounding box to the corresponding total quantity value as the integrity coefficient of each edge line;

[0026] In each bounding box, take the value obtained by normalizing the product of the integrity coefficient of each edge line and the crack prominence degree value as the integrity factor of each edge line, and take the maximum integrity factor of all edge lines in the bounding box as the integrity degree value of the bounding box;

[0027] In each bounding box, multiply the value obtained by performing negative correlation mapping and normalization on the integrity coefficient of each edge line by the crack prominence degree value, and take the value obtained by normalizing the resulting product as the missing factor of each edge line, and take the maximum missing factor of all edge lines in the bounding box as the missing degree value of the bounding box.

[0028] Further, the method for obtaining the adjusted weight includes:

[0029] In each bounding box, take the edge line corresponding to the maximum missing factor as the target edge line;

[0030] Group the bounding boxes to which the target edge line belongs to the same edge line in the component image into the same category of bounding boxes;

[0031] In each category of bounding boxes, determine the reference weight of each bounding box according to the integrity degree value and the missing degree value of each bounding box, where the integrity degree value and the reference weight are positively correlated, and the missing degree value and the reference weight are negatively correlated;

[0032] Optionally select a bounding box as the target bounding box, and take the remaining bounding boxes of the same category as the target bounding box as the reference bounding boxes;

[0033] The difference in confidence between the target bounding box and each reference bounding box is used as the confidence deviation factor, the difference in the integrity value between the target bounding box and each reference bounding box is used as the integrity deviation factor, and the product of the confidence deviation factor and the integrity deviation factor is used as the positive correlation factor;

[0034] The positive correlation factors are weighted and fused and normalized by using the reference weights of all the reference bounding boxes corresponding to the target bounding box to obtain a positive correlation coefficient, and the value obtained by performing a negative correlation mapping on the positive correlation coefficient is used as the adjustment weight of the target bounding box.

[0035] Furthermore, the method for obtaining the adjustment threshold includes:

[0036] Multiply the value obtained by performing a negative correlation mapping on the preset initial threshold by the adjustment weight of each bounding box to obtain an adjustment amplitude;

[0037] The sum value of the preset initial threshold and the corresponding adjustment amplitude of each bounding box is used as the adjustment threshold of each bounding box.

[0038] Furthermore, the preset initial threshold is 0.5.

[0039] Furthermore, the method for obtaining the edge line includes:

[0040] Process the component image based on the Canny operator to obtain all the edge lines.

[0041] An automatic defect recognition system for key cast components of an automobile includes a processor and a memory. At least one instruction, at least one program, a code set or an instruction set is stored in the memory. When at least one instruction, at least one program, the code set or the instruction set is loaded and executed by the processor, the steps of the automatic defect recognition method for key cast components of an automobile are implemented.

[0042] The present invention has the following beneficial effects:

[0043] Obtain the component image and based on the pre-trained neural network, obtain the bounding box in the component image and the confidence corresponding to the bounding box. In order to avoid misjudgment or missed detection when using NMS to filter the bounding boxes, since there may be different crack distribution characteristics in the bounding boxes, it is necessary to judge the specific crack defect distribution in the bounding boxes. Given that cracks have certain extension characteristics, that is, the gray-scale change shows an obvious gradual change characteristic, so in each bounding box, compare the position distribution characteristics and gray-scale change of the edge pixel points to determine the crack prominence value of the edge line, which can reflect the degree of manifestation of the edge line as a crack. The confidence of the bounding box is usually related to the positioning accuracy of the bounding box, so the integrity of the edge line representing the crack in the bounding box can be further judged, so as to quantify the integrity value and missing value of the bounding box. Specifically, the number distribution characteristics of the edge pixel points can be analyzed and quantified in combination with the crack prominence value of the edge line. Then, according to the integrity value, missing value of the bounding box, and the correlation between the integrity value and the confidence, analyze the possibility of multiple crack defects in each bounding box to obtain the adjustment weight of each bounding box. Finally, in order to reduce the possibility that the bounding boxes with multiple crack defects are simply judged as a single incomplete large defect and filtered out, when using NMS to filter the bounding boxes, adjust the preset initial threshold according to the adjustment weight of each bounding box to obtain the adaptive adjustment threshold for each bounding box, so as to perform more accurate filtering of the bounding boxes, obtain the final crack bounding boxes, and effectively reduce the occurrence probability of misjudgment or missed detection. Description of the Drawings

[0044] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0045] Figure 1 It is a flowchart of a method for automatically identifying defects in key cast components of an automobile provided by an embodiment of the present invention;

[0046] Figure 2 It is a schematic diagram of the result after a component image is recognized by a neural network provided by an embodiment of the present invention;

[0047] Figure 3 It is a flowchart of a method for obtaining the crack prominence value provided by an embodiment of the present invention;

[0048] Figure 4The system block diagram of an automatic defect recognition system for key cast components of an automobile provided by an embodiment of the present invention;

[0049] Figure 5 The schematic structural diagram of an automatic defect recognition system for key cast components of an automobile provided by an embodiment of the present invention. Detailed implementation manners

[0050] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following combines the accompanying drawings and preferred embodiments to elaborate in detail on an automatic defect recognition method and system for key cast components of an automobile proposed according to the present invention, its specific implementation manners, structures, features, and effects. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0052] The following specifically describes the specific solutions of an automatic defect recognition method and system for key cast components of an automobile provided by the present invention with reference to the accompanying drawings.

[0053] Please refer to Figure 1 , which shows the flowchart of a method for automatically recognizing defects in key cast components of an automobile provided by an embodiment of the present invention. The method includes the following steps:

[0054] Step S1: Obtain the component image, and obtain all the bounding boxes and the confidence levels corresponding to the bounding boxes in the component image based on a pre-trained neural network.

[0055] In the automotive manufacturing industry, the surface quality of key cast components is crucial, especially crack defects, which directly affect the strength and safety of the components. Currently, deep learning algorithms, such as YOLOV5, have been widely used in the field of automatic defect recognition for key cast components of automobiles due to their high efficiency and accuracy. With its strong feature extraction ability and real-time detection speed, YOLOV5 has shown significant advantages in identifying crack defects.

[0056] However, since the surfaces of cast components are often complex and variable, including various textures, shadows, and reflections, these features increase the difficulty of defect identification. When YOLOV5 processes such complex surfaces, it may misidentify the same defect as multiple independent defects, or mistake multiple adjacent defects for a large defect. Therefore, to optimize the recognition results of YOLOV5, the post-processing NMS (Non-Maximum Suppression) technique is mainly used at present. NMS compares the confidence levels and overlaps of bounding boxes, removes overlapping bounding boxes based on a fixed threshold, and retains the most likely detection results. This method is effective in most cases, but due to the possible existence of various crack distribution characteristics in the bounding boxes, such as single complete cracks, single incomplete cracks, multiple complete cracks, multiple incomplete cracks, and the coexistence of complete and incomplete cracks. When there are coexisting complete and incomplete cracks in a certain bounding box, due to its low confidence level, NMS may mistake them for a large defect, resulting in the incorrect discarding of the bounding box, thus causing missed detections and misjudgments.

[0057] In the embodiment of the present invention, in order to avoid screening out the bounding boxes with crack defects, resulting in missed detections or misjudgments, the crack distribution in the bounding boxes is analyzed, and then the preset initial threshold is adjusted, so as to obtain an adaptive adjustment threshold for each bounding box to screen the final crack bounding boxes.

[0058] First, to perform subsequent defect identification, high-definition images of key automotive cast components need to be obtained. An industrial camera can be arranged above the production line, and according to the surface material of the component and the detection requirements, a suitable light source is configured to reduce shadow and reflection interference. The key automotive cast component is placed on the conveyor belt of the production line, and the industrial camera is used to take and obtain the component image (after image preprocessing into a grayscale image, the preprocessing process is a well-known technology and will not be elaborated here).

[0059] Next, the component image is processed using a pre-trained neural network. This neural network is usually built based on a deep learning framework and trained with a large amount of component image data pre-annotated with crack defects. During the training process, the neural network will learn various features of the components, including shape, texture, color, etc., and the associations between these features and crack defects. At this time, when the neural network receives a new component image, it will traverse each pixel point of the image, classify the pixel points using the learned features, and determine whether they belong to a part of the crack defect. Moreover, the neural network will further generate a series of bounding boxes, each bounding box surrounding a possible crack area; at the same time, the neural network will also assign a confidence level to each bounding box, and this value represents the probability that the network believes there is indeed a crack defect within the bounding box.

[0060] So far, the component images, all the bounding boxes in the component images, and the confidence levels corresponding to each bounding box can be obtained. Please refer to Figure 2 , which shows a schematic diagram of the result after the component image in an embodiment of the present invention is recognized by a neural network. The red boxes in the figure are the bounding boxes, and the confidence levels corresponding to each bounding box are not shown.

[0061] It should be noted that the training process of the neural network is a well-known technology, and the specific process will not be elaborated here; in this embodiment of the present invention, the backbone network of YOLOv5 is used to extract features for the training of the neural network. The bounding box regression loss function uses GIoU Loss, the classification loss function uses binary cross-entropy loss, and the confidence loss function uses BCE Loss.

[0062] Step S2: Obtain the edge lines in the component image; in each bounding box, compare the position distribution characteristics and gray-scale change conditions of the edge pixel points between the edge lines to determine the crack prominence value of each edge line.

[0063] When detecting crack defects on the surface of key cast components of an automobile, since the crack itself shows a strong texture state, there will be textures with crack characteristics in the bounding box where the crack exists. Therefore, the edge lines in the component image can be obtained first. Since there will also be textures on the component surface, among the obtained edge lines, most of them will be the edge lines shown by the textures on the component surface. However, compared with the relatively regular texture distribution characteristics on the surface of the cast component, the edge lines formed by the crack have higher gray-scale changes and unique extension characteristics. Therefore, the position distribution characteristics and gray-scale change conditions between the edge pixel points in the bounding box can be analyzed to determine the crack prominence value of the edge line, which is used to characterize the possibility that the edge line is a crack.

[0064] First, obtain the edge lines in the component image. Preferably, in an embodiment of the present invention, the method for obtaining the edge lines includes:

[0065] Process the component image based on the Canny operator to obtain all the edge lines. It should be noted that obtaining edge lines based on the Canny operator is a well-known technology, and the specific process will not be elaborated here.

[0066] Cracks have their own unique extension characteristics, specifically manifested as cracks often starting from a certain point and spreading around. Moreover, as the degree of crack opening increases, the crack width at the starting point of cracking will increase, which is more obvious in the image, and the absorption and scattering effects on light are enhanced, resulting in a decrease in the gray value. Then, the gray change near the cracking point is more obvious. As the crack spreads around from the starting point of cracking, the crack width gradually decreases, that is, the crack is less obvious in the image, and the gray change of the edge line is less obvious. Therefore, in each bounding box, the position distribution characteristics and gray change of the edge pixel points between the edge lines are compared to determine the crack prominence value of each edge line, reflecting the possibility that the edge line corresponds to the edge line of the crack.

[0067] Preferably, in one embodiment of the present invention, the method for obtaining the crack prominence value includes:

[0068] Please refer to Figure 3 , which shows the flowchart of the method for obtaining the crack prominence value in one embodiment of the present invention. The method includes the following steps:

[0069] Step S201: In each bounding box, take the endpoints of each edge line as target points.

[0070] In each bounding box, take all the endpoints of each edge line as target points. By taking the endpoints as target points, it is easier to trace the extension path of the crack.

[0071] Step S202: Segment each edge line to obtain multiple line segments.

[0072] During the extension process of the crack, its width and gray gradient may change. The segmentation process can capture these changes and provide a basis for subsequent feature extraction and crack degree evaluation.

[0073] Therefore, each edge line is segmented to obtain multiple line segments: on each edge line, based on the DBSCAN clustering algorithm, clustering analysis is performed on the edge pixel points to obtain multiple clustering clusters, where the neighborhood radius and the minimum number of points are both preset values.

[0074] Finally, the edge pixel points in each clustering cluster are combined into a line segment.

[0075] It should be noted that the clustering radius is set to 0.3, the minimum number of points within the radius is set to 5, and both can be adjusted according to the implementation scenario and are not limited here; the DBSCAN clustering algorithm is a well-known technology, and the specific process is not elaborated here.

[0076] Step S203: On each edge line, analyze the gray change of the edge pixel points between the segments to determine the crack manifestation value of each edge line.

[0077] The gray-scale change at the crack is relatively obvious, so the gray-scale gradient is relatively large. Therefore, the gray-scale gradient of the edge pixels is obtained based on the Canny operator (a well-known technique, not elaborated here), and on each line segment, the average value of the gray-scale gradients of all edge pixels is used as the gray-scale change factor. The larger the gray-scale change factor, the more obvious the gray-scale change of this line segment on the edge line.

[0078] Cracks usually appear as continuous regions with significant gray-scale value changes in the image, and the gray-scale change factor near the crack initiation point is larger. When spreading from the initiation point to the surrounding area, the width of the crack gradually decreases, and then the gray-scale change factor also becomes smaller; however, the texture features of the component itself will have a more regular gray-scale change.

[0079] Therefore, by comparing the differences in gray-scale change factors between line segments on the edge line, the manifestation degree of the edge line as a crack can be evaluated, and the crack manifestation degree value is obtained: On each edge line, for any line segment, the line segments passed between this line segment and each target point are used as comparison segments, and the difference between the gray-scale change factor of this line segment and that of each comparison segment is calculated as the gray-scale change difference. The larger the gray-scale change difference, the more the path between this line segment and the target point conforms to the extension characteristics of the gray-scale change shown by the crack. The average value of the gray-scale change differences between this line segment and all comparison segments is used as the crack trend factor. At this time, the larger the crack trend factor, the more likely the path between this line segment and the target point is the path corresponding to the real crack.

[0080] At this time, there is a crack trend factor between this line segment and each target point. The maximum value of all crack trend factors corresponding to this line segment is used as the crack trend degree value of this line segment. Based on the foregoing analysis, the maximum value can be regarded as the maximum possibility that the extension path is a crack.

[0081] Finally, on each edge line, the crack manifestation degree value of each edge line is obtained by weighted fusion of the crack trend degree value using the gray-scale change factor of the line segment: Given that the gray-scale change of the edge line belonging to the crack is relatively obvious, that is, the gray-scale change factor is relatively large, the gray-scale change factor is normalized using the softmax() function to obtain the gray-scale weight. The product of the gray-scale weight of each line segment on the edge line and the crack trend degree value is used as the weighted factor. The larger the weighted factor, the more likely it is to be a real crack. The normalized value of the sum of the weighted factors of all line segments on each edge line is used as the crack manifestation degree value of each edge line. The larger this value, the higher the possibility that this edge line is a crack. Among them, normalization is a well-known technical means in the art, and the choice of the normalization function can be linear normalization or standard normalization, etc. The specific normalization method is not limited here.

[0082] Step S204: In each bounding box, compare the differences in the crack manifestation degree values between different edge lines to determine the crack prominence degree value of each edge line.

[0083] In a bounding box, there may be multiple edge lines, but not all edge lines are cracks. By comparing the crack manifestation degree values between edge lines and quantifying the crack prominence degree value of each edge line, the possibility that it is a crack can be further reflected.

[0084] In each bounding box, calculate the difference in the crack manifestation degree value between each edge line and each of the remaining edge lines as the prominence factor. The larger the positive prominence factor, the higher the possibility that the edge line is a crack edge line.

[0085] Finally, the normalized value of the sum of all prominence factors corresponding to each edge line is used as the crack prominence degree value of each edge line. The larger the crack prominence degree value of a certain edge line, the more prominent the crack feature of this edge line compared to other edge lines in the same bounding box, and the more likely it is to be a real crack edge line. Since the sum of the prominence factors here may be positive or negative, the sigmoid() function can be used as the normalization method.

[0086] Step S3: Based on the crack prominence degree values of the edge lines in each bounding box and in combination with the quantity distribution characteristics of the edge pixels on the edge lines, determine the integrity value and the missing degree value of each bounding box respectively; comprehensively analyze the integrity value, the missing degree value of the bounding box, and the correlation between the integrity value and the confidence level, so as to determine the adjustment weight of each bounding box.

[0087] The confidence of YOLOv5 is related to the positioning accuracy of the bounding box. If the model correctly locates the bounding box of the defect and has a high overlap with the actual defect area, the confidence is high. If the overlap between the bounding box and the actual defect area is small or the deviation is large, the confidence is low. Therefore, the confidence of the bounding box can accurately reflect the positioning accuracy of the bounding box. Therefore, by analyzing the integrity of the edge lines with a higher crack prominence value within the bounding box, the missing degree value and the integrity degree value of the bounding box can be judged. Furthermore, in normal cases, there is a correlation between the confidence of the bounding box and the integrity degree value of the cracks therein, that is, when the cracks in the bounding box are complete, the confidence is high, and when the cracks are incomplete, the confidence is low. Therefore, by comprehensively analyzing the integrity degree value, the missing degree value of the bounding box, and the aforementioned correlation between the integrity degree value and the confidence, it is possible to more accurately judge whether there are two or more crack defects within the bounding box and the integrity of the crack defects, so as to determine the adjustment weight of each bounding box, which is helpful for more accurately evaluating the possibility of each bounding box being a crack bounding box in the subsequent process of screening the bounding boxes, and avoiding misjudgment or missed detection caused by screening out the bounding boxes with complete cracks.

[0088] First, based on the crack prominence value of the edge lines in each bounding box and combined with the quantity distribution characteristics of the edge pixel points on the edge lines, the integrity degree value and the missing degree value of each bounding box are respectively determined. Preferably, in an embodiment of the present invention, the method for obtaining the integrity degree value and the missing degree value of the bounding box includes:

[0089] In the component image, the number of edge pixel points on each edge line is used as the total quantity value.

[0090] In order to measure the integrity degree of the edge lines in the bounding box, in each bounding box, the ratio of the number of edge pixel points of each edge line within the bounding box to the total quantity value corresponding to this edge line in the component image is used as the integrity coefficient of each edge line. The larger the integrity coefficient, the more the bounding box can enclose the edge line to a greater extent, that is, the higher the integrity degree of the edge line in the bounding box.

[0091] Based on the analysis in the foregoing step S2, in a bounding box, the larger the crack prominence value of an edge line, the more likely it is that this edge line is a true crack edge line compared to other edge lines in the same bounding box. Therefore, the value obtained by normalizing the product of the integrity coefficient and the crack prominence value of each edge line in the bounding box is used as the integrity factor of each edge line. The integrity factor comprehensively considers the possibility that the edge line is a crack and the integrity degree of the edge line. Thus, the larger the integrity factor, the more completely the bounding box can enclose the edge line representing the true crack. Therefore, the maximum integrity factor of all edge lines in the bounding box is used as the integrity degree value of the bounding box. Taking the maximum integrity factor as the integrity degree value, the larger the integrity degree value, the more complete and crack-representing edge lines exist in the bounding box. Here, normalization is a well-known technical means in the art. The choice of the normalization function can be linear normalization or standard normalization, etc. The specific normalization method is not limited herein.

[0092] Then, in each bounding box, the value obtained by multiplying the value obtained by negatively correlating and normalizing the integrity coefficient of each edge line with the crack prominence value, and then normalizing the resulting product, is used as the missing factor of each edge line. The larger the product here, the larger the missing factor, indicating that the integrity degree of the edge line representing the true crack in the bounding box is lower, and the bounding box cannot more completely enclose the edge line representing the true crack. The maximum missing factor of all edge lines in the bounding box is used as the missing degree value of the bounding box. The larger the missing degree value, the more incomplete and crack-representing edge lines exist in the bounding box. The negative correlation mapping and normalization can be performed using the formula exp(-x), where exp() represents the exponential function with the natural constant e as the base, and x represents the independent variable.

[0093] Under normal circumstances, for a bounding box, there are the following five situations:

[0094] (1) If there is only one edge line corresponding to a complete true crack inside it, then the integrity degree value will be larger, the missing degree value will be smaller, and the confidence level will also be larger; (2) If there is only one edge line corresponding to an incomplete true crack inside it, then the integrity degree value will be smaller, the missing degree value will be larger, and the confidence level will also be smaller; (3) If there are multiple edge lines corresponding to true cracks inside it and all are complete, then the integrity degree value will be larger, the missing degree value will be smaller, and the confidence level will also be larger; (4) If there are multiple edge lines corresponding to true cracks inside it and all are incomplete, then the integrity degree value will be smaller, the missing degree value will be larger, and the confidence level will also be smaller; (5) If there are both edge lines corresponding to complete true cracks and edge lines corresponding to incomplete true cracks inside the bounding box, then the larger the integrity degree value, the larger the missing degree value will be, and the confidence level will be smaller.

[0095] When the NMS filters the bounding boxes, it sorts the bounding boxes from high to low according to the confidence level and selects the bounding boxes with higher confidence levels as the objects to be retained. Then, the intersection over union (IoU) of the subsequent bounding boxes is calculated for elimination. However, based on the foregoing analysis, it can be seen that the confidence level of the bounding boxes when complete cracks and incomplete cracks coexist is small. Therefore, during the NMS operation, it may be misjudged as a redundant bounding box due to a large IoU. However, since there are complete crack defects inside, the threshold needs to be increased to retain the bounding box and avoid the bounding box with complete cracks being screened out, resulting in misjudgment or missed detection.

[0096] By analyzing the aforementioned 5 cases, it is not difficult to understand that among the three cases with a small confidence level, when complete cracks and incomplete cracks coexist in the bounding box, the confidence level and the integrity value will show a unique negative correlation. Therefore, the characteristics of this correlation can be used as an index to judge that the crack distribution in the bounding box is the 5th case, and combined with the integrity value and the missing degree value of the bounding box, the adjustment weight of the bounding box is determined, which helps to obtain an adaptive adjustment threshold subsequently.

[0097] Preferably, in an embodiment of the present invention, the method for obtaining the adjustment weight includes:

[0098] In the embodiment of the present invention, it mainly aims at the case where complete cracks and incomplete cracks coexist in the bounding box. Therefore, in each bounding box, the edge line corresponding to the maximum missing factor is used as the target edge line. At this time, each bounding box with incomplete cracks corresponds to a target edge line. The bounding boxes whose target edge lines belong to the same edge line in the component image are classified as the same type of bounding boxes; the purpose of classifying the bounding boxes is mainly to screen the bounding boxes for reference, and the bounding boxes with the same edge line have higher reference value.

[0099] Then, in each type of bounding box, according to the integrity value and the missing degree value of each bounding box, the reference weight of each bounding box is determined. The integrity value and the reference weight are positively correlated, and the missing degree value and the reference weight are negatively correlated. The formula model of the reference weight includes:

[0100]

[0101] Wherein, CQ represents the reference weight; QS represents the missing degree value; WZ represents the integrity value; ε represents a preset first parameter.

[0102] In the formula model of the reference weight, considering that the reference value of the bounding box with more complete real cracks should be higher, the integrity value is used as the numerator part and the missing degree value is used as the denominator part, so as to obtain the reference weight of each bounding box. The larger the reference weight, the higher the integrity degree of the cracks in the bounding box, the higher the possibility of containing only one crack, the lower the interference, and the greater the reference value.

[0103] It should be noted that the function of presetting the first parameter ε is to prevent the denominator from being 0, and it can take the value of 0.001. The specific value can be adjusted according to the implementation scenario and is not limited here.

[0104] Arbitrarily select a bounding box as the target bounding box, and regard the remaining bounding boxes of the same category as the reference bounding boxes as the reference bounding boxes, so as to analyze the correlation between the confidence of the bounding box and the integrity value:

[0105] Take the difference in confidence between the target bounding box and each reference bounding box as the confidence deviation factor, and take the difference in integrity value between the target bounding box and each reference bounding box as the integrity deviation factor. When the confidence deviation factor and the integrity deviation factor have the same sign at this time, it can be explained that there is a positive correlation between the confidence and the integrity value. Therefore, take the product of the confidence deviation factor and the integrity deviation factor as the positive correlation factor. The larger the positive correlation factor, the lower the possibility of coexistence of complete cracks and incomplete cracks in the target bounding box.

[0106] Use the reference weights of all reference bounding boxes corresponding to the target bounding box to perform weighted fusion and normalization processing on the positive correlation factor to obtain the positive correlation coefficient: Multiply the value obtained by normalizing the reference weights of the reference bounding boxes using the softmax() function by the positive correlation factor corresponding to the reference bounding box to obtain the weighted correlation factor, and take the value obtained by normalizing the sum of the weighted correlation factors of all reference bounding boxes as the positive correlation coefficient. The larger the weighted correlation factor, the larger the positive correlation coefficient, indicating that the reference value of the reference bounding box is higher, and the lower the possibility of coexistence of complete cracks and incomplete cracks in the target bounding box. Therefore, perform a negative correlation mapping process on the positive correlation coefficient to correct the logical relationship, so as to obtain the adjustment weight of the target bounding box. The larger the adjustment weight at this time, the greater the possibility of coexistence of complete cracks and incomplete cracks in the target bounding box. The negative correlation mapping here can adopt the formula 1 - x, where x represents the independent variable. Then, in order to avoid erroneously screening out this target bounding box in the subsequent NMS process, the initial threshold can be adjusted based on the adjustment weight of this bounding box.

[0107] Step S4: When using NMS and the confidence of the bounding box to screen the bounding box, adjust the preset initial threshold based on the adjustment weight of the bounding box to obtain an adjusted threshold for screening, so as to obtain the crack bounding box.

[0108] When using NMS (Non-Maximum Suppression) to screen bounding boxes, calculate the intersection over union (IoU) between the bounding box with the highest confidence and any other bounding box. The IoU is the intersection of the two bounding boxes divided by their union, which is used to quantify the overlapping degree of the bounding boxes. Use a preset initial threshold to screen out the bounding boxes with a relatively high IoU. If it exceeds the preset initial threshold, the bounding box is considered redundant and needs to be removed.

[0109] Based on the foregoing steps, the adjustment weights of all bounding boxes can be obtained, and the adjustment weights to a certain extent reflect the possibility of coexistence of complete cracks and incomplete cracks in the bounding box. Given that the bounding box with the coexistence of complete cracks and incomplete cracks can also reflect the location of the crack defect, so in this step, the preset initial threshold in the NMS process can be adjusted using the adjustment weights of the bounding boxes to obtain the adjusted threshold corresponding to each bounding box to determine whether it is a redundant bounding box, so as to screen out the final crack bounding boxes.

[0110] Preferably, in an embodiment of the present invention, the method for obtaining the adjusted threshold includes:

[0111] Multiply the value obtained by performing a negative correlation mapping on the preset initial threshold by the adjustment weight of each bounding box to obtain the adjustment amplitude. Take the sum of the preset initial threshold and the adjustment amplitude corresponding to each bounding box as the adjusted threshold for each bounding box. The formula model of the adjusted threshold includes:

[0112] TY = CY+(1 - CY)×TQ

[0113] Where, TY represents the adjusted threshold for each bounding box; CY represents the preset initial threshold; TQ represents the adjustment weight.

[0114] In the formula model of the adjusted threshold, performing a negative correlation mapping process on the preset initial threshold gives (1 - CY). The role of this process is to ensure that the final adjusted threshold is not greater than 1. Then, when the adjustment weight is larger, it means that the possibility of coexistence of complete cracks and incomplete cracks in the bounding box is higher. Then, in order to keep this bounding box, the threshold in the NMS process for this bounding box should be increased. Therefore, multiply (1 - CY) by the adjustment weight to obtain the adjustment amplitude (1 - CY)×TQ. The larger the adjustment amplitude, the greater the degree to which the threshold needs to be increased. Finally, take the sum of the adjustment amplitude and the preset initial threshold as the adjusted threshold for the bounding box. The larger the adjusted threshold, the higher the possibility of coexistence of complete cracks and incomplete cracks in this bounding box.

[0115] It should be noted that the preset initial threshold in the embodiment of the present invention is 0.5. In other embodiments of the present invention, the preset initial threshold can be adjusted according to the implementation scenario and is not limited here.

[0116] Based on the foregoing steps, the adjustment threshold corresponding to each bounding box can be obtained, and then NMS can be used to screen the bounding boxes to obtain the crack bounding boxes: First, all the bounding boxes are sorted in descending order of confidence to obtain a sorted sequence. In the sorted sequence, the bounding box with the highest confidence is taken as the retained object. Then, for the bounding boxes to be traversed after the retained object in the sorted sequence, the intersection over union (IoU value) between the currently traversed bounding box to be traversed and the retained object is checked in turn. If the IoU is greater than the adjustment threshold of the bounding box to be traversed, it is considered that the currently traversed bounding box to be traversed is redundant and is removed; the above sorting and suppression steps are repeated until all the bounding boxes are processed. The final remaining set of bounding boxes is the result after NMS and is used as the crack bounding boxes.

[0117] An example of this process is as follows: Suppose there is a set containing 5 bounding boxes, and the sorted sequence after sorting in descending order of confidence is (A, B, C, D, E). Next, iterative calculations are performed: In the first round: A has the highest confidence and is taken as the retained object, and the remaining bounding boxes BCDE are taken as the bounding boxes to be traversed. If the IoU between A and B > the adjustment threshold corresponding to B, then B will be removed. The same applies to C, D, and E. If BD are removed in the first round (indicating that ABD detect the same target, and the bounding box A with the highest confidence is retained; while CE may detect another target), A is removed from the set and used as a crack bounding box. At this time, only the bounding boxes CE remain in the set, and then the second round begins: C is taken as the retained object, and E is taken as the bounding box to be traversed. If the IoU between C and E > the adjustment threshold corresponding to E, then E will be removed, and C is removed from the set and used as a crack bounding box. At this time, the set of bounding boxes is empty and the process stops. So far, two crack bounding boxes, namely A and E, are obtained among the 5 bounding boxes.

[0118] It should be noted that the NMS process is a well-known technology, and only a brief description of its process is given in the embodiments of the present invention for illustration.

[0119] In summary, component images are obtained, and bounding boxes and corresponding confidence levels in the component images are obtained based on a pre-trained neural network. To avoid misjudgment or missed detection when using NMS to filter the bounding boxes, since there may be different crack distribution characteristics in the bounding boxes, it is necessary to determine the specific crack distribution in the bounding boxes. Given that cracks have a certain extension characteristic, that is, the gray-scale change shows an obvious gradual change characteristic, in each bounding box, the position distribution characteristics and gray-scale change of the edge pixel points are compared to determine the crack prominence value of the edge line, which can reflect the degree of manifestation of the edge line as a crack. The confidence level of the bounding box is usually related to the positioning accuracy of the bounding box, so the integrity of the edge line representing the crack in the bounding box can be further judged, thereby quantifying the integrity value and missing value of the bounding box. Specifically, the number distribution characteristics of the edge pixel points can be analyzed and quantified in combination with the crack prominence value of the edge line. Then, based on the integrity value, missing value of the bounding box, and the correlation between the integrity value and the confidence level, the possibility of multiple crack defects in each bounding box is analyzed to obtain the adjustment weight of each bounding box. Finally, in order to reduce the possibility that the bounding boxes with multiple crack defects are simply judged as a single incomplete large defect and filtered out, when using NMS to filter the bounding boxes, the preset initial threshold is adjusted according to the adjustment weight of each bounding box to obtain an adaptive adjustment threshold for each bounding box, so as to perform more accurate filtering of the bounding boxes, obtain the final crack bounding boxes, and effectively reduce the occurrence probability of misjudgment or missed detection.

[0120] The embodiment of the present invention further provides an automatic defect recognition system for key cast components of an automobile. Please refer to Figure 4 , which shows a system block diagram, including a data acquisition module 401 for implementing step S1 in the above method embodiment; a crack analysis module 402 for implementing step S2 in the above method embodiment; a weight adjustment module 403 for implementing step S3 in the above method embodiment; and a crack bounding box determination module 404 for implementing step S4 in the above method embodiment.

[0121] It should be noted that the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, an automatic defect recognition system for key cast components of an automobile and an embodiment of an automatic defect recognition method for key cast components of an automobile provided in the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0122] Please refer to Figure 5, which shows a schematic structural diagram of an automatic defect recognition system for key cast components of an automobile provided by an embodiment of the present invention, including a processor 500, a memory 501, a bus 502, and a communication interface 503. The processor 500, the communication interface 503, and the memory 501 are connected through the bus 502. Among them, the memory 501 may include a high-speed random access memory. The bus 502 may be an ISA bus, a PCI bus, an EISA bus, etc. The processor 500 may be an integrated circuit chip with signal processing capabilities. The memory 501 stores at least one instruction, at least one program, a code set, or an instruction set. When at least one instruction, at least one program, a code set, or an instruction set is loaded and executed by the processor, the steps in an automatic defect recognition method for key cast components of an automobile are implemented.

[0123] It should be noted that the above sequence of embodiments of the present invention is only for description and does not represent the superiority or inferiority of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0124] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. An automatic defect recognition method for key cast components of an automobile, characterized in that, The method includes: Obtain a component image, and based on a pre-trained neural network, obtain all bounding boxes in the component image and the confidence levels corresponding to the bounding boxes; Obtain the edge lines in the component image; in each bounding box, compare the position distribution characteristics and gray-scale change conditions of the edge pixel points between the edge lines to determine the crack prominence value of each edge line; Based on the crack prominence values of the edge lines in each bounding box, and in combination with the quantity distribution characteristics of the edge pixel points on the edge lines, respectively determine the integrity value and the missing value of each bounding box; comprehensively analyze the integrity value, the missing value of the bounding box, and the correlation between the integrity value and the confidence level, so as to determine the adjustment weight of each bounding box; When screening the bounding boxes by using NMS and the confidence levels of the bounding boxes, adjust a preset initial threshold based on the adjustment weight of the bounding box to obtain an adjusted threshold for screening, so as to obtain the crack bounding boxes.

2. The automatic defect recognition method for key casting parts of an automobile according to claim 1, characterized in that, The method for obtaining the crack prominence value includes: In each bounding box, take all the end points of each edge line as target points; Segment each edge line to obtain a plurality of line segments, and take the mean value of the gray-scale gradients of all the edge pixel points on each line segment as the gray-scale change factor; On each edge line, analyze the difference situation of the gray-scale change factors between the line segments to determine the crack manifestation value of each edge line; Calculate the difference between the crack manifestation values of each edge line and each of the remaining edge lines as the prominence factor; The value obtained by normalizing the sum value of all the prominence factors corresponding to each edge line is used as the crack prominence value of each edge line.

3. The automatic defect recognition method for key cast components of an automobile according to claim 2, characterized in that, The segmenting each edge line to obtain a plurality of line segments includes: On each edge line, perform clustering analysis on the edge pixel points based on the DBSCAN clustering algorithm to obtain a plurality of clustering clusters, where the neighborhood radius and the minimum number of points are both preset values; Form the edge pixel points in each clustering cluster into a line segment.

4. The automatic defect recognition method for key cast components of an automobile according to claim 2, characterized in that, The method for obtaining the crack manifestation value includes: On each edge line, for any one line segment, take the line segments passed between this line segment and each target point as comparison segments, calculate the difference between the gray-scale change factor of this line segment and that of each comparison segment as the gray-scale change difference, and take the mean value of the gray-scale change differences between this line segment and all the comparison segments as the crack trend factor; Take the maximum value of all the crack trend factors corresponding to this line segment as the crack trend degree value of this line segment; On each edge line, perform weighted fusion on the crack trend degree value by using the gray-scale change factor of the line segment to obtain the crack manifestation value of each edge line.

5. The automatic defect recognition method for key cast components of an automobile according to claim 1, characterized in that, The respectively determining the integrity value and the missing value of each bounding box based on the crack prominence values of the edge lines in each bounding box and in combination with the quantity distribution characteristics of the edge pixel points on the edge lines includes: In the component image, take the number of edge pixel points on each edge line as the total quantity value; In each bounding box, take the ratio of the number of edge pixel points of each edge line within the bounding box to the corresponding total quantity value as the integrity coefficient of each edge line; In each bounding box, the value obtained by normalizing the product of the integrity coefficient of each edge line and the crack prominence value is used as the integrity factor of each edge line, and the maximum integrity factor of all edge lines in the bounding box is used as the integrity degree value of the bounding box; In each bounding box, the value obtained by multiplying the value obtained by negatively correlating and normalizing the integrity coefficient of each edge line with the crack prominence value, and then normalizing the obtained product, is used as the missing factor of each edge line, and the maximum missing factor of all edge lines in the bounding box is used as the missing degree value of the bounding box.

6. The automatic defect recognition method for key casting parts of an automobile according to claim 5, wherein The method for obtaining the adjustment weight includes: In each bounding box, the edge line corresponding to the maximum missing factor is used as the target edge line; The bounding boxes whose target edge lines belong to the same edge line in the component image are grouped into the same category of bounding boxes; In each category of bounding boxes, according to the integrity degree value and the missing degree value of each bounding box, the reference weight of each bounding box is determined. The integrity degree value and the reference weight are positively correlated, and the missing degree value and the reference weight are negatively correlated; Optionally select a bounding box as the target bounding box, and use the remaining bounding boxes of the same category as the reference bounding boxes as the target bounding box; The difference in confidence between the target bounding box and each reference bounding box is used as the confidence deviation factor, the difference in integrity degree value between the target bounding box and each reference bounding box is used as the integrity deviation factor, and the product of the confidence deviation factor and the integrity deviation factor is used as the positive correlation factor; The positive correlation factor is weighted and fused and normalized by using the reference weights of all reference bounding boxes corresponding to the target bounding box to obtain a positive correlation coefficient, and the value obtained by negatively correlating the positive correlation coefficient is used as the adjustment weight of the target bounding box.

7. The automatic defect recognition method for key cast components of an automobile according to claim 1, characterized in that, The method for obtaining the adjustment threshold includes: Multiply the value obtained by negatively correlating the preset initial threshold by the adjustment weight of each bounding box to obtain the adjustment amplitude; The sum value of the preset initial threshold and the corresponding adjustment amplitude of each bounding box is used as the adjustment threshold of each bounding box.

8. The automatic defect recognition method for key cast components of an automobile according to claim 1, characterized in that, The preset initial threshold is 0.

5.

9. The automatic defect recognition method for key casting parts of an automobile according to claim 1, characterized in that The method for obtaining the edge line includes: Process the component image based on the Canny operator to obtain all edge lines.

10. An automatic defect recognition system for key cast components of an automobile, characterized in that, It includes a processor and a memory. At least one instruction, at least one program, a code set or an instruction set is stored in the memory. When at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor, the steps of an automatic defect recognition method for key cast components of an automobile as described in any one of claims 1-9 are implemented.

Images