A deep learning-based automatic detection and evaluation method for FDS joint forming quality indicators

Abstract

The application discloses a kind of FDS joint forming quality index automatic detection and evaluation method based on deep learning, and the profile morphology test figure of FDS joint is obtained by test, and it is input into the profile morphology automatic segmentation module trained, and the automatic segmentation figure is output;Automatic segmentation figure is input into forming quality index automatic measurement module, and the upper plate, lower plate, screw and scale area are identified, then the scale is detected and the conversion ratio of pixel to millimeter is determined, then the binary mask extraction of upper plate, lower plate and screw is carried out, and finally the calculation of each forming quality index is completed on binary mask;FDS joint forming quality index is input into forming quality index automatic evaluation module, and the strategy of item compliance determination and comprehensive weighted score is used to evaluate each forming quality index.The present application significantly reduces the manpower investment and human error of obtaining FDS profile forming quality index, and improves the detection speed and process quality evaluation efficiency.

Description

Technical Field

[0001] This invention belongs to the field of connector forming quality analysis technology, specifically relating to an automatic detection and evaluation method for FDS connector forming quality indicators based on deep learning. Background Technology

[0002] Flow Drill Screw (FDS) joining technology is a key technology driving the development of multi-material hybrid vehicle bodies, especially suitable for joining box-type enclosed structures. The cross-sectional morphology of the joint directly affects its mechanical properties. The forming quality of FDS joints is typically evaluated using indicators such as placement gap, inter-plate gap, maximum inter-plate gap, effective thread count, and filler ratio. Therefore, rapid measurement and evaluation of forming indicators are important research areas for the quality evaluation and design of FDS joint joining processes.

[0003] While traditional methods for obtaining joint forming quality indicators through cross-sectional metallographic observation or numerical simulation are effective, they suffer from high costs and long cycles. Chinese patent (CN116630343A) discloses a deep learning-based method for segmenting cross-sectional images of rivetless riveted joints, and Chinese patent (CN113971676A) discloses a deep learning-based method for automatically segmenting cross-sectional images of connectors. Both methods primarily focus on cross-sectional image segmentation of connectors and indirectly improve the efficiency of measuring joint cross-sectional forming quality indicators. Chinese patent (CN121766133A) proposes a method for predicting connector forming quality; however, this method is based on geometric dimension measurement of the image and struggles to directly measure non-geometric dimension indicators such as the number of effective threads and fill rate in FDS joints.

[0004] In recent years, deep learning has become an important technical means for analyzing the forming quality of connectors. Existing technologies mostly focus on cross-sectional image segmentation or geometric dimension extraction. However, after obtaining the index values ​​from the metallographic test images of FDS joint cross-sections, there is still a lack of corresponding technical solutions for how to combine process specifications for quantitative evaluation and quickly generate evaluation output. Furthermore, on-site engineering still relies on manual experience for interpretation, resulting in low evaluation efficiency and susceptibility to human evaluation errors. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes an automatic detection and evaluation method for FDS joint forming quality indicators based on deep learning.

[0006] The present invention achieves the above-mentioned technical objectives through the following technical means.

[0007] An automatic detection and evaluation method for FDS joint forming quality indicators based on deep learning:

[0008] Experimental images of the cross-sectional morphology of the FDS joint were obtained through experiments and input into the trained automatic cross-sectional morphology segmentation module to output an automatic segmentation image of the FDS joint cross-sectional morphology.

[0009] The FDS joint cross-sectional morphology automatic segmentation map is input into the forming quality index automatic measurement module. The upper plate, lower plate, screw and scale area are identified according to the color / category convention. Then the scale is detected and the conversion ratio from pixel to millimeter is determined. Then the binary mask of the upper plate, lower plate and screw is extracted. Finally, the forming quality index is calculated on the binary mask.

[0010] The forming quality index of the FDS joint is input into the automatic evaluation module for forming quality index. The automatic evaluation module for forming quality index adopts a hybrid strategy that combines sub-item compliance judgment and comprehensive weighted score to evaluate each forming quality index.

[0011] The forming quality evaluation indicators include placement gap, plate-to-plate gap, maximum plate-to-plate gap, number of effective threads, and filling rate.

[0012] Furthermore, the automatic profile segmentation module adopts a U-Net architecture, which consists of an encoder and a decoder. The encoder uses EfficientNet-B4 as the backbone network and extracts multi-scale features through progressive downsampling layers and convolutional feature extraction layers. The decoder upsamples the deepest features through progressive upsampling layers and then splices and fuses them with the same-scale features from the encoder through skip connection layers.

[0013] Furthermore, the detection of the scale and determination of the pixel-to-millimeter conversion ratio specifically involves: extracting the region corresponding to the scale within the area of ​​the automatically segmented cross-sectional topography map, obtaining the binary edge map of that region, extracting the set of line segments, selecting the line segment with the longest Euclidean length as the scale line segment, and using the ratio of the nominal physical length of the scale to the pixel length of the line segment as the pixel-to-millimeter conversion ratio.

[0014] Furthermore, the binary mask extraction of the upper plate, lower plate, and screw is specifically as follows: based on the color / category convention, the upper plate, lower plate, and screw are segmented into corresponding binary mask images by performing interval thresholding in the HSV color space; wherein, for the area where the screw passes through both the upper and lower plates, two interval thresholding segments are used and then merged bitwise; when extracting the binary mask image of the lower plate, the area containing the scale in the automatic segmentation image of the cross-sectional shape is zeroed out by masking.

[0015] Furthermore, the calculation of various forming quality indicators on the binary mask includes the calculation of the placement gap. The placement gap is calculated as follows: the left and right measurement halves are divided by the left and right extreme values ​​and the midpoint of the maximum contour of the screw periphery; in the left or right measurement half, the column-direction scan of the repaired upper plate binary mask is performed to obtain the upper surface column set; after sorting the column set by the horizontal coordinate, the column subset within the (1 / 5, 2 / 3) division zone from the outside to the inside of the left or right upper plate is taken, and the point with the largest vertical coordinate of the division zone is taken as the reference horizontal line A; the lowest point of the annular disk structure is searched within the quarter-width column range on the left or right side of the screw head, and the reference horizontal line B is determined by the lowest point; the left or right placement gap is obtained by multiplying the vertical pixel distance between the two reference horizontal lines by the conversion ratio, and the average of the left and right placement gaps is taken as the output.

[0016] Furthermore, the calculation of various forming quality indicators on the binary mask includes the calculation of inter-plate gaps. The inter-plate gap calculation is as follows: the left and right measurement half-zones are divided by the left and right extreme values ​​and the midpoint of the maximum outer contour of the screw; the minimum ordinate of the upper plate is read at the leftmost and rightmost vertical line positions of the screw as the lower surface of the upper plate, and the maximum ordinate of the lower plate is read at the leftmost and rightmost vertical line positions of the screw as the upper surface of the lower plate. The difference between the minimum ordinate of the upper plate and the maximum ordinate of the lower plate on the same side is multiplied by the conversion ratio to obtain the inter-plate gaps of the left and right sides and the average is taken.

[0017] Furthermore, the calculation of various forming quality indicators on the binary mask includes the calculation of the maximum inter-plate gap. The calculation of the maximum inter-plate gap is as follows: the left and right measurement half-regions are divided by the left and right extreme values ​​and the midpoint of the maximum outline of the screw periphery; within the range of all pixel columns that appear simultaneously on the upper and lower plates, the vertical gap area between the lower edge of the upper plate and the upper edge of the lower plate is taken column by column, the gap area is combined into a binary image, the eight-neighbor connected component labeling is performed on the binary image, the independent gap area is identified, the connected component with the largest area is selected, and its maximum vertical span is taken as the maximum inter-plate gap on that side.

[0018] Furthermore, the calculation of various forming quality indicators on the binary mask includes the calculation of the effective number of threads. The calculation of the effective number of threads is as follows: the left and right measurement halves are divided by the left and right extreme values ​​and the midpoint of the maximum outer contour of the screw; the binary mask of the lower plate is scanned column by column on the left or right side, and the minimum value of the vertical coordinate of the upper surface point of each column is taken as a horizontal line, and the maximum value of the vertical coordinate of the lower surface point of each column is taken as a horizontal line, and the effective vertical axis range is defined by the two horizontal lines; the horizontal projection of the screw binary mask image is performed within the effective vertical axis range to enhance the tooth profile texture of the thread, and then the peak and valley detection is performed to identify the number of complete threads; after counting the number of threads on the left and right sides respectively, the smaller value is taken as the effective number of threads.

[0019] Furthermore, the calculation of various forming quality indicators on the binary mask includes the calculation of the fill rate. The fill rate is calculated as follows: the left and right measurement halves are divided by the left and right extreme values ​​and the midpoint of the maximum outer contour of the screw; rectangular areas are defined on the left and right sides of the screw, and the left and right boundaries of the rectangular areas are determined by the peaks of the effective threads of the screw and the effective threads of the lower plate. The upper and lower boundaries of the rectangles are determined by drawing horizontal lines from the intersections of the outermost vertical line of the screw and the upper and lower points of the lower plate; the total area of ​​the lower plate pixels and screw pixels within the rectangular area is calculated and divided by the total area of ​​the rectangular area to obtain the fill rate.

[0020] Furthermore, the automatic evaluation module for forming quality indicators includes an evaluation rule configuration unit, a sub-item judgment unit, and a comprehensive scoring unit. The evaluation rule configuration unit reads the evaluation rule configuration file, which records the threshold range of the sub-item status identifiers corresponding to each forming quality indicator. The sub-item status identifiers include qualified, warning, and unqualified. The sub-item judgment unit compares each forming quality evaluation indicator with its corresponding threshold range and outputs the sub-item status identifier. When the sub-item status identifier is qualified or warning, the comprehensive scoring unit maps each forming quality indicator to a sub-item score according to the interval and then performs a weighted summation to obtain a comprehensive score, thereby determining the forming quality indicator level.

[0021] The beneficial effects of this invention are as follows:

[0022] (1) This invention overcomes the problem that existing automatic measurement methods mainly rely on geometric dimension calculations, making it difficult to cover non-geometric indicators such as the number of effective threads and the filling rate in FDS connectors. Based on color / category conventions, the upper plate, lower plate, screws, and scale areas are identified. Then, the scale is detected and the pixel-to-millimeters conversion ratio is determined. Next, the binary masks of the upper plate, lower plate, and screws are extracted. Finally, the calculation of each forming quality indicator is completed on the binary mask. This invention can automatically and uniformly output the forming quality indicators of FDS connectors, improving the completeness of indicator coverage and engineering practicality.

[0023] (2) Based on the automatic detection of cross-sectional morphology test image indicators, this invention introduces an automatic evaluation mechanism that combines sub-item compliance judgment and comprehensive weighted score, and quickly outputs traceable FDS cross-sectional morphology quality evaluation conclusions, which is conducive to reducing manual measurement differences and improving the efficiency of batch cross-sectional morphology detection and process quality evaluation. Attached Figure Description

[0024] Figure 1 This is a flowchart of the method described in this invention;

[0025] Figure 2 This is a schematic diagram of the U-Net network structure in the automatic profile topography segmentation module used in this embodiment of the invention;

[0026] Figure 3(a) is a cross-sectional morphology test diagram in an embodiment of the present invention;

[0027] Figure 3(b) is a manually segmented mask diagram in an embodiment of the present invention;

[0028] Figure 3(c) is an automatic segmentation mask diagram in an embodiment of the present invention;

[0029] Figure 4(a) is a schematic diagram of screw area extraction using the forming quality index measurement module in the embodiment of the present invention;

[0030] Figure 4(b) is a schematic diagram of the upper plate area extraction of the forming quality index measurement module used in the embodiment of the present invention;

[0031] Figure 4(c) is a schematic diagram of the extraction of the lower plate area of ​​the forming quality index measurement module used in the embodiment of the present invention;

[0032] Figure 4(d) is a schematic diagram of the extraction of the gap area between the upper and lower plates on the left side of the forming quality index measurement module used in the embodiment of the present invention;

[0033] Figure 4(e) is a schematic diagram of the extraction of the gap area between the upper and lower plates on the right side of the forming quality index measurement module used in the embodiment of the present invention;

[0034] Figure 5(a) is a positioning diagram for measuring the placement gap in an embodiment of the present invention;

[0035] Figure 5(b) is a diagram showing the positioning of the inter-plate gap measurement in an embodiment of the present invention;

[0036] Figure 5(c) is a diagram showing the measurement and positioning of the maximum inter-plate gap in an embodiment of the present invention;

[0037] Figure 5(d) is a positioning diagram for filling rate measurement in an embodiment of the present invention;

[0038] Figure 5(e) is a positioning diagram for measuring the number of effective threads in an embodiment of the present invention. Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0040] Figure 1This invention provides an overall flowchart of an automatic detection and evaluation method for FDS joint forming quality indicators based on deep learning. The overall technical approach uses cross-sectional morphology test images obtained from FDS cross-sectional metallographic observation experiments as the object, and mainly includes an automatic cross-sectional morphology segmentation module, an automatic forming quality indicator measurement module, and an automatic forming quality indicator evaluation module. It should be noted that the automatic cross-sectional morphology segmentation module is used to process the cross-sectional morphology test images to quickly obtain cross-sectional morphology mask images, providing input for subsequent automatic measurements. The innovative focus of this embodiment lies in the rapid automatic measurement of forming quality indicators and the comprehensive evaluation mechanism for forming quality indicators. Specifically, it includes the following eight steps:

[0041] Step 1: Construct a profile topography segmentation dataset.

[0042] In this embodiment, metallographic observation experiments under different process parameters were used to obtain cross-sectional morphology test images of FDS joints of steel and aluminum plates. A manual segmentation tool was used to finely annotate and segment the cross-sectional morphology test images to obtain segmentation mask images. These manual segmentation mask images include five categories: background, upper plate, lower plate, screw, and ruler. The process parameters include upper plate thickness, lower plate thickness, FDS equipment spindle pressure and speed during the hot-melt drilling stage, FDS equipment spindle pressure and speed during the tapping and screwing stage, and FDS equipment spindle pressure and speed during the screw tightening stage. The manual segmentation tool includes Labelme, CVAT, or SAM interactive tools; in this embodiment, Labelme is preferred. The screw structure, based on the conventional structure, also has an outwardly protruding annular disc structure at the bottom of its head.

[0043] Figure 3(a) is a metallographic observation test diagram of the cross-sectional morphology of the FDS joint of steel and aluminum plates obtained under one of the process parameters, and Figure 3(b) is the segmentation mask diagram corresponding to Figure 3(a).

[0044] In this embodiment, the cross-sectional morphology segmentation dataset includes experimental images of the cross-sectional morphology of FDS joints of steel and aluminum plates obtained from metallographic observation experiments, as well as manually segmented and labeled cross-sectional morphology images (i.e., segmentation mask images). 80% of the total data in the cross-sectional morphology segmentation dataset is randomly selected as the training set, and the remainder is used as the test set for model training and validation.

[0045] Step 2: Build an automatic cross-sectional topography segmentation module.

[0046] In this embodiment, the automatic profile segmentation module adopts the U-Net architecture. The encoder of the U-Net architecture is directly initialized using pre-trained weight parameters from the Image Net dataset (the pre-trained weight parameters are sourced from the official PyTorch simulation library). The U-Net network structure is as follows: Figure 2 As shown.

[0047] In this embodiment, the U-Net architecture consists of an encoder and a decoder: the encoder uses EfficientNet-B4 as the backbone network and extracts multi-scale features through progressive downsampling layers and convolutional feature extraction layers; the decoder upsamples the deepest features through progressive upsampling layers and merges them with the same-scale features from the encoder through skip connection layers, thereby taking into account both global semantics and boundary details.

[0048] In this embodiment, the ruler category in the manual and automatic segmentation process is a line segment and rendered as a fixed line width area.

[0049] In this embodiment, training the automatic profile topography segmentation module specifically includes the following three steps:

[0050] Step 1: Data size standardization and preprocessing.

[0051] The original image (connector cross-sectional morphology experiment) was scaled to the module input size (1024×512) using nearest neighbor interpolation to maintain a consistent aspect ratio; in addition, the original image was normalized using Image Net mean-variance to adapt it to encoder training.

[0052] Step 2: Train the automatic profile topography segmentation module.

[0053] In this embodiment, the training parameters include: training optimizer, learning rate, batch size, training epochs, cross-entropy loss weights, and Dice loss weights.

[0054] In this embodiment, the Adam W optimizer is preferably used, and the encoder learning rate is set to 0.1 times the decoder learning rate. The learning rate scheduling preferably adopts a cosine annealing strategy. The mIoU (mean intersection-union ratio) of the validation set is used as the basis for early stopping to improve training efficiency and avoid overfitting.

[0055] In this embodiment, the training loss preferably adopts a combination of cross-entropy loss and Dice loss to balance pixel classification accuracy and segmentation stability of small targets / fine structures.

[0056] In this embodiment, the objective function of the automatic profile topography segmentation module is:

[0057]

[0058] In the formula, For the total value function, It is the model's cross-entropy loss. It is the Dice loss of the model. These are the weights of the model's cross-entropy loss. These are the Dice loss weights of the model. To determine the probability that the i-th pixel belongs to category c (i.e., background, top plate, bottom plate, screw, and ruler, which are five categories), To determine whether the i-th pixel truly belongs to category c (if it does), (otherwise it is 0). For the smoothing term, this embodiment takes... .

[0059] In this embodiment, the verification stage preferably outputs mIoU, mDice (average Dice coefficient), pixel accuracy, and IoU / Dice for each category, and can also output a confusion matrix to analyze the confusion between categories; at the same time, it can save the comparison map of the original image / predicted mask / ground mask for manual verification.

[0060] The input of the automatic segmentation module for cross-sectional morphology after training and verification is a three-channel RGB image (i.e., the cross-sectional morphology test image in step one), and the output is an automatic segmentation image (Figure 3(c)), which includes five categories: background, upper plate, lower plate, screw, and ruler.

[0061] Step 3: Verify the accuracy of the automatic profile segmentation module by comparing and analyzing the measurement indicators of the manual profile segmentation map and the automatic profile segmentation map.

[0062] The automatically segmented and manually segmented profile images were manually measured, and the results were compared and analyzed. Five sets of quality index measurement results from the automatically segmented profile images were randomly selected and compared with the corresponding results from the manually segmented profile images. The relative errors were calculated, and the comparison results are shown in Table 1. Note that the fill rate could not be manually measured and was not used for accuracy verification of the automatically segmented profile module in this embodiment.

[0063] Table 1

[0064]

[0065] As can be seen from Table 1, the measurement errors of the placement gap, maximum inter-plate gap, inter-plate gap and effective number of threads in both the automatic and manual segmentation diagrams are less than 10%, indicating that the automatic segmentation image obtained by the automatic segmentation module of cross-sectional morphology has good accuracy.

[0066] Step 4: Determine the FDS forming quality evaluation index.

[0067] Furthermore, the FDS forming quality evaluation index is selected based on its ability to characterize the fatigue life, preload reliability, impact fracture resistance, and ultimate bearing strength of the connection structure.

[0068] Furthermore, in this embodiment, the FDS forming quality evaluation indexes are selected as placement gap, plate gap, maximum plate gap, effective thread count, and filling rate to represent the forming quality of FDS.

[0069] Step 5: Build an automatic measurement module for forming quality indicators.

[0070] In this embodiment, the automatic measurement module for forming quality indicators takes the automatic segmentation map of the cross-sectional morphology as input and identifies the upper plate, lower plate, screw, and scale areas according to color / category conventions (in this embodiment, upper plate: blue, ID1; lower plate: yellow, ID2; screw: red, ID3; scale: yellow, ID4). The following operations are then performed sequentially: detecting the scale and determining the pixel-to-millimeter conversion ratio → extracting the binary mask for the upper plate, lower plate, and screw → performing quantitative calculations of each forming quality indicator on the binary mask according to geometric and morphological rules → structured output. The pixel-to-millimeter conversion is calibrated using the scale.

[0071] In this embodiment, the automatic measurement module for forming quality indicators is based on color threshold segmentation and geometric constraint rules, and specifically includes the following process:

[0072] (1) Ruler recognition and pixel-to-millimeter conversion.

[0073] First, in the upper right region (including the scale) of the automatic segmentation image of the cross-sectional morphology, the yellow area corresponding to the scale is extracted according to the HSV interval threshold segmentation. Then, Canny edge detection is performed on the yellow area to obtain a binary edge map. Then, probabilistic Hough line detection is used to extract the set of line segments. The line segment with the longest Euclidean length is selected as the scale line segment. The ratio of the nominal physical length of the scale (specified by the metallographic observation test equipment) to the pixel length of the line segment is used as the conversion ratio from pixel to millimeter.

[0074] (2) Region mask extraction.

[0075] Based on color / category conventions, interval thresholding is performed on the blue upper plate, yellow lower plate, and red screw in the HSV color space to obtain corresponding binary mask images. Specifically, for the area where the red screw passes through both the blue upper plate and the yellow lower plate, two interval thresholding segments are used, and the segments are either bitwise (operated independently on each pixel position of the two binary masks) or fused. When extracting the binary mask image of the lower plate, the upper right corner area of ​​the automatic segmentation image of the profile shape (the area where the scale is located) is zeroed out to eliminate the interference of the same color area of ​​the scale on the recognition of the geometric boundaries of the upper and lower plates.

[0076] (3) Calculation of the gap between the landing positions.

[0077] The left and right measurement halves are divided by the left and right extreme values ​​and the midpoint of the maximum outer contour of the screw. In the left (right) measurement half, morphological closing operation is first performed on the binary mask of the upper plate to repair small fractures. Then, column scanning is performed on the repaired mask to obtain the column set of the upper surface. After sorting the column set by the horizontal coordinate, the column subset within the (1 / 5, 2 / 3) division zone of the upper plate on the left (right) side is taken. The point with the largest vertical coordinate of the division zone is taken as the reference horizontal line A. The lowest point of the annular disk structure is searched within the quarter-width column range on the left (right) side of the screw head. The reference horizontal line B is determined by the lowest point. The left (right) side positioning gap is obtained by multiplying the vertical pixel distance between the two reference horizontal lines by the conversion ratio. The average of the left and right positioning gaps is taken as the output.

[0078] (4) Calculation of gap between plates.

[0079] The left and right measurement halves are divided by the left and right extreme values ​​and the midpoint of the maximum outer contour of the screw. The minimum ordinate of the upper plate is read at the leftmost and rightmost vertical line positions of the screw, respectively, as the lower surface of the upper plate. The maximum ordinate of the lower plate is read at the leftmost and rightmost vertical line positions of the screw, respectively, as the upper surface of the lower plate. The gap between the left and right plates is obtained by multiplying the difference between the minimum and maximum ordinates of the upper and lower plates on the same side by the conversion ratio and taking the average. In this implementation case, the gap between the plates is represented as 0 if it is less than 0.05mm.

[0080] (5) Calculation of maximum inter-plate gap.

[0081] The left and right measurement halves are divided by the left and right extreme values ​​and the midpoint of the maximum outer contour of the screw. Within the range of all pixel columns that appear simultaneously on the upper and lower plates, the vertical gap regions between the lower edge of the upper plate and the upper edge of the lower plate are taken column by column. These gap regions are combined into a binary image. Morphological closing operation is first performed on the binary image to bridge small breaks, and then eight-neighbor connected component labeling is performed to identify independent gap regions. The connected component with the largest area is selected, and its maximum vertical span is taken as the maximum inter-plate gap on that side. After performing the above operation on the left and right sides respectively, abnormal sides that exceed the normal range (e.g., greater than 1 mm) and excessively small values ​​that are less than the minimum effective threshold (e.g., 0.1 mm) are removed. Only the effective results are averaged as the final maximum inter-plate gap output.

[0082] (6) Calculation of the number of effective threads.

[0083] The left and right measurement halves are divided by the left and right extreme values ​​and the midpoint of the maximum outer contour of the screw. The binary mask of the lower plate is scanned column by column on the left (right) side. The minimum value of the vertical coordinate of the upper surface point of each column is taken as a horizontal line, and the maximum value of the vertical coordinate of the lower surface point of each column is taken as a horizontal line. The effective vertical axis range is defined by the two horizontal lines. The horizontal projection of the screw binary mask image is performed within the effective vertical axis range. After combining the vertical gradient amplitude of the grayscale image to enhance the tooth texture of the thread, the peak and valley detection is performed to identify the number of complete threads. The smaller value after counting on the left and right sides is taken as the effective number of threads.

[0084] (7) Fill rate calculation.

[0085] The left and right measurement halves are divided by the left and right extreme values ​​and the midpoint of the screw's outermost contour. Rectangular statistical areas are then defined on the left and right sides of the screw: the left and right boundaries of the rectangles are determined by the peaks of the effective threads and the effective threads on the lower plate, while the upper and lower boundaries are determined by horizontal lines drawn from the intersections of the outermost vertical line of the screw and the upper and lower points of the lower plate. The total area of ​​the lower plate pixels and screw pixels within this rectangular area (i.e., the total number of pixels occupied by both materials) is calculated, and this ratio, divided by the total area of ​​the rectangular area, is the fill rate.

[0086] (8) Visualization and structured output.

[0087] Based on the binary mask image, output a visualized image containing key lines and key point annotations, and output a structured data file containing all forming quality indicators for result traceability and error analysis.

[0088] Figure 4(a) is a schematic diagram of screw area extraction using the forming quality index measurement module in the embodiment of the present invention; Figure 4(b) is a schematic diagram of upper plate area extraction using the forming quality index measurement module in the embodiment of the present invention; Figure 4(c) is a schematic diagram of lower plate area extraction using the forming quality index measurement module in the embodiment of the present invention; Figure 4(d) is a schematic diagram of gap area extraction between upper and lower plates on the left side using the forming quality index measurement module in the embodiment of the present invention; Figure 4(e) is a schematic diagram of gap area extraction between upper and lower plates on the right side using the forming quality index measurement module in the embodiment of the present invention. Figure 5(a) is an automatic measurement and positioning diagram of the placement gap; Figure 5(b) is an automatic measurement and positioning diagram of the inter-plate gap; Figure 5(c) is an automatic measurement and positioning diagram of the maximum inter-plate gap; Figure 5(d) is an automatic measurement and positioning diagram of the fill rate; Figure 5(e) is an automatic measurement and positioning diagram of the number of effective threads.

[0089] Step 6: Verify the accuracy of the automatic measurement module for forming quality indicators.

[0090] The automatically measured forming quality indicators were compared and analyzed with the manually measured results of the cross-sectional metallographic experiments. Five sets of automatically segmented cross-sectional morphology images were randomly selected, and the measured quality indicators obtained by automatic measurement were compared with the manually measured results of the cross-sectional morphology test images under the corresponding process conditions. The relative errors were calculated, and the comparison results are shown in Table 2.

[0091] Table 2

[0092]

[0093] The results in Table 2 show that the automatic measurement method of the present invention has high accuracy, and the relative error between its measurement results and the manually measured values ​​is within 10%.

[0094] Step 7: Automatic evaluation and output of forming quality indicators.

[0095] In this embodiment, the automatic measurement results of the forming quality indicators output in step five are input into the automatic evaluation module for forming quality indicators. The automatic evaluation module for forming quality indicators adopts a hybrid strategy that combines sub-item compliance judgment with comprehensive weighted scoring: First, the evaluation rule configuration unit reads the evaluation rule configuration file, which records the sub-item status indicator threshold ranges corresponding to each forming quality indicator in a structured manner. The sub-item status indicators include qualified, warning, and unqualified. Then, the sub-item judgment unit compares the placement gap, plate gap, maximum plate gap, filling rate, and effective thread count obtained in step five with the corresponding threshold ranges and outputs the sub-item status indicators. When the sub-item status indicator is qualified or warning, the comprehensive scoring unit maps each forming quality indicator to sub-item scores according to the intervals and then performs a weighted sum to obtain the comprehensive score. The weighting coefficients can be configured according to the process stage or material grade. In this embodiment, when the forming quality indicator is in the qualified range, the sub-item score is 100 points; when the forming quality indicator is in the warning range, the sub-item score is 60 points. When the veto rule is met, the overall evaluation conclusion is directly unqualified. The evaluation results are linked with the measured structured data, sample numbers, and process metadata and then written into the evaluation report file for quality archiving and traceability.

[0096] In this embodiment, for ease of understanding, an example of the evaluation rule configuration is given in Table 3, and an example of the evaluation output for the same batch of samples as in Table 2 is given in Table 4. The thresholds in Table 3 are only examples; in actual applications, they are provided by process specifications or enterprise standard libraries and can be adjusted online.

[0097] Table 3 Evaluation Thresholds and Weight Configuration Table

[0098]

[0099] Table 4 Automatic Evaluation Results of Indicators

[0100]

[0101] In this embodiment, a batch processing workflow of "segmentation → measurement → evaluation → visualization → result summarization" is supported for single or multiple cross-sectional topography test images via script. The measurement and evaluation results are directly output as structured files, greatly shortening the measurement speed of FDS cross-sectional topography images. To verify the efficiency improvement of this method in engineering applications, the average time consumption for manually measuring 100 FDS cross-sectional topography test images is compared with the average time consumption for measuring 100 FDS cross-sectional topography test images using the method of this invention. The comparison results are shown in Table 5. It can be seen that this invention can quickly measure FDS cross-sectional topography test images, reducing manpower and improving efficiency.

[0102] Table 5

[0103]

[0104] Step 8: Obtain cross-sectional morphology test images of other FDS joints, input them into the automatic cross-sectional morphology segmentation module to obtain the automatic cross-sectional morphology segmentation image, then input them into the automatic forming quality index measurement module to obtain each forming quality index, and finally obtain the evaluation results through the automatic forming quality index evaluation module.

[0105] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for automatic detection and evaluation of FDS joint forming quality indicators based on deep learning, characterized in that: Experimental images of the cross-sectional morphology of the FDS joint were obtained through experiments and input into the trained automatic cross-sectional morphology segmentation module to output an automatic segmentation image of the FDS joint cross-sectional morphology. The FDS joint cross-sectional morphology automatic segmentation map is input into the forming quality index automatic measurement module, and the upper plate, lower plate, screw and scale area are identified according to the color / category convention. Then the scale is detected and the conversion ratio from pixel to millimeter is determined. Then the binary mask of the upper plate, lower plate and screw is extracted. Finally, the forming quality index is calculated on the binary mask. The forming quality index of the FDS joint is input into the automatic evaluation module for forming quality index. The automatic evaluation module for forming quality index uses a hybrid strategy that combines sub-item compliance judgment and comprehensive weighted score to evaluate each forming quality index. The FDS joint forming quality evaluation indicators include placement gap, plate-to-plate gap, maximum plate-to-plate gap, number of effective threads, and filling rate.

2. The automatic detection and evaluation method for FDS joint forming quality indicators according to claim 1, characterized in that, The automatic profile segmentation module adopts the U-Net architecture, which consists of an encoder and a decoder. The encoder uses EfficientNet-B4 as the backbone network and extracts multi-scale features through progressive downsampling layers and convolutional feature extraction layers. The decoder upsamples the deepest features through progressive upsampling layers and then splices and fuses them with the same-scale features from the encoder through skip connection layers.

3. The automatic detection and evaluation method for FDS joint forming quality indicators according to claim 1, characterized in that, The detection of the scale and determination of the pixel-to-millimeter conversion ratio are specifically as follows: in the area containing the scale in the automatic segmentation map of the cross-sectional shape, the area corresponding to the scale is extracted, the binary edge map of the area is obtained, and then the set of line segments is extracted. The line segment with the longest Euclidean length is selected as the scale line segment, and the ratio of the nominal physical length of the scale to the pixel length of the line segment is used as the pixel-to-millimeter conversion ratio.

4. The automatic detection and evaluation method for FDS joint forming quality indicators according to claim 1, characterized in that, The extraction of the binary mask for the upper plate, lower plate, and screw is specifically as follows: based on the color / category convention, the upper plate, lower plate, and screw are segmented into corresponding binary mask images by performing interval thresholding in the HSV color space; wherein, for the area where the screw passes through both the upper and lower plates, two interval thresholding segments are used and then merged bitwise; when extracting the binary mask image of the lower plate, the area containing the scale in the automatic segmentation image of the cross-sectional shape is zeroed out by masking.

5. The automatic detection and evaluation method for FDS joint forming quality indicators according to claim 4, characterized in that, The calculation of various forming quality indicators on the binary mask includes the calculation of the placement gap. The placement gap is calculated as follows: the left and right measurement half-zones are divided by the left and right extreme values ​​and the midpoint of the maximum outline of the screw periphery; in the left or right measurement half-zone, the column-oriented scan of the repaired upper plate binary mask is performed to obtain the upper surface column set; after sorting the column set by the horizontal coordinate, the column subset within the (1 / 5, 2 / 3) division zone from the outside to the inside of the left or right upper plate is taken, and the point with the largest vertical coordinate of the division zone is taken as the reference horizontal line A; the lowest point of the annular disk structure is searched within the left or right quarter-width column range of the screw head, and the reference horizontal line B is determined by the lowest point; the left or right placement gap is obtained by multiplying the vertical pixel distance between the two reference horizontal lines by the conversion ratio, and the average of the left and right placement gaps is taken as the output.

6. The automatic detection and evaluation method for FDS joint forming quality indicators according to claim 4, characterized in that, The calculation of various forming quality indicators on the binary mask includes the calculation of the inter-plate gap. The inter-plate gap is calculated as follows: the left and right measurement half-zones are divided by the left and right extreme values ​​and the midpoint of the maximum outline of the screw's outer perimeter; the minimum ordinate of the upper plate is read at the leftmost and rightmost vertical line positions of the screw as the lower surface of the upper plate, and the maximum ordinate of the lower plate is read at the leftmost and rightmost vertical line positions of the screw as the upper surface of the lower plate. The difference between the minimum ordinate of the upper plate and the maximum ordinate of the lower plate on the same side is multiplied by the conversion ratio to obtain the inter-plate gap of the left and right sides and the average is taken.

7. The automatic detection and evaluation method for FDS joint forming quality indicators according to claim 4, characterized in that, The calculation of various forming quality indicators on the binary mask includes the calculation of the maximum inter-plate gap. The calculation of the maximum inter-plate gap is as follows: the left and right measurement half-regions are divided by the left and right extreme values ​​and the midpoint of the maximum outline of the screw periphery; within the range of all pixel columns that appear simultaneously on the upper and lower plates, the vertical gap area between the lower edge of the upper plate and the upper edge of the lower plate is taken column by column, and the gap area is combined into a binary image. The binary image is marked with eight neighbor connected components to identify independent gap areas. The connected component with the largest area is selected from them, and its maximum vertical span is taken as the maximum inter-plate gap on that side.

8. The automatic detection and evaluation method for FDS joint forming quality indicators according to claim 4, characterized in that, The calculation of various forming quality indicators on the binary mask includes the calculation of the effective number of threads. The calculation of the effective number of threads is as follows: the left and right measurement half-zones are divided by the left and right extreme values ​​and the midpoint of the maximum outer contour of the screw; the binary mask of the lower plate is scanned column by column on the left or right side, and the minimum value of the vertical coordinate of the upper surface point of each column is taken as a horizontal line, and the maximum value of the vertical coordinate of the lower surface point of each column is taken as a horizontal line, and the effective vertical axis range is defined by the two horizontal lines; the horizontal projection of the screw binary mask image is performed within the effective vertical axis range to enhance the tooth profile texture of the thread, and then the peak and valley detection is performed to identify the number of complete threads; after counting the number of threads on the left and right sides respectively, the smaller value is taken as the effective number of threads.

9. The automatic detection and evaluation method for FDS joint forming quality indicators according to claim 4, characterized in that, The calculation of various forming quality indicators on the binary mask includes the calculation of the fill rate. The fill rate is calculated as follows: the left and right measurement halves are divided by the left and right extreme values ​​and the midpoint of the maximum outer contour of the screw; rectangular areas are defined on the left and right sides of the screw, and the left and right boundaries of the rectangular areas are determined by the peaks of the effective threads of the screw and the effective threads of the lower plate. The upper and lower boundaries of the rectangles are determined by drawing horizontal lines from the intersections of the outermost vertical line of the screw and the upper and lower points of the lower plate; the total area of ​​the lower plate pixels and screw pixels within the rectangular area is calculated and divided by the total area of ​​the rectangular area to obtain the fill rate.

10. The automatic detection and evaluation method for FDS joint forming quality indicators according to claim 1, characterized in that, The automatic evaluation module for forming quality indicators includes an evaluation rule configuration unit, a sub-item judgment unit, and a comprehensive scoring unit. The evaluation rule configuration unit reads the evaluation rule configuration file, which records the threshold range of the sub-item status identifiers corresponding to each forming quality indicator. The sub-item status identifiers include qualified, warning, and unqualified. The sub-item judgment unit compares each forming quality evaluation indicator with its corresponding threshold range and outputs the sub-item status identifier. When the sub-item status identifier is qualified or warning, the comprehensive scoring unit maps each forming quality indicator to a sub-item score according to the interval and then performs a weighted summation to obtain a comprehensive score, thereby determining the forming quality indicator level.

Images