A deep learning-based detection result dynamic optimization display system and method

Abstract

This invention relates to the field of appearance inspection, specifically to a deep learning-based dynamic optimization and display system and method for inspection results. The system includes an appearance inspection module, a vision processing module, a defect recognition module, a modeling and display module, and a 3D reconstruction module. The appearance inspection module acquires images of the workpiece surface; the vision processing module segments defect areas; the defect recognition module outputs feature parameters of defect samples; the modeling and display module establishes a visualized 3D model; and the 3D reconstruction module performs surface reconstruction and visualization output. This invention can identify various defects on crankshaft surfaces and has advantages such as low power consumption, low cost, easy installation, high detection accuracy, and good real-time performance. It improves the detection accuracy for small target workpieces, ensures network generalization ability, enhances the reliability of detection results, improves the accuracy of surface reconstruction and the efficiency of defect recognition, and facilitates manual reading and analysis.

Description

Technical Field

[0001] This invention relates to the field of appearance inspection, specifically to a system and method for dynamically optimizing and displaying inspection results based on deep learning. Background Technology

[0002] Industrial appearance inspection is a technique for examining the external features of product parts on industrial production lines to detect defects and determine whether the products meet quality standards. Appearance inspection typically involves manual visual inspection or machine vision inspection, and includes dimensional and assembly checks, surface defect detection, and printing and label inspection to identify surface defects in the workpiece. It is a crucial step in the precision manufacturing industry.

[0003] Because workpieces such as bearings, injection molded parts, and die-cast parts have complex appearances, small defect volumes, and large background interference, manual defect inspection suffers from high costs, low inspection efficiency, and low accuracy. Industrial cameras cannot completely cover the workpiece, and machine inspection requires a lot of cost for image acquisition. It is characterized by high difficulty in defect identification, difficult inspection process design, low accuracy, and cumbersome labeling process, resulting in insufficient automation of inspection.

[0004] In addition, the data obtained after inspecting the product surface needs to be presented to users in an intuitive way to help them understand the product and better adjust the production line. This process involves a precise modeling process of the workpiece. The existing modeling process and image annotation process are relatively cumbersome and have a low level of 3D reconstruction of complex curved surfaces. Summary of the Invention

[0005] The purpose of this invention is to provide a system and method for dynamically optimizing and displaying detection results based on deep learning, so as to solve the problems mentioned in the background art.

[0006] To address the aforementioned technical problems, the present invention provides the following technical solution: a deep learning-based dynamic optimization and display system for detection results, comprising: an appearance detection module, a visual processing module, a defect identification module, a modeling and display module, and a three-dimensional reconstruction module; The appearance inspection module is used to place the workpiece to be inspected on a tray, and through a compensating light source, a variable background plate and a displacement camera, collect the appearance end faces of the workpiece, stitch the end faces of the workpiece, remove non-detection target areas, and obtain the workpiece surface image. The vision processing module is used to convert the workpiece crankshaft surface into a rectangular region through image polar coordinate transformation, stretch the convex defects into line segments, detect cracks or convex defects on the crankshaft surface through height feature screening, recover the image through inverse coordinate transformation to obtain the defect region, extract the standard ROI region containing the most defect points based on the roundness and area of ​​the defect region, perform edge extraction on the obtained ROI region, segment the defect region and determine the structural element size. The defect identification module is used to acquire images of the inner wall of the workpiece using dark field illumination, train a scale discrimination network based on the connected components of the end face ROI, segment the defect area of ​​the inner wall of the workpiece using a contour tracking algorithm, obtain defect samples, and output the feature parameters of the defect samples. The modeling and display module is used to obtain the surface contour of the part at the current angle, group the contours, calculate the minimum bounding boundary of each group of contours, divide each group of contours according to the minimum bounding boundary, generate a local triangular mesh, increase the weight of the defect position in the triangular mesh according to the feature parameters of the defect sample, and construct the contour convex hull by scanning the mesh point by point to obtain a visualized three-dimensional model of the workpiece. The 3D reconstruction module is used to divide the model surface. During the division process, the color values ​​of defect nodes are set according to the mesh weight. The color of the 3D surface is reconstructed by inverse distance interpolation, and the surface reconstruction is completed and visualized.

[0007] Furthermore, the appearance inspection module includes: a contour registration unit and an end face positioning unit; The contour registration unit is used to acquire the inner hole contour of the workpiece, detect the image area where the inner hole center is located, determine the assembly position, perform circle fitting on the inner hole center point, locate the inner hole position, establish the transformation function of the base coordinate system, pallet coordinate system and sensor coordinate system, adjust the camera acquisition parameters according to the previous recognition results, and compensate and correct the pallet connection error and equipment assembly error. The end face positioning unit is used to reflect the center of vision of each end face onto the parallel surface of the tray surface, stitch together the corrected end face images to form a crankshaft surface unfolded image, enhance image contrast through homomorphic filtering, perform threshold segmentation and contour detection on the workpiece template, and determine the integrity of the workpiece surface image.

[0008] Furthermore, the vision processing module includes: a coordinate transformation unit, a region detection unit, and a defect segmentation unit; The coordinate transformation unit is used to perform polar coordinate transformation with the crankshaft axis inside the workpiece as the polar axis, the circumferential direction as the rectangular horizontal direction, and the axial mapping direction as the vertical direction to obtain line segments on the image. Based on the geometric features of the defect and the local gray-level gradient, the defect line segments are selected, and the defect area is determined by inverse transformation. The region detection unit is used to construct a sub-image for the ROI region, add auxiliary surfaces as boundary conditions, obtain the region boundary through depth traversal, fit the region background image through the line grayscale distribution curve, subtract the ROI region image from the background image, search for the maximum value of the subtraction result, segment out the small defects, and sharpen and smooth the defect region. The defect segmentation unit is used to eliminate holes in areas connected to the background through opening and closing operations, draw a polygonal diagram of the surface width and excess height of the defect area, perform point cloud detection on the surface appearance, analyze the inner and outer contour diameters of the workpiece, classify defects according to their distribution range, and use the defect area image as a dataset to train an image classification model.

[0009] Furthermore, the defect identification module includes: a network training unit, a frequency domain difference unit, and a feature detection unit; The network training unit is used to detect corner points and extract the inner wall ROI. The Canny operator is used to extract the edge of the inner wall ROI, adjust the kernel binding coefficient and membership adjustment coefficient, segment the defects in the inner wall and end face ROI, and fill the defect voids. The frequency domain difference unit is used to transform the image to the frequency domain, process the frequency domain image with a low-pass filter, and inversely transform to obtain a low-frequency background image. The detection area is divided into partitions and different thresholds are set. The difference is obtained from the original image to obtain a defect candidate image, extract the defects, and label the inner wall defect dataset. The feature detection unit is used to randomly extract intact samples from non-defective regions, couple the difference features between intact samples and defective samples to the input of the generator, feed them back to the generative adversarial network, and output feature parameters, which include the area, perimeter, aspect ratio, and grayscale mean of the defective samples.

[0010] Furthermore, the modeling and display module includes: a boundary detection unit and a mesh modeling unit; The boundary detection unit is used to perform triangulation by scanning the workpiece with a laser, use a vertical filter to filter the image mean, a horizontal filter to weaken the boundary, remove invalid boundaries of the contour line, locate the rotation detection area, eliminate common edges containing the current point, and extract the workpiece boundary by setting a threshold. The mesh modeling unit is used to perform two-dimensional triangulation on the point set in each group to generate local triangular meshes. At each vertex of the triangular mesh, a weight is set according to the distance from the center of the defect region and the area of ​​the defect region. The vertex density is adjusted according to the weight, all local triangular meshes are merged, and a visualization model is output.

[0011] Furthermore, the three-dimensional reconstruction module includes: a surface subdivision unit and a visualization processing unit; The surface subdivision unit is used to perform planar mapping subdivision of horizontal surfaces in the three-dimensional model, reconstruct the surface shape, perform three-dimensional convex shell subdivision of rotating surfaces or discontinuous surfaces, fuse features of different scales, predict the occupancy rate of points based on the spatial location information of points, and extract triangular meshes using the moving cube algorithm. The visual processing unit is used to guide the 3D reconstruction process according to the calibrated camera intrinsic and extrinsic parameters, perform texture mapping and 3D reconstruction on the acquired images, restore the 3D structure layer by layer from the boundary of the 2D region according to the surface topology, add direction and order parameters to each surface edge, and reconstruct the model.

[0012] A method for dynamically optimizing and displaying detection results based on deep learning includes the following steps: Step S1. Place the workpiece to be tested on the tray, collect images of each of the workpiece's exterior end faces, stitch together the end faces of each workpiece, remove non-detection target areas, and obtain the workpiece surface image. Step S2. By transforming the image into polar coordinates, the surface of the workpiece crankshaft is converted into a rectangular region. Cracks or bumps on the crankshaft surface are detected by height feature screening. The image is restored by inverse transformation to obtain the defect region. The standard ROI region containing the most defect points is extracted and edge extraction is performed to obtain the defect region. Step S3. Acquire images of the inner wall of the workpiece, train a scale discriminant network based on the connected components of the end face ROI, segment the defect region of the inner wall of the workpiece, obtain defect samples, detect and output the feature parameters of the defect samples; Step S4. Obtain the surface contour of the part, group the contours, calculate the minimum bounding boundary of each group of contours, generate a local triangular mesh according to the minimum bounding boundary, adjust the weight of the defect position in the triangular mesh according to the feature parameters of the defect sample, scan the mesh point by point to construct the contour convex hull, and obtain the visualized three-dimensional model of the workpiece. Step S5. Mesh the surface of the 3D model. During the meshing process, set the color values ​​of the defect nodes according to the mesh weights, reconstruct the color of the 3D surface using inverse distance interpolation, complete the surface reconstruction, and output the visualization.

[0013] Furthermore, step S1 includes: Step S11. Obtain the outline of the inner hole of the workpiece, detect the image area where the center of the inner hole is located, determine the assembly position, and perform circle fitting on the center point of the inner hole to locate the position of the inner hole. Establish the transformation function of the base coordinate system, the pallet coordinate system and the sensor coordinate system. Adjust the camera acquisition parameters according to the previous recognition results and compensate and correct the pallet connection error and the equipment assembly error. Step S12. Reflect the center of vision of each end face onto the parallel surface of the tray surface, stitch together the corrected images of each end face to form a crankshaft surface unfolded image, enhance image contrast through homomorphic filtering, perform threshold segmentation and contour detection on the workpiece template, and determine the integrity of the workpiece surface image.

[0014] Furthermore, step S2 includes: Step S21. Using the crankshaft axis inside the workpiece as the polar axis, the circumferential direction as the horizontal direction of the rectangle, and the axial mapping direction as the vertical direction, perform polar coordinate transformation to obtain line segments on the image. Based on the geometric features of the defect and the local grayscale gradient, filter the defect line segments and perform inverse transformation to determine the defect area. Step S22. Construct a sub-image for the ROI region, add auxiliary surfaces as boundary conditions, obtain the region boundary through depth traversal, fit the region background image through the line grayscale distribution curve, subtract the ROI region image from the background image, search for the maximum value of the subtraction result, segment out the small defects, and sharpen and smooth the defect area. Step S22. After opening and closing operations, eliminate holes in the area connected to the background, draw a polygonal diagram of the surface width and excess height of the defect area, perform point cloud detection on the surface appearance, analyze the inner and outer contour diameters of the workpiece, classify the defects according to the distribution range of the defects, and use the defect area image as a dataset to train an image classification model.

[0015] Furthermore, step S3 includes: Step S31. Detect corner points and extract inner wall ROI. Use the Canny operator to extract the edges of the inner wall ROI, adjust the kernel binding coefficient and membership adjustment coefficient, segment the defects in the inner wall and end face ROI, and fill the defect holes. After transforming the image to the frequency domain, use a low-pass filter to process the frequency domain image, and inverse transform to obtain the low-frequency background image. Divide the detection area into partitions and set different thresholds. Differ from the original image to obtain the defect candidate image, extract the defects, and label the inner wall defect dataset. Step S32. Randomly extract intact samples from non-defective regions, couple the difference features between intact samples and defective samples to the input of the generator, feed them back to the generative adversarial network, and output feature parameters, including the area, perimeter, aspect ratio, and grayscale mean of the defective samples.

[0016] Furthermore, step S4 includes: Step S41. Perform triangulation by scanning the workpiece with a laser, use a vertical filter to filter the image mean, use a horizontal filter to weaken the boundary, remove invalid boundaries of the contour line, locate the rotation detection area, eliminate common edges containing the current point, and extract the workpiece boundary by setting a threshold. Step S42. Perform two-dimensional triangulation on the point set in each group to generate local triangular meshes. Set weights at each vertex of the triangular mesh according to the distance from the center of the defect region and the area of ​​the defect region. Adjust the vertex density according to the weights, merge all local triangular meshes, and output a visualization model.

[0017] Furthermore, step S5 includes: Step S51. Perform planar mapping subdivision on the horizontal surface within the 3D model to reconstruct the surface shape. Perform 3D convex shell subdivision on the surface of revolution or discontinuous surface. Fuse features of different scales, predict the occupancy rate of points based on the spatial location information of points, and extract triangular meshes using the moving cube algorithm. Step S52. Based on the calibrated camera intrinsic and extrinsic parameters, guide the 3D reconstruction process, perform texture mapping and 3D reconstruction on the acquired images, restore the 3D structure layer by layer from the boundary of the 2D region according to the surface topology, add direction and order parameters to each surface edge, and reconstruct the model.

[0018] Compared with the prior art, the beneficial effects achieved by the present invention are: 1. This invention reflects the visual center of each end face onto a parallel surface of the tray surface, stitches together the end faces of each workpiece to obtain an image of the workpiece surface, and uses image polar coordinate transformation to segment out minute defects and identify the defect area ROI. It can adjust the light source angle according to the workpiece material and surface reflectivity to effectively identify various defects on the crankshaft surface. It has advantages such as low power consumption, low cost, convenient installation, high detection accuracy, and good real-time performance, effectively improving the detection accuracy for small target workpieces.

[0019] 2. This invention can train a scale-based discriminant network based on the connected components of the end face ROI, segment defects in the inner wall and the end face ROI, and fill defect voids, realizing the quantitative detection of defects in the inner wall of the pipeline. It balances detection efficiency and accuracy, ensures the network's generalization ability, improves the reliability of detection results, reduces workpiece maintenance costs, and enhances the operational accuracy and stability of appearance inspection.

[0020] 3. This invention reconstructs the three-dimensional surface shape of the model by performing planar mapping and subdivision of the horizontal surface within the three-dimensional model, adding direction and sequence parameters, and regularizing the planar and rotating regions. This reduces the impact of noise, improves the accuracy of surface reconstruction and the efficiency of defect identification, ensures the structural integrity of the model, facilitates manual reading and analysis, and enables quality traceability of the inspected products. Attached Figure Description

[0021] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the structure of a deep learning-based dynamic optimization and display system for detection results according to the present invention. Figure 2 This is a schematic diagram illustrating the steps of a method for dynamically optimizing and displaying detection results based on deep learning, as described in this invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please see Figures 1 to 2 The present invention provides a technical solution: a deep learning-based detection result dynamic optimization display system, comprising: an appearance detection module, a visual processing module, a defect recognition module, a modeling display module, and a three-dimensional reconstruction module; The appearance inspection module is used to place the workpiece to be inspected on a tray, and through a compensating light source, a variable background plate and a displacement camera, collect the appearance end faces of the workpiece, stitch the end faces of the workpiece, remove non-detection target areas, and obtain the workpiece surface image. The appearance inspection module includes: a contour registration unit and an end face positioning unit; The contour registration unit is used to acquire the inner hole contour of the workpiece, detect the image area where the inner hole center is located, determine the assembly position, perform circle fitting on the inner hole center point, locate the inner hole position, establish the transformation function of the base coordinate system, pallet coordinate system and sensor coordinate system, adjust the camera acquisition parameters according to the previous recognition results, and compensate and correct the pallet connection error and equipment assembly error. The end face positioning unit is used to reflect the center of vision of each end face onto the parallel surface of the tray surface, stitch together the corrected end face images to form a crankshaft surface unfolded image, enhance image contrast through homomorphic filtering, perform threshold segmentation and contour detection on the workpiece template, and determine the integrity of the workpiece surface image.

[0024] The vision processing module is used to convert the workpiece crankshaft surface into a rectangular region through image polar coordinate transformation, stretch the convex defects into line segments, detect cracks or convex defects on the crankshaft surface through height feature screening, recover the image through inverse coordinate transformation to obtain the defect region, extract the standard ROI region containing the most defect points based on the roundness and area of ​​the defect region, perform edge extraction on the obtained ROI region, segment the defect region and determine the structural element size. The vision processing module includes: a coordinate transformation unit, a region detection unit, and a defect segmentation unit; The coordinate transformation unit is used to perform polar coordinate transformation with the crankshaft axis inside the workpiece as the polar axis, the circumferential direction as the rectangular horizontal direction, and the axial mapping direction as the vertical direction to obtain line segments on the image. Based on the geometric features of the defect and the local gray-level gradient, the defect line segments are selected, and the defect area is determined by inverse transformation. The region detection unit is used to construct a sub-image for the ROI region, add auxiliary surfaces as boundary conditions, obtain the region boundary through depth traversal, fit the region background image through the line grayscale distribution curve, subtract the ROI region image from the background image, search for the maximum value of the subtraction result, segment out the small defects, and sharpen and smooth the defect region. The defect segmentation unit is used to eliminate holes in areas connected to the background through opening and closing operations, draw a polygonal diagram of the surface width and excess height of the defect area, perform point cloud detection on the surface appearance, analyze the inner and outer contour diameters of the workpiece, classify defects according to their distribution range, and use the defect area image as a dataset to train an image classification model.

[0025] The defect identification module is used to acquire images of the inner wall of the workpiece using dark field illumination, train a scale discrimination network based on the connected components of the end face ROI, segment the defect area of ​​the inner wall of the workpiece using a contour tracking algorithm, obtain defect samples, and output the feature parameters of the defect samples. The defect identification module includes: a network training unit, a frequency domain difference unit, and a feature detection unit; The network training unit is used to detect corner points and extract the inner wall ROI. The Canny operator is used to extract the edge of the inner wall ROI, adjust the kernel binding coefficient and membership adjustment coefficient, segment the defects in the inner wall and end face ROI, and fill the defect voids. The frequency domain difference unit is used to transform the image to the frequency domain, process the frequency domain image with a low-pass filter, and inversely transform to obtain a low-frequency background image. The detection area is divided into partitions and different thresholds are set. The difference is obtained from the original image to obtain a defect candidate image, extract the defects, and label the inner wall defect dataset. The feature detection unit is used to randomly extract intact samples from non-defective regions, couple the difference features between intact samples and defective samples to the input of the generator, feed them back to the generative adversarial network, and output feature parameters, which include the area, perimeter, aspect ratio, and grayscale mean of the defective samples.

[0026] The modeling and display module is used to obtain the surface contour of the part at the current angle, group the contours, calculate the minimum bounding boundary of each group of contours, divide each group of contours according to the minimum bounding boundary, generate a local triangular mesh, increase the weight of the defect position in the triangular mesh according to the feature parameters of the defect sample, and construct the contour convex hull by scanning the mesh point by point to obtain a visualized three-dimensional model of the workpiece. The modeling and display module includes: a boundary detection unit and a mesh modeling unit; The boundary detection unit is used to perform triangulation by scanning the workpiece with a laser, use a vertical filter to filter the image mean, a horizontal filter to weaken the boundary, remove invalid boundaries of the contour line, locate the rotation detection area, eliminate common edges containing the current point, and extract the workpiece boundary by setting a threshold. The mesh modeling unit is used to perform two-dimensional triangulation on the point set in each group to generate local triangular meshes. At each vertex of the triangular mesh, a weight is set according to the distance from the center of the defect region and the area of ​​the defect region. The vertex density is adjusted according to the weight, all local triangular meshes are merged, and a visualization model is output.

[0027] The 3D reconstruction module is used to divide the model surface. During the division process, the color values ​​of defect nodes are set according to the mesh weight. The color of the 3D surface is reconstructed by inverse distance interpolation, and the surface reconstruction is completed and visualized.

[0028] The three-dimensional reconstruction module includes: a surface segmentation unit and a visual processing unit; The surface subdivision unit is used to perform planar mapping subdivision of horizontal surfaces in the three-dimensional model, reconstruct the surface shape, perform three-dimensional convex shell subdivision of rotating surfaces or discontinuous surfaces, fuse features of different scales, predict the occupancy rate of points based on the spatial location information of points, and extract triangular meshes using the moving cube algorithm. The visual processing unit is used to guide the 3D reconstruction process according to the calibrated camera intrinsic and extrinsic parameters, perform texture mapping and 3D reconstruction on the acquired images, restore the 3D structure layer by layer from the boundary of the 2D region according to the surface topology, add direction and order parameters to each surface edge, and reconstruct the model.

[0029] A method for dynamically optimizing and displaying detection results based on deep learning includes the following steps: Step S1. Place the workpiece to be tested on the tray, collect images of each of the workpiece's exterior end faces, stitch together the end faces of each workpiece, remove non-detection target areas, and obtain the workpiece surface image. Step S1 includes: Step S11. Obtain the outline of the inner hole of the workpiece, detect the image area where the center of the inner hole is located, determine the assembly position, and perform circle fitting on the center point of the inner hole to locate the position of the inner hole. Establish the transformation function of the base coordinate system, the pallet coordinate system and the sensor coordinate system. Adjust the camera acquisition parameters according to the previous recognition results and compensate and correct the pallet connection error and the equipment assembly error. Step S12. Reflect the center of vision of each end face onto the parallel surface of the tray surface, stitch together the corrected images of each end face to form a crankshaft surface unfolded image, enhance image contrast through homomorphic filtering, perform threshold segmentation and contour detection on the workpiece template, and determine the integrity of the workpiece surface image.

[0030] Step S2. By transforming the image into polar coordinates, the surface of the workpiece crankshaft is converted into a rectangular region. Cracks or bumps on the crankshaft surface are detected by height feature screening. The image is restored by inverse transformation to obtain the defect region. The standard ROI region containing the most defect points is extracted and edge extraction is performed to obtain the defect region. Step S2 includes: Step S21. Using the crankshaft axis inside the workpiece as the polar axis, the circumferential direction as the horizontal direction of the rectangle, and the axial mapping direction as the vertical direction, perform polar coordinate transformation to obtain line segments on the image. Based on the geometric features of the defect and the local grayscale gradient, filter the defect line segments and perform inverse transformation to determine the defect area. Step S22. Construct a sub-image for the ROI region, add auxiliary surfaces as boundary conditions, obtain the region boundary through depth traversal, fit the region background image through the line grayscale distribution curve, subtract the ROI region image from the background image, search for the maximum value of the subtraction result, segment out the small defects, and sharpen and smooth the defect area. Step S22. After opening and closing operations, eliminate holes in the area connected to the background, draw a polygonal diagram of the surface width and excess height of the defect area, perform point cloud detection on the surface appearance, analyze the inner and outer contour diameters of the workpiece, classify the defects according to the distribution range of the defects, and use the defect area image as a dataset to train an image classification model.

[0031] Step S3. Acquire images of the inner wall of the workpiece, train a scale discriminant network based on the connected components of the end face ROI, segment the defect region of the inner wall of the workpiece, obtain defect samples, detect and output the feature parameters of the defect samples; Step S3 includes: Step S31. Detect corner points and extract inner wall ROI. Use the Canny operator to extract the edges of the inner wall ROI, adjust the kernel binding coefficient and membership adjustment coefficient, segment the defects in the inner wall and end face ROI, and fill the defect holes. After transforming the image to the frequency domain, use a low-pass filter to process the frequency domain image, and inverse transform to obtain the low-frequency background image. Divide the detection area into partitions and set different thresholds. Differ from the original image to obtain the defect candidate image, extract the defects, and label the inner wall defect dataset. Step S32. Randomly extract intact samples from non-defective regions, couple the difference features between intact samples and defective samples to the input of the generator, feed them back to the generative adversarial network, and output feature parameters, including the area, perimeter, aspect ratio, and grayscale mean of the defective samples.

[0032] Step S4. Obtain the surface contour of the part, group the contours, calculate the minimum bounding boundary of each group of contours, generate a local triangular mesh according to the minimum bounding boundary, adjust the weight of the defect position in the triangular mesh according to the feature parameters of the defect sample, scan the mesh point by point to construct the contour convex hull, and obtain the visualized three-dimensional model of the workpiece. Step S4 includes: Step S41. Perform triangulation by scanning the workpiece with a laser, use a vertical filter to filter the image mean, use a horizontal filter to weaken the boundary, remove invalid boundaries of the contour line, locate the rotation detection area, eliminate common edges containing the current point, and extract the workpiece boundary by setting a threshold. Step S42. Perform two-dimensional triangulation on the point set in each group to generate local triangular meshes. Set weights at each vertex of the triangular mesh according to the distance from the center of the defect region and the area of ​​the defect region. Adjust the vertex density according to the weights, merge all local triangular meshes, and output a visualization model.

[0033] Step S5. Mesh the surface of the 3D model. During the meshing process, set the color values ​​of the defect nodes according to the mesh weights, reconstruct the color of the 3D surface using inverse distance interpolation, complete the surface reconstruction, and output the visualization.

[0034] Step S5 includes: Step S51. Perform planar mapping subdivision on the horizontal surface within the 3D model to reconstruct the surface shape. Perform 3D convex shell subdivision on the surface of revolution or discontinuous surface. Fuse features of different scales, predict the occupancy rate of points based on the spatial location information of points, and extract triangular meshes using the moving cube algorithm. Step S52. Based on the calibrated camera intrinsic and extrinsic parameters, guide the 3D reconstruction process, perform texture mapping and 3D reconstruction on the acquired images, restore the 3D structure layer by layer from the boundary of the 2D region according to the surface topology, add direction and order parameters to each surface edge, and reconstruct the model.

[0035] Example: An image acquisition system is built to acquire images of the workpiece end face. Perspective correction and image registration are performed. The images of each end face are stitched together through centroidal reflection to form a unfolded image of the workpiece surface. Polar coordinate transformation is applied to the crankshaft surface. Edge detection is used to identify long crack defects with a fixed angle to the axis and convex line segments with a length below a threshold. Inverse coordinate transformation is performed to restore the defect area. Image enhancement and morphological processing are performed on the defect area. Region of Interest (ROI) is extracted. Edge extraction is performed on the ROI area. The image background is fitted. Difference and segmentation are performed relative to the background to determine the defect structural elements. The inner wall of the workpiece is illuminated at a low angle to extract the ROI of the end face. The scale discrimination network is trained to obtain the concave and convex defects of the inner surface. The contour tracking algorithm is used to extract all connected components of the defects. Corner point detection and precise ROI localization are performed on the inner wall region. GAN enhancement is used to extract the feature of the defect samples. The workpiece contour is grouped, the minimum bounding boundary is calculated, local triangular meshes are generated, defect weights are fused, the overall convex hull is constructed, and a visualization model is generated.

[0036] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0037] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for dynamically optimizing and displaying detection results based on deep learning, characterized in that, The method includes the following steps: Step S1. Place the workpiece to be tested on the tray, collect images of each of the workpiece's exterior end faces, stitch together the end faces of each workpiece, remove non-detection target areas, and obtain the workpiece surface image. Step S2. By transforming the image into polar coordinates, the surface of the workpiece crankshaft is converted into a rectangular region. Cracks or bumps on the crankshaft surface are detected by height feature screening. The image is restored by inverse transformation to obtain the defect region. The standard ROI region containing the most defect points is extracted and edge extraction is performed to obtain the defect region. Step S3. Acquire images of the inner wall of the workpiece, train a scale discriminant network based on the connected components of the end face ROI, segment the defect region of the inner wall of the workpiece, obtain defect samples, detect and output the feature parameters of the defect samples; Step S4. Obtain the surface contour of the part, group the contours, calculate the minimum bounding boundary of each group of contours, generate a local triangular mesh according to the minimum bounding boundary, adjust the weight of the defect position in the triangular mesh according to the feature parameters of the defect sample, and construct the contour convex hull by scanning the mesh point by point to obtain the visualized 3D model of the workpiece.

2. The method for dynamically optimizing and displaying detection results based on deep learning according to claim 1, characterized in that: Step S1 includes: Step S11. Obtain the outline of the inner hole of the workpiece, detect the image area where the center of the inner hole is located, determine the assembly position, and perform circle fitting on the center point of the inner hole to locate the position of the inner hole. Establish the transformation function of the base coordinate system, the pallet coordinate system and the sensor coordinate system. Adjust the camera acquisition parameters according to the previous recognition results and compensate and correct the pallet connection error and the equipment assembly error. Step S12. Reflect the center of vision of each end face onto the parallel surface of the tray surface, stitch together the corrected images of each end face to form a crankshaft surface unfolded image, enhance image contrast through homomorphic filtering, perform threshold segmentation and contour detection on the workpiece template, and determine the integrity of the workpiece surface image.

3. The method for dynamically optimizing and displaying detection results based on deep learning according to claim 2, characterized in that: Step S2 includes: Step S21. Using the crankshaft axis inside the workpiece as the polar axis, the circumferential direction as the horizontal direction of the rectangle, and the axial mapping direction as the vertical direction, perform polar coordinate transformation to obtain line segments on the image. Based on the geometric features of the defect and the local grayscale gradient, filter the defect line segments and perform inverse transformation to determine the defect area. Step S22. Construct a sub-image for the ROI region, add auxiliary surfaces as boundary conditions, obtain the region boundary through depth traversal, fit the region background image through the line grayscale distribution curve, subtract the ROI region image from the background image, search for the maximum value of the subtraction result, segment out the small defects, and sharpen and smooth the defect area. Step S22. After opening and closing operations, eliminate holes in the area connected to the background, draw a polygonal diagram of the surface width and excess height of the defect area, perform point cloud detection on the surface appearance, analyze the inner and outer contour diameters of the workpiece, classify the defects according to the distribution range of the defects, and use the defect area image as a dataset to train an image classification model.

4. The method for dynamically optimizing and displaying detection results based on deep learning according to claim 3, characterized in that: Step S3 includes: Step S31. Detect corner points and extract inner wall ROI. Use the Canny operator to extract the edges of the inner wall ROI, adjust the kernel binding coefficient and membership adjustment coefficient, segment the defects in the inner wall and end face ROI, and fill the defect holes. After transforming the image to the frequency domain, use a low-pass filter to process the frequency domain image, and inverse transform to obtain the low-frequency background image. Divide the detection area into partitions and set different thresholds. Differ from the original image to obtain the defect candidate image, extract the defects, and label the inner wall defect dataset. Step S32. Randomly extract intact samples from non-defective regions, couple the difference features between intact samples and defective samples to the input of the generator, feed them back to the generative adversarial network, and output feature parameters, including the area, perimeter, aspect ratio, and grayscale mean of the defective samples.

5. The method for dynamically optimizing and displaying detection results based on deep learning according to claim 4, characterized in that: Step S4 includes: Step S41. Perform triangulation by scanning the workpiece with a laser, use a vertical filter to filter the image mean, use a horizontal filter to weaken the boundary, remove invalid boundaries of the contour line, locate the rotation detection area, eliminate common edges containing the current point, and extract the workpiece boundary by setting a threshold. Step S42. Perform two-dimensional triangulation on the point set in each group to generate local triangular meshes. Set weights at each vertex of the triangular mesh according to the distance from the center of the defect region and the area of ​​the defect region. Adjust the vertex density according to the weights, merge all local triangular meshes, and output a visualization model.

6. A deep learning-based system for dynamically optimizing and displaying detection results, characterized in that, The system includes the following modules: appearance inspection module, vision processing module, defect identification module, modeling and display module, and 3D reconstruction module; The appearance inspection module is used to place the workpiece to be inspected on a tray, and through a compensating light source, a variable background plate and a displacement camera, collect the appearance end faces of the workpiece, stitch the end faces of the workpiece, remove non-detection target areas, and obtain the workpiece surface image. The vision processing module is used to convert the workpiece crankshaft surface into a rectangular region through image polar coordinate transformation, stretch the convex defects into line segments, detect cracks or convex defects on the crankshaft surface through height feature screening, recover the image through inverse coordinate transformation to obtain the defect region, extract the standard ROI region containing the most defect points based on the roundness and area of ​​the defect region, perform edge extraction on the obtained ROI region, segment the defect region and determine the structural element size. The defect identification module is used to acquire images of the inner wall of the workpiece using dark field illumination, train a scale discrimination network based on the connected components of the end face ROI, segment the defect area of ​​the inner wall of the workpiece using a contour tracking algorithm, obtain defect samples, and output the feature parameters of the defect samples. The modeling and display module is used to obtain the surface contour of the part at the current angle, group the contours, calculate the minimum bounding boundary of each group of contours, divide each group of contours according to the minimum bounding boundary, generate a local triangular mesh, increase the weight of the defect position in the triangular mesh according to the feature parameters of the defect sample, and construct the contour convex hull by scanning the mesh point by point to obtain a visualized three-dimensional model of the workpiece. The 3D reconstruction module is used to divide the model surface. During the division process, the color values ​​of defect nodes are set according to the mesh weight. The color of the 3D surface is reconstructed by inverse distance interpolation, and the surface reconstruction is completed and visualized.

7. The detection result dynamic optimization and display system based on deep learning according to claim 6, characterized in that: The appearance inspection module includes: a contour registration unit and an end face positioning unit; The contour registration unit is used to acquire the inner hole contour of the workpiece, detect the image area where the inner hole center is located, determine the assembly position, perform circle fitting on the inner hole center point, locate the inner hole position, establish the transformation function of the base coordinate system, pallet coordinate system and sensor coordinate system, adjust the camera acquisition parameters according to the previous recognition results, and compensate and correct the pallet connection error and equipment assembly error. The end face positioning unit is used to reflect the center of vision of each end face onto the parallel surface of the tray surface, stitch together the corrected end face images to form a crankshaft surface unfolded image, enhance image contrast through homomorphic filtering, perform threshold segmentation and contour detection on the workpiece template, and determine the integrity of the workpiece surface image.

8. The deep learning-based detection result dynamic optimization and display system according to claim 7, characterized in that: The vision processing module includes: a coordinate transformation unit, a region detection unit, and a defect segmentation unit; The coordinate transformation unit is used to perform polar coordinate transformation with the crankshaft axis inside the workpiece as the polar axis, the circumferential direction as the rectangular horizontal direction, and the axial mapping direction as the vertical direction to obtain line segments on the image. Based on the geometric features of the defect and the local gray-level gradient, the defect line segments are selected, and the defect area is determined by inverse transformation. The region detection unit is used to construct a sub-image for the ROI region, add auxiliary surfaces as boundary conditions, obtain the region boundary through depth traversal, fit the region background image through the line grayscale distribution curve, subtract the ROI region image from the background image, search for the maximum value of the subtraction result, segment out the small defects, and sharpen and smooth the defect region. The defect segmentation unit is used to eliminate holes in areas connected to the background through opening and closing operations, draw a polygonal diagram of the surface width and excess height of the defect area, perform point cloud detection on the surface appearance, analyze the inner and outer contour diameters of the workpiece, classify defects according to their distribution range, and use the defect area image as a dataset to train an image classification model.

9. The deep learning-based detection result dynamic optimization and display system according to claim 8, characterized in that: The defect identification module includes: a network training unit, a frequency domain difference unit, and a feature detection unit; The network training unit is used to detect corner points and extract the inner wall ROI. The Canny operator is used to extract the edge of the inner wall ROI, adjust the kernel binding coefficient and membership adjustment coefficient, segment the defects in the inner wall and end face ROI, and fill the defect voids. The frequency domain difference unit is used to transform the image to the frequency domain, process the frequency domain image with a low-pass filter, and inversely transform to obtain a low-frequency background image. The detection area is divided into partitions and different thresholds are set. The difference is obtained from the original image to obtain a defect candidate image, extract the defects, and label the inner wall defect dataset. The feature detection unit is used to randomly extract intact samples from non-defective regions, couple the difference features between intact samples and defective samples to the input of the generator, feed them back to the generative adversarial network, and output feature parameters, which include the area, perimeter, aspect ratio, and grayscale mean of the defective samples.

10. The deep learning-based detection result dynamic optimization and display system according to claim 9, characterized in that: The modeling and display module includes: a boundary detection unit and a mesh modeling unit; The boundary detection unit is used to perform triangulation by scanning the workpiece with a laser, use a vertical filter to filter the image mean, a horizontal filter to weaken the boundary, remove invalid boundaries of the contour line, locate the rotation detection area, eliminate common edges containing the current point, and extract the workpiece boundary by setting a threshold. The mesh modeling unit is used to perform two-dimensional triangulation on the point set in each group to generate a local triangular mesh. At each vertex of the triangular mesh, a weight is set according to the distance from the center of the defect region and the area of ​​the defect region. The vertex density is adjusted according to the weight, all local triangular meshes are merged, and a visualization model is output. The three-dimensional reconstruction module includes: a surface segmentation unit and a visual processing unit; The surface subdivision unit is used to perform planar mapping subdivision of horizontal surfaces in the three-dimensional model, reconstruct the surface shape, perform three-dimensional convex shell subdivision of rotating surfaces or discontinuous surfaces, fuse features of different scales, predict the occupancy rate of points based on the spatial location information of points, and extract triangular meshes using the moving cube algorithm. The visual processing unit is used to guide the 3D reconstruction process according to the calibrated camera intrinsic and extrinsic parameters, perform texture mapping and 3D reconstruction on the acquired images, restore the 3D structure layer by layer from the boundary of the 2D region according to the surface topology, add direction and order parameters to each surface edge, and reconstruct the model.

Images