Workpiece defect repairing method and device, electronic equipment and medium

Abstract

The invention relates to the technical field of vehicles, and discloses a workpiece defect repairing method and device, electronic equipment and a medium, and the method comprises the steps: recognizing an initial defect region based on a surface image of a to-be-repaired workpiece, and extracting the image features of the initial defect region; performing fusion enhancement on the image features through a neural network comprising a detail enhancement branch and a geometric adaptive branch to generate enhanced image features; correcting the boundary of the initial defect area based on the enhanced image features to obtain a defect position, and determining a defect depth corresponding to the defect position from the point cloud data of the to-be-repaired workpiece; and generating a repair strategy according to the defect position and the defect depth, and controlling repair equipment to repair the to-be-repaired workpiece based on the repair strategy. The defect positioning precision is improved. And meanwhile, the problem of connectivity deficiency between detection and repair links is solved, and an integrated automation technology of workpiece defect detection and defect repair is formed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to a workpiece defect repairing method and device, electronic equipment and medium. BACKGROUND

[0002] In the whole vehicle manufacturing process, the surface quality of the stamping sheet metal part is directly related to the safety, aesthetic value and market competitiveness of the whole vehicle.

[0003] In the field of metal stamping part surface defect identification and repair, related technologies collect the surface image of the workpiece through an image acquisition device, and directly determine the workpiece defect position. Since the surface image is a two-dimensional plane image, the three-dimensional spatial information of the workpiece is missing, and the defect position may be missed or incorrectly identified, resulting in low identification accuracy of the defect position. At the same time, the repair of the workpiece defect position mostly depends on manual work, and there is a lack of integrated automatic technology for workpiece defect detection and defect repair. SUMMARY

[0004] In view of the above problems, the present application provides a workpiece defect repairing method and device, electronic equipment and medium, which solves the problems of lack of connectivity between sheet metal part surface defect detection and repair links and lack of repair process automation in the stamping process.

[0005] The first aspect of the present application provides a workpiece defect repairing method, comprising: identifying an initial defect area based on a surface image of a workpiece to be repaired, and extracting image features of the initial defect area; fusing and enhancing the image features through a neural network containing a detail enhancement branch and a geometric self-adaptive branch to generate enhanced image features; correcting the boundary of the initial defect area based on the enhanced image features to obtain a defect position, and determining a defect depth corresponding to the defect position from point cloud data of the workpiece to be repaired; generating a repair strategy according to the defect position and the defect depth, and controlling a repair device to repair the workpiece to be repaired based on the repair strategy.

[0006] In an optional embodiment, the enhanced image features are generated by fusing and enhancing the image features through a neural network containing a detail enhancement branch and a geometric self-adaptive branch, comprising: enhancing the texture details of the image features through the detail enhancement branch to obtain first image features; enhancing the geometric shapes of the image features through the geometric self-adaptive branch to obtain second image features; and weighting and fusing the first image features and the second image features to obtain the enhanced image features.

[0007] In an optional embodiment, the enhanced image features include detail features and geometric shape features; and the modifying the boundary of the initial defect region based on the enhanced image features to obtain the defect position includes: modifying a defect range in the initial defect region based on the detail features, and modifying the boundary of the defect range based on the geometric shape features to obtain the defect position.

[0008] In an optional embodiment, the repair strategy is generated according to the defect position and the defect depth, and the repair device is controlled to repair the workpiece to be repaired based on the repair strategy, including: mapping the defect position and the defect depth into a virtual model corresponding to the workpiece to be repaired to obtain a defect coordinate set; generating a motion path of the repair device according to the defect coordinate set, and determining a process parameter of the repair device according to the defect depth and the motion path; and controlling the repair device to repair the workpiece to be repaired based on the motion path and the process parameter.

[0009] In an optional embodiment, the motion path of the repair device is generated according to the defect coordinate set, and the process parameter of the repair device is determined according to the defect depth and the motion path, including: determining three-dimensional information and a surface normal vector of the defect position according to the defect coordinate set; substituting the three-dimensional information and the surface normal vector into a preset algorithm to generate the motion path of the repair device; and determining a spraying pressure and a moving speed of the repair device according to the defect depth of each defect point in the motion path.

[0010] In an optional embodiment, the repair method further includes: performing visual detection and acoustic resonance detection on the repaired workpiece to determine whether the repair is successful; and determining that the repair fails if the visual detection result indicates that the surface of the repaired workpiece has defects or the acoustic resonance detection result indicates that the surface of the repaired workpiece has defects.

[0011] In an optional embodiment, the initial defect region is identified based on the surface image of the workpiece to be repaired, including: obtaining a surface image of each workpiece; determining a workpiece to be repaired whose surface has defects according to the surface image of each workpiece, and determining an initial defect position of the workpiece to be repaired.

[0012] A second aspect of the present application provides a workpiece defect repair device, including: a feature extraction module configured to identify an initial defect region based on a surface image of a workpiece to be repaired, and extract image features of the initial defect region; a feature enhancement module configured to fuse and enhance the image features through a neural network including a detail enhancement branch and a geometric self-adaptive branch to generate enhanced image features; a modification module configured to modify a boundary of the initial defect region based on the enhanced image features to obtain a defect position, and determine a defect depth corresponding to the defect position from point cloud data of the workpiece to be repaired; and a repair module configured to generate a repair strategy according to the defect position and the defect depth, and control a repair device to repair the workpiece to be repaired based on the repair strategy.

[0013] The third aspect of the present application provides an electronic device, comprising: a controller; a memory for storing one or more programs, which, when executed by the controller, cause the controller to implement the above repair method.

[0014] The fourth aspect of the present application provides a computer-readable storage medium having computer-readable instructions stored thereon, which, when executed by a processor of a computer, cause the computer to perform the above repair method.

[0015] The present application identifies the initial defect area of the workpiece to be repaired based on the surface image, extracts image features, uses a neural network containing a detail enhancement branch and a geometric self-adaptive branch to fuse and enhance the image features, generates enhanced image features, further corrects the initial defect area boundary to obtain accurate defect positions, and determines the defect depth corresponding to the defect positions in combination with the point cloud data, and finally generates a repair strategy according to the defect positions and depths and controls the repair equipment to perform repair. The present application significantly improves the detail retention capability of micro defects and the capture capability of irregular geometric shapes by introducing a double-branch collaborative enhancement mechanism, thereby improving the defect positioning accuracy. At the same time, the connectivity problem between the detection and repair links is solved, forming an integrated automatic technology of workpiece defect detection and defect repair.

[0016] The above description is only a summary of the technical solutions of the present application. In order to enable the technical means of the present application to be more clearly understood, and to be implemented according to the content of the description, and in order to enable the above and other purposes, features and advantages of the present application to be more apparent and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0017] The drawings herein are incorporated into the specification and form part of the specification, show embodiments consistent with the present application, and together with the specification serve to explain the principles of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained from these drawings without creative labor for those skilled in the art.

[0018] Figure 1 is a flowchart of a workpiece defect repair method according to an exemplary embodiment of the present application.

[0019] Figure 2 is a flowchart of another workpiece defect repair method according to an exemplary embodiment of the present application. Figure 1

[0020] Figure 3 is a process flowchart of various modules in the repair system of the present application.

[0021] ​Figure 4 is a structural schematic diagram of a workpiece defect repairing device according to an example embodiment of the present application.

[0022] Figure 5 is a structural schematic diagram of a computer system of an electronic device according to an example embodiment of the present application. DETAILED DESCRIPTION

[0023] The example embodiments will be described in detail herein with reference to the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following example embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0024] The block diagrams shown in the accompanying drawings are merely functional entities, and do not necessarily correspond to physically independent entities. That is, the functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0025] The flowcharts shown in the accompanying drawings are merely example illustrations, and do not necessarily include all contents and operations / steps, nor are they necessarily executed in the order described. For example, some operations / steps can be further divided, and some operations / steps can be combined or partially combined, so that the actual execution order can be changed according to actual situations.

[0026] In the present application, "a plurality of" means two or more. The association relationship of "and / or" describes the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally means that the associated objects before and after are in an "or" relationship.

[0027] In the field of metal stamping part surface defect identification and repair, related technologies acquire the surface image of the workpiece through an image acquisition device, and directly determine the workpiece defect position. Since the surface image is a two-dimensional plane image, the three-dimensional spatial information of the workpiece is missing, and the defect position may be missed or incorrectly identified, resulting in low identification accuracy of the defect position. At the same time, the repair of the workpiece defect position mostly depends on manual work, and there is a lack of integrated automatic technology for workpiece defect detection and defect repair.

[0028] Therefore, one aspect of the present application provides a workpiece defect repairing method, which is used to solve the problems of missing connectivity between the surface defect detection and repair of sheet metal parts in the stamping process and missing automation of the repair process. For details, please refer toFigure 1 , Figure 1 is a flowchart of a workpiece defect repair method according to an example embodiment of the present application. The repair method includes at least S110 to S140, which are described in detail as follows: S110: Identify the initial defect area based on the surface image of the workpiece to be repaired, and extract the image features of the initial defect area.

[0029] The workpiece to be repaired is a sheet metal part with defects that need to be repaired. The surface image of each workpiece is a single frame or multiple frames of images captured by a high-resolution industrial camera under the synchronous illumination of a multi-spectral ring light source.

[0030] The initial defect area is a defect area identified according to the surface image, i.e., determining whether the workpiece surface has defects according to the two-dimensional planar image of the workpiece, and taking the workpiece with defects as the workpiece to be repaired.

[0031] In some embodiments, the surface image of each workpiece is obtained; the workpiece to be repaired with defects on the surface is determined according to the surface image of each workpiece, and the initial defect position of the workpiece to be repaired is determined. For example, input the surface image of each workpiece into an image detection model (such as YOLOv8n or a common AI model), and the surface image with the initial defect position and the workpiece to be repaired can be automatically determined.

[0032] The image features are shallow features of the initial defect area extracted from the surface image. For example, input the surface image into the YOLOv8n model, the surface image is divided into multiple rectangular regions, and the image features are shallow features output by the Backbone network after being cropped and normalized in the corresponding rectangular region.

[0033] S120: Fuse and enhance the image features through a neural network containing a detail enhancement branch and a geometric adaptive branch to generate enhanced image features.

[0034] A multi-scale detail enhancement module (MDEM) is constructed in the neural network of the present application, which is integrated between the Neck layer and the Head layer of YOLOv8. The MDEM contains three core components: a detail enhancement branch, a geometric adaptive branch, and a cross-scale feature fusion unit (CS-Fusion).

[0035] The detail enhancement branch is a neural network substructure for enhancing the high-frequency information of the image, such as a high-resolution shallow feature extraction branch (HR-Branch), which adopts a depthwise separable convolution (Depthwise Separable Convolution) structure to extract high-resolution feature maps from the shallow layers of the backbone network (i.e., to enhance the image features output by the backbone network), to retain the texture details of the tiny defects and significantly enhance the edge gradient response of the tiny defects.

[0036] The geometric adaptive branch refers to a neural network substructure with spatial deformation modeling capability, such as an adaptive receptive field adjustment branch (ARF-Branch), which embeds a deformable convolution (Deformable Convolution v2) to dynamically adjust the spatial position and weight of each sampling point, so that the convolution kernel can adaptively fit the irregular defect contours such as ellipses, zigzags, and rings, to dynamically adjust the shape of the convolution kernel and adapt to the geometric features of defects of different sizes, thereby enhancing the ability to capture irregular tiny targets.

[0037] The fusion enhancement refers to a channel-level concatenation of the output features of the two branches, i.e., a weighted fusion of the texture details of the HR-Branch and the deformation-aware semantic features of the ARF-Branch, to generate enhanced multi-scale feature maps (i.e., enhanced image features) that contain both the high-fidelity texture details provided by the HR-Branch and the geometric topological constraints extracted by the ARF-Branch.

[0038] By way of example, the texture details of the image features are enhanced by the detail enhancement branch to obtain first image features; the geometric shape of the image features is enhanced by the geometric adaptive branch to obtain second image features; and the first image features and the second image features are weightedly fused to obtain enhanced image features.

[0039] The first image features can be understood as image features with enhanced texture details of the defects, and the second image features can be understood as image features with enhanced geometric (defect shape) features of the defects.

[0040] The weighted fusion refers to a learnable linear / non-linear combination of the first image feature and the second image feature in the channel dimension or the spatial dimension, and the weight is generated by a cross-scale feature fusion unit (CS-Fusion). The unit first performs element-wise addition or splicing on the first image feature and the second image feature, and then generates a channel-level weight vector through a channel attention mechanism (such as the Channel Attention Sub-module in CBAM). After the vector is activated by Sigmoid, it is multiplied with the spliced feature channel by channel, realizing the differential association of the detail information and the geometric information.

[0041] S130: Correcting the boundary of the initial defect region based on the enhanced image feature to obtain a defect position, and determining a defect depth corresponding to the defect position from the point cloud data of the workpiece to be repaired.

[0042] The defect position is the position of the defect in the workpiece to be repaired, which is a closed area with a boundary being a critical line between the defect part and the non-defect part.

[0043] The point cloud data is a point cloud set obtained by scanning the workpiece to be repaired by a 3D line laser scanner, which can be understood as a three-dimensional coordinate point set representing the three-dimensional appearance of the workpiece to be repaired.

[0044] The defect depth is the longitudinal depth of the defect in the region where the defect is located, i.e., the Z-axis coordinate set of the point cloud.

[0045] Since the initial defect region is a defect region determined only according to a two-dimensional surface image, the precision of the defect size, shape, and region boundary in the initial defect region is low, and there may be missing small texture defects and missing region boundaries. The embodiment uses enhanced image features to find and fill in missing small texture defects, and clearly defines the defect region.

[0046] Exemplarily, the enhanced image features include detail features and geometric shape features; the defect range in the initial defect region is corrected based on the detail features, and the boundary of the defect range is corrected based on the geometric shape features to obtain the defect position.

[0047] The detail feature is a detail enhancement of the image feature output by the Backbone network, i.e., a multi-scale detail feature output after high-resolution feature extraction by a high-resolution shallow feature extraction branch (HR-Branch).

[0048] The geometric shape feature is a deformation perception semantic feature map output by an adaptive receptive field adjustment branch (ARF-Branch).

[0049] The defect position is a point set of a defect region and a boundary contour in a two-dimensional coordinate system after the defect range in the initial defect region is corrected based on a detail feature and the boundary of the defect range is corrected based on a geometric feature.

[0050] The defect range in the initial defect region is corrected based on the detail feature, which can be understood as defect range leak detection based on the detail feature, ensuring defect coverage integrity and avoiding missed detection due to texture weakening. The boundary of the defect range is corrected based on the geometric feature, which can be understood as clearly defining the contour line of the boundary based on the geometric feature, ensuring the accuracy of spatial positioning and avoiding the situation of defect range expansion error. The newly added area formed by the detail feature expansion provides a more sufficient contour sampling basis for the geometric feature, and the high-confidence boundary after the geometric feature refinement reversely restricts the invalid expansion direction of the detail feature. This inside-out and coarse-to-fine correction mechanism solves the defect positioning drift problem caused by the disconnection between detection and repair.

[0051] S140: Generate a repair strategy according to the defect position and the defect depth, and control the repair device to repair the workpiece to be repaired based on the repair strategy.

[0052] The repair strategy is an execution strategy for the repair device, including but not limited to the motion path, process parameters, repair time, etc. of the repair device.

[0053] The repair device is a device for repairing the workpiece. For example, the repair device is a six-axis collaborative robot equipped with a low-temperature cold spraying device, the spray gun is connected to the end of the robot arm, and the pressure regulating system is integrated with the spray gun through a high-pressure hose to adjust the spraying pressure.

[0054] The embodiment identifies the initial defect region of the workpiece to be repaired based on the surface image, extracts image features, uses a neural network containing a detail enhancement branch and a geometric adaptive branch to fuse and enhance the image features, generates enhanced image features, and then corrects the boundary of the initial defect region to obtain an accurate defect position. In combination with point cloud data, the defect depth corresponding to the defect position is determined, and finally the repair strategy is generated according to the defect position and the depth and the repair device is controlled to perform repair. The embodiment introduces a double-branch collaborative enhancement mechanism, which significantly improves the detail retention capability of micro-defects and the capture capability of irregular geometric shapes, thereby improving the defect positioning accuracy. At the same time, the connectivity problem between the detection and repair links is solved, forming an integrated and automated technology for workpiece defect detection and defect repair.

[0055] In another exemplary embodiment, the repaired workpiece is further subjected to defect re-inspection to form a closed-loop system of defect detection-repair-re-inspection, thereby optimizing the entire process flow. For example, the repaired workpiece is subjected to visual inspection and acoustic resonance inspection to determine whether the repair was successful; if the visual inspection result indicates that a defect exists on the surface of the repaired workpiece, or the acoustic resonance inspection result indicates that a defect exists on the surface of the repaired workpiece, then the repair is determined to have failed.

[0056] Visual inspection uses a white light interferometer to detect the surface roughness of the repaired area of ​​the workpiece in order to detect the defect repair status.

[0057] Acoustic resonance testing is a method of detecting the coating quality of a workpiece using acoustic resonance to determine whether the coating is acceptable after defect repair.

[0058] In this example, the repair of the workpiece can only be considered successful if both the visual detection results and the acoustic resonance detection results indicate that the repaired workpiece is free of defects; if either detection result indicates that the repaired workpiece has defects, the repair is considered a failure.

[0059] For example, a closed-loop quality inspection station is set up in the production line. The core components of this station include a white light interferometric surface profiler and an acoustic resonance detection probe, which are fixed to an independent gantry frame by an anti-vibration air-floating platform. An XYZ direction motion device is integrated, which is 400mm away from the conveyor belt surface and perpendicular to the direction of sheet metal movement.

[0060] When the repaired sheet metal part is sent to the closed-loop inspection station, the inspection mechanism is triggered. The location information of the previous defect is read, and a white light interferometer is used to perform surface roughness inspection and 3D model reconstruction of the repaired area. The coating quality is also inspected through acoustic resonance. Workpieces that pass the inspection enter the off-line process, while unqualified workpieces are returned to the repair station for defect repair again.

[0061] Visual inspection ensures the surface integrity of the repaired area from the perspectives of macro-geometry and micro-morphology, while acoustic resonance inspection ensures the quality of the interface bonding between the repair layer and the substrate from the perspective of elastic wave propagation. Together, they cover the two most critical failure scenarios after sheet metal repair: appearance defects (affecting assembly and painting) and hidden debonding (affecting fatigue strength and corrosion resistance).

[0062] In another exemplary embodiment of this application, a repair strategy is generated based on the defect location and defect depth, and the repair equipment is controlled to repair the workpiece based on the repair strategy. Please refer to [link to relevant documentation] for details. Figure 2 , Figure 2 Based on Figure 1 The exemplary embodiment shown illustrates a flowchart of another method for repairing workpiece defects. This repair method, in... Figure 1The S140 shown includes S210 to S230, which are described in detail below: S210: Map the defect location and defect depth to the virtual model corresponding to the workpiece to be repaired to obtain the defect coordinate set.

[0063] The defect location is the set of center points and contour vertices of the defect region in a two-dimensional coordinate system after the defect range and boundary are corrected by enhancing image features.

[0064] Defect depth is the normal distance value corresponding to the defect location extracted from the point cloud data of the workpiece to be repaired. It is also the longitudinal depth of the defect within the area where the defect is located, i.e., the set of Z-axis coordinates of the point cloud.

[0065] The virtual model is a lightweight digital twin built based on the CAD design model of the workpiece to be repaired. The virtual model includes a complete geometric topology, material property labels, and a preset coordinate system.

[0066] The mapping process is achieved through a homogeneous transformation matrix: First, the two-dimensional image coordinates of the defect location are back-projected into a three-dimensional point cloud candidate set in the world coordinate system through the camera intrinsic parameter matrix and the distortion correction model; then, combined with the defect depth constraint, matching points with an angle of less than 15° with the surface normal of the virtual model and the smallest Euclidean distance are selected from the point cloud candidate set; finally, the matching point and its neighboring points are uniformly transformed to the local coordinate system of the virtual model to form a defect coordinate set with spatial consistency.

[0067] The defect coordinate set is a set of discrete three-dimensional point coordinates that are mapped to the virtual model and have a unified coordinate system (such as the workpiece CAD model coordinate system or digital twin coordinate system). Each three-dimensional point coordinate corresponds to a point within the defect area or a boundary point of the defect location.

[0068] S220: Generate the motion path of the repair equipment based on the defect coordinate set, and determine the process parameters of the repair equipment based on the defect depth and the motion path.

[0069] The spatial distribution characteristics of each coordinate point in the defect coordinate set directly determine the path generation strategy: For isolated single-point defects, spherical interpolation (Slerp) is used to generate a circular arc path with continuous end effector posture; for linear defects (such as scratches), B-spline curve interpolation is used, with the number of nodes adaptively set according to the defect length, and curvature continuity C² ensuring a smooth and abrupt spray gun trajectory; for planar defects (such as pits), a spiral progressive filling algorithm is used: starting from the defect center, it expands outward according to the Archimedes' spiral equation r=a+bθ, and the step angle increment Δθ is dynamically adjusted with the defect depth gradient. For example, if the depth gradient is large, Δθ decreases to ensure uniform edge transition.

[0070] For example, the three-dimensional information and surface normal vector of the defect location are determined based on the defect coordinate set; the three-dimensional information and surface normal vector are substituted into a preset algorithm to generate the motion path of the repair equipment; and the spraying pressure and moving speed of the repair equipment are determined based on the defect depth of each defect point in the motion path.

[0071] The three-dimensional information includes the X, Y, and Z coordinates of each point at the defect location and their absolute positional relationship in the global coordinate system, used to characterize the spatial distribution of the defect. The surface normal vector is obtained by least-squares fitting of the neighborhood point cloud surface and calculating the gradient direction. It can be adaptively adjusted according to the defect scale, ensuring stable normal estimation capability in scenarios with small pits (such as pinholes with a diameter <0.3mm) and large-scale ripples (such as stretch marks with a length >50mm). The surface normal vector also serves as a basis for determining the defect geometry type, thereby dynamically adjusting the spray gun posture according to different defect geometry types to ensure vertical spraying.

[0072] The preset algorithms include a B-spline curve interpolation algorithm and a spiral progressive filling algorithm. The 3D coordinates and corresponding surface normal vectors of all points in the defect coordinate set are substituted into the B-spline curve interpolation algorithm to construct a non-uniform rational B-spline (NURBS) curve with normal constraints. This curve is a central guiding path covering all defect points. Using this central path as the axis, and based on the defect depth distribution and the spiral progressive filling algorithm, multiple concentric spiral trajectories are generated, thus obtaining the motion path of the repair equipment. B-spline interpolation ensures path smoothness to suppress robotic arm vibration, while spiral progressive filling improves the uniformity of material accumulation.

[0073] The movement path contains defect points of varying depths. To ensure the uniformity and smoothness of the workpiece surface after spraying, it is necessary to adaptively adjust the amount of paint and the spraying depth for defect points of different depths. This can be achieved by controlling the spraying pressure and moving speed of the repair equipment. For example, the speed can be reduced and the pressure increased when the defect depth increases.

[0074] S230: The repair equipment is controlled based on motion path and process parameters to repair the workpiece.

[0075] The repair equipment is a six-axis collaborative robotic arm equipped with a cryogenic cold spraying device. The spray gun is connected to the end effector of the robotic arm, and the pressure regulation system is integrated with the spray gun through a high-pressure hose to adjust the spraying pressure. The control method adopts a master-slave dual-loop architecture: the outer loop is a path tracking controller that receives the pose sequence and outputs the target angles of each joint, controlling the robotic arm to move based on the motion path; the inner loop is a process parameter servo controller that receives process parameters such as spraying pressure and movement speed, controlling the spray gun to spray the workpiece to be repaired.

[0076] In another exemplary embodiment of this application, the application scenario of the above-mentioned method for repairing multiple workpiece defects is illustrated. Please refer to the following for details. Figure 3 , Figure 3 This is a schematic diagram of the process flow of each module in the repair system of this application. The repair method is configured in the repair system 300, which includes a multimodal vision inspection module 310, an edge computing control module 320, a repair execution module 330, and a closed-loop quality inspection module 340. They can be connected to each other by wired or wireless communication. This application does not limit the connection method between them.

[0077] The 310 multimodal vision inspection module integrates a high-resolution industrial camera, a 3D line laser scanner, and a multispectral ring light source. The high-resolution industrial camera and 3D line laser scanner are rigidly connected by a truss, positioned 400mm from the workpiece surface at a 30° tilt angle. The multispectral ring light source is mounted around the camera and synchronously triggers the illumination mode (diffuse / polarized / multispectral) via a CAN bus. Figure 3 As shown, the multimodal vision inspection module 310 can acquire surface images (2D images) of each sheet metal part to detect the workpiece to be repaired and the corresponding initial defect area, and can also acquire 3D point cloud data of the workpiece to be repaired.

[0078] The hardware computing devices of the edge computing control module 320 include an edge computing unit (industrial computer) and a communication interface. It embeds an AI defect recognition model and path planning algorithm. To achieve more accurate recognition, a multi-scale detail enhancement module (MDEM) is constructed and integrated between the Neck and Head layers of YOLOv8. Specifically, MDEM contains three core components: (1) High-resolution shallow feature extraction branch (HR-Branch): uses depthwise separable convolution to extract high-resolution feature maps from the shallow layer of the backbone, preserving the texture details of small defects; (2) Adaptive receptive field adjustment branch (ARF-Branch): introduces deformable convolution to dynamically adjust the shape of the convolution kernel, adapting to the geometric features of defects of different sizes, and enhancing the ability to capture irregular small targets; (3) Cross-scale feature fusion unit (CS-Fusion): uses channel attention to weightedly fuse the detailed information of HR-Branch and the semantic information of ARF-Branch to generate enhanced multi-scale feature maps. Through the dual-branch high-resolution feature preservation branch and the adaptive receptive field fusion mechanism, the edge computing control module 320 can significantly improve the model's ability to detect small defects in large-area workpieces. Figure 3 As shown, the edge computing control module 320 can generate motion paths and process parameters based on the defect location and defect depth to control the repair execution module 330 to repair the workpiece to be repaired.

[0079] The repair execution module 330 includes a six-axis collaborative robotic arm and a cold spraying device. The spray gun of the cold spraying device is connected to the end of the robotic arm, and the pressure regulating system is integrated with the spray gun through a high-pressure hose. After receiving the motion path and process parameters transmitted by the edge computing control module 320, the robotic arm and the cold spraying device perform repair work on the stamped parts placed on the precision positioning worktable according to the motion path and process parameters.

[0080] The closed-loop quality inspection module 340 includes a white light interferometric surface profilometer and an acoustic resonance detection probe. It is fixed to an independent gantry via a vibration-damping air-floating platform and integrates an XYZ direction motion device. It is positioned 400mm from the conveyor belt surface, perpendicular to the sheet metal part's movement direction. When a repaired sheet metal part is transferred to the closed-loop inspection station, the inspection program is triggered. It reads the location information of previous defects, activates the white light interferometer to perform surface roughness detection and 3D model reconstruction at the repair location, and uses acoustic resonance to detect coating quality. Workpieces that pass inspection proceed to the off-line process, while unqualified workpieces are returned to the repair station. The inspection results are fed back to the edge computing control module 320, which adjusts the motion path and process parameters, and re-executes the repair program. All quality inspection results are fed back to the edge computing control module 320 for optimization of the process parameter matching model.

[0081] Another aspect of this application provides a device for repairing workpiece defects, such as... Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the structure of a workpiece defect repair apparatus according to an exemplary embodiment of this application. The repair apparatus 400 includes: The feature extraction module 410 is used to identify the initial defect region based on the surface image of the workpiece to be repaired, and to extract the image features of the initial defect region.

[0082] The feature enhancement module 430 is used to fuse and enhance image features through a neural network that includes detail enhancement branches and geometric adaptation branches to generate enhanced image features.

[0083] The correction module 450 is used to correct the boundary of the initial defect area based on the enhanced image features to obtain the defect location, and to determine the defect depth corresponding to the defect location from the point cloud data of the workpiece to be repaired.

[0084] Repair module 470 is used to generate repair strategies based on defect location and defect depth, and control repair equipment to repair the workpiece to be repaired based on the repair strategies.

[0085] In another exemplary embodiment, the feature enhancement module 430 includes: The first enhancement unit is used to enhance the texture details of the image features through the detail enhancement branch to obtain the first image features.

[0086] The second enhancement unit is used to enhance the geometry of image features through geometric adaptive branching to obtain second image features.

[0087] The fusion unit is used to perform weighted fusion of the first image features and the second image features to obtain enhanced image features.

[0088] In another exemplary embodiment, the enhanced image features include detail features and geometric features; the correction module 450 includes: The correction unit is used to correct the defect range in the initial defect region based on detailed features and to correct the boundary of the defect range based on geometric features, so as to obtain the defect location.

[0089] In another exemplary embodiment, the repair module 470 includes: The mapping unit is used to map the defect location and defect depth to the virtual model corresponding to the workpiece to be repaired, thereby obtaining a set of defect coordinates.

[0090] The generation and determination unit is used to generate the motion path of the repair equipment based on the defect coordinate set, and to determine the process parameters of the repair equipment based on the defect depth and the motion path.

[0091] The repair unit is used to control the repair equipment to repair the workpiece based on the motion path and process parameters.

[0092] In another exemplary embodiment, the generation and determination unit includes: The first determining module is used to determine the three-dimensional information of the defect location and the surface normal vector based on the defect coordinate set.

[0093] The generation module is used to input 3D information and surface normal vectors into a preset algorithm to generate the motion path of the repair equipment.

[0094] The second determining module is used to determine the spraying pressure and moving speed of the repair equipment based on the defect depth at each defect point in the movement path.

[0095] In another exemplary embodiment, the repair device 400 further includes: The repair and inspection module is used to perform visual inspection and acoustic resonance inspection on the repaired workpiece to determine whether the repair was successful.

[0096] The repair failure determination module is used to determine repair failure if the visual detection results indicate that there are defects on the surface of the repaired workpiece, or if the acoustic resonance detection results indicate that there are defects on the surface of the repaired workpiece.

[0097] In another exemplary embodiment, the feature extraction module 410 includes: The acquisition unit is used to acquire surface images of each workpiece.

[0098] The initial defect location determination unit is used to determine the workpiece to be repaired with surface defects based on the surface image of each workpiece, and to determine the initial defect location of the workpiece to be repaired.

[0099] This repair device identifies the initial defect region of the workpiece to be repaired based on surface image recognition, extracts image features, and uses a neural network with detail enhancement and geometric adaptation branches to fuse and enhance these features, generating enhanced image features. It then corrects the boundary of the initial defect region to obtain the precise defect location, and combines point cloud data to determine the defect depth corresponding to the defect location. Finally, it generates a repair strategy based on the defect location and depth and controls the repair equipment to perform the repair. This repair device, by introducing a dual-branch collaborative enhancement mechanism, significantly improves the ability to preserve details of minute defects and capture irregular geometric shapes, thereby improving defect positioning accuracy. Simultaneously, it solves the problem of lack of connectivity between the detection and repair stages, forming an integrated automated technology for workpiece defect detection and repair.

[0100] It should be noted that the repair device provided in the above embodiments and the repair method provided in the foregoing embodiments belong to the same concept. The specific way in which each module and unit performs operations has been described in detail in the method embodiments, and will not be repeated here.

[0101] Another aspect of this application provides an electronic device, including: a controller; and a memory for storing one or more programs, which, when executed by the controller, perform the above-described repair method.

[0102] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer system for an electronic device according to an exemplary embodiment of this application, illustrating a schematic diagram of the structure of a computer system suitable for implementing the embodiments of this application.

[0103] It should be noted that, Figure 5 The computer system 500 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0104] like Figure 5 As shown, the computer system 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 502 or programs loaded from storage portion 508 into Random Access Memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An Input / Output (I / O) interface 505 is also connected to the bus 504.

[0105] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 510 as needed so that computer programs read from them can be installed into storage section 508 as needed.

[0106] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs various functions defined in the system of this application.

[0107] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0110] Another aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned repair method. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0111] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the repair methods provided in the various embodiments described above.

[0112] According to one aspect of the embodiments of this application, a computer system is also provided, including a Central Processing Unit (CPU), which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from storage into random access memory (RAM), such as performing the methods described above. Various programs and data required for system operation are also stored in the RAM. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0113] The following components are connected to the I / O interface: input components including keyboards, mice, etc.; output components including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage components including hard drives; and communication components including network interface cards such as LAN (Local Area Network) cards and modems. The communication components perform communication processing via networks such as the Internet. Drives are also connected to the I / O interface as needed. Removable media, such as disks, optical discs, magneto-optical discs, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage components as required.

[0114] The above description is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. A method for repairing defects in a workpiece, characterized in that, The repair method includes: The initial defect region is identified based on the surface image of the workpiece to be repaired, and the image features of the initial defect region are extracted. The image features are fused and enhanced using a neural network that includes detail enhancement and geometry adaptation branches to generate enhanced image features; The boundary of the initial defect region is corrected based on the enhanced image features to obtain the defect location, and the defect depth corresponding to the defect location is determined from the point cloud data of the workpiece to be repaired. A repair strategy is generated based on the defect location and the defect depth, and the repair equipment is controlled to repair the workpiece based on the repair strategy.

2. The repair method according to claim 1, characterized in that, The image features are fused and enhanced using a neural network that includes detail enhancement and geometry adaptation branches to generate enhanced image features, including: The texture details of the image features are enhanced by the detail enhancement branch to obtain the first image feature; The geometry of the image features is enhanced by geometric adaptive branching to obtain a second image feature; The first image features and the second image features are weighted and fused to obtain enhanced image features.

3. The repair method according to claim 1, characterized in that, The enhanced image features include detail features and geometric shape features; Based on the enhanced image features, the boundary of the initial defect region is corrected to obtain the defect location, including: The defect range in the initial defect region is corrected based on the detailed features, and the boundary of the defect range is corrected based on the geometric features to obtain the defect location.

4. The repair method according to claim 1, characterized in that, A repair strategy is generated based on the defect location and the defect depth, and a repair device is controlled to repair the workpiece based on the repair strategy, including: The defect location and the defect depth are mapped to the virtual model corresponding to the workpiece to be repaired to obtain a set of defect coordinates; The movement path of the repair equipment is generated based on the set of defect coordinates, and the process parameters of the repair equipment are determined based on the defect depth and the movement path. The repair equipment is controlled to repair the workpiece based on the motion path and the process parameters.

5. The repair method according to claim 4, characterized in that, The motion path of the repair equipment is generated based on the set of defect coordinates, and the process parameters of the repair equipment are determined based on the defect depth and the motion path, including: The three-dimensional information and surface normal vector of the defect location are determined based on the defect coordinate set; The three-dimensional information and the surface normal vector are substituted into a preset algorithm to generate the motion path of the repair device; The spraying pressure and moving speed of the repair equipment are determined based on the defect depth at each defect point in the movement path.

6. The repair method according to any one of claims 1 to 5, characterized in that, The repair method also includes: Visual and acoustic resonance tests are performed on the repaired workpiece to determine whether the repair was successful. If the visual detection results indicate that there are defects on the surface of the repaired workpiece, or the acoustic resonance detection results indicate that there are defects on the surface of the repaired workpiece, then the repair is determined to have failed.

7. The repair method according to any one of claims 1 to 5, characterized in that, The initial defect region is identified based on the surface image of the workpiece to be repaired, including: Obtain surface images of each workpiece; Based on the surface images of each workpiece, the workpieces with surface defects to be repaired are identified, and the initial defect locations of the workpieces to be repaired are determined.

8. A device for repairing defects in a workpiece, characterized in that, The repair device includes: The feature extraction module is used to identify the initial defect region based on the surface image of the workpiece to be repaired, and to extract the image features of the initial defect region; The feature enhancement module is used to fuse and enhance the image features through a neural network that includes a detail enhancement branch and a geometric adaptation branch, to generate enhanced image features; The correction module is used to correct the boundary of the initial defect region based on the enhanced image features to obtain the defect location, and to determine the defect depth corresponding to the defect location from the point cloud data of the workpiece to be repaired. The repair module is used to generate a repair strategy based on the defect location and the defect depth, and control the repair equipment to repair the workpiece to be repaired based on the repair strategy.

9. An electronic device, characterized in that, include: Controller; A memory for storing one or more programs, which, when executed by a controller, cause the controller to implement the repair method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that, when executed by the computer's processor, cause the computer to perform the repair method according to any one of claims 1 to 7.

Images