Vision-based curved metal surface defect online detection method and device

Abstract

The invention provides a vision-based curved metal surface defect online detection method, which comprises the following steps of: selecting a plurality of vision scanning discrete optimization imaging points from a CAD (Computer Aided Design) model of a to-be-detected metal workpiece in a visual simulation platform, and generating a continuous vision scanning track; aiming at the metal workpiece obtained in the current processing procedure, enabling a machine tool to move according to a continuous visual scanning track, and obtaining a plurality of to-be-detected images with the same shooting angle and different incident light source conditions from the surface of the metal workpiece at each optimized imaging point; the optimized imaging points are input into a highlight removal model, and a fused image after highlight removal is output for each optimized imaging point; inputting the fused image into an interactive image segmentation model, and segmenting a to-be-detected area image of a single part from the fused image in cooperation with interactive prompt; and inputting the data into a defect detection model with an attention mechanism to obtain a defect detection result. According to the invention, real-time, efficient and high-precision defect detection of the complex curved surface metal workpiece in the machining process can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of complex metal surface defect detection, and in particular to a curved surface metal surface defect online detection method and device based on vision. BACKGROUND

[0002] The integral impeller of the core component of the aviation power equipment adopts a complex curved surface configuration, and the precision machining quality thereof directly affects the performance and service life of the equipment. In the five-axis milling forming process, under the superimposed action of the dynamic cutting force fluctuation and the tool wear, the surface of the blade type is prone to hidden defects such as micro-cracks, vibration marks and geometric distortion, thereby constituting potential quality hazards.

[0003] The existing quality detection system has double technical bottlenecks: the artificial visual inspection method is limited by the visual resolution and the subjective judgment deviation, and it is difficult to identify the micro-defect characteristics; and the traditional machine vision system is prone to feature mis-extraction and defect omission when dealing with metal processing texture interference, curved surface reflection distortion and other industrial scenes. Especially in the machine detection environment, the dynamic interference such as splashing of cutting fluid and sudden change of ambient light further reduces the detection reliability.

[0004] The existing visual detection technology cannot simultaneously solve the three technical contradictions of curved surface reflection suppression, micro-defect enhancement and online anti-interference, resulting in the pain point that the precision and efficiency of the in-machine quality inspection of the complex component cannot be achieved. It is urgent to develop an intelligent detection method that combines dynamic environment adaptability and feature decoupling capability to realize the machining-detection closed-loop quality control. SUMMARY

[0005] The present application aims to at least solve one of the technical problems existing in the prior art.

[0006] To this end, the present application provides a curved surface metal surface defect online detection method and device based on vision, which can realize real-time, efficient and high-precision defect detection of complex curved surface metal workpieces in the machining process, solve the problems of low efficiency and poor consistency of traditional manual detection, and effectively overcome the problems of metal surface highlights, strong edge interference and small features.

[0007] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0008] The present application provides a curved surface metal surface defect online detection method based on vision, which comprises the following steps:

[0009] Step S1, selecting a plurality of visual scanning discrete optimization imaging points of a to-be-detected metal workpiece from a CAD model of the to-be-detected metal workpiece with curved surface characteristics in a visual simulation platform, and generating a continuous visual scanning track based on all the optimization imaging points;

[0010] Step S2, for the metal workpiece obtained in the current machining process, the machine tool moves according to the continuous visual scanning track, and a plurality of to-be-detected images of the metal workpiece surface under the same shooting angle and different incident light source conditions are acquired at each optimized imaging point;

[0011] Step S3, input all the to-be-detected images into a pre-trained high light removal model, the high light removal model fuses each to-be-detected image located at the same optimized imaging point, realizes high light removal, and respectively outputs a fused image after high light removal for each optimized imaging point;

[0012] Step S4, input the fused image into a pre-trained interactive image segmentation model, and segment a to-be-detected region image of a single part from the fused image with interactive prompts;

[0013] Step S5, input the to-be-detected region image into a defect detection model with an attention mechanism to obtain a defect detection result.

[0014] In some embodiments, step S1 comprises:

[0015] Step S11, respectively set simulation models of the machine tool, the metal workpiece as a whole and the camera in the visual simulation platform, and the relative position relationship of each simulation model is set according to the actual machining scene, wherein the machine tool simulation model is used to simulate the movement of each axis of the machine tool, the metal workpiece as a whole simulation model is set on the machining table in the machine tool simulation model and is used to reflect the geometric characteristics of the to-be-detected metal workpiece as a whole, and the camera simulation model is arranged on the main shaft of the machine tool simulation model and is used to acquire a geometric model of the to-be-detected metal workpiece as a whole under camera imaging, and the geometric model is defined as a camera imaging geometric model;

[0016] Step S12, manually select the visual scanning discrete imaging points according to the geometric characteristics of the to-be-detected metal workpiece as a whole and the camera imaging geometric model in the visual simulation platform, and determine the visibility and / or interference points to obtain N visual scanning discrete optimized imaging points;

[0017] Step S13, obtain the coordinates of the N visual scanning discrete optimized imaging points in the machine tool coordinate system according to the machine tool simulation model, and generate a continuous visual scanning track according to the coordinates.

[0018] In some embodiments, in step S2, the image of the region to be detected is obtained by using a visual defect detection device clamped on the spindle of a machine tool, the visual defect detection device comprising a clamping mechanism, a camera, a communication module, a power supply and an image display module mounted on the clamping mechanism, and a multi-channel timing light source mounted on the camera; the clamping mechanism is fixed on the spindle of the machine tool through a tool holder, and the coaxiality between the lens of the camera and the spindle of the machine tool is ensured through the clamping mechanism; the camera is connected and fastened to the middle part of the clamping mechanism, used for shooting the surface of the metal workpiece to be detected during the movement of the machine tool, obtaining the image to be detected and transmitting it to the outside through the communication module; the multi-channel timing light source is fastened to the side of the camera lens, used for providing incident light sources in different directions, when the spindle of the machine tool reaches an optimized imaging point in the continuous visual scanning track, each light source of the multi-channel timing light source flashes in a set order, and at the same time, the camera shoots at the corresponding light source flashing, and M images are shot at the same camera angle.

[0019] In some embodiments, before shooting the image to be detected, the visual defect detection device needs to be calibrated, and the calibration parameters include a virtual tool length L and a spindle positioning angle θ, the virtual tool length L is the distance between the tool holder mounting surface and the camera focal plane, and the spindle positioning angle θ is the rotation angle of the camera coordinate system and the machine tool coordinate system around the machine tool Z axis, and the theoretical spindle positioning angle is 0°.

[0020] The calibration of the virtual tool length L includes:

[0021] The visual defect detection device is installed on the spindle of the machine tool, and the camera is used to continuously collect images; the machine tool Z axis is moved so that the camera images the machine tool rotary table; the image definition B of the collected image is calculated, and the Z coordinate of the position with the maximum B value is recorded as the virtual tool length L.

[0022] The calibration of the spindle positioning angle θ includes:

[0023] The visual defect detection device is installed on the spindle of the machine tool, the camera is moved so that it is aligned with an edge of the machine tool rotary table for shooting, and an edge detection algorithm is used to detect the image edge of the shot image; the included angle between the detected image edge and the machine tool X axis direction is calculated as the spindle positioning angle.

[0024] In some embodiments, the training of the highlight removal model includes:

[0025] A plurality of metal surface image pairs under different exposure conditions are generated based on a generative adversarial data set expansion algorithm, the metal surface image pair is composed of a highlight image and a corresponding highlight-free image, a label-free real highlight image for unsupervised training is obtained, and the metal surface image pair and the label-free real highlight image are used to constitute a training data set.

[0026] A high-light removal model is built based on the MEF-Net and by introducing a detail enhancement module and a color enhancement module; the detail enhancement module is embedded between a down-sampling layer of the MEF-Net and a feature prediction network, and reduces high-frequency texture loss through multi-scale context feature fusion; the detail enhancement module adopts a bidirectional detail enhancement strategy: for underexposed areas, a standard deviation is used to adaptively adjust contrast; for overexposed areas, an original image is inverted and a standard deviation is used to adaptively adjust contrast to restore details; the color enhancement module is connected at a stage of chroma channel fusion in a YCbCr color space of the MEF-Net, and learns a chroma mapping relationship through an encoder-decoder structure;

[0027] The high-light removal model is trained by using the training data set and an unsupervised learning strategy, to obtain a trained high-light removal model.

[0028] In some embodiments, an interactive image segmentation model is constructed by using the SAM-Adapter model, and the interactive image segmentation model includes an image encoder, a prompt encoder and a mask decoder connected in sequence; the image encoder is used to perform multi-scale feature extraction on an input fused image after high-light removal, to generate a high-dimensional semantic feature map; the prompt encoder encodes an input interactive prompt, to generate a prompt embedding vector aligned with the semantic feature map; and the mask decoder dynamically generates a segmentation mask according to the semantic feature map generated by the image encoder and the prompt embedding vector generated by the prompt encoder.

[0029] In some embodiments, the input of the fused image into the pre-trained interactive image segmentation model, and the segmentation of a single part to-be-detected region image from the fused image in cooperation with an interactive prompt, includes:

[0030] The fused image is input into the pre-trained interactive image segmentation model, and a metal workpiece overall region and irrelevant background in the fused image are segmented according to an interactive prompt 1 input by a user, to obtain a metal workpiece overall region image; the interactive prompt 1 adopts any one or a combination of multiple of non-iterative prompts, iterative prompts, ambiguous prompts, prophetic mechanisms and variability prompts;

[0031] The interactive image segmentation model is caused to segment a single part to-be-detected and irrelevant parts in the metal workpiece overall region image according to an interactive prompt 2 input by a user, to obtain a single part to-be-detected region image; the interactive prompt 2 adopts an iterative prompt strategy, and in the metal workpiece overall region image, for a single part, a positive prompt is generated at a key position in the metal workpiece overall region, and multiple negative prompts are generated around the positive prompt; through multiple iterations of the prompts, irrelevant part regions are removed, so that the single part to-be-detected and irrelevant parts are segmented.

[0032] The interactive image segmentation model is caused to segment a to-be-detected region and an irrelevant detection region in a single part region image to be detected according to an interactive prompt 3 input by a user, to obtain a to-be-detected region image of a single part; the interactive prompt 3 adopts an iterative prompt strategy, generates a positive prompt in a part to-be-detected region and a negative prompt in an irrelevant region of the part in the single part region image, adjusts the prompt position and the positive and negative prompt distribution multiple times, and segments the to-be-detected region and the irrelevant region of the single part, the to-be-detected region being a non-edge region of a part surface in which a defect has occurred, and the irrelevant region being a region in which an interference pixel is located on the part surface.

[0033] In some embodiments, the defect detection model with an attention mechanism replaces a Concat module in a feature fusion part of an existing single-level target detection model with a bidirectional feature pyramid network, inserts a convolution block attention module after a convolution module in the feature fusion stage, and introduces a cross-level skip connection in a head network in the existing single-level target detection model.

[0034] When training the defect detection model with an attention mechanism, multiple training images are randomly spliced into a single composite image based on a mosaic data enhancement strategy, as input of the defect detection model with an attention mechanism; and a total loss function adopted is a weighted sum of a classification loss, a positioning loss and a confidence loss, and the positioning loss adopts an EIoU loss.

[0035] In some embodiments, before inputting the to-be-detected region image into the defect detection model with an attention mechanism, the to-be-detected region image is subjected to mosaic enhancement and normalization, to generate a multi-scale input image.

[0036] The multi-scale input image is extracted by a backbone network of the defect detection model with an attention mechanism to obtain multi-scale features, channel and spatial attention weights are fused by the bidirectional feature pyramid network and the convolution block attention module, and an enhanced defect feature map is output; a detection head of the defect detection model with an attention mechanism generates a candidate box based on the enhanced defect feature map, combines EIoU-NMS screening, and outputs a defect category, a position and a confidence, as the defect detection result.

[0037] The second aspect of the present application provides a visual-based on-line detection device for defects on a curved metal surface, comprising:

[0038] A continuous visual scanning trajectory generation module is configured to select a plurality of visual scanning discrete optimized imaging points of a to-be-detected metal workpiece from a CAD model of the to-be-detected metal workpiece with curved features in a visual simulation platform, and generate a continuous visual scanning trajectory based on all the optimized imaging points.

[0039] An image to be detected acquisition module is configured to, for a metal workpiece obtained in a current machining process, make a machine tool move according to the continuous visual scanning track, and acquire multiple images to be detected of the metal workpiece surface at each optimized imaging point under the condition of the same shooting angle and different incident light sources.

[0040] A high light removal module is configured to input all the images to be detected into a pre-trained high light removal model, the high light removal model fuses each image to be detected located at the same optimized imaging point, realizes high light removal, and outputs a fused image after high light removal for each optimized imaging point.

[0041] An interactive image segmentation module is configured to input the fused image into a pre-trained interactive image segmentation model, and segment a single part image to be detected from the fused image according to an interactive prompt.

[0042] A defect detection module is configured to input the image to be detected into a defect detection model with an attention mechanism to obtain a defect detection result.

[0043] Compared with the prior art, the present application has the following advantages:

[0044] 1、The present application automatically generates a continuous scanning path by integrating a visualization platform and an intelligent trajectory planning algorithm, and supports a manual teaching mode, thereby significantly improving detection efficiency and positioning accuracy, solving the problems of low efficiency and incomplete path coverage of traditional manual programming, and realizing full-automatic high-precision detection of complex curved surfaces.

[0045] 2、The present application solves the technical bottleneck that traditional fixed detection equipment cannot be compatible with special-shaped workpieces by designing a visual defect detection device, and is particularly suitable for online detection of complex curved metal parts. The high reliability of the detection data is ensured by the embedded wireless transmission system, and the real-time image processing pipeline is developed to realize the synchronization of the machining process and quality detection. Compared with the traditional offline detection method, the risk of misjudgment caused by manual intervention is reduced.

[0046] 3、The present application effectively eliminates the interference of metal surface reflection by using a multi-image fusion high light suppression technology, completely retains the microscopic features of the workpiece surface, overcomes the defect of detail loss in the high light area compared with the traditional single-frame image processing method, ensures the uniform imaging quality of the metal workpiece surface with different curvatures, and improves the imaging effect of special structures such as deep holes and grooves compared with the single light source detection scheme.

[0047] 4、The detection architecture combining the region segmentation algorithm with the single-stage target detection model with the attention mechanism is adopted, local feature enhancement and edge noise suppression are carried out, small defects are accurately recognized, and compared with the conventional convolutional neural network method, the false positive rate in the strong edge interference scene is obviously reduced. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 It is the whole flow chart of the online detection method for the curved metal surface defects based on vision provided by the first aspect embodiment of the application;

[0049] Figure 2 It is Figure 1 The flow chart of generating the continuous imaging track in the detection method shown in the figure;

[0050] Figure 3 In (a) and (b) of the figure, the visual defect detection equipment used in the detection method shown in the figure is the structural schematic diagram under the first view angle and the second view angle; Figure 1

[0051] Figure 4 In (a) and (b) of the figure, the virtual tool length L and the main shaft positioning angle θ set by the visual defect detection equipment shown in the figure are shown; Figure 3

[0052] Figure 5 It is Figure 1 The flow chart of carrying out the multi-image fusion and the highlight removal processing on the acquired detection image in the detection method shown in the figure;

[0053] Figure 6 It is Figure 1 The flow chart of carrying out the image segmentation processing on the image after the highlight removal processing in the detection method shown in the figure;

[0054] Figure 7 It is the schematic diagram of the image segmentation of the original image by using different types of non-iterative prompts by the embodiment of the application, wherein (a) is the original image, (b)-(e) are respectively single-point non-iterative, double-point non-iterative, single-frame non-iterative and single-frame multi-region non-iterative prompts;

[0055] Figure 8 It is the schematic diagram of the image segmentation of the original image by using iterative prompts by the embodiment of the application, wherein (a) is the original image, (b)-(d) are respectively the first time iterative prompt, the second time iterative prompt and the third time iterative prompt;

[0056] Figure 9 It is the schematic diagram of the image segmentation of the original image by using ambiguous prompts by the embodiment of the application. DETAILED DESCRIPTION

[0057] ​​In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and not to limit the present application. The following solutions are only used to explain the present application and the specific solutions are not limited thereto. In addition, only the parts related to the present application are shown in the drawings for convenience of description, rather than all the processes.

[0058] On the contrary, the present application covers any alternative, modification, equivalent method and solution defined by the claims within the essence and scope of the present application. Further, in order to make the public better understand the present application, some specific details are described in the following detailed description of the present application. The present application can also be completely understood without the description of these details by those skilled in the art.

[0059] Referring to Figure 1 The first aspect embodiment of the present application provides a visual-based online detection method for surface defects of a curved metal surface, comprising the following steps:

[0060] Step S1, in a visual simulation platform, a plurality of visual scanning discrete optimization imaging points of a metal workpiece to be detected are manually selected from a CAD model of the metal workpiece to be detected having a curved surface feature, and a continuous visual scanning trajectory is generated based on all the optimization imaging points;

[0061] Step S2, for a metal workpiece obtained in a current machining process, the machine tool moves according to the continuous visual scanning trajectory, and at each optimization imaging point, a visual defect detection device clamped on the main shaft of the machine tool is used to take pictures of the metal workpiece surface under the same camera angle and different incident light source conditions, thereby obtaining a plurality of images to be detected;

[0062] Step S3, all the images to be detected obtained in step S2 are input into a pre-trained high light removal model, the high light removal model fuses each image to be detected located at the same optimization imaging point, realizes high light removal, and outputs one fused image after high light removal for each optimization imaging point;

[0063] Step S4, the fused image after high light removal is input into a pre-trained interactive image segmentation model, and an interactive prompt is used to segment the image of a detection area of a single part from the fused image;

[0064] Step S5, the image of the detection area is input into a defect detection model with an attention mechanism, and a defect detection result is obtained.

[0065] It should be noted that the "online" in the method of the embodiment refers to that the detection process is synchronized with the metal workpiece processing process, that is, after the workpiece completes the machining process of the machine tool, the surface defect detection of the workpiece is directly completed through the visual defect detection equipment clamped on the spindle of the machine tool without taking the workpiece off the machine tool, and the detection is directly entered into the next machining process to realize the "machining-detection" closed-loop integration. In addition, the detection method assumes that the defects on the surface of the metal workpiece only occur at the non-edge of the surface of the metal workpiece, and for the case that the edge of the metal workpiece has defects, it is considered that there is a major mistake in the machining process of the machine tool, which is not within the research scope of the method of the application.

[0066] The specific implementation process of the curved surface metal surface defect online detection method proposed in the application will be further described in detail below in combination with the application example of a five-axis numerical control machine tool machining an integral impeller.

[0067] In some embodiments, referring to Figure 2 , the step S1 specifically comprises:

[0068] The step S11 sets simulation models of a machine tool, a to-be-machined integral impeller and a camera in a visual simulation platform, respectively, and sets relative position relationships of the simulation models according to an actual machining scene, wherein the machine tool simulation model is used to simulate movement of each axis of the machine tool, the to-be-machined integral impeller simulation model (which can use a CAD model) is set on a machining table in the machine tool simulation model to reflect geometric features of the to-be-detected integral impeller, and the camera simulation model is arranged on a spindle of the machine tool simulation model to obtain a geometric model of the to-be-detected integral impeller under camera imaging, and the geometric model is defined as a camera imaging geometric model.

[0069] The step S12 manually selects visual scanning discrete imaging points and judges visibility and / or interference points according to geometric features of the to-be-detected integral impeller and the camera imaging geometric model in the visual simulation platform to obtain N visual scanning discrete optimization imaging points.

[0070] The step S13 obtains coordinates of the N visual scanning discrete optimization imaging points in a machine tool coordinate system according to the machine tool simulation model, compiles nc code according to the coordinates, and generates a continuous visual scanning track.

[0071] Further, the visual simulation platform is built by using an existing commercial simulation software to create a visual environment for manual selection of discrete imaging points and judgment of visibility and / or interference points. The purpose of the judgment of the visibility and / or interference points is to ensure that a plurality of to-be-detected images obtained by visual scanning can completely cover all surface type information of the integral impeller.

[0072] It can be understood that the embodiment of the present application presents the three-dimensional model of the metal workpiece to be detected and the image thereof under the camera visual angle through the visual simulation platform, and the tool path and machining trajectory in the numerical control machining process can be intuitively understood through the graphical display, the complex machining process is presented in an intuitive manner, and the scanning trajectory planning of the complex shape workpiece is realized.

[0073] In some embodiments, step S2 specifically comprises:

[0074] S21, installation and calibration of the visual defect detection device

[0075] Installation of the visual defect detection device: a visual defect detection device is clamped on the main shaft of the machine tool, as shown in (a) and (b) of Figure 3 The visual defect detection device comprises a clamping mechanism 3, a camera 2, a communication module and a power supply 4 and an image display module 5 installed on the clamping mechanism 3, and a multi-channel time sequence light source 1 installed on the camera 2; wherein the clamping mechanism 3 serves as the main carrier of the device, is fixed on the main shaft of the machine tool through a tool handle, realizes the installation and positioning of the visual defect detection device and the machine tool, and ensures the coaxiality between the lens of the camera 2 and the main shaft of the machine tool; the camera 2 adopts a high-resolution industrial camera, is fastened in the middle of the clamping mechanism 3 through a screw, is used for shooting the surface of the metal workpiece to be detected during the movement of the machine tool, acquiring the detected image and transmitting the detected image to the industrial computer through the communication module; the image display module 5 is an LCD screen, which is used for real-time display of the detected image shot by the camera 2 for the operator, facilitating online preview; in the communication module and the power supply 4, the communication module is used for wireless transmission of data, and the power supply is used for power supply for the camera 2, the multi-channel time sequence light source 1 and the communication module and other electronic devices; the multi-channel time sequence light source 1 adopts a four-channel multi-angle ring LED multi-time sequence light source, is fastened on the side of the camera lens through a screw, and is used for providing incident light sources in different directions to ensure that the camera 2 acquires a clear image.

[0076] The visual defect detection device provided by the embodiment has high integration and is small and light, and can be adapted to different types of machine tools through the tool handle. After the machining of the workpiece of the current process is completed, the visual defect detection device is taken out from the tool magazine of the machine tool and the surface image of the workpiece to be detected is acquired, which is used for subsequent defect detection; after the current defect detection is completed, the tool is taken out from the tool magazine of the machine tool for machining of the next process.

[0077] Calibration of the visual defect detection device: the parameters of the visual defect detection device include the geometric size of the visual defect detection device, the image positioning accuracy, the imaging pixel, the virtual tool length L, the main shaft positioning angle θ and the camera positioning error, and the key parameters are the virtual tool length L and the main shaft positioning angle θ. Before the visual defect detection device is used to acquire the detected image, the two parameters of the visual defect detection device need to be calibrated, as shown inFigure 4 Wherein (a), (b) are the definitions of virtual tool length L and spindle orientation angle θ, respectively. Wherein:

[0078] The virtual tool length L is defined as the distance between the tool holder mounting surface and the camera focal plane. This parameter affects the definition of the visual image, so it needs to be calibrated before image acquisition. The calibration process is as follows: install the visual defect detection device on the machine tool spindle, continuously acquire images using an industrial camera; manually move the machine tool Z axis so that the camera 2 images the machine tool rotary table; calculate the image definition B of the acquired image, and record the Z coordinate of the maximum B value, which is the virtual tool length L. There are more than one way to calculate the image definition B, such as Laplacian variance, Brenner function, information entropy function, etc. In this embodiment, the Brenner function is used to evaluate the image definition, and the calculation formula is as follows:

[0079]

[0080] In the formula, I(x, y) is the gray value of the image at position (x, y), and I(x+2, y) is the gray value of the image at position (x+2, y).

[0081] The spindle orientation angle θ is defined as the rotation angle of the camera coordinate system and the machine tool coordinate system around the machine tool Z axis. The theoretical spindle orientation angle is 0°. The calibration process of the spindle orientation angle θ is as follows: install the visual defect detection device on the machine tool spindle, turn on the camera 2 to prepare for image acquisition; manually move the camera 2 so that the camera 2 is aligned with the edge of the machine tool rotary table for shooting; use edge detection algorithm to detect the image edge of the shot image; calculate the angle between the detected image edge and the machine tool X axis direction as the spindle orientation angle, and adjust the spindle orientation angle by manually rotating the machine tool spindle until the spindle orientation angle is 0°.

[0082] S22, image acquisition

[0083] After the installation and calibration of the visual defect detection equipment are completed, the machine tool is controlled to move according to the continuous visual scanning trajectory generated in step S1, and the visual defect detection equipment clamped on the machine tool spindle is used to capture the overall impeller surface under the same camera angle and different incident light source conditions at the N optimized imaging points, so as to obtain M images to be detected, and a total of NXM images to be detected are obtained. Specifically, when the machine tool spindle reaches a certain optimized imaging point in the continuous visual scanning trajectory, each light source of the multi-path time sequence light source 1 flashes in turn, and the camera 2 captures at the corresponding light source flashing time, and M images are captured under the same camera angle. Each image obtained at the N visual scanning imaging points covers a part of the overall impeller surface to be detected, and finally the whole surface type information of the overall impeller to be detected is contained in the N images. In this embodiment, M=5 images to be detected are captured at each optimized imaging point, of which 4 images are captured when each light source of the multi-path time sequence light source 1 is turned on in turn, and the remaining 1 image is captured when the light sources of the multi-path time sequence light source 1 are turned on at the same time. The images captured by the camera 2 are transmitted to the industrial computer through the communication module by using a high-speed data transmission protocol, and subsequent image processing is performed. In this embodiment, multiple images are captured at each optimized imaging point, and the light source angle of each imaging is different, so that the highlight in the collected impeller image appears at different positions, which provides a basis for subsequent highlight removal processing.

[0084] In some embodiments, in step S3, all the images to be detected obtained in step S2 are input into a highlight removal model, which is based on a MEF-Net (trained by unmarked highlight image data) introducing a Detail Enhance Model (DEM) and a Color Enhance Model (CEM) to process the highlight metal image sequence under different exposure angles: after down-sampling, the low-resolution weight map is predicted by the CAN network, combined with the original image and the multi-scale feature map, and the high-resolution weight map is obtained by using the guide filter, W k represents the weight map of the original image X k , and Y is the unhighlighted image obtained by final weighted fusion. By fusing the multi-scale images, the highlight-removed picture is obtained.

[0085] Further, referring to Figure 5 , step S3 specifically includes:

[0086] Step S31, construction and training of the highlight removal model

[0087] Training dataset acquisition: The acquisition of the training dataset is based on the data set expansion algorithm (DCGAN) of the generative confrontation, which solves the problem of the scarcity of high light image data in the real scene by generating metal surface image pairs under different exposure conditions. This algorithm uses a generator to synthesize metal images containing simulated highlights (such as specular reflection, overexposure), and optimizes it with a discriminator to generate realistic highlight-no highlight image pairs, i.e. metal surface image pairs. The unmarked real highlight image in the training dataset is a real metal image without labeled highlight area, which is used for unsupervised training.

[0088] Building a highlight removal model: The highlight removal model architecture is based on MEF-Net, which introduces a depth perception enhancement module (DPM) containing a detail enhancement module (DEM) and a color enhancement module (CEM); wherein,

[0089] The DEM module is embedded between the downsampling layer of MEF-Net and the feature prediction network (CAN), reducing high-frequency texture loss through multi-scale context feature fusion. In the specific design, a bidirectional detail enhancement strategy is adopted: for underexposed areas, the contrast is adjusted adaptively by standard deviation, and for overexposed areas, the details are restored by inverting the original image and adjusting the contrast adaptively using standard deviation;

[0090] The CEM module is connected to the YCbCr color space chroma channel fusion stage of MEF-Net, learning the chroma mapping relationship through an encoder-decoder structure (4 branches per layer). The input contains the luminance and chrominance components of the original image, and the output is the adaptively corrected chrominance information, solving the color distortion problem caused by the nonlinear relationship between luminance and chrominance in traditional methods.

[0091] The overall process of the highlight removal model is as follows: after the input multi-exposure image is encoded by MEF-Net, the texture details are enhanced by DEM, and then the chrominance components are corrected by CEM, finally the highlight-free image is generated by fusion.

[0092] The metal surface image pairs synthesized by DCGAN and the unmarked real metal images are input into the above highlight removal model, and are trained using unsupervised learning methods. Specifically, an unsupervised learning strategy is adopted, and the loss function is replaced by the multi-scale structural similarity (MSSSIM) instead of the traditional SSIM, which improves the perception quality of the highlight area by calculating the contrast and structural similarity in multiple scales. The total loss function weight is set to λ1=15 (detail preservation) and λ2=0.5 (color consistency) through experimental optimization. The training parameters include the Adam optimizer (learning rate 0.0001, momentum β1=0.5), Batch-size equal to the number of exposure images, and the termination condition is 200 epochs or the loss decreases by less than 1e-4 for 5 consecutive iterations.

[0093] Step S32, highlight removal

[0094] The plurality of to-be-detected images obtained in step S2 are input into the trained high-light removal model in batches, wherein each to-be-detected image located at the same optimized imaging point is input into the high-light removal model as a batch, and a high-light-removed fused image for the corresponding optimized imaging point is output.

[0095] It can be understood that image high-light removal is an important link of defect detection, which is caused by the characteristics of the detection object of the embodiment, i.e., the overall impeller. On the one hand, the material of the overall impeller is often aluminum alloy, stainless steel, titanium alloy, etc. After milling, the surface often has a mirror effect. The characteristics of the metal material cause the high light of the surface of the metal workpiece to cover part of the texture details. For underexposed images, high dynamic range information is squeezed in a limited range, and for overexposed images, information is shifted up and truncated, both of which can cause the image contrast to decrease and the details to be destroyed, thereby seriously affecting the effect of defect detection. On the other hand, the overall impeller belongs to a complex curved surface. No matter how the optical system is designed, local high light will appear. Therefore, the design idea of a common optical system cannot remove high light from the root. The single-image self-optimization method has poor high-light removal effect and can cause a large amount of texture information to be lost. Based on the above reasons, the embodiment designs a high-light removal model based on a multi-image fusion high-light removal method. This method can almost perfectly eliminate the high light of the metal surface while preserving the details of the metal surface of the image as much as possible. In summary, through step S2, multiple images are taken at a single position, and the angles of the light sources for each imaging are different, so that the high light in the collected impeller images appears at different positions. Then, a high-light-free image can be synthesized through the multi-image fusion method, thereby achieving the purpose of removing the high light of the image.

[0096] In some embodiments, after the high-light removal processing is completed, the N fused images obtained are respectively subjected to image segmentation using a pre-trained interactive image segmentation model and in cooperation with interactive prompts, and the to-be-detected regions of a single blade are extracted from each fused image, thereby obtaining all to-be-detected region images of a single blade. Now, the specific steps of step S4 are described in detail taking one of the fused images as an example. Figure 6 The specific steps of step S4 are described in detail taking one of the fused images as an example.

[0097] Step S41, construction and training of an interactive image segmentation model

[0098] The interactive image segmentation model is constructed, the image segmentation model adopts the SAM-Adapter model, and includes an image encoder, a prompt encoder and a mask decoder connected in turn; the image encoder is an image encoder of the SAM, performs multi-scale feature extraction on the input fusion image after high light removal, generates a high-dimensional semantic feature map, the image encoder encodes the global context information of the input image to capture complex edge, texture and illumination change features, and provides basic visual representation for subsequent segmentation; in the embodiment, the image encoder adopts a ViT-H16 architecture, the framework has 14x14 window attention and four equidistant global attention blocks, in the training process, the weights of the pre-trained image encoder are kept unchanged, and the mask encoder of the SAM is also used, which is composed of an improved transformer decoder module and a dynamic mask prediction head, the weights of the mask decoder are initialized using the weights of the pre-trained SAM, and the mask decoder is adjusted during training, and no prompt is input in the original mask encoder of the SAM. The prompt encoder encodes the interactive prompt (such as point, box, text, etc.) input by the user to generate a prompt embedding vector aligned with the semantic feature map obtained by the image encoder, and the prompt encoder aims to convert the user's intention (such as foreground / background label) into spatial attention weight to guide the interactive image segmentation model to focus on the target area; the mask decoder adopts an improved transformer decoder, dynamically generates a segmentation mask according to the semantic feature map generated by the image encoder and the prompt embedding vector generated by the prompt encoder, and the mask decoder fuses image and prompt information through cross-attention mechanism, outputs an accurate target area binary mask, and suppresses irrelevant background at the same time. The training data set of the interactive image segmentation model adopts multi-scene images containing complex curved metal workpieces and corresponding fine segmentation masks, covering different light conditions, curvature of curved surface and surface state; the training strategy adopts supervised learning, calculates the difference between the predicted mask and the real mask by using the cross entropy loss function, and uses the Adam optimizer for iterative training to improve the segmentation accuracy of the model for metal curved surface features.

[0099] Step S42, input the fusion image into the interactive image segmentation model, and segment the overall impeller region and irrelevant background in the fusion image according to the user input interactive prompt 1 to obtain an overall impeller region image. The interactive prompt 1 can adopt any one or a combination of interactive prompt strategies such as a non-iterative prompt (single input prompt such as a point / box, directly generating a result, suitable for a scene with clear target and simple structure), an iterative prompt (gradually optimizing a mask through multiple rounds of positive / negative prompts to solve complex occlusion or weak edge problems), an ambiguous prompt (gradually optimizing a mask through multiple rounds of positive / negative prompts to solve complex occlusion or weak edge problems), a prophetic mechanism (when a prompt points to multiple possible objects, the model outputs Top-3 candidate masks for the user to select, and automatically recommends an optimal solution in combination with an IoU score), and a variability prompt (analyzing prompt position sensitivity, layering a target region according to the distance from the edge, and preferentially selecting a central sub-region prompt to improve stability); see Figure 7 Fig. 2(a)-(e) are schematic diagrams of the original image and the images after single-point non-iterative, double-point non-iterative, single-box non-iterative, and single-box multi-region non-iterative prompts, respectively, and a box can be selected to tightly surround the impeller mask in the fusion image; see Figure 8 Fig. 3(a)-(d) are schematic diagrams of the original image and the images after the first, second, and third iterative prompts, respectively, and the interactive image segmentation model can be prompted once to segment the overall impeller, and the segmentation result can be gradually improved through subsequent prompts to remove irrelevant background, thereby segmenting the overall impeller region and irrelevant background; see Figure 9 Fig. 4 is a process schematic diagram of an ambiguous prompt.

[0100] Step S43, make the interactive image segmentation model segment the single blade to be detected and irrelevant blades in the overall impeller region image according to the user input interactive prompt 2 to obtain a single blade region image to be detected; the interactive prompt 2 adopts an iterative prompt strategy, in the overall impeller region image, for a single blade, positive prompts are generated at key positions such as the center of the overall impeller region, and multiple negative prompts are generated around the positive prompts, through multiple iterative prompts, unnecessary irrelevant blade regions are removed, and thus the single blade to be detected and irrelevant blades are segmented.

[0101] Step S44, the interactive image segmentation model is caused to segment the region to be detected of the single blade in the single blade region image to be detected according to the interactive prompt 3 input by the user, such as the blade suction surface, the blade leading edge and the blade trailing edge and the irrelevant detection region, and a final segmentation image, that is, the single blade region image to be detected, is obtained, which is used for subsequent defect detection. The interactive prompt 3 adopts an iterative prompting strategy, positive prompts are generated in the blade region to be detected, and negative prompts are generated in the irrelevant region of the blade, the position of the prompt and the distribution of the positive and negative prompts are adjusted for multiple iterations, and the single blade region to be detected and the irrelevant region of the blade are segmented. The region to be detected is a non-edge region (excluding strong edges) on the surface of the blade, and the middle region prone to defects is focused on; the irrelevant region is a region where the edge, shadow, scratch and other interference pixels are located.

[0102] It can be understood that the interactive image segmentation model provided by the embodiment of the application can realize segmentation of an impeller image containing multiple blades to obtain an image containing only a single blade region to be detected. For a metal workpiece with complex edge features, the metal workpiece has a complex image structure, which will seriously affect the subsequent defect detection process. After convolution processing, the defect is easy to be confused with the edge feature, and therefore the image edge needs to be removed before defect detection. In the embodiment, the interactive prompting strategy is designed to realize the segmentation of the impeller image based on the current mainstream image segmentation model. In order to solve the problem of different segmentation granularities in the image segmentation process, the single blade region to be detected is gradually located from the fusion image obtained in step S3.

[0103] In some embodiments, step S5 specifically comprises:

[0104] Step S51, model architecture improvement:

[0105] The feature fusion part in the original lightweight network One-stage (single-stage target detection) model is improved. The original feature fusion part adopts a Concat module for multi-scale feature splicing. An high-precision defect detection model is constructed in the embodiment of the application. The defect detection model replaces the Concat module in the existing single-stage target detection model with a bidirectional feature pyramid network (BiFPN), enhances cross-scale feature interaction through weighted fusion, and optimizes the context information transmission of micro defects.

[0106] A convolutional block attention module (CBAM) is inserted after the convolution module in the feature fusion stage, specifically at the three key feature layers (such as 80x80, 40x40, and 20x20 scales) output by the backbone network. The CBAM is composed of a channel attention mechanism (CAM) and a spatial attention mechanism (SAM). The CAM performs global max-pooling and average-pooling on the input feature map (HxWxC), generates channel weights through a shared multi-layer perceptron (Shared MLP), and then multiplies the original features element by element after Sigmoid activation to strengthen the defect-related channels.

[0107] The CAM: Global max-pooling and average-pooling are performed on the input feature map (HxWxC), channel weights are generated through a shared multi-layer perceptron (Shared MLP), and then the original features are multiplied element by element after Sigmoid activation to strengthen the defect-related channels.

[0108] The SAM: The features output by the CAM are subjected to max-pooling and average-pooling in the channel dimension, concatenated into an HxWx2 feature map, and then subjected to 7x3 convolution to generate spatial weights. The original features are then weighted after Sigmoid activation to focus on the spatial positions of the defect regions.

[0109] Finally, the network structure is adjusted by replacing the standard convolution module (Conv) at the end of the backbone network with a CBAM. Specifically, a single CBAM module is introduced to complete the convolutional enhancement in the feature fusion stage and replace the convolution at the end of the backbone network. This design not only strengthens the channel and spatial attention weights in the feature fusion process but also optimizes the feature extraction capability at the end of the backbone network, ultimately improving the detection accuracy of micro-defects. Meanwhile, cross-level skip connections are introduced in the head network to enhance the transmission efficiency of micro-defect features.

[0110] Step S52, training strategy optimization:

[0111] In the data preprocessing stage, the Mosaic data augmentation strategy is used to randomly splice four training images into a single composite image, which is used as the input of the defect detection model with attention mechanism. This strategy expands the diversity of data distribution and alleviates the problem of insufficient small sample defect data.

[0112] EIoU (Enhanced IoU) is used instead of CIoU as the positioning loss to accelerate model convergence and improve the regression accuracy of the prediction box. The total loss function is the weighted sum of the classification loss (Focal Loss), the positioning loss (EIoU), and the confidence loss.

[0113] And the Adam optimizer (learning rate 0.001, momentum β1=0.9) is used, the batch-size is 16, the training period is 300 epochs, the learning rate is fixed for the first 250 epochs, and the learning rate is linearly decayed to 0 for the last 50 epochs.

[0114] Step S53, actual detection:

[0115] First, input preprocessing is performed, and the image of the to-be-detected region segmented in step S4 is subjected to Mosaic enhancement and normalization to generate a multi-scale input image (such as 640*640).

[0116] Then, the multi-scale input image is extracted through the backbone network (CSPDarknet53) of the defect detection model with attention mechanism to obtain multi-scale features, and the channel and spatial attention weights are fused through the BiFPN and CBAM modules to output enhanced defect feature maps.

[0117] Finally, the detection head of the defect detection model with attention mechanism is used for prediction, specifically, the detection head generates candidate boxes based on the enhanced defect feature maps, combines EIoU-NMS screening, and outputs the defect class, position and confidence.

[0118] It can be understood that after preprocessing, that is, the to-be-detected region image of each single leaf after removing highlights based on interactive prompts and image segmentation will be input into the defect detection model, which can efficiently identify various defects that may exist on the surface of the leaf, such as cracks, pits, scratches, etc. Finally, the image after defect detection will be marked with defect information to obtain the defect detection result and form an image with defect identification for subsequent analysis and processing. This series of processes not only ensures efficient detection of metal curved surface defects, but also realizes online detection, thereby greatly improving production efficiency and reducing the cost and error of manual detection.

[0119] The second aspect embodiment of the present application provides a visual-based curved metal surface defect online detection device, comprising:

[0120] A continuous visual scanning trajectory generation module is configured to select a plurality of visual scanning discrete optimization imaging points of a to-be-detected metal workpiece from a CAD model of the to-be-detected metal workpiece with curved surface characteristics in a visualization simulation platform, and generate a continuous visual scanning trajectory based on all optimization imaging points;

[0121] A to-be-detected image acquisition module is configured to, for a metal workpiece obtained in a current machining process, make a machine tool move according to the continuous visual scanning trajectory, and acquire a plurality of to-be-detected images of the metal workpiece surface under the same shooting angle and different incident light source conditions at each optimization imaging point;

[0122] The high light removal module is configured to input all the to-be-detected images into a pre-trained high light removal model, the high light removal model fuses each to-be-detected image located at a same optimized imaging point, removes high light, and respectively outputs a fused image after high light removal for each optimized imaging point;

[0123] The interactive image segmentation module is configured to input the fused image into a pre-trained interactive image segmentation model, and segment a to-be-detected region image of a single part from the fused image in cooperation with an interactive prompt;

[0124] The defect detection module is configured to input the to-be-detected region image into a defect detection model with an attention mechanism to obtain a defect detection result.

[0125] It should be noted that the foregoing embodiment of the method for on-line detection of defects on a curved metal surface based on vision is also applicable to the on-line detection device for defects on a curved metal surface based on vision in the embodiment, and will not be described here again.

[0126] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms must be directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, different embodiments or examples described in the present specification and the features of different embodiments or examples can be combined and combined by those skilled in the art without contradiction.

[0127] Although the embodiments of the present application have been shown and described above, it should be understood that the above-described embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments within the scope of the present application.

Claims

1. A vision-based online detection method for defects on curved metal surfaces, characterized in that, include: Step S1: In the visualization simulation platform, select multiple visual scanning discrete optimization imaging points of the metal workpiece to be inspected from the CAD model of the metal workpiece to be inspected with curved surface features, and generate a continuous visual scanning trajectory based on all optimization imaging points. Step S2: For the metal workpiece obtained in the current processing step, make the machine tool move according to the continuous visual scanning trajectory, and acquire multiple images of the metal workpiece surface at the same shooting angle and different incident light source conditions at each optimized imaging point; Step S3: Input all images to be detected into a pre-trained specular removal model. The specular removal model fuses the images to be detected located at the same optimized imaging point to achieve specular removal. For each optimized imaging point, it outputs a fused image after specular removal. Step S4: Input the fused image into a pre-trained interactive image segmentation model, and with interactive prompts, segment the detection region image of a single part from the fused image; Step S5: Input the image of the region to be detected into a defect detection model with an attention mechanism to obtain the defect detection result.

2. The online detection method for defects on curved metal surfaces according to claim 1, characterized in that, Step S1 includes: Step S11: In the visualization simulation platform, simulation models of the machine tool, the overall metal workpiece, and the camera are set respectively. The relative positional relationship of each simulation model is set according to the actual processing scenario. Among them, the machine tool simulation model is used to simulate the movement of each axis of the machine tool. The overall metal workpiece simulation model is set on the processing table in the machine tool simulation model to reflect the geometric features of the overall metal workpiece to be inspected. The camera simulation model is set on the spindle of the machine tool simulation model to obtain the geometric model of the overall metal workpiece to be inspected under camera imaging. This geometric model is defined as the camera imaging geometric model. Step S12: Based on the overall geometric features of the metal workpiece to be inspected and the camera imaging geometric model, manually select discrete imaging points for visual scanning in the visualization simulation platform, and determine the visibility and / or interference points to obtain N discrete optimized imaging points for visual scanning. Step S13: Obtain the coordinates of N visual scanning discrete optimized imaging points in the machine tool coordinate system based on the machine tool simulation model, and generate a continuous visual scanning trajectory based on these coordinates.

3. The online detection method for defects on curved metal surfaces according to claim 1, characterized in that, In step S2, a visual defect detection device mounted on the machine tool spindle is used to acquire an image of the area to be inspected. The visual defect detection device includes a clamping mechanism, a camera, a communication module, a power supply and image display module mounted on the clamping mechanism, and a multi-channel time-sequential light source mounted on the camera. The clamping mechanism is fixed to the machine tool spindle by a tool holder, and the clamping mechanism ensures the coaxiality between the camera lens and the machine tool spindle. The camera is connected and fastened to the middle of the clamping mechanism and is used to take pictures of the surface of the metal workpiece to be inspected during the movement of the machine tool, acquire the image to be inspected, and transmit it to the outside through the communication module. The multi-channel time-sequential light source is fastened to the side of the camera lens and is used to provide incident light sources from different directions. When the machine tool spindle reaches a certain optimized imaging point in the continuous visual scanning trajectory, the light sources of the multi-channel time-sequential light source flash in a set order, and the camera takes pictures when the corresponding light source flashes, capturing M images at the same camera angle.

4. The online detection method for defects on curved metal surfaces according to claim 3, characterized in that, Before capturing the image to be detected, the visual defect detection device needs to be calibrated. The calibration parameters include virtual tool length L and spindle positioning angle θ. The virtual tool length L is the distance between the tool holder mounting surface and the camera focal plane. The spindle positioning angle θ is the rotation angle of the camera coordinate system and the machine tool coordinate system around the machine tool Z-axis. The theoretical spindle positioning angle is 0°. The calibration of the virtual tool length L includes: The visual defect detection device is installed on the machine tool spindle, and the camera is used to continuously acquire images; the machine tool Z-axis is moved so that the camera images the machine tool rotary table; the image sharpness B of the acquired image is calculated, and the Z coordinate of the position with the maximum B value is recorded as the virtual tool length L; The calibration of the spindle positioning angle θ includes: The visual defect detection device is installed on the machine tool spindle. The camera is moved so that it is aimed at a certain edge of the machine tool turntable to take a picture. The edge detection algorithm is used to detect the edge of the captured image. The angle between the detected image edge and the X-axis of the machine tool is calculated as the spindle positioning angle.

5. The online detection method for defects on curved metal surfaces according to claim 1, characterized in that, The training of the specular removal model includes: A dataset augmentation algorithm based on generative adversarial methods generates multiple pairs of metal surface images under different exposure conditions. Each metal surface image pair consists of a highlight image and a corresponding non-highlight image. Unlabeled real highlight images are obtained for unsupervised training. The metal surface image pairs and the unlabeled real highlight images are used to form a training dataset. A specular removal model is built based on MEF-Net and incorporates detail enhancement and color enhancement modules. The detail enhancement module is embedded between the downsampling layer and the feature prediction network of MEF-Net, reducing high-frequency texture loss through multi-scale contextual feature fusion. The detail enhancement module employs a bidirectional detail enhancement strategy: for underexposed areas, contrast is adaptively adjusted using standard deviation; for overexposed areas, details are restored by inverting the original image and using standard deviation to adaptively adjust contrast. The color enhancement module is connected to the chroma channel fusion stage of the YCbCr color space of MEF-Net, learning the chroma mapping relationship through an encoder-decoder structure. The specular removal model is trained using the training dataset and an unsupervised learning strategy to obtain the trained specular removal model.

6. The online detection method for defects on curved metal surfaces according to claim 1, characterized in that, An interactive image segmentation model is constructed using the SAM-Adapter model. The interactive image segmentation model includes an image encoder, a cue encoder, and a mask decoder connected in sequence. The image encoder is used to extract multi-scale features from the input fused image after specular removal to generate a high-dimensional semantic feature map. The cue encoder encodes the input interactive cue to generate a cue embedding vector aligned with the semantic feature map. The mask decoder dynamically generates a segmentation mask based on the semantic feature map generated by the image encoder and the cue embedding vector generated by the cue encoder.

7. The online detection method for defects on curved metal surfaces according to claim 1, characterized in that, The step of inputting the fused image into a pre-trained interactive image segmentation model and, with the aid of interactive prompts, segmenting the detection region image of a single part from the fused image includes: The fused image is input into a pre-trained interactive image segmentation model, and the overall region of the metal workpiece and the irrelevant background in the fused image are segmented according to the user-input interactive prompt 1 to obtain an image of the overall region of the metal workpiece; the interactive prompt 1 adopts any one or more of the following combinations: non-iterative prompt, iterative prompt, ambiguous prompt, oracle mechanism and variability prompt. The interactive image segmentation model segments the individual parts to be detected and irrelevant parts in the overall image of the metal workpiece based on the user-input interactive prompts 2, resulting in an image of the individual part to be detected. The interactive prompts 2 adopt an iterative prompting strategy. In the overall image of the metal workpiece, for an individual part, positive prompts are generated at key positions in the overall region of the metal workpiece, and multiple negative prompts are generated around it. Through multiple iterations of prompts, unwanted irrelevant part regions are eliminated, thereby segmenting the individual parts to be detected and irrelevant parts. The interactive image segmentation model segments the target region and irrelevant detection region in the image of a single part based on the user-input interactive prompt 3, resulting in the target region image of a single part. The interactive prompt 3 employs an iterative prompting strategy, generating positive prompts in the target region and negative prompts in the irrelevant region of the part in the single part region image. The prompt positions and the distribution of positive and negative prompts are adjusted through multiple iterations to segment the target region and irrelevant region of the single part. The target region is the non-edge region on the surface of the part where defects have occurred; the irrelevant region is the region where interfering pixels are located on the surface of the part.

8. The online detection method for defects on curved metal surfaces according to claim 1, characterized in that, The defect detection model with attention mechanism replaces the Concat module used in the feature fusion part of the existing single-level target detection model with a bidirectional feature pyramid network, inserts a convolutional block attention module after the convolutional module in the feature fusion stage, and introduces cross-level skip connections in the head network of the existing single-level target detection model. When training the defect detection model with attention mechanism, multiple training images are randomly stitched into a single composite image based on the mosaic data augmentation strategy, which is used as the input of the defect detection model with attention mechanism; the total loss function used is the weighted sum of classification loss, localization loss and confidence loss, and the localization loss adopts EIoU loss.

9. The online detection method for defects on curved metal surfaces according to claim 8, characterized in that, Before inputting the image of the region to be detected into the defect detection model with attention mechanism, the image of the region to be detected is first enhanced with mosaic and normalized to generate a multi-scale input image; The multi-scale input image is processed by the backbone network of the defect detection model with attention mechanism to extract multi-scale features. Channel and spatial attention weights are then fused through the bidirectional feature pyramid network and the convolutional block attention module to output an enhanced defect feature map. The detection head of the defect detection model with attention mechanism generates candidate boxes based on the enhanced defect feature map. After screening with EIoU-NMS, the defect category, location, and confidence score are output as the defect detection result.

10. A vision-based online detection device for defects on curved metal surfaces, characterized in that, include: The continuous visual scanning trajectory generation module is configured to select multiple visual scanning discrete optimized imaging points of the metal workpiece to be inspected from the CAD model of the metal workpiece to be inspected with curved surface features in the visualization simulation platform, and generate a continuous visual scanning trajectory based on all optimized imaging points. The image acquisition module is configured to move the machine tool according to the continuous visual scanning trajectory for the metal workpiece obtained in the current processing step, and acquire multiple images of the metal workpiece surface at the same shooting angle and different incident light source conditions at each optimized imaging point. The highlight removal module is configured to input all images to be detected into a pre-trained highlight removal model. The highlight removal model fuses the images to be detected located at the same optimized imaging point to achieve highlight removal, and outputs a fused image after highlight removal for each optimized imaging point. An interactive image segmentation module is configured to input the fused image into a pre-trained interactive image segmentation model and, with the aid of interactive prompts, segment out the detection region image of a single part from the fused image. The defect detection module is configured to input the image of the region to be detected into a defect detection model with an attention mechanism to obtain the defect detection result.

Images