A multi-camera fusion workpiece surface defect detection method and related device

Abstract

The application provides a workpiece surface defect detection method based on multi-camera fusion and related equipment, and the method comprises the following steps: a plurality of industrial cameras are used to collect videos of the surface of a rotary workpiece to generate video data of the surface of the rotary workpiece; the video data is decoded to obtain each frame of image in the video; the obtained frame of image is cropped and spliced to form an expanded image of the overall or partial workpiece surface; and the expanded image is subjected to image processing analysis to detect defects on the workpiece surface. The application combines the flexible configuration of multiple industrial cameras, adopts high-speed video acquisition technology, and realizes the optical flexibility that cannot be matched by ordinary line scanning cameras. This configuration is not only suitable for various complex industrial scenes, but also has a large depth of field feature, which can ensure that clear images can be obtained at different working distances. The application has strong universality for detecting the surface defects of the outer wall of a metal rotary workpiece with a curved surface and high reflectivity.

Description

Technical Field

[0001] This application relates to the field of machine vision inspection technology, and in particular to a method and related equipment for detecting surface defects in workpieces using multi-camera fusion. Background Technology

[0002] In the current field of industrial inspection, machine vision technology is widely used in product inspection. It primarily involves using industrial cameras to photograph the products being inspected, typically storing these images in a computer. The image signals captured by the camera are first converted into digital signals that the computer can process, and then transmitted to the computer's memory via a high-speed bus. Based on this, machine vision software performs in-depth analysis of these digital images to determine the product's quality and whether it meets preset standards, thus achieving the purpose of image inspection. Therefore, imaging quality plays a crucial role in machine vision inspection.

[0003] When using an area scan camera for imaging, if the product being inspected is a highly reflective curved metal object, variations in the surface curvature and reflectivity can lead to uneven brightness distribution in the image, increasing the difficulty of image processing. Line scan cameras differ from common area scan cameras in their working principle. Area scan cameras can acquire a complete 2D image in a single exposure, while line scan cameras capture only one line of the object at a time (this line is typically one pixel wide). During the imaging process, the line scan camera gradually combines these lines by scanning line by line to ultimately form a complete 2D image. Although line scan cameras have certain advantages in imaging highly reflective curved objects, their practical applications require extremely precise installation positioning; even a tiny positional deviation (greater than 0.05 mm) can significantly affect the imaging results.

[0004] Given the limitations of area scan cameras in processing images of highly reflective curved metal objects and the strict requirements of line scan cameras on installation position, this application provides a multi-camera fusion method for detecting surface defects in workpieces, offering a new solution for workpiece inspection. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this application provides a multi-camera fusion method and related equipment for detecting surface defects in workpieces, thereby at least solving the problems of imaging difficulties and limited defect detection for high-reflectivity metal rotating bodies in existing technologies.

[0006] To achieve the above objectives and other advantages, this application adopts the following technical solution:

[0007] In a first aspect, this application provides a multi-camera fusion method for detecting surface defects in workpieces, applied to the defect detection of machined parts of rotation made of metal with curved surfaces and high light reflectivity. The method includes:

[0008] Several industrial cameras are used to capture video of the surface of the rotary workpiece to generate video data of the surface of the rotary workpiece.

[0009] The video data is decoded to obtain each frame of the video.

[0010] The acquired frame images are cropped and stitched together to form an unfolded image of the entire or partial workpiece surface;

[0011] The unfolded image is subjected to image processing and analysis to detect defects on the surface of the workpiece.

[0012] According to the multi-camera fusion method for detecting surface defects on a workpiece provided in this application, the step of using several industrial cameras to acquire video data of the surface of the workpiece to generate video data of the workpiece surface includes:

[0013] Several industrial cameras are arranged above or around the rotary workpiece, and each industrial camera is configured to capture video of a specific local position of the rotary workpiece.

[0014] The industrial camera is activated to simultaneously capture video of various specific local locations on the rotary workpiece, ensuring that the shooting range of each camera covers an independent area with overlapping edges on the surface of the rotary workpiece.

[0015] The local video data captured by each of the industrial cameras are stitched together to form video data of the surface of the rotating workpiece.

[0016] According to the multi-camera fusion method for detecting defects on the surface of a workpiece provided in this application, the surface of the rotary workpiece includes multiple rotary surfaces of different diameters, and the diameter of the rotary surface corresponding to a specific local position captured by each of the industrial cameras is different.

[0017] The step of setting up a plurality of industrial cameras above or around the rotary workpiece, each industrial camera being configured to capture video of a specific local location of the rotary workpiece, includes:

[0018] Determine the diameter of each different rotating surface on the surface of the machined rotary body;

[0019] A maximum processing frame rate is set for the rotating surface with the largest diameter. Using the maximum processing frame rate as a reference standard, the corresponding processing frame rate of the industrial camera is set according to the diameter. A rotating surface with a larger diameter corresponds to a higher processing frame rate, and a rotating surface with a smaller diameter corresponds to a lower processing frame rate.

[0020] Different processing frame rates are configured in the parameters of the corresponding industrial cameras, so that each industrial camera can capture images of the corresponding rotating surface on the rotating workpiece according to the set processing frame rate.

[0021] According to the multi-camera fusion method for detecting surface defects on a workpiece provided in this application, the step of decoding the video data to obtain each frame of the video includes:

[0022] Receive a video stream after encoding and compressing the video data, the video stream containing video parameter information and compressed image data, the parameter information including: video parameters and encoding format;

[0023] Based on the encoding format and the video parameters, the video stream is parsed, and image data in the video is obtained frame by frame.

[0024] Traverse each frame of image data and determine the frame image to be extracted according to the set frame extraction parameters. The frame extraction parameters include: start frame, end frame, frame extraction interval, and number of frames.

[0025] Each selected frame image is saved as a separate image file.

[0026] According to the multi-camera fusion method for detecting surface defects on a workpiece provided in this application, the step of parsing the video bitstream based on the encoding format and the video parameters, and acquiring image data from the video frame by frame, includes:

[0027] For the image frame data in the video bitstream, the transform coding technique used in the video encoding process is used to perform inverse quantization and inverse transform processing during the video decoding process. The quantized coefficients are mapped back to the original or near-original transform coefficient values, and the transform coefficient values ​​are converted back to the spatial domain to restore the uncompressed original image data.

[0028] Meanwhile, during the video decoding process, the predictive coding technique used in the video encoding process is used to perform intra-frame prediction of the correlation between adjacent pixels in the same frame image and inter-frame prediction of the motion information between adjacent frames, so as to reconstruct the image data by using the reverse process of the predictive coding technique.

[0029] The video decoding process is performed frame by frame to acquire and reconstruct the image data in the video until the entire video stream is decoded.

[0030] According to the multi-camera fusion method for detecting surface defects on a workpiece provided in this application, the step of cropping and stitching the acquired frame images to form an unfolded image of the entire or partial workpiece surface includes:

[0031] The frame image is cropped according to preset cropping parameters for the workpiece parts that need to be inspected for defects.

[0032] Multiple cropped frames are stitched together according to a predetermined stitching order and rules to form an unfolded image of the entire or partial surface of the workpiece.

[0033] Save the unfolded image of the workpiece surface after stitching as an image file.

[0034] According to the multi-camera fusion method for detecting surface defects on a workpiece provided in this application, the method further includes: using parallel processing technology with a multi-core CPU and CUDA architecture to perform real-time image stitching of local video data from multiple industrial cameras to form video data of the surface of a rotating workpiece; and decoding and stitching the video data to form an unfolded image of the workpiece surface.

[0035] Secondly, this application provides an electronic device, the electronic device comprising:

[0036] One or more processors; and a memory storing computer program instructions that, when executed, cause the processors to perform the multi-camera fusion workpiece surface defect detection method as described above.

[0037] Thirdly, this application provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the multi-camera fusion method for detecting workpiece surface defects as described above.

[0038] Fourthly, this application provides a computer program product, including a computer program / instructions, which, when executed by a processor, implements the multi-camera fusion method for detecting surface defects on a workpiece as described above.

[0039] This application provides a multi-camera fusion method and related equipment for detecting surface defects on workpieces. It is applied to defect detection of curved, highly reflective metal parts made of metal. The method includes: using several industrial cameras to capture video of the surface of the workpiece to generate video data; decoding the video data to obtain each frame; cropping and stitching the acquired frame images to form a unfolded image of the entire or partial workpiece surface; and performing image processing and analysis on the unfolded image to detect defects on the workpiece surface. This application combines a flexible configuration of multiple industrial cameras to capture video of the surface of the workpiece, ensuring image information is obtained from multiple angles and positions. The use of high-speed video acquisition technology achieves optical flexibility unmatched by ordinary line scan cameras. This configuration is not only suitable for various complex industrial scenarios but also possesses a large depth of field, ensuring clear images are obtained at different working distances. It has strong versatility for detecting the outer walls of curved, highly reflective metal parts of rotation. By configuring the number of camera workstations and adjusting the focal length, surface defect detection of rotary workpieces of any length can be easily achieved. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other implementation methods can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic flowchart of the multi-camera fusion method for detecting surface defects on a workpiece provided in an embodiment of this application;

[0042] Figure 2 This is a dimensional schematic diagram of the rotary machined part provided in the embodiments of this application;

[0043] Figure 3 This is the original drawing and the corresponding surface development drawing of a certain part of the rotary machined part provided in the embodiments of this application;

[0044] Figure 4 This is the original drawing and the corresponding surface development drawing of another part of the rotary machined part provided in the embodiments of this application;

[0045] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0046] The above description is merely an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings.

[0047] It should be noted that those skilled in the art will understand, explicitly and implicitly, that the embodiments described in this application can be combined with other embodiments without conflict. Unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by those skilled in the art. The terms "a," "an," "an," "the," and similar words used in this application do not indicate quantity limitation and can represent singular or plural. The terms "comprising," "including," "having," and any variations thereof used in this application are intended to cover non-exclusive inclusion; the terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific ordering of objects.

[0048] Metal rotating parts have highly reflective surfaces, which easily cause reflections and glare when captured by area scan cameras, affecting image clarity and contrast. Furthermore, variations in the curvature and reflective properties of the metal surface lead to uneven image brightness distribution, increasing the difficulty of image processing. Therefore, area scan cameras face limitations in processing ordinary images of highly reflective metal surfaces, including limitations in image quality, measurement accuracy, applicable scenarios, cost, and technical difficulty. This application provides a multi-camera fusion method and related equipment for workpiece surface defect detection, taking corresponding measures to optimize the image acquisition and processing process to improve the accuracy and efficiency of defect detection.

[0049] Reference Figure 1 As shown in the figure, this application provides a multi-camera fusion method for detecting surface defects on a workpiece, applicable to the defect detection of a machined part of rotation made of metal with a curved surface shape and high light reflectivity. The method includes:

[0050] Step S1: Use several industrial cameras to capture video of the surface of the rotating workpiece to generate video data of the surface of the rotating workpiece.

[0051] In this embodiment, step S1 specifically includes:

[0052] Step S101: Set up several industrial cameras above or around the rotary workpiece, each industrial camera being configured to capture video of a specific local location of the rotary workpiece.

[0053] For example, such as Figure 2 The design drawing of the machined part shown depicts a typical rotary body, rotating continuously around a fixed axis. This rotary part has multiple surfaces of rotation with different diameters. That is, the circumference of the part is not a uniform diameter, but rather exhibits different diameters at different locations according to design requirements. At one end of the part (e.g., the left end), there is a surface of rotation with a larger diameter, which gradually decreases as the circumference extends to the other end (e.g., the right end). At the other end of the part, there is a surface of rotation with a smaller diameter.

[0054] Because rotary workpieces have multiple rotating surfaces of varying diameters, the shooting height of industrial cameras needs to be adjusted according to these different diameters. When shooting a rotating surface with a larger diameter, the camera needs to be adjusted to a higher shooting height to capture the entire area and expand the field of view. Conversely, when shooting a rotating surface with a smaller diameter, the camera can be lowered appropriately.

[0055] In this embodiment, step S101 specifically includes:

[0056] Step S1011: Determine the diameter of each different rotating surface on the surface of the workpiece.

[0057] Step S1012: Set the maximum processing frame rate for the rotating surface with the largest diameter. Using the maximum processing frame rate as a reference standard, set the corresponding processing frame rate of the industrial camera according to the diameter. The rotating surface with a larger diameter corresponds to a higher processing frame rate, and the rotating surface with a smaller diameter corresponds to a lower processing frame rate.

[0058] Step S1013: Configure the different processing frame rates in the parameters of the corresponding industrial cameras so that each industrial camera can capture images of the corresponding rotating surface on the rotating workpiece according to the set processing frame rate.

[0059] Specifically, the diameter of each rotating surface is determined based on the dimensions marked on the design drawings. After determining the diameter of each rotating surface, a suitable maximum processing frame rate needs to be selected. This frame rate is usually determined based on the shooting requirements of the rotating surface with the largest diameter to ensure that the image of that rotating surface can be captured clearly and continuously. Using the maximum processing frame rate as a reference standard, the processing frame rate of the corresponding industrial camera is set proportionally according to the diameter of the other rotating surfaces.

[0060] For example, three to four cameras are used. Figure 2 The outer surface of the rotating workpiece shown is imaged, and the resulting image is cropped to 1280×30 pixels. This cropping operation significantly improves the camera's processing frame rate, which can be increased from the standard 216.25fps to 1000fps.

[0061] In this working environment, the camera's field of view was set to 96mm × 76.8mm. Based on this field of view and the cropped pixel size, the calculated single-pixel accuracy is: 96mm / 1280pixel = 0.075mm / pixel, which means that each pixel in the image corresponds to 0.075 millimeters on the actual object.

[0062] To capture video, the camera is adjusted to video capture mode. Taking the first camera as an example, the diameter of the product being captured by the camera is φ87.8mm (approximately φ88mm). Figure 2 The perimeter of the location marked ① is calculated using the formula π×D and is 276.32 mm. Under the cropping effect, the camera's video capture speed is: 1000fps×0.075mm / pixel=75mm / s.

[0063] To ensure the captured video remains synchronized with the product's scrolling, the rotational speed of the outer surface needs to be precisely set. This speed is calculated by multiplying the capture speed of 75 mm / s by 60 (converting the speed unit from mm / s to mm / min), and then dividing by the product's circumference of 276.32 mm. The calculated rotational speed of the outer surface is: 60 × capture speed / circumference = 60 × 75 / 276.32 = 16.285 r / min (minutes).

[0064] After setting the aforementioned rotation speed, the video recording time can be further determined. This time is obtained by dividing the product's circumference of 276.32 mm by the acquisition speed of 75 mm / s, resulting in 276.32 / 75 = 3.684 seconds. Based on the settings of the first camera, the frame rate settings for the other cameras can be deduced (frame rate settings can be configured in the camera software or by using a programming interface to access the camera's configuration interface), ensuring that all cameras acquire data at the same rotation speed, thereby guaranteeing the synchronization and accuracy of the entire system. Figure 2 The radius of the half-plane of the rotating surface at point ① is 44mm and the radius of the half-plane of the rotating surface at point ② is 22mm. The camera recording parameters are shown in Table 1 below.

[0065] Table 1 Recording parameters for surfaces of revolution with different diameters

[0066]

[0067] Step S102: Start the industrial camera to simultaneously capture video of various specific local locations on the rotating workpiece, ensuring that the shooting range of each camera covers an independent area with overlapping edges on the surface of the rotating workpiece.

[0068] Specifically, each camera is equipped with a corresponding lens to capture different areas of the workpiece's outer surface. A light source provides the necessary illumination to ensure clear images. The rotating workpiece is clamped on a rolling mechanism, driven by a belt. As the workpiece rotates, its outer surface sequentially passes through the camera's shooting area. Multiple cameras are mounted in fixed positions, their lenses aimed at the rotating workpiece. When the workpiece rotates into the camera's shooting range, the camera captures an image of that area. Because there are multiple cameras, they can each capture different parts of the workpiece's outer surface. The rotation speed of the rolling belt is adjusted according to the required recording time, precisely controlling the rotation speed of the workpiece surface. Camera shooting can be synchronously triggered based on the belt's movement, ensuring the workpiece surface is captured from the optimal position. The position and intensity of the light source can be adjusted according to shooting requirements to ensure clear images are obtained from different shooting angles. This helps reduce shadows and reflections, improving image contrast and detail.

[0069] It's important to note that the shooting range of each camera needs careful design to ensure they cover a distinct area of ​​the workpiece surface, with some overlap between these areas. This edge overlap is crucial for seamless stitching of subsequent video data, preventing information loss or misalignment.

[0070] For example, the industrial camera could be a high-frame-rate industrial camera, with a frame rate up to 1000fps. This industrial camera uses USB 3.0 transmission speeds, a significant improvement over USB 2.0 and GigE (Gigabit Ethernet) cameras. With a transmission bandwidth of 5Gbps, combined with a high-speed sensor, data transfer time can be as low as 1.2 milliseconds under shooting conditions of 300,000 pixels and 800 frames per second (FPS). Even with a high-resolution 20-megapixel camera, a transmission speed of up to 18 frames per second can be achieved. Benefiting from the ample transmission bandwidth, USB 3.0 technology allows a single computer to maintain a high frame rate while being compatible with and connecting multiple cameras, undoubtedly improving system flexibility and scalability. Furthermore, it performs excellently in terms of CPU utilization, employing Direct Memory Access (DMA) transmission, meaning that data transfer consumes almost no CPU resources, thus ensuring efficient system operation and rational resource utilization.

[0071] Step S103: Stitch together the local video data captured by each industrial camera to form video data of the surface of the rotating workpiece.

[0072] Specifically, this is a crucial step in multi-camera fusion. Image processing algorithms stitch together local video data captured by each camera to form a single video dataset covering the entire or partial surface of the workpiece. During this process, information about overlapping edges is used to calibrate and adjust the positions between different video frames, ensuring the stitched video data is continuous in both time and space. Before video stitching, the cameras need to be calibrated to obtain their internal and external parameters. These parameters are used for subsequent image registration and fusion. It is essential to ensure that the video streams from different cameras are synchronized in time to avoid misalignment or breaks after stitching.

[0073] For example, video streams are acquired from various cameras, and each frame is preprocessed, including denoising and resizing. Keypoints and descriptors are extracted from each frame using feature detection algorithms (such as SIFT, SURF, or ORB). Then, the optimal matching points are found using feature matching algorithms (such as FLANN or BFMatcher). A homography matrix is ​​calculated based on the matching points, and perspective transformation is applied to the images to align them to the same coordinate system. The aligned images are then fused to generate a seamlessly stitched video. The video stitching process typically also involves multi-band fusion and image smoothing techniques to reduce the visibility of seams. Therefore, through the above steps and the application of key algorithms, local video data captured by various cameras can be stitched together to form video data covering the entire or partial surface of the workpiece.

[0074] Step S2: Decode the video data to obtain each frame of the video.

[0075] In this embodiment, step S2 specifically includes:

[0076] Step S201: Receive the video bitstream after encoding and compressing the video data. The video bitstream contains video parameter information and compressed image data. The parameter information includes: video parameters and encoding format.

[0077] Specifically, a video stream is encoded and compressed video data that contains both video parameter information and actual image data. The parameter information is crucial for the subsequent decoding process, informing the decoder how to correctly parse and reconstruct the video data. This parameter information typically includes video parameters (such as resolution and frame rate) and encoding formats (such as H.264, H.265, etc.).

[0078] Step S202: Parse the video stream based on the encoding format and video parameters, and obtain the image data in the video frame by frame.

[0079] In this embodiment, step S202 specifically includes:

[0080] For image frame data in the video bitstream, the transform coding technique used in the video encoding process is used to perform inverse quantization and inverse transform processing during the video decoding process. The quantized coefficients are mapped back to the original or near-original transform coefficient values, and the transform coefficient values ​​are converted back to the spatial domain to restore the uncompressed original image data.

[0081] Meanwhile, during the video decoding process, the predictive coding technique used in the video encoding process is used to perform intra-frame prediction of the correlation between adjacent pixels in the same frame image and inter-frame prediction of the motion information between adjacent frames, so as to reconstruct the image data by using the reverse process of the predictive coding technique.

[0082] The video decoding process is performed frame by frame to acquire and reconstruct the image data in the video until the entire video stream is decoded.

[0083] Specifically, in video encoding, transform coding techniques transform and encode pixel values ​​in an image or video frame, such as Discrete Cosine Transform (DCT) or Discrete Wavelet Transform (DWT), and then quantize the transformed coefficients to reduce data volume. During decoding, these quantized coefficients are first dequantized, mapping the quantized coefficient values ​​back to the original or near-original transform coefficient values. The dequantized transform coefficient values ​​are still in the transform domain (e.g., the frequency domain), and need to be converted back to the spatial domain (i.e., the pixel domain) through inverse transform processing to restore the uncompressed original image data. Inverse transform processing typically corresponds to the transform algorithm used in the encoding process, such as the inverse of the Discrete Cosine Transform or the inverse of the Discrete Wavelet Transform.

[0084] To remove temporal and spatial redundancy in video, predictive coding techniques are employed in video encoding. During video decoding, predictive coding techniques from the video encoding process are used for intra-frame and inter-frame prediction. For intra-frame prediction, since adjacent pixels within the same frame typically exhibit strong correlations, the decoder utilizes this correlation to predict the current pixel value based on already decoded adjacent pixel values. By calculating the residual between the predicted and true values ​​and decoding this residual, the accurate value of the current pixel can be obtained. For inter-frame prediction, due to the temporal continuity of the video sequence, motion information exists between adjacent frames. The decoder uses inter-frame prediction techniques to predict the pixel value of the current frame by identifying motion vectors between adjacent frames. This typically involves motion estimation and compensation: generating a predicted image of the current frame based on already decoded adjacent frame images and motion vectors, and then decoding the residual to obtain the accurate image of the current frame.

[0085] The decoder executes the above decoding process frame by frame in the video stream. For each frame of image data, it sequentially performs inverse quantization, inverse transform processing, intra-frame prediction, and inter-frame prediction to reconstruct the uncompressed original image data. The reconstructed original image data is uncompressed and undamaged, thus preserving more detailed information and making the image clearer and more realistic. Using the original image data for defect detection reduces the impact of noise and distortion introduced during image compression or processing on the detection results.

[0086] Step S203: Traverse each frame of image data and determine the frame image to be extracted according to the set frame extraction parameters. The frame extraction parameters include: start frame, end frame, frame extraction interval and number of frames.

[0087] Step S204: Save each selected frame image as a separate image file.

[0088] The decoder determines the frames to extract based on the set frame extraction parameters. These parameters include the start frame, end frame, extraction interval, and number of frames. By iterating through each frame of image data and filtering according to these parameters, the decoder can accurately extract the frames required by the user. Finally, the decoder saves each selected frame as a separate image file. These image files are used for subsequent stitching processing. The save format can be selected according to user needs, such as JPEG or PNG.

[0089] Step S3: Crop and stitch the acquired frame images to form an unfolded image of the entire or partial workpiece surface.

[0090] In this embodiment, step S3 specifically includes:

[0091] Step S301: Based on the workpiece parts that need to be inspected for defects, crop the acquired frame image according to the preset cropping parameters;

[0092] Step S302: The cropped multiple frame images are stitched together according to a predetermined stitching order and rules to form an unfolded image of the entire or partial workpiece surface;

[0093] Step S303: Save the unfolded image of the spliced ​​workpiece surface as an image file.

[0094] Specifically, the location of the workpiece to be inspected can be identified first, such as the entire workpiece or a specific part of it. Based on the identified workpiece location, corresponding trimming parameters can be preset, such as the size, shape, and position of the trimming area. Image processing tools can then be used to batch process the acquired frame images.

[0095] Before stitching, the stitching order and rules between the cropped frames need to be determined. If the frames were extracted in chronological order from the video recording, then the stitching must also follow this chronological order to maintain the continuity of the workpiece surface changes. Stitching rules are typically determined by the shape, texture, defect features, and inspection requirements of the workpiece surface. If the workpiece surface is flat, the stitching rules may focus on ensuring precise alignment of adjacent images in the horizontal or vertical direction. For curved workpieces, the stitching rules need to consider perspective distortion and surface fitting to ensure the stitched image remains visually coherent and accurate. Texture consistency and repeatability can help identify overlapping areas between adjacent images and serve as a reference during stitching. Defect features are also a key factor to consider in the stitching rules; for example, if the defect manifests as a small crack or scratch, the stitching rules should avoid excessive smoothing or image fusion in the defect area.

[0096] As needed, the stitched image can be further processed and adjusted, such as color correction, brightness adjustment, and contrast enhancement, to improve image quality and detection results. After stitching and adjustment, the final unfolded image of the workpiece surface is saved as an image file. An appropriate image format is selected based on the image resolution and quality parameters. The unfolded image of the stitched workpiece surface is shown below. Figure 3 , Figure 4 As shown in the figure. Figure a shows the original view of a certain part of the machined part of the rotating body, and Figure b shows the two-dimensional view of the surface of that part after it has been unfolded.

[0097] pass Figure 3 , Figure 4 As can be seen, the images generated by a series of algorithms have uniform illumination and clear defect contrast, providing a reliable guarantee for defect detection and identification.

[0098] Step S4: Perform image processing analysis on the unfolded image to detect defects on the workpiece surface. The color image can be converted to a grayscale image to simplify calculations and highlight defect features. An image segmentation algorithm is used to segment the image based on the grayscale difference between the defect and the background, setting an appropriate threshold to separate the defect from the background. Based on the extracted features, a classifier (such as a support vector machine or neural network) is used to detect and identify defects in the image. According to the characteristics and type of the defect, it can be classified into different defect categories, such as cracks, scratches, and dents. Therefore, by performing image processing analysis on the unfolded image, defects on the workpiece surface can be automatically detected. This replaces traditional manual inspection methods, improving inspection efficiency and accuracy while reducing inspection costs.

[0099] This embodiment also includes: using parallel processing technology with multi-core CPU and CUDA architecture to perform real-time image stitching of local video data from multiple industrial cameras to form video data of the surface of the rotating workpiece, and decoding and stitching the video data to form an unfolded image of the workpiece surface.

[0100] Specifically, multi-core CPUs can process multiple tasks simultaneously, improving processing speed through parallel computing. CUDA (Compute Unified Device Architecture) is a parallel computing architecture launched by NVIDIA, which significantly improves computing performance by utilizing the processing power of graphics processing units (GPUs). By combining multi-core CPUs with the parallel processing technology of the CUDA architecture, video fusion and image processing algorithms from multiple workstations can be highly parallelized. Parallel computing significantly accelerates image stitching speed, enabling real-time image stitching of local video data from multiple industrial cameras, providing an efficient and accurate solution for quality inspection of the surface of rotating parts.

[0101] In summary, the multi-camera fusion method for detecting surface defects in workpieces provided in this application is applied to the defect detection of curved, highly reflective metal parts made of metal. The method includes: using several industrial cameras to capture video of the surface of the workpiece to generate video data; decoding the video data to obtain each frame; cropping and stitching the acquired frame images to form an unfolded image of the entire or partial workpiece surface; and performing image processing and analysis on the unfolded image to detect defects on the workpiece surface. This application combines the flexible configuration of multiple industrial cameras to capture video of the surface of the workpiece, ensuring that image information of the workpiece surface is obtained from multiple angles and positions. The use of high-speed video acquisition technology achieves optical flexibility unmatched by ordinary line scan cameras. This configuration is not only suitable for various complex industrial scenarios but also possesses a large depth of field, ensuring clear images are obtained at different working distances. It has strong versatility for detecting the outer walls of curved, highly reflective metal parts of rotation. By configuring the number of workstations and adjusting the focal length, surface defect detection of rotary parts of any length can be easily achieved.

[0102] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0103] Furthermore, some embodiments of this application also provide an electronic device. The electronic device can be various forms of digital computer, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device can also be various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0104] The electronic device includes: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the multi-camera fusion workpiece surface defect detection method provided in any one or more of the above embodiments. Figure 5 An exemplary structural diagram of the electronic device is disclosed. For example... Figure 5 As shown, the electronic device includes one or more processors 1101, a memory 1102, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0105] The electronic device may further include an input device 1103 and an output device 1104. The processor 1101, memory 1102, input device 1103, and output device 1104 may be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0106] Input device 1103 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 1104 may include a display device, auxiliary lighting device (e.g., LED), and haptic feedback device (e.g., vibration motor). The display device may include, but is not limited to, a liquid crystal display (LCD), a light-emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0107] To provide interaction with the user, the electronic device can be a computer. The computer has: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0108] In this embodiment, a computer-readable medium stores a computer program / instructions, which, when executed by a processor, implement the multi-camera fusion method for detecting surface defects on workpieces provided in any one or more of the above embodiments. This computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into that device. The aforementioned computer-readable medium carries one or more computer-readable instructions.

[0109] The memory 1102 can serve as a non-transitory computer-readable storage medium, used to store non-transitory software programs, non-transitory computer-executable programs, and modules. The processor 1101 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 1102, thereby implementing the program instructions / modules corresponding to the methods provided in any one or more of the embodiments described above in this application.

[0110] The memory 1102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 1102 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1102 may optionally include memory remotely located relative to the processor 1101, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0111] It should be noted that more specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0112] Computer-readable storage media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only optical disc (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0113] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0114] In the above embodiments, all or part of the implementation can be achieved through software, hardware, firmware, or any combination thereof. For example, it can be implemented using an Application-Specific Integrated Circuit (ASIC), a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of this application can be executed by a processor to implement the above steps or functions. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, and similar devices. In addition, some steps or functions of this application can be implemented in hardware, for example, as circuitry that cooperates with a processor to perform the various steps or functions.

[0115] The computer program product provided in this application includes one or more computer programs / instructions. When executed by a processor, these computer programs / instructions generate, in whole or in part, the processes or functions described in this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk, SSD), etc.

[0116] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0117] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily made by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims, and the above embodiments should be regarded as exemplary and non-limiting.

Claims

1. A method for detecting surface defects in a workpiece using multi-camera fusion, characterized in that, The method is applicable to defect detection of machined rotating parts made of metal with curved surfaces and high light reflectivity, and includes: Several industrial cameras are used to capture video of the surface of the rotary workpiece to generate video data of the surface of the rotary workpiece, including: Several industrial cameras are arranged above or around the rotary workpiece. Each industrial camera is configured to capture video of a specific local position of the rotary workpiece. The surface of the rotary workpiece includes multiple rotating surfaces of different diameters. The diameter of the rotating surface corresponding to the specific local position captured by each industrial camera is different. The step of setting up a plurality of industrial cameras above or around the rotary workpiece, each industrial camera being configured to capture video of a specific local location of the rotary workpiece, includes: Determine the diameter of each different rotating surface on the surface of the machined rotary body; The images captured by the industrial camera are cropped according to a preset cropping size, and the single-pixel accuracy of the industrial camera is determined based on the field of view of the industrial camera and the pixel size after cropping. A maximum processing frame rate is set for the rotating surface with the largest diameter. Using the maximum processing frame rate as a reference standard, the video acquisition speed is determined based on the maximum processing frame rate and the single-pixel accuracy. The rotational speed of the outer surface of the rotating workpiece is determined based on the video acquisition speed and the circumference of the rotating surface with the largest diameter. After setting the rotational speed, the processing frame rate of the corresponding industrial camera is set according to the diameter. A larger diameter rotating surface corresponds to a higher processing frame rate, and a smaller diameter rotating surface corresponds to a lower processing frame rate. The processing frame rates of other industrial cameras are derived by reversing the parameter setting method of the industrial camera that captures the rotating surface with the largest diameter. Different processing frame rates are configured in the parameters of the corresponding industrial cameras so that each industrial camera can capture images of the corresponding rotating surface on the rotating body workpiece at the set processing frame rate under the same rotation speed conditions. The video data is decoded to obtain each frame of the video. The acquired frame images are cropped and stitched together to form an unfolded image of the entire or partial surface of the workpiece. The unfolded image is subjected to image processing and analysis to detect defects on the surface of the workpiece.

2. The multi-camera fusion method for detecting surface defects in a workpiece according to claim 1, characterized in that, The step of using several industrial cameras to capture video of the surface of the rotary workpiece to generate video data of the surface of the rotary workpiece further includes: The industrial camera is activated to simultaneously capture video of various specific local locations on the rotary workpiece, ensuring that the shooting range of each camera covers an independent area with overlapping edges on the surface of the rotary workpiece. The local video data captured by each of the industrial cameras are stitched together to form video data of the surface of the rotating workpiece.

3. The multi-camera fusion method for detecting surface defects in a workpiece according to claim 1, characterized in that, The step of decoding the video data to obtain each frame of the video includes: Receive a video stream after encoding and compressing the video data, the video stream containing video parameter information and compressed image data, the parameter information including: video parameters and encoding format; Based on the encoding format and the video parameters, the video stream is parsed, and image data in the video is obtained frame by frame. Traverse each frame of image data and determine the frame image to be extracted according to the set frame extraction parameters. The frame extraction parameters include: start frame, end frame, frame extraction interval, and number of frames. Each selected frame image is saved as a separate image file.

4. The multi-camera fusion method for detecting surface defects in a workpiece according to claim 3, characterized in that, The step of parsing the video bitstream based on the encoding format and the video parameters, and obtaining image data from the video frame by frame, includes: For the image frame data in the video bitstream, the transform coding technique used in the video encoding process is used to perform inverse quantization and inverse transform processing during the video decoding process. The quantized coefficients are mapped back to the original or near-original transform coefficient values, and the transform coefficient values ​​are converted back to the spatial domain to restore the uncompressed original image data. Meanwhile, during the video decoding process, the predictive coding technique used in the video encoding process is used to perform intra-frame prediction of the correlation between adjacent pixels in the same frame image and inter-frame prediction of the motion information between adjacent frames, so as to reconstruct the image data by using the reverse process of the predictive coding technique. The video decoding process is performed frame by frame to acquire and reconstruct the image data in the video until the entire video stream is decoded.

5. The multi-camera fusion method for detecting surface defects in a workpiece according to claim 1, characterized in that, The step of cropping and stitching the acquired frame images to form an unfolded image of the entire or partial workpiece surface includes: The frame image is cropped according to preset cropping parameters for the workpiece parts that need to be inspected for defects. Multiple cropped frames are stitched together according to a predetermined stitching order and rules to form an unfolded image of the entire or partial surface of the workpiece. Save the unfolded image of the workpiece surface after stitching as an image file.

6. The multi-camera fusion method for detecting surface defects in a workpiece according to claim 1, characterized in that, Also includes: Parallel processing technology using a multi-core CPU and CUDA architecture is employed to perform real-time image stitching of local video data from multiple industrial cameras to form video data of the surface of a rotating workpiece. The video data is then decoded and stitched together to form an unfolded image of the workpiece surface.

7. An electronic device, characterized in that, The electronic device includes: One or more processors; and a memory storing computer program instructions, which, when executed, cause the processors to perform the multi-camera fusion method for detecting surface defects on a workpiece as described in any one of claims 1-6.

8. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the multi-camera fusion method for detecting surface defects on workpieces as described in any one of claims 1-6.

9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the multi-camera fusion method for detecting surface defects in a workpiece as described in any one of claims 1-6.

Images