Movement appearance detection method, apparatus, device, medium and program product

Abstract

The application is suitable for the technical field of movement appearance detection, and provides a movement appearance detection method, device, equipment, medium and program product. The method comprises the following steps: acquiring a first original image of a movement through a front camera and acquiring a plurality of second original images of the movement through a plurality of side cameras, and respectively pre-processing the first original image and the plurality of second original images to obtain a first standard image and a plurality of second standard images; performing feature fusion on the first standard image and the plurality of second standard images to obtain a fusion feature map covering the top surface and all side surfaces of the movement; and performing comparison detection on the fusion feature map and outputting a movement appearance detection result. The application solves the technical problem of a geometric occlusion blind area caused by the three-dimensional structure of the movement under a single front view angle, realizes single-time full-coverage detection of multiple types of heterogeneous defects such as missing parts, misplacement and deformation, and improves the accuracy of movement appearance defect detection.

Description

Technical Field

[0001] This invention relates to the field of movement appearance inspection technology, and in particular to a movement appearance inspection method, apparatus, equipment, medium and program product. Background Technology

[0002] As the core moving component of watches and precision instruments, the movement has numerous parts and a complex spatial structure. The production line's outgoing inspection process requires comprehensive testing for various appearance defects, such as missing parts, misalignment, and deformation. Current movement appearance inspection mainly relies on manual visual inspection, which is inefficient, produces inconsistent results, and cannot meet the cycle time requirements of high-capacity production lines.

[0003] Current technology uses a single top-mounted camera to capture images of the top surface of the movement from the front. Because of the occlusion between the sidewall parts and the top surface parts, a geometric blind spot inevitably occurs under a single frontal view, making it impossible to detect defects in the side and occluded areas, resulting in a high false negative rate. Furthermore, current technology uses only a single type of light source, leading to significant differences in lighting conditions between the top and side surfaces of the movement. This results in the coexistence of local shadows and saturated highlights in the image, severely impacting the detectability of edge features and further contributing to high false positive and false negative rates. Summary of the Invention

[0004] This invention provides a method, apparatus, equipment, medium, and program product for inspecting the appearance of watch movements. This invention solves the technical problem of the geometric blind spot caused by the three-dimensional structure of the watch movement under a single frontal view. It achieves full coverage detection of various heterogeneous defects such as missing parts, misalignment, and deformation in a single operation, thereby improving the accuracy of defect detection for watch movement appearance.

[0005] In a first aspect, embodiments of this application provide a method for inspecting the appearance of a watch movement, including: A first original image of the movement is acquired by a front-view camera and multiple second original images of the movement are acquired by multiple side-view cameras. The first original image and the multiple second original images are preprocessed respectively to obtain a first standard image and multiple second standard images. The first standard image and the plurality of second standard images are fused to obtain a fused feature map covering the top surface and all sides of the movement. The fused feature map is compared and detected, and the appearance detection result of the movement is output.

[0006] Optionally, in a first implementation of the first aspect of the present invention, the step of acquiring a first original image of the movement using a front-view camera and acquiring multiple second original images of the movement using multiple side-view cameras, and preprocessing the first original image and the multiple second original images respectively to obtain a first standard image and multiple second standard images, includes: The shutters of a front-view camera and multiple side-view cameras are opened simultaneously. The front-view camera captures a first raw image of the camera mechanism, and the multiple side-view cameras capture multiple second raw images of the camera mechanism. The first original image and the plurality of second original images are filtered and distorted respectively to obtain a first standard image and a plurality of second standard images.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the step of performing filtering and distortion correction on the first original image and the plurality of second original images respectively to obtain a first standard image and a plurality of second standard images includes: The first original image and the plurality of second original images are filtered respectively to obtain a first denoised image and a plurality of second denoised images; Histogram equalization and interpolation fusion are performed on the first denoised image and the plurality of second denoised images respectively to obtain a first enhanced image and a plurality of second enhanced images; Based on the first intrinsic parameter matrix of the front-view camera and the second intrinsic parameter matrices of each side-view camera, distortion correction is performed on the first enhanced image and the plurality of second enhanced images to obtain a first standard image and a plurality of second standard images.

[0008] Optionally, in a third implementation of the first aspect of the present invention, the step of performing distortion correction on the first enhanced image and the plurality of second enhanced images based on the first intrinsic parameter matrix of the front-viewing camera and the second intrinsic parameter matrices of each side-viewing camera to obtain a first standard image and a plurality of second standard images includes: Obtain the first intrinsic parameter matrix of the front-viewing camera, the first intrinsic parameter matrix including a first radial distortion coefficient and a first tangential distortion coefficient; obtain the second intrinsic parameter matrix of each side-viewing camera, the second intrinsic parameter matrix including a second radial distortion coefficient and a second tangential distortion coefficient; Based on the first radial distortion coefficient and the first tangential distortion coefficient, a first mapping lookup table is constructed for each first pixel in the first enhanced image, and bilinear interpolation resampling is performed on the first enhanced image based on the first mapping lookup table to obtain a first standard image. Based on the second radial distortion coefficient and the second tangential distortion coefficient, a second mapping lookup table is constructed for each second pixel in the second enhanced image, and bilinear interpolation resampling is performed on the second enhanced image according to the second mapping lookup table to obtain multiple second standard images.

[0009] Optionally, in a fourth implementation of the first aspect of the present invention, the step of fusing features of the first standard image and the plurality of second standard images to obtain a fused feature map covering the top surface and all sides of the movement includes: Based on the first extrinsic rotation matrix and the first translation vector of the front-view camera and the second extrinsic rotation matrix and the second translation vector of each side-view camera, feature alignment is performed on the first standard image and the plurality of second standard images to obtain alignment feature maps of each viewpoint. The contour feature values ​​and gradient magnitudes at the same world coordinate position in the aligned feature maps from each viewpoint are superimposed to obtain a fused feature map covering the top surface and all sides of the movement.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, the step of performing feature alignment on the first standard image and the plurality of second standard images based on the first extrinsic rotation matrix and the first translation vector of the front-viewing camera and the second extrinsic rotation matrix and the second translation vector of each side-viewing camera to obtain an aligned feature map for each viewpoint includes: The first camera coordinates are calculated based on the first pixel coordinates of each first pixel in the first standard image and the first intrinsic parameter matrix; the second camera coordinates are calculated based on the second pixel coordinates of each second pixel in each second standard image and the second intrinsic parameter matrix. The first camera coordinates are subjected to a left multiplication matrix transformation with the first extrinsic rotation matrix of the front-viewing camera, and then a first translation vector is superimposed to obtain the first ray direction vector of each first pixel in the unified world coordinate system; the second camera coordinates are subjected to a left multiplication matrix transformation with the second extrinsic rotation matrix of each side-viewing camera, and then a second translation vector is superimposed to obtain the second ray direction vector of each second pixel in the unified world coordinate system. Based on the first ray direction vector and the second ray direction vector, the first standard image and the plurality of second standard images are feature aligned to obtain the aligned feature map of each viewpoint.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, the step of performing feature alignment on the first standard image and the plurality of second standard images based on the first ray direction vector and the second ray direction vector to obtain alignment feature maps for each viewpoint includes: Based on the first ray direction vector, calculate the first world coordinate intersection point of the first pixel point on the Z=0 plane; based on the second ray direction vector, calculate the second world coordinate intersection point of the second pixel point on the corresponding side perpendicular sub-plane. Write the intersection points of the first and second world coordinates into the corresponding grid positions in the unified world coordinate system to obtain the alignment feature maps of each viewpoint.

[0012] Optionally, in the seventh implementation of the first aspect of the present invention, the step of comparing and detecting the fused feature map and outputting the movement appearance detection result includes: Perform contour matching detection on the target part region of the contour feature layer in the fused feature map to obtain a first detection result; BLOB connected component detection is performed on the target part's region of interest in the gradient feature layer of the fused feature map to obtain a second detection result; Based on the first and second test results, the output mechanism appearance test result is qualified if all test items have no defect markings, and unqualified if the test items have defect markings.

[0013] Optionally, in the eighth implementation of the first aspect of the present invention, the step of comparing and detecting the fused feature map and outputting the movement appearance detection result includes: The measured edge contours of each target part region are extracted from the contour feature layer of the fused feature map; The measured edge contours are compared with the contours of the good product template to generate a first detection result. The first detection result includes the target part area being a deformation defect and the target part area being a misalignment defect.

[0014] Secondly, embodiments of this application provide a movement appearance inspection device, comprising: The image acquisition module is used to acquire a first original image of the mechanism through a front-view camera and a plurality of second original images of the mechanism through a plurality of side-view cameras, and to preprocess the first original image and the plurality of second original images respectively to obtain a first standard image and a plurality of second standard images. The feature fusion module is used to perform feature fusion on the first standard image and the plurality of second standard images to obtain a fused feature map covering the top surface and all sides of the movement. The comparison and detection module is used to compare and detect the fused feature map and output the appearance detection results of the movement.

[0015] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described mechanism appearance inspection method.

[0016] Fourthly, embodiments of this application provide a readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described mechanism appearance inspection method.

[0017] Fifthly, embodiments of this application provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the above-described mechanism appearance inspection method.

[0018] In one of the solutions provided by the aforementioned methods, devices, equipment, media, and programs for inspecting the appearance of the movement, a multi-view camera array combined with a TTL hardware synchronous triggering acquisition mechanism solves the technical problem of geometric occlusion blind spots caused by the three-dimensional structure of the movement under a single frontal view, achieving 360° full-circumferential appearance information acquisition of the top surface and all sides of the movement; by independently configuring the ring LED light source and the strip LED light source in zones and using independent PWM modulation and lookup table brightness constraints, a consistent grayscale dynamic range is ensured for each image; and by using a preprocessing link based on the intrinsic parameter matrix of each camera, including denoising, contrast enhancement, and pixel-by-pixel distortion correction, the impact of sensor noise and lens distortion on subsequent contour feature extraction is eliminated. To mitigate the interference of precision issues, a multi-view coordinate projection transformation and feature-level superposition fusion based on the rotation matrix and translation vector of each camera's extrinsic parameters were used to uniformly express the contour and gradient features of each independent viewpoint in the world coordinate system. This reconstructed a complete appearance feature map covering the entire circumference of the movement, overcoming the inherent limitation that single-view two-dimensional images can only express local appearance information. By executing two complementary detection algorithms in parallel on the fused feature map—contour Hu moment similarity comparison and BLOB connected component geometric feature threshold comparison—the detection conclusions containing defect type codes and position coordinates were summarized and output. This achieved single-time full-coverage detection of various heterogeneous defects such as missing parts, misalignment, and deformation, improving the accuracy of movement appearance defect detection. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of a movement appearance inspection system according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a method for inspecting the appearance of a movement in one embodiment of the present invention; Figure 3 yes Figure 2 A schematic diagram of the implementation process of step S10; Figure 4 yes Figure 2 A schematic diagram of the implementation process of step S20; Figure 5 yes Figure 4A schematic diagram of the implementation process of step S21; Figure 6 yes Figure 2 A schematic diagram of the implementation process of step S30; Figure 7 This is a schematic diagram of a movement appearance inspection device according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. It should also be understood that, as used in this specification and the appended claims, the term "and / or" refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0023] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0024] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0025] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0026] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0027] To address the problems mentioned above in the background art, this application provides a method, apparatus, device, medium, and program product for inspecting the appearance of watch movements. The watch movement appearance inspection method provided by this invention can be applied to, for example... Figure 1 The movement appearance inspection system shown includes a client and a server.

[0028] In one embodiment, such as Figure 2 As shown, a method for inspecting the appearance of a watch movement is provided, which is then applied to... Figure 1 Taking the movement appearance inspection system in China as an example, the following steps are included: S10: The first original image of the mechanism is acquired by the front-view camera and the second original images of the mechanism are acquired by the multiple side-view cameras. The first original image and the multiple second original images are preprocessed respectively to obtain the first standard image and the multiple second standard images. S20: Perform feature fusion on the first standard image and multiple second standard images to obtain a fused feature map covering the top surface and all sides of the movement; S30: Compare and detect the fused feature maps, and output the detection results of the movement appearance.

[0029] In this embodiment, a multi-view camera array combined with a TTL hardware synchronous trigger acquisition mechanism solves the technical problem of geometric occlusion blind spots caused by the three-dimensional structure of the movement under a single frontal view, achieving 360° full-circumferential appearance information acquisition of the top surface and all sides of the movement; by independently configuring the ring LED light source and the strip LED light source and independently modulating them with PWM and using a lookup table for brightness constraints, a consistent grayscale dynamic range is ensured for each image; by using a preprocessing link based on the intrinsic parameter matrix of each camera, including denoising, contrast enhancement, and pixel-by-pixel distortion correction, the interference of sensor noise and lens distortion on the accuracy of subsequent contour feature extraction is eliminated; by using a multi-view camera array combined with a TTL hardware synchronous trigger acquisition mechanism, the technical problem of geometric occlusion blind spots caused by the three-dimensional structure of the movement is solved, achieving 360° full-circumferential appearance information acquisition of the top surface and all sides of the movement ... is solved, achieving 360° full-circumferential appearance information acquisition of the top surface and all sides of the movement is solved, achieving 360° full-circumferential appearance information acquisition of the top surface and all sides of the movement is solved, achieving 360° full-circumferential appearance information acquisition of the top surface and all sides of the movement is solved, achieving 360° full-circumferential appearance information acquisition of the top surface and all sides of the movement is solved, achieving 360° full-circumferential appearance information acquisition of the top The multi-view coordinate projection transformation and feature-level superposition fusion of the external parameter rotation matrix and translation vector unifies the contour features and gradient features of each independent viewpoint in the world coordinate system, reconstructing a complete appearance feature map covering the entire circumference of the movement, breaking through the inherent limitation that single-view two-dimensional images can only express local appearance information; by executing two complementary detection algorithms in parallel on the fused feature map—contour Hu moment similarity comparison and BLOB connected component geometric feature threshold comparison—the detection conclusion containing defect type encoding and position coordinates is summarized and output, realizing single-time full-coverage detection of various heterogeneous defects such as missing parts, misalignment, and deformation, improving the accuracy of movement appearance defect detection.

[0030] In one embodiment, such as Figure 3 As shown, step S10 specifically includes the following steps: S11: Control the front-view camera and multiple side-view cameras to open the shutter simultaneously. The front-view camera captures the first raw image of the camera mechanism, and the multiple side-view cameras capture multiple second raw images of the camera mechanism. S12: Filter and correct distortion on the first original image and multiple second original images respectively to obtain a first standard image and multiple second standard images.

[0031] In this embodiment, after the movement arrives at the inspection station, the conveying and positioning mechanism clamps and positions the movement, ensuring that the top surface of the movement is within the effective depth of field of the front-view camera, while simultaneously covering the side walls of the movement with multiple side-view cameras. The front-view camera uses an industrial camera with at least 5 megapixels; the higher resolution helps preserve details such as tiny gears, pressure plates, screws, and clamping plate edges on the top surface of the movement. The multiple side-view cameras use industrial cameras with at least 1.3 megapixels. These side-view cameras primarily handle image acquisition of the movement's side walls and obstructed edge areas, with resolution configuration focused on meeting the needs for identifying side contours, lateral missing parts, and assembly deviations. The side-view cameras are arranged at equal angular intervals along the circumference of the movement, typically with four cameras. The overlap area of ​​the field of view of adjacent side-view cameras is set to be at least 10%, ensuring that the overlap covers potential blind spots at the boundary between adjacent viewpoints, allowing for continuous representation of the side appearance information in the circumferential direction.

[0032] In this embodiment, after the positioning sensor in the detection station detects that the mechanism has entered the predetermined position, it sends a level transition positioning signal to the synchronous trigger control module. After confirming that the positioning mechanism has clamped and that the brightness of the ring LED light source and the strip LED light source has reached a stable state, the synchronous trigger control module outputs a TTL high-level trigger signal to the front-view camera and multiple side-view cameras simultaneously through a single multi-channel TTL trigger bus. Each camera opens its shutter synchronously after receiving the rising edge of the trigger signal. The TTL hardware synchronous trigger method compresses the synchronization error between the actual acquisition time of each camera to the microsecond level, which is 3 to 4 orders of magnitude better than the software trigger command method (the typical difference in software trigger timing is tens of milliseconds). This ensures that the images from each viewpoint correspond to the same static pose state in the time domain. A ring-shaped LED light source is positioned near the lens of the front-view camera and coaxially mounted with the camera's optical axis. Utilizing its large solid-angle diffuse reflection characteristics, it provides uniform frontal illumination to the top surface of the mechanism, eliminating high-saturation areas caused by specular reflection from the top surface. Strip-shaped LED light sources are positioned corresponding to the side-view cameras, with incident angles set between 30° and 45°. They utilize the grazing effect generated by oblique incidence to enhance the contrast between the edges and contours of side parts, while suppressing large-scale shadow areas caused by changes in side curvature. The output brightness of each light source is independently adjusted using PWM (Pulse Width Modulation). During calibration, each light source is individually illuminated, and grayscale histograms of the corresponding detection areas are collected. The average grayscale value difference between the image center and edge areas is calculated. A lookup table mapping method is used to correct the PWM duty cycle of each light source. Iterative adjustments are made until the brightness deviation within a single area of ​​each image is constrained to within ±5%. The converged PWM parameters of each light source are then written into the light source controller firmware as fixed light source configuration parameters for the detection phase. The front-view camera acquires the first original image of the top surface of the movement after synchronous triggering, while multiple side-view cameras acquire multiple second original images of different sides of the movement. At the same time, metadata such as acquisition timestamp, camera number, exposure time, gain value, light source channel number and product batch number are written to the first original image and multiple second original images.

[0033] In this embodiment, after image acquisition is completed, a first original image and multiple second original images are read respectively. The corresponding preprocessing parameters are then called according to the camera number to perform filtering and distortion correction on each image, forming a first standard image and multiple second standard images. The filtering process mainly reduces industrial camera sensor noise, local salt-and-pepper noise, and random grayscale disturbances in weakly textured areas, ensuring that the edges of parts, hole boundaries, and sidewall contours maintain a more stable grayscale structure in subsequent processing. Distortion correction, based on pre-calibrated camera intrinsic parameters and lens distortion parameters, maps images acquired by different cameras to a standard image space closer to the pinhole camera model, reducing the impact of lens distortion on contour position, edge distance, and subsequent multi-view fusion accuracy.

[0034] In one embodiment, step S12 specifically includes the following steps: S121; Filter the first original image and the multiple second original images respectively to obtain the first denoised image and the multiple second denoised images; S122; Perform histogram equalization and interpolation fusion on the first denoised image and multiple second denoised images respectively to obtain the first enhanced image and multiple second enhanced images; S123: Based on the first intrinsic parameter matrix of the front-viewing camera and the second intrinsic parameter matrix of each side-viewing camera, distortion correction is performed on the first enhanced image and multiple second enhanced images respectively to obtain the first standard image and multiple second standard images.

[0035] In this embodiment, the image processing computer reads the first original image output by the front-view camera and the second original images output by multiple side-view cameras according to the camera number, and calls the preprocessing parameters corresponding to the front-view and side-view perspectives based on the acquired metadata. When performing filtering processing on the first original image and the multiple second original images, for the salt-and-pepper noise generated by the industrial camera CMOS sensor under high gain or long exposure conditions, median filtering is performed on the image using a fixed-size filtering window (5×5 pixel window for the front-view camera and 3×3 pixel window for the side-view cameras). Specifically, the filtering window slides pixel by pixel on the image, and the gray values ​​of all pixels within the window are sorted in ascending order. The gray value of the center pixel is replaced by the gray value of the center pixel after sorting. Due to the non-linear characteristics of median filtering, the abnormal gray values ​​of isolated salt-and-pepper noise pixels are replaced by the normal gray values ​​of other pixels within the window. Meanwhile, the edge gradient features in the image are not blurred due to the filtering operation; for Gaussian random noise introduced by the sensor readout circuit, Gaussian filtering is performed on the image after median filtering. The image is convolved and smoothed with a pre-calibrated Gaussian kernel size and standard deviation σ=0.8. A 5×5 Gaussian kernel is used for the front-view camera and a 3×3 Gaussian kernel is used for the side-view camera. After the sequential processing of median filtering and Gaussian filtering, the first original image is converted into the first denoised image, and multiple second original images are converted into multiple second denoised images respectively. The first denoised image and multiple second denoised images still retain the real contour structure under their respective viewpoints.

[0036] In this embodiment, when performing histogram equalization processing on the first denoised image and multiple second denoised images, the Limiting Contrast Adaptive Histogram Equalization (CLAHE) algorithm is adopted. The specific process is as follows: the image is divided into 8×8 non-overlapping sub-blocks, and a local gray-level histogram is calculated for each sub-block. Before performing histogram equalization mapping, frequencies exceeding the contrast limit parameter clipLimit (set to 2.0) in the sub-block histogram are clipped and evenly distributed to the remaining gray levels of the histogram, thereby constraining the upper limit of the local contrast amplification factor and preventing noise from being amplified after excessive enhancement of weak contrast areas. Subsequently, the equalization mapping function of adjacent sub-blocks is double-lined. Bilinear interpolation, which calculates bilinear interpolation weights for any pixel in the image based on its distance from the centers of its four neighboring sub-blocks, and then sums the equalization mapping results of the four sub-blocks according to their weights to generate a continuous and smooth contrast-enhanced image of the whole image. This ensures that the enhanced edge transition remains continuous and avoids abrupt changes at the boundaries of the sub-blocks. After CLAHE processing, the edge contours of the parts in the curved area on the side of the movement and the local shadow area on the top surface change from a weak contrast state to a high contrast detectable state, while the overall highlight area of ​​the image does not become oversaturated due to the equalization operation, thus preserving the edge gradient features of the highly reflective parts on the top surface, resulting in the first enhanced image and multiple second enhanced images.

[0037] In one embodiment, step S123 specifically includes the following steps: Obtain the first intrinsic parameter matrix of the front-viewing camera, which includes the first radial distortion coefficient and the first tangential distortion coefficient; obtain the second intrinsic parameter matrix of each side-viewing camera, which includes the second radial distortion coefficient and the second tangential distortion coefficient. Based on the first radial distortion coefficient and the first tangential distortion coefficient, a first mapping lookup table is constructed for each first pixel in the first enhanced image, and bilinear interpolation resampling is performed on the first enhanced image according to the first mapping lookup table to obtain the first standard image; Based on the second radial distortion coefficient and the second tangential distortion coefficient, a second mapping lookup table is constructed for each second pixel in the second enhanced image, and bilinear interpolation resampling is performed on the second enhanced image according to the second mapping lookup table to obtain multiple second standard images.

[0038] In this embodiment, during the installation and deployment phase, a checkerboard calibration board with known grid dimensions is used to perform intrinsic parameter calibration on the front-view camera and each side-view camera. At least 15 calibration images are acquired under different poses, and the Zhang calibration method is used to solve for the intrinsic parameter matrix and distortion coefficients corresponding to each camera. The calibration results are then persistently stored in the form of a parameter file. The matrix elements of the first intrinsic parameter matrix include the horizontal focal length fx and vertical focal length fy of the front-view camera (in pixels), and the horizontal coordinate cx and vertical coordinate cy of the image principal point (in pixels). The first radial distortion coefficient includes three coefficients, k1, k2, and k3, which describe the barrel or pincushion distortion distribution in the radial direction of the lens (radiating outward from the optical axis). The first tangential distortion coefficient includes two coefficients, p1 and p2, which describe the asymmetric offset in the tangential direction caused by lens assembly eccentricity or imaging plane tilt.

[0039] In this embodiment, the specific process of constructing the first mapping lookup table is as follows: For each first pixel in the first enhanced image, its pixel coordinates (u, v) are used as input, and the radial distortion correction amount is calculated based on the first radial distortion coefficients k1, k2, and k3. Specifically, the normalized coordinates x' = (u - cx) / fx and y' = (v - cy) / fy are set, and the squared normalized distance r between the point and the principal point is calculated. 2 =x' 2 +y' 2 The radial distortion correction factor is (1+k1·r) 2 +k2·r 4 +k3·r 6 Multiply the normalized coordinates by the correction factor to obtain the normalized coordinates after radial distortion correction; then calculate the tangential distortion correction based on the first tangential distortion coefficients p1 and p2. The tangential correction in the horizontal direction is 2·p1·x'·y' + p2·(r 2 +2·x' 2 The vertical tangential correction is p1·(r) 2 +2·y' 2)+2·p2·x'·y', the above tangential correction amount is superimposed on the normalized coordinates after radial correction to obtain the normalized coordinates after complete distortion correction; finally, the normalized coordinates after correction are multiplied by the corresponding focal length and added back to the principal point coordinates to obtain the corresponding position of the pixel in the normalized image coordinate system after distortion correction, and the above mapping relationship from distorted image coordinates to normalized image coordinates is stored pixel by pixel as the first mapping lookup table. During the detection phase, based on the non-integer mapping positions recorded in the first mapping lookup table, the gray values ​​of adjacent pixels in the first enhanced image are selected for bilinear interpolation resampling: For the non-integer coordinate position (x_mapped, y_mapped) in the mapping lookup table, the gray values ​​of the four surrounding integer pixel coordinate positions (floor(x_mapped), floor(y_mapped)), (floor(x_mapped)+1, floor(y_mapped)), (floor(x_mapped), floor(y_mapped)+1), (floor(x_mapped)+1, floor(y_mapped)+1) are taken. The bilinear interpolation weights are calculated based on the horizontal distance dx and vertical distance dy between the non-integer position and the four integer positions. The gray values ​​of the four integer positions are weighted and summed according to the weights of (1-dx)·(1-dy), dx·(1-dy), (1-dx)·dy, and dx·dy to obtain the resampled gray value of the corresponding position after correction, thereby generating the first standard image. For multiple second enhanced images, the corresponding second intrinsic parameter matrix is ​​read one by one according to the side-view camera number. Based on the second radial distortion coefficient and the second tangential distortion coefficient of each side-view camera, a second mapping lookup table is established in the same way for each path. Bilinear interpolation resampling is then performed on the corresponding second enhanced images to obtain multiple second standard images. Since the side-view cameras are arranged circumferentially around the camera body, the installation angle, lens center and distortion distribution of different side-view cameras are different. Multiple second mapping lookup tables need to be generated and called separately to avoid incorrectly applying the calibration parameters of one side-view camera to another side-view image.

[0040] In one embodiment, such as Figure 4 As shown, step S20 specifically includes the following steps: S21: Based on the first extrinsic rotation matrix and the first translation vector of the front-view camera and the second extrinsic rotation matrix and the second translation vector of each side-view camera, feature alignment is performed on the first standard image and multiple second standard images to obtain the alignment feature map of each viewpoint; S22: Superimpose the contour feature values ​​and gradient magnitudes at the same world coordinate position in the aligned feature maps from each viewpoint to obtain a fused feature map covering the top surface and all sides of the movement.

[0041] In this embodiment, during the installation phase, a three-dimensional calibration target (composed of three mutually orthogonal planar checkerboard squares, each square having a side length of 5mm, with the normal directions of the three faces of the target parallel to the X-axis, Y-axis, and Z-axis, respectively) is placed at the inspection station. All cameras simultaneously acquire data from the three-dimensional calibration target. Based on the premise that the intrinsic parameters of each camera have been calibrated, the PnP algorithm is used to solve for the rotation matrix and translation vector of each camera's coordinate system relative to a unified world coordinate system with the geometric center of the three-dimensional calibration target as the origin. The calibration results are then persistently stored. During the inspection phase, the first extrinsic parameter rotation matrix and the first translation vector corresponding to the front-view camera are read, and the second extrinsic parameter rotation matrix and the second translation vector corresponding to each side-view camera are read. The corresponding extrinsic parameter parameters are called according to the camera number, and features such as edges, corners, holes, and contour segments in the first standard image and multiple second standard images are mapped to their corresponding positions in the unified world coordinate system, forming alignment feature maps for each viewpoint.

[0042] In this embodiment, before performing overlay fusion, contour feature values ​​and gradient magnitudes are extracted from standard images from each viewpoint. The contour feature values ​​are obtained as follows: after grayscale processing of the standard images from each viewpoint, edge detection is performed using the Canny operator. Specifically, the grayscale image is first Gaussian smoothed to suppress noise, then the gradient magnitude and gradient direction of each pixel in the image are calculated, non-maximum suppression is performed on the gradient magnitude map to refine the edge width, and finally, double threshold hysteresis is performed on the retained edge pixels. Pixels with edge response intensity higher than the high threshold are identified as strong edge pixels, and pixels with edge response intensity between the low and high thresholds and adjacent to strong edge pixels are identified as weak edge pixels. A binary contour feature map containing strong edge pixels and weak edge pixels that meet the connectivity condition is output. The contour feature value of each pixel in the map is its corresponding edge response intensity value. The gradient magnitude is obtained as follows: The Sobel operator is used to perform first-order difference convolution operations on the grayscale image in both the horizontal and vertical directions to obtain the horizontal gradient component Gx and the vertical gradient component Gy. Then, root mean square operations are performed on Gx and Gy for each pixel to obtain its gradient magnitude. Simultaneously, the gradient direction angle for each pixel is calculated, resulting in gradient magnitude and gradient direction maps for each viewpoint's standard image. After extracting the contour feature values ​​and gradient magnitudes for each viewpoint, based on the correspondence between the world coordinate intersection points obtained from coordinate transformation of each pixel and the unified world coordinate system grid, the contour feature values ​​and gradient magnitudes of each pixel are written into the corresponding grid positions of the unified world coordinate system feature map. When feature values ​​from multiple viewpoints exist at the same world coordinate grid location, the contour feature values ​​from each viewpoint at that grid location are summed and written to the corresponding grid in the contour feature layer. Similarly, the gradient magnitudes from each viewpoint are summed and written to the corresponding grid in the gradient feature layer, completing the overlay writing. In a preferred embodiment, the cosine of the angle between the optical axis of each camera and the normal direction of the surface corresponding to the grid location can be used as the overlay weight. Weighted overlay and normalized overlay are then applied to the feature values ​​of each viewpoint to ensure that the image angle closer to the normally incident viewpoint receives a higher feature contribution weight. For grid locations covered by only a single viewpoint, the feature values ​​of that viewpoint are directly used for filling, without performing overlay operations. After completing the overlay writing for all grid locations, the overlay result is normalized to a uniform grayscale range, resulting in a fused feature map covering the top surface and all sides of the camera body. The fused feature map consists of two independent layers: a contour feature layer, which records the superposition and fusion results of the part edge contour features from the front-view camera and the side-view camera at each grid position in the unified world coordinate system, and is used for subsequent contour matching defect detection; and a gradient feature layer, which records the superposition and fusion results of the gradient magnitudes from the front-view camera and the side-view camera at each grid position in the unified world coordinate system, and is used for subsequent BLOB connected component defect detection.The grid resolution of the fused feature map is preset according to the spatial scale of the world coordinate system and the detection accuracy requirements. The fused feature map completely covers the projection range of the top surface of the movement on the Z=0 plane and the projection range of all sides of the movement on their respective vertical sub-planes.

[0043] In one embodiment, such as Figure 5 As shown, step S21 specifically includes the following steps: S211: Calculate the coordinates of the first camera based on the first pixel coordinates and the first intrinsic parameter matrix of each first pixel in the first standard image; calculate the coordinates of the second camera based on the second pixel coordinates and the second intrinsic parameter matrix of each second standard image; S212: Perform a left multiplication matrix transformation on the first camera coordinates and the first extrinsic parameter rotation matrix of the front-viewing camera, and then superimpose the first translation vector to obtain the first ray direction vector of each first pixel in the unified world coordinate system; perform a left multiplication matrix transformation on the second camera coordinates and the second extrinsic parameter rotation matrix of each side-viewing camera, and then superimpose the second translation vector to obtain the second ray direction vector of each second pixel in the unified world coordinate system. S213: Based on the first ray direction vector and the second ray direction vector, perform feature alignment on the first standard image and multiple second standard images to obtain the alignment feature map of each viewpoint.

[0044] In this embodiment, during the installation and deployment phase, a checkerboard calibration board with known grid dimensions is used to perform intrinsic parameter calibration on the front-view camera and each side-view camera. At least 15 calibration images are acquired under different poses, and the Zhang calibration method is used to solve for the intrinsic parameter matrix and distortion coefficients corresponding to each camera. The calibration results are then persistently stored in parameter files. During the detection phase, the first intrinsic parameter matrix corresponding to the front-view camera and the second intrinsic parameter matrix corresponding to each side-view camera are read from the stored parameter files according to the camera number. The first intrinsic parameter matrix is ​​a 3x3 upper triangular matrix. Matrix elements include the horizontal focal length fx and vertical focal length fy of the front-view camera (in pixels), and the horizontal coordinate cx and vertical coordinate cy of the image principal point (in pixels). For each first pixel in the first standard image, its first pixel coordinates are represented as (u, v), where u is the column coordinate and v is the row coordinate, both in pixels. The specific process of matching the first pixel coordinates (u, v) of the first pixel point with the first intrinsic parameter matrix is ​​as follows: Subtract the horizontal coordinate cx of the image principal point from the column coordinate u to obtain the horizontal pixel offset (u-cx). Then divide this offset by the horizontal focal length fx to obtain the normalized horizontal camera coordinate component xc, i.e., xc = (u-cx) / fx; Subtract the vertical coordinate cy of the image principal point from the row coordinate v to obtain the vertical pixel offset (v-cy). Then divide this offset by the vertical focal length fy to obtain the normalized vertical camera coordinate component yc, i.e., yc = (v-cy) / fy; Set the depth normalized component zc = 1, and combine xc, yc, and zc into a three-dimensional column vector [xc, yc, 1]. T This refers to the first camera coordinates of the first pixel in the front-view camera coordinate system. Its direction describes the normalized direction of the 3D ray pointing from the optical center of the front-view camera to the corresponding spatial position of the first pixel in the camera coordinate system. The aforementioned first camera coordinates are non-normalized direction vectors along the ray direction. These will be further transformed to a unified world coordinate system and normalized in the extrinsic parameter transformation step to obtain the first ray direction vector. For each second pixel in each second standard image, the second pixel coordinates are decentered and normalized in the same way based on the focal length parameter and principal point coordinates in the second intrinsic parameter matrix of the corresponding side-view camera, resulting in the second camera coordinates of each second pixel in the corresponding side-view camera coordinate system. Since multiple side-view cameras are arranged circumferentially along the camera core, the focal length, principal point position, and mounting posture of different side-view cameras differ. Therefore, the second camera coordinates of each side-view camera need to be calculated separately based on its own independent second intrinsic parameter matrix to avoid scale deviation or center offset between different side-view images.

[0045] In this embodiment, the first camera coordinates [xc, yc, 1] of each first pixel are obtained. TNext, the first extrinsic parameter rotation matrix R1 and the first translation vector T1 corresponding to the orthographic camera are invoked. The first extrinsic parameter rotation matrix R1 is a 3x3 orthogonal rotation matrix, describing the rotation relationship between the orthographic camera coordinate system and the unified world coordinate system; the first translation vector T1 is a 3x1 column vector, describing the position of the origin of the unified world coordinate system in the orthographic camera coordinate system. The coordinates of the orthographic camera's optical center in the world coordinate system can be obtained from -R1. T T1 is calculated, where R1 T Let R1 be the transpose of the matrix. For each first pixel, the first camera coordinates are [xc, yc, 1]. T Then perform a left multiplication matrix transformation on it and the first extrinsic parameter rotation matrix R1, that is, calculate R1·[xc, yc, 1] T Then, the above rotation transformation results are superimposed on the first translation vector T1, i.e., R1·[xc, yc, 1] is executed. T +T1; Divide the result by its L2 norm to perform normalization, obtaining the unit direction vector from the optical center of the frontal camera to the spatial position corresponding to the first pixel, which is the first ray direction vector d1. Using the world coordinates of the optical center of the frontal camera O1=-R1... T • T1 is the ray origin, and d1 is the ray direction, forming the parametric ray P = O1 + t·d1 of the first pixel in the unified world coordinate system, where t is the depth parameter along the ray direction. For each side-view camera, the second camera coordinates of each second pixel are adjusted in the same way according to the second extrinsic parameter rotation matrix R2 and the second translation vector T2, performing R2·[xc2, yc2, 1]. T After normalization using +T2, the second ray direction vector d2 of each second pixel is obtained, with the world coordinates of the optical center of the side-view camera being O2=-R2. T • T2 is the starting point of the ray that forms the parameterized ray for each second pixel point.

[0046] In one embodiment, step S213 specifically includes the following steps: Based on the first ray direction vector, calculate the first world coordinate intersection point of the first pixel point on the Z=0 plane; based on the second ray direction vector, calculate the second world coordinate intersection point of the second pixel point on the corresponding side perpendicular sub-plane. The intersection points of the first-world coordinate system and the intersection points of the second-world coordinate system are written into the corresponding grid positions in the unified world coordinate system to obtain the alignment feature maps of each viewpoint.

[0047] In this embodiment, for each first pixel in the first standard image, taking the world coordinates O1 of the optical center of the front-viewing camera as the ray origin and the first ray direction vector d1 as the ray direction, let the Z component of P in the ray parameter equation P=O1+t·d1 be equal to the Z=0 plane value where the top surface of the mechanism is located, to obtain O1z+t·d1z=0, where O1z is the Z component of the ray origin O1 and d1z is the Z component of the first ray direction vector d1; when the absolute value of d1z is greater than the preset minimum threshold (taken as 1×10), -6 When solving for the parameter t = -(O1z / d1z), t is substituted into the ray parameter equation to calculate the X component Px = O1x + t·d1x and the Y component Py = O1y + t·d1y respectively, and the first world coordinate intersection point (Px, Py, 0) of the first pixel point on the Z=0 plane is obtained. When the absolute value of d1z is less than or equal to the preset minimum threshold, it indicates that the ray direction is approximately parallel to the Z=0 plane, and the corresponding first pixel point is marked as an invalid projection point and does not participate in the generation of the fusion feature map. For each second pixel in each second standard image, the side perpendicular subplane is determined by the direction of the sidewall of the mechanism covered by the corresponding side-view camera, defined by the unit vector n of the outward normal direction of the side and the coordinates P0 of the pre-calibrated reference point on the side; taking the world coordinates O2 of the optical center of the side-view camera as the ray origin and the second ray direction vector d2 as the ray direction, let the ray and the side perpendicular subplane satisfy the plane equation n·(P-P0)=0, substituting into the ray parameter equation, we get n·(O2+t·d2-P0)=0, and rearranging, we get t=[(P 0-O2)·n] / (d2·n), where "·" represents the vector dot product operation; when the absolute value of d2·n is greater than the preset minimum threshold, solve the parameter equation of the input ray P=O2+t·d2 after t according to the above formula, and calculate the second world coordinate intersection point of the second pixel point on the corresponding side vertical sub-plane; when the absolute value of d2·n is less than or equal to the preset minimum threshold, it indicates that the ray direction is approximately parallel to the corresponding side vertical sub-plane, and the corresponding second pixel point is marked as an invalid projection point and does not participate in the generation of the fusion feature map.

[0048] In this embodiment, after calculating the intersection points of the first and second world coordinate systems, the continuous spatial coordinates are converted into discrete grid positions in a unified world coordinate system based on the preset world coordinate grid resolution. The grid origin corresponds to the origin of the world coordinate system. The grid unit size is preset according to the core size and detection accuracy requirements. The world coordinate intersection points (Px, Py, Pz) of each pixel are converted into grid coordinates (floor(Px / grid_size+map_w / 2), floor(Py / grid_size+map_h / 2)) according to the grid unit size, where grid_size is the grid unit size, and map_w and map_h are the width and height of the fused feature map (in units of grid numbers), respectively. The contour feature value, gradient magnitude, and camera number of the corresponding pixel are written into the corresponding grid position. For pixels where the ray fails to fall into the effective top plane or the effective side perpendicular sub-plane, the corresponding pixel is marked as an invalid projection point, and invalid projection points are prevented from participating in the generation of the alignment feature map. The first standard image forms the alignment feature map of the top view, and multiple second standard images form the alignment feature maps of each side view. The alignment feature maps of each view have a consistent spatial index relationship in the unified world coordinate system.

[0049] In one embodiment, such as Figure 6 As shown, step S30 specifically includes the following steps: S31: Perform contour matching detection on the target part region of the contour feature layer in the fused feature map to obtain the first detection result; S32: Perform BLOB connected component detection on the region of interest of the target part in the gradient feature layer of the fused feature map to obtain the second detection result; S33: Based on the first and second test results, the appearance test result of the output mechanism with no defect markings in all test items is deemed qualified, and the appearance test result of the output mechanism with defect markings in the test items is deemed unqualified.

[0050] In this embodiment, defect detection is performed on different feature layers of the fused feature map according to the pre-registered target part region, target part region of interest, and detection item configuration table. Step S31 locates the target part region from the contour feature layer of the fused feature map, extracts the edge contour, contour center position, and contour morphology features within the target part region, and matches and compares the measured contour corresponding to the target part region with the good product template contour to form a first detection result characterizing misalignment defects, deformation defects, and contour anomalies. Step S32 reads the target part region of interest from the gradient feature layer of the fused feature map, generates a binary detection region based on the gradient response intensity and region grayscale distribution, performs BLOB connected component labeling on the binary detection region, extracts the area, perimeter, roundness, aspect ratio, and other geometric features of each connected component, and compares the extraction results with the statistical threshold range of the good product samples to form a second detection result.

[0051] In this embodiment, the establishment of the good product template contour feature library and the geometric feature threshold range of good product samples is as follows: Before online inspection, a complete set of multi-view images of no less than 20 confirmed qualified mechanisms are collected. Corresponding fusion feature maps are generated according to the preprocessing pipeline and feature fusion pipeline. The edge contours of each target part region are extracted from the contour feature layers of each fusion feature map. A 7-dimensional Hu moment feature vector (Hu moments have rotation, scaling, and translation invariance) and a normalized Fourier descriptor vector (retaining the first 32 coefficients) are calculated for each contour curve. The Hu moments of the same part contour in the 20 good product samples are then... The mean values ​​of the vector and the Fourier descriptor vector are respectively used to construct the good product template feature vector, and the mean and standard deviation of the center coordinates of the contour of each part are calculated. The area, perimeter, roundness and aspect ratio of the connected regions within the region of interest of each target part are extracted from the gradient feature layers of each fused feature map, and the mean and standard deviation of the corresponding 20 good product samples are calculated. The lower threshold of each geometric feature is set to the mean minus 3 times the standard deviation, and the upper threshold is set to the mean plus 3 times the standard deviation, and a geometric feature threshold range table of the region of interest of each target part is constructed. The above template feature vector and threshold range table are persistently stored for use in the detection stage.

[0052] Step S33 summarizes the first and second inspection results according to the target part number, inspection item number, defect type code, and world coordinate position, forming a comprehensive judgment record for each target part. When no defect mark is generated for any inspection item, the output movement appearance inspection result is qualified, and the inspection timestamp (Unix time, accurate to milliseconds), camera view source, product batch number, and qualified conclusion are written into the inspection record; when any inspection item has a defect mark, the output movement appearance inspection result is unqualified, and the defect type code (01=part missing, 02=part misalignment, 03=part deformation, 04=foreign object), the world coordinate position of the defect, the associated target part area, and the source of the detection algorithm that triggered the defect judgment are recorded simultaneously. The inspection conclusion is output in real time to the programmable controller to drive the actuator to classify and sort according to the defect type code.

[0053] In one embodiment, step S31 specifically includes the following steps: Extract the measured edge contours of each target part region from the contour feature layer of the fused feature map; The measured edge contours are compared with the contours of the good product template to generate the first detection result. The first detection result includes whether the target part area is a deformation defect or a misalignment defect.

[0054] In this embodiment, each target part region is located in the contour feature layer of the fused feature map according to the detection item configuration table. Connected contour tracking is performed on the contour feature map of each target part region, so that the edge pixels belonging to the outer boundary of the same target part are connected into the measured edge contour according to the spatial adjacency relationship. For contour segments with local breaks, local completion is performed according to the adjacent edge direction, endpoint distance and the preset boundary range of the target part region.

[0055] In this embodiment, a 7-dimensional Hu moment feature vector is calculated for the extracted measured edge contour: the image moments of the contour region are calculated, the zero-order moment m00 is the total gray value of the pixels in the contour area (used to calculate the coordinates of the contour center), the first-order moments m10 and m01 are used to calculate the horizontal coordinate cx_actual=m10 / m00 and the vertical coordinate cy_actual=m01 / m00 of the contour center, and the second-order and third-order central moments are used to calculate the 7 Hu invariant moment components; then the normalized Fourier descriptor vector of the contour curve is calculated: the contour point sequence is represented as a complex sequence z(t)=x(t)+j·y(t), the discrete Fourier transform is performed on the complex sequence to obtain the coefficient sequence Z(k), the first 32 coefficients of the amplitude sequence |Z(k)| of Z(k) are taken, and all coefficients are normalized with |Z(1)| to obtain the normalized Fourier descriptor vector, which has rotation, scaling and translation invariance. The cosine similarity is calculated between the Hu moment feature vector of the measured edge contour and the Hu moment feature vector of the pre-stored good product template contour. The cosine similarity is calculated as (A·B) / (|A|·|B|), where A and B are the feature vectors of the measured and template contours, respectively. The cosine similarity is also calculated between the measured Fourier descriptor vector and the pre-stored Fourier descriptor vector of the good product template. If the Hu moment cosine similarity is lower than the pre-stored similarity threshold of 0.85 or the Fourier descriptor cosine similarity is lower than the pre-stored similarity threshold of 0.80, a deformation defect is determined in the target part area. Defect type code 03 is output, and the world coordinate position of the deformation area is recorded. The Euclidean distance dist = sqrt((cx_actual - cx_template)) between the center coordinates of the measured contour (cx_actual, cy_actual) and the center coordinates of the pre-stored good product template contour (cx_template, cy_template) is calculated. 2 +(cy_actual-cy_template) 2 The distance is compared with the upper limit of the position tolerance (the Euclidean distance corresponding to the mean ± 3 times the standard deviation). When dist exceeds the upper limit of the position tolerance, it is determined that there is a misalignment defect in the target part area, the defect type code 02 is output, and the world coordinate position corresponding to the misalignment direction and offset and the associated target part number are recorded. For the same target part area, if both the contour similarity is insufficient and the position offset exceeds the limit, the deformation defect and the misalignment defect mark are written into the first detection result at the same time, and the defect type with higher risk is retained as the main defect code according to the detection rules.

[0056] In this embodiment, the following BLOB connected component detection process is performed on the regions of interest of each target part in the gradient feature layer of the fused feature map: perform Otsu adaptive threshold binarization on the gradient magnitude map of the region of interest. The Otsu algorithm traverses all possible thresholds T (the value range is 0 to 255), and calculates the between-class variance σ when dividing the pixels into two categories: foreground (gradient magnitude ≥ T) and background (gradient magnitude < T) for each threshold T. 2 B(T)=w0(T)·w1(T)·[μ0(T)-μ1(T)] 2 , where w0(T) and w1(T) are the proportions of foreground and background pixels respectively, and μ0(T) and μ1(T) are the means of foreground and background pixels respectively. Select the threshold T that maximizes the between-class variance as the optimal segmentation threshold, set the pixels with gradient magnitude greater than T in the region of interest to 255, and the rest to 0, to obtain the binarized detection region. Perform an 8-connected connected component labeling algorithm on the binarized detection region: adopt a two-pass scanning method. In the first pass, scan pixel by pixel from left to right and from top to bottom, and check the connected component labels of the left neighbor pixel and the upper neighbor pixel (and the upper left and upper right diagonal neighbor pixels) for each pixel with a value of 255. If there is no labeled pixel in the neighborhood, assign a new label to this pixel. If there is a labeled pixel in the neighborhood, assign the smallest label among them to the current pixel and record the label equivalence relationship. In the second pass, unify and merge the equivalent labels into the same connected component number according to the equivalence relationship table to obtain the number mapping map of all connected components in the binarized detection region. Extract the following geometric features for each connected component: area A (the total number of pixels in the connected component), perimeter P (the number of pixels on the outer boundary of the connected component), circularity C = 4π·A / P 2 (the value range is 0 to 1, and the closer the value is to 1, the closer the shape is to a circle), and the aspect ratio AR = bw / bh corresponding to the width bw and height bh of the bounding rectangle. Compare each geometric feature value with the statistical threshold range of the corresponding region of interest in the pre-stored good sample item by item: when the area of the largest connected component in the region of interest is lower than the lower limit threshold of the pre-stored area, it is determined as a part missing defect, and the defect type code 01 is output; when the number of connected components in the region of interest exceeds the statistical upper limit of the good sample, or the circularity or aspect ratio of any connected component exceeds the corresponding threshold range, it is determined as a foreign object or deformation defect, and the defect type code 03 or 04 is output; summarize the detection results of all regions of interest to form the second detection result, and record the defect type code of each defect item, the world coordinate position corresponding to the center coordinates of the bounding rectangle of the defect connected component, and the region of interest number of the associated target part.

[0057] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution is prior or posterior. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0058] In one embodiment, a movement appearance inspection device is provided, which corresponds one-to-one with the movement appearance inspection method described in the above embodiments. For example... Figure 7 As shown, the movement appearance inspection device includes: The image acquisition module 701 is used to acquire a first original image of the camera mechanism through a front-view camera and a plurality of second original images of the camera mechanism through a plurality of side-view cameras, and to preprocess the first original image and the plurality of second original images respectively to obtain a first standard image and a plurality of second standard images. The feature fusion module 702 is used to perform feature fusion on the first standard image and multiple second standard images to obtain a fused feature map covering the top surface and all sides of the movement. The comparison and detection module 703 is used to compare and detect the fused feature map and output the appearance detection results of the movement.

[0059] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0060] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0061] This application also provides a computer device, such as... Figure 8 As shown, the computer device includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor. When the processor executes the computer program, it implements the steps in any of the above method embodiments, or when the processor executes the computer program, it implements the functions of each module / unit in the above device embodiments.

[0062] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the computer device.

[0063] Those skilled in the art will understand that Figure 8 The computer device described is merely an example and does not constitute a limitation on the computer device. It may include more or fewer components than shown, or combine certain components, or different components. For example, the computer device may also include input / output devices, network access devices, buses, etc.

[0064] The aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0065] The memory can be an internal storage unit of the computer device, such as a hard drive or RAM. The memory can also be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory can include both internal and external storage units of the computer device.

[0066] This application also provides a readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0067] This application provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0068] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0069] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0070] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0071] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0072] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0073] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for inspecting the appearance of a watch movement, characterized in that, include: A first original image of the movement is acquired by a front-view camera and multiple second original images of the movement are acquired by multiple side-view cameras. The first original image and the multiple second original images are preprocessed respectively to obtain a first standard image and multiple second standard images. Feature fusion is performed on the first standard image and the plurality of second standard images to obtain a fused feature map covering the top surface and all sides of the movement; specifically, this includes: calculating the first camera coordinates based on the first pixel coordinates and the first intrinsic parameter matrix of each first pixel in the first standard image; calculating the second camera coordinates based on the second pixel coordinates and the second intrinsic parameter matrix of each second pixel in each second standard image; performing a left multiplication matrix transformation on the first camera coordinates and the first extrinsic parameter rotation matrix of the front-view camera, and then superimposing a first translation vector to obtain the first ray direction vector of each first pixel in a unified world coordinate system; and performing a left multiplication matrix transformation on the second camera coordinates and the second extrinsic parameter rotation matrix of each side-view camera respectively. After performing a left multiplication matrix transformation and superimposing a second translation vector, the second ray direction vectors of each second pixel point in the unified world coordinate system are obtained. Based on the first ray direction vector, the first world coordinate intersection point of the first pixel point on the Z=0 plane is calculated. Based on the second ray direction vector, the second world coordinate intersection points of the second pixel point on the corresponding side perpendicular sub-plane are calculated. The first world coordinate intersection point and the second world coordinate intersection point are written into the corresponding grid position in the unified world coordinate system to obtain the alignment feature map of each viewpoint. The contour feature value and gradient magnitude of the same world coordinate position in the alignment feature map of each viewpoint are superimposed to obtain the fused feature map covering the top surface of the movement and all sides. The fused feature map is compared and detected, and the appearance detection result of the movement is output.

2. The movement appearance inspection method as described in claim 1, characterized in that, The process involves acquiring a first original image of the movement using a front-view camera and multiple second original images of the movement using multiple side-view cameras. Preprocessing the first original image and the multiple second original images yields a first standard image and multiple second standard images, including: The shutters of a front-view camera and multiple side-view cameras are opened simultaneously. The front-view camera captures a first raw image of the camera mechanism, and the multiple side-view cameras capture multiple second raw images of the camera mechanism. The first original image and the plurality of second original images are filtered and distorted respectively to obtain a first standard image and a plurality of second standard images.

3. The movement appearance inspection method as described in claim 2, characterized in that, The step of filtering and distortion correction on the first original image and the plurality of second original images respectively to obtain a first standard image and a plurality of second standard images includes: The first original image and the plurality of second original images are filtered respectively to obtain a first denoised image and a plurality of second denoised images; Histogram equalization and interpolation fusion are performed on the first denoised image and the plurality of second denoised images respectively to obtain a first enhanced image and a plurality of second enhanced images; Based on the first intrinsic parameter matrix of the front-view camera and the second intrinsic parameter matrices of each side-view camera, distortion correction is performed on the first enhanced image and the plurality of second enhanced images to obtain a first standard image and a plurality of second standard images.

4. The movement appearance inspection method as described in claim 3, characterized in that, The distortion correction is performed on the first enhanced image and the plurality of second enhanced images based on the first intrinsic parameter matrix of the front-view camera and the second intrinsic parameter matrices of each side-view camera, respectively, to obtain a first standard image and a plurality of second standard images, including: Obtain the first intrinsic parameter matrix of the front-viewing camera, the first intrinsic parameter matrix including a first radial distortion coefficient and a first tangential distortion coefficient; obtain the second intrinsic parameter matrix of each side-viewing camera, the second intrinsic parameter matrix including a second radial distortion coefficient and a second tangential distortion coefficient; Based on the first radial distortion coefficient and the first tangential distortion coefficient, a first mapping lookup table is constructed for each first pixel in the first enhanced image, and bilinear interpolation resampling is performed on the first enhanced image based on the first mapping lookup table to obtain a first standard image. Based on the second radial distortion coefficient and the second tangential distortion coefficient, a second mapping lookup table is constructed for each second pixel in the second enhanced image, and bilinear interpolation resampling is performed on the second enhanced image according to the second mapping lookup table to obtain multiple second standard images.

5. The movement appearance inspection method as described in claim 1, characterized in that, The comparison and detection of the fused feature map, and the output of the movement appearance detection result, includes: Perform contour matching detection on the target part region of the contour feature layer in the fused feature map to obtain a first detection result; BLOB connected component detection is performed on the target part's region of interest in the gradient feature layer of the fused feature map to obtain a second detection result; Based on the first and second test results, the output mechanism appearance test result is qualified if all test items have no defect markings, and unqualified if the test items have defect markings.

6. The movement appearance inspection method as described in claim 5, characterized in that, The comparison and detection of the fused feature map, and the output of the movement appearance detection result, includes: The measured edge contours of each target part region are extracted from the contour feature layer of the fused feature map; The measured edge contours are compared with the contours of the good product template to generate a first detection result. The first detection result includes the target part area being a deformation defect and the target part area being a misalignment defect.

7. A movement appearance inspection device, characterized in that, The steps for implementing the movement appearance inspection method as described in any one of claims 1 to 6 include: The image acquisition module is used to acquire a first original image of the mechanism through a front-view camera and a plurality of second original images of the mechanism through a plurality of side-view cameras, and to preprocess the first original image and the plurality of second original images respectively to obtain a first standard image and a plurality of second standard images. The feature fusion module is used to perform feature fusion on the first standard image and the plurality of second standard images to obtain a fused feature map covering the top surface and all sides of the movement. The comparison and detection module is used to compare and detect the fused feature map and output the appearance detection results of the movement.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the movement appearance inspection method as described in any one of claims 1 to 6.

9. A readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the movement appearance inspection method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the movement appearance inspection method as described in any one of claims 1 to 6.

Images