An electronic cabin assembly quality detection method based on three-dimensional vision

Abstract

The application discloses a kind of electronic cabin assembly quality detection methods based on three-dimensional vision, first for standard electronic cabin generation for detecting time comparison data;Then for the electronic cabin to be detected in all objects to be detected, obtain the depth image and color image of each object to be detected by shooting;Then according to the depth image, color image of the object to be detected combined with the data for detecting time comparison of standard electronic cabin, in turn detect whether the screw of the electronic cabin to be detected is missing and not installed in place, whether cable plug is missing and not installed in place, whether connector and cable plug in detection wiring unit are matched error, whether cable bending radius is too small.The application realizes three-dimensional quantitative detection of assembly quality by fusing depth information, significantly improves detection accuracy, reliability and automation level.

Description

Technical Field

[0001] This invention relates to the field of automated inspection technology, and in particular to a method for inspecting the assembly quality of an electronic cabin based on three-dimensional vision. Background Technology

[0002] In the manufacturing of high-end equipment such as aerospace and precision instruments, assembly quality is a key factor determining the final performance and reliability of a product. Especially in assembly environments with complex structures and confined spaces, such as missile electronics compartments, the installation accuracy of components and the correctness of cable connections directly affect the functionality and safety of the entire system.

[0003] To replace inefficient and subjectively-dependent manual visual inspection, machine vision inspection technology has been widely used in automated quality control. Existing automated inspection methods typically use industrial cameras to acquire high-resolution two-dimensional images of the target object and then use image processing and pattern recognition algorithms to determine assembly quality. For example, template matching and edge detection technologies can identify whether parts are misassembled or missing; optical character recognition (OCR) technology can read and verify cable label information.

[0004] To address the ever-increasing demands for inspection accuracy, more complex solutions have emerged in existing technologies, such as the visual inspection method for complex assembly structures disclosed in Chinese Patent CN119131332A. This method represents an advanced technical approach in the field of two-dimensional visual inspection. Its core idea is to adaptively select a multi-scale decomposition algorithm (such as wavelet transform) to decompose a two-dimensional image into components of different frequencies, and extract deep features from the high-frequency components that best reflect detailed characteristics to identify minute defects that are difficult to detect using conventional methods. This method aims to achieve extremely high detection accuracy within the framework of two-dimensional information by deeply mining the detailed information of the image.

[0005] However, in practice, it has been found that the existing technologies that rely on pure two-dimensional image analysis are not sensitive to assembly defects in the depth direction when applied to assembly scenarios with complex three-dimensional spatial characteristics, such as electronic cabins.

[0006] During assembly, defects such as screws not being fully tightened or plugs not being properly inserted often occur, indicating "improper installation." These defects may appear very faint in two-dimensional images, sometimes even indistinguishable from images of a properly installed device. Pure two-dimensional vision technology, unable to acquire depth information, struggles to accurately determine the spatial position of a target along the Z-axis (depth direction), leading to high rates of missed and false detections of these critical defects, posing a serious threat to the long-term reliability of equipment. For critical assembly parameters like the bending radius of flexible cables, which are essentially three-dimensional geometric parameters, pure two-dimensional vision systems can only acquire the cable's projection onto the imaging plane, unable to directly measure its spatial orientation and degree of bending. While indirect inference can be made by analyzing light, shadow, and contour information in the image, this method is complex, lacks precision, and is highly susceptible to lighting conditions and shooting angles, making it difficult to meet the stringent assembly quality requirements of high-end equipment.

[0007] Therefore, there is an urgent need in this field for a new technical solution to overcome the limitations of existing two-dimensional vision inspection technology in dealing with assembly defects related to three-dimensional space, and to achieve reliable automated inspection of assembly quality in narrow spaces. Summary of the Invention

[0008] The technical problem to be solved by this invention is that, in special environments such as missile electronic compartments where the field of view is narrow, complex, and limited, existing pure two-dimensional vision inspection technology cannot accurately quantify and evaluate three-dimensional geometry and is insensitive to assembly defects in the depth direction. This invention provides a three-dimensional vision-based electronic compartment assembly quality inspection method to overcome the shortcomings of the prior art.

[0009] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0010] An assembly quality inspection method for an electronic cabin based on three-dimensional vision is disclosed. The wiring unit within the electronic cabin includes connectors and corresponding cable plugs. Each wiring unit has the same label on both the connector and its corresponding cable plug, but the labels on the connectors in each wiring unit are different. The method includes the following steps:

[0011] Step 1), for the standard electronic chamber, generate data for comparison during testing;

[0012] Step 1.1) Set and record the corresponding camera positions and shooting angles for all objects to be tested in the standard electronic cabin; the objects to be tested include screws, wiring units, and cables to be tested;

[0013] Step 1.2): For each object to be inspected in the standard electronic cabin, an industrial camera is used to take pictures according to its corresponding camera position and shooting angle to obtain the depth image and color image of each object to be inspected, which are used as the template depth image and template color image of the object to be inspected, respectively.

[0014] Step 1.3), for each screw to be inspected:

[0015] Step 1.3.1), record its number and its corresponding template depth image and template color image; record the bounding rectangle of the pixels occupied by the screw to be detected in the template color image of the screw to be detected, and use it as the ROI region of the screw to be detected;

[0016] Step 1.3.2) Extract HOG features from the ROI region of the screw to be detected in its template color image and apply a Hanning window to the HOG features to obtain the template features of the screw to be detected. Use the template features of the screw to be detected to train a discriminative correlation filter to obtain the correlation filter of the screw to be detected.

[0017] Step 1.4), for each wiring unit to be tested:

[0018] Step 1.4.1), record its connector number, its cable plug number, and the corresponding template depth image and template color image; record the pixel coordinates of the bounding rectangle of the occupied pixels of the cable plug in the template color image of the wiring unit as the ROI area of ​​the cable plug of the wiring unit.

[0019] Step 1.4.2) Extract HOG features from the ROI region of the cable plug of the wiring unit in its template color image and apply Hanning window to the HOG features to obtain the template features of the cable plug of the wiring unit to be detected. Use the template features of the cable plug to train a discriminative correlation filter to obtain the correlation filter of the cable plug of the wiring unit to be detected.

[0020] Step 2), for the electronic cabin to be inspected, according to the recorded camera positions and shooting angles of the objects to be inspected, use an industrial camera to take pictures of all the objects to be inspected inside, and obtain the depth image and color image of each object to be inspected, as its depth image and color image to be inspected.

[0021] Step 3) Check whether the screws of the electronic compartment to be tested are missing or improperly installed. For each screw of the electronic compartment to be tested:

[0022] Step 3.1) Extract the HOG features of the ROI region of the screw to be detected in the color image to be detected and apply the Hanning window to the HOG features to obtain the sample features of the screw. Substitute the template features and sample features of the screw to be detected into the Gaussian kernel correlation formula to calculate the kernel correlation matrix of the spatial domain of the screw to be detected. Substitute the kernel correlation matrix and the correlation filter of the screw to be detected into the correlation filter response formula to calculate the initial two-dimensional response map of the screw to be detected.

[0023] Step 3.2): Calculate the depth deviation of each pixel in the ROI region of the screw to be detected in the template depth image and the depth image to be detected, and then calculate the average depth deviation. Substitute the average depth deviation into the exponential decay function to calculate the depth weight coefficient of the screw to be detected. Multiply the initial two-dimensional response map of the screw to be detected by the depth weight coefficient of the screw to be detected to obtain the depth information weighted three-dimensional enhanced response map of the screw to be detected. Compare the response peak of the three-dimensional enhanced response map of the screw to be detected with a preset first threshold and a preset second threshold, where the first threshold is greater than the second threshold. If the response peak of the three-dimensional enhanced response map of the screw to be detected is greater than or equal to the first threshold, the screw to be detected is determined to be correctly installed; if the response peak of the three-dimensional enhanced response map of the screw to be detected is less than the first threshold but greater than or equal to the second threshold, the screw to be detected is determined to be not installed properly; if the response peak of the three-dimensional enhanced response map of the screw to be detected is less than the second threshold, the screw to be detected is determined to be not installed.

[0024] Step 4), check whether the cable plugs are missing or improperly installed. For each wiring unit to be tested in the electronic compartment:

[0025] Step 4.1) Extract the HOG features of the ROI region of the cable plug in the color image of the wiring unit to be tested, and apply the Hanning window to the HOG features to obtain the sample features of the cable plug. Substitute the template features and sample features of the cable plug into the Gaussian kernel correlation formula to calculate the kernel correlation matrix of the cable plug in the spatial domain. Substitute the kernel correlation matrix and the correlation filter of the cable plug of the wiring unit to be tested into the correlation filter response formula to calculate the initial two-dimensional response map of the cable plug of the wiring unit to be tested.

[0026] Step 4.2) Calculate the depth deviation of each pixel in the ROI region of the cable plug of the wiring unit to be tested in its template depth image and the depth image to be tested, and then calculate the average depth deviation. Substitute it into the exponential decay function to calculate the depth weight coefficient of the cable plug. Multiply the initial two-dimensional response map of the cable plug of the wiring unit to be tested by the depth weight coefficient of the cable plug to obtain the depth information weighted three-dimensional enhanced response map of the cable plug. Compare the response peak of the three-dimensional enhanced response map of the cable plug with the preset third threshold and the preset fourth threshold, where the third threshold is greater than the fourth threshold. If the response peak of the three-dimensional enhanced response map of the cable plug is greater than or equal to the third threshold, the cable plug is determined to be correctly installed; if the response peak of the three-dimensional enhanced response map of the cable plug is less than the third threshold but greater than or equal to the fourth threshold, the cable plug is determined to be not installed properly; if the response peak of the three-dimensional enhanced response map of the cable plug is less than or equal to the fourth threshold, the cable plug is determined to be not installed.

[0027] Step 5), check whether the connectors and cable plugs in the wiring unit are mismatched. For each wiring unit to be tested in the electronics compartment:

[0028] Step 5.1) Use the PP-OCRv2 model to identify the label text of the cable plug and the label text of the connector in the color image of the wiring unit to be inspected;

[0029] Step 5.2) Based on the principle of spatial proximity, compare the label text of the cable plug with the label text of the nearest connector. If they are the same, the connector and cable plug are matched correctly; if they are different, the connector and cable plug are not matched correctly.

[0030] Step 6), check if the cable bending radius is too small. For each cable to be tested in the electronic compartment:

[0031] Step 6.1) Use depth threshold-based segmentation technology to segment the cable region of the cable to be detected in the depth image to obtain the cable mask, preprocess the cable mask, and then extract the cable skeleton from the preprocessed cable mask through a thinning algorithm. Optimize the cable skeleton, remove redundant pixels on the cable skeleton and disconnect the intersections of the cable skeleton.

[0032] Step 6.2) Two-dimensional point sampling is performed on the cable skeleton. The depth Z of the sampling point is obtained from the pixel coordinates of the sampling point in the depth image to be detected. Combined with the intrinsic parameters of the industrial camera, the X and Y coordinates of the sampling point in the camera coordinate system are calculated by back projection to obtain the three-dimensional information of the sampling point. Based on the sampling points containing three-dimensional information, the three-dimensional shape of the flexible cable is reconstructed by fitting a spatial curve. The radius of curvature at each sampling point is calculated and the minimum value is compared with the preset radius of curvature threshold. If the minimum value is less than the preset radius of curvature threshold, it is determined that the bending radius of the cable at that sampling point is too small.

[0033] As a further optimization of the electronic cabin assembly quality inspection method based on three-dimensional vision of the present invention, the Gaussian kernel correlation formulas mentioned in steps 3.1) and 4.1) are as follows:

[0034] ;

[0035] In the formula, The kernel correlation matrix represents the spatial domain. Indicates template features, Indicates sample features, Indicates the inverse Fourier transform. Representative template feature The complex conjugate of each channel in the frequency domain. Indicates the sample feature number Frequency domain representation of each channel, This indicates element-wise multiplication. The total number of elements of the template feature in the spatial dimension; This is the standard deviation parameter of the Gaussian kernel function, i.e., the kernel function standard deviation, which is used to control the response width of the kernel function.

[0036] As a further optimization of the electronic cabin assembly quality inspection method based on three-dimensional vision of the present invention, the correlation filter response formulas in steps 3.1) and 4.1) are as follows:

[0037] ;

[0038] In the formula, This is the initial two-dimensional response diagram. For the pre-computed and stored frequency domain representation of the filter, Frequency domain representation of nuclear correlation results.

[0039] As a further optimization of the electronic cabin assembly quality inspection method based on three-dimensional vision of the present invention, the exponential decay function formula mentioned in steps 3.2) and 4.2) is as follows:

[0040] ;

[0041] This is the depth weighting coefficient; This is a preset depth influence factor used to control the intensity of the influence of depth differences on the weight decay rate. This represents the average depth deviation.

[0042] As a further optimization of the electronic cabin assembly quality inspection method based on three-dimensional vision of the present invention, the detailed steps of preprocessing the cable mask in step 6.1) are as follows:

[0043] Using the OpenCV library, morphological closing operations are used to remove holes inside the cable mask and smooth the edges of the cable mask area. Then, the connectedComponentsWithStats function is used to remove target areas within the cable mask whose area is smaller than a preset fifth threshold to eliminate noise.

[0044] As a further optimization of the electronic cabin assembly quality inspection method based on three-dimensional vision of the present invention, the specific steps for optimizing the cable skeleton in step 6.1) are as follows:

[0045] Step 6.1.1): For each pixel on the cable skeleton, if there is a pixel in the lower right and no pixel in the upper right, or a pixel in the upper right and no pixel in the lower right, or a pixel in the lower left and no pixel in the upper left, or a pixel in the upper left and no pixel in the lower left, then the pixel is determined to be a redundant pixel and removed from the cable skeleton.

[0046] Step 6.1.2) Create a 3x3 convolution kernel, where the weight of the center point is much greater than that of the surrounding points. Convolve the kernel with the cable skeleton after removing redundant pixels to obtain the convolution value of each pixel in the cable skeleton after removing redundant pixels. Compare each convolution value with the preset sixth threshold. Set the values ​​of the pixels in the 3x3 area around the pixel whose convolution value in the cable skeleton after removing redundant pixels is greater than the preset sixth threshold to 0, and disconnect the intersection of the cable skeleton.

[0047] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0048] This invention significantly enhances the ability to identify depth-sensitive defects and reduces the false negative rate: The proposed 3D enhanced response strategy uses depth information to weight and adjust the response results of 2D correlation filtering, enabling the system to accurately identify defects such as "improper installation" of screws and plugs, which are deviations in the depth direction. This method effectively compensates for the shortcomings of existing technologies in being insensitive to such defects, and significantly reduces the false negative rate of critical defects. Furthermore, this invention uses a depth camera to acquire depth data, performs 3D reconstruction and spline curve fitting on flexible cables, and achieves direct 3D measurement and calculation of their bending radius. This method elevates detection from qualitative "identification" in a 2D plane to quantitative "measurement" in 3D space, fundamentally solving the problem that pure 2D vision cannot accurately assess the quality of 3D geometric shapes, and significantly improving the reliability of detection.

[0049] A systematic automated inspection framework has been constructed, improving inspection efficiency and coverage: This invention organically integrates technologies such as correlation filtering and optical character recognition to form an automated inspection process capable of solving various common assembly defects in three core component categories: parts, circuit board connectors and plugs, and cables. This solution requires no manual intervention, is highly comprehensive, improves inspection efficiency, shortens production cycles, and reduces reliance on highly skilled assembly workers. Compared to technical solutions that can only solve single problems, it has higher engineering application value. Attached Figure Description

[0050] Figure 1 This is an overall flowchart of the present invention;

[0051] Figure 2 This is a detailed flowchart of the detection process for missing or improperly installed objects in this invention.

[0052] Figure 3 This is a detailed flowchart of the connector and cable plug mismatch detection in this invention;

[0053] Figure 4 This is a detailed flowchart of the cable bending radius detection process in this invention. Detailed Implementation

[0054] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings:

[0055] This invention can be implemented in many different forms and should not be considered limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully express the scope of the invention to those skilled in the art. In the drawings, components are enlarged for clarity.

[0056] This invention provides a method for inspecting the assembly quality of an electronic compartment based on three-dimensional vision. Please refer to [link / reference]. Figure 1It demonstrates the overall process of the present invention, which specifically includes the following steps:

[0057] Step 1) Generates data for comparison during testing, targeting the standard electronic module. This step is the preparation phase of the testing process, aiming to establish an accurate and quantifiable benchmark for subsequent electronic modules to be tested.

[0058] Step 1.1): For a standard electronic cabin that has been confirmed to be fully and correctly assembled, plan and set the camera positions and shooting angles for all objects to be inspected inside, and record them. The objects to be inspected include three categories: screws, wiring units, and cables. By presetting fixed camera poses, it can be ensured that the template data and the inspection data are compared in the same coordinate system, eliminating errors caused by changes in viewing angle.

[0059] Step 1.2): For each object to be inspected within the standard electronic cabin, an industrial camera is used at its corresponding preset camera position and adjusted to a preset shooting angle for image capture. The camera acquires depth and color images of each object to be inspected, which are then stored as template depth and template color images for that object, respectively.

[0060] Step 1.3): Perform template data processing for each screw to be inspected.

[0061] Step 1.3.1): Assign a unique number to each screw to be inspected and record its number and its corresponding template depth image and template color image. In the template color image of the screw to be inspected, determine the bounding rectangle of the pixel area occupied by the screw to be inspected by manual annotation. Record the pixel coordinates of this rectangle in the template color image and define it as the region of interest (ROI) of the screw to be inspected.

[0062] Step 1.3.2) extracts the Histogram of Oriented Gradients (HOG) features from the Region of Interest (ROI) of the screw to be detected within its template color image. HOG features effectively describe the edge and shape information of an object and have good robustness to illumination changes. To reduce spectral leakage and boundary effects, the extracted HOG features are weighted using a Hanning window to obtain the template features of the screw to be detected. Subsequently, a discriminative correlation filter is trained using these template features to obtain a correlation filter specifically for locating the screw to be detected. This filter is stored and associated with the screw number.

[0063] Step 1.4): Perform template data processing for each wiring unit to be tested.

[0064] The wiring unit includes a fixed connector and a mating cable plug. In a standard electronics compartment, each wiring unit's connector and its corresponding cable plug have the same label, but the labels on the connectors of each wiring unit are different for unique identification.

[0065] Step 1.4.1): Record the connector number, cable plug number, and corresponding template depth image and template color image of the wiring unit. Similar to screw processing, in the template color image of the wiring unit, determine the outer rectangle of the pixel area occupied by the cable plug, and record the pixel coordinates of the rectangle as the ROI region of the cable plug of the wiring unit.

[0066] Step 1.4.2): Extract HOG features from the ROI region of the cable plug of the wiring unit in its template color image, and apply a Hanning window to the HOG features to obtain the template features of the cable plug of the wiring unit to be detected. Use these template features to train a discriminative correlation filter to obtain the correlation filter for the cable plug of the wiring unit to be detected.

[0067] Step 2) Acquire images of the electronic cabin to be tested.

[0068] For the electronic compartment to be inspected, use an industrial camera to photograph all objects inside it, strictly following the camera positions and shooting angles recorded in step 1.1). After shooting, obtain a depth image and a color image for each object to be inspected, which will be used as its depth image and color image to be inspected, respectively.

[0069] Step 3) Check if any screws in the electronics compartment to be tested are missing or improperly installed. Please refer to [link / reference]. Figure 2 This is a detailed flowchart for detecting missing or improperly installed components. This step is performed on each screw in the electronic compartment to be tested, as follows:

[0070] Step 3.1): In the color image of the screw to be detected, the HOG features of its ROI region are extracted, and a Hanning window is applied to the HOG features to obtain the sample features of the screw. Substituting the template features of the screw to be detected obtained in Step 1.3.2) and the currently acquired sample features into the Gaussian kernel correlation formula, the kernel correlation matrix of the screw in the spatial domain is calculated. The Gaussian kernel correlation formula is as follows:

[0071] ;

[0072] In the formula, The kernel correlation matrix represents the spatial domain; Indicates template features; Indicates sample characteristics; Indicates the inverse Fourier transform; The complex conjugate of the frequency domain representation of the i-th channel of the template feature; Indicates the sample feature number Frequency domain representation of each channel; This indicates element-wise multiplication; The total number of elements of the template feature in the spatial dimension; This is the standard deviation parameter of the Gaussian kernel function, i.e., the kernel function standard deviation, which is used to control the response width of the kernel function.

[0073] Subsequently, the calculated kernel correlation matrix and the correlation filter of the screw to be detected trained in step 1.3.2) are substituted into the correlation filter response formula to calculate the initial two-dimensional response map of the screw to be detected. The correlation filter response formula is as follows:

[0074] ;

[0075] In the formula, This is the initial two-dimensional response diagram. For the pre-computed and stored frequency domain representation of the filter, Frequency domain representation of nuclear correlation results.

[0076] Step 3.2) To distinguish between depth-sensitive defects such as "correct installation" and "incorrect installation," depth information is introduced for weighting. The depth deviation of each corresponding pixel within the ROI region of the screw to be inspected in both the template depth image and the inspection depth image is calculated. The average depth deviation of all valid pixels is obtained by averaging the depth deviations. This average depth deviation is then substituted into an exponential decay function to calculate the depth weighting coefficient of the screw to be inspected. The formula for the exponential decay function is as follows:

[0077] ;

[0078] This is the depth weighting coefficient; This is a preset depth influence factor used to control the intensity of the influence of depth differences on the weight decay rate. This represents the average depth deviation. This function ensures that the larger the depth deviation, the smaller the weighting coefficient, and the stronger the suppression effect on the response map.

[0079] The initial two-dimensional response map obtained in step 3.1) is multiplied by the depth weighting coefficient to obtain a depth-weighted three-dimensional enhanced response map of the screw to be detected. Finally, the response peak of the three-dimensional enhanced response map is judged. A first threshold and a second threshold are preset, with the first threshold being greater than the second threshold.

[0080] If the peak value of the response map of the three-dimensional enhanced response map of the screw to be tested is greater than or equal to the first threshold, it indicates that the target matches the template height in both appearance and depth, and the screw to be tested is determined to be correctly installed.

[0081] If the peak value of the response map of the three-dimensional enhanced response map of the screw to be tested is less than the first threshold and greater than or equal to the second threshold, it indicates that the target matches in appearance, but there is a significant depth deviation, and it is determined that the screw to be tested is not installed in place.

[0082] If the peak value of the response map of the three-dimensional enhanced response map of the screw to be tested is less than the second threshold, it indicates that the target is seriously deviated from the template in appearance or depth, and the screw to be tested is determined to be not installed (missing).

[0083] Step 4) Check if the cable plugs are missing or improperly installed. This step is similar to Step 3), and the cable plugs of each wiring unit to be tested are checked:

[0084] Step 4.1): In the color image of the cable plug to be detected, extract the HOG features of its ROI region and apply a Hanning window to obtain the sample features of the cable plug. Substitute the template features and sample features of the cable plug into the Gaussian kernel correlation formula to calculate the kernel correlation matrix of the cable plug in the spatial domain. Then, substitute this kernel correlation matrix and the correlation filter of the cable plug obtained in step 1.4.2) into the correlation filter response formula to calculate the initial two-dimensional response map.

[0085] Step 4.2) Calculate the average depth deviation of the cable plug of the wiring unit to be detected within the ROI region of its template depth image and the depth image to be detected, and substitute it into the exponential decay function to calculate the depth weighting coefficient of the cable plug. Multiply the initial two-dimensional response map by this depth weighting coefficient to obtain the depth-weighted three-dimensional enhanced response map of the cable plug. Compare the response peak of this three-dimensional enhanced response map with preset third and fourth thresholds, where the third threshold is greater than the fourth threshold.

[0086] If the peak value of the three-dimensional augmented response map of the cable plug is greater than or equal to the third threshold, the cable plug is determined to be installed correctly.

[0087] If the peak value of the three-dimensional augmented response map of the cable plug is less than the third threshold but greater than or equal to the fourth threshold, the cable plug is determined to be not installed properly.

[0088] If the peak value of the response of the three-dimensional augmented response map of the cable plug is less than the fourth threshold, it is determined that the cable plug is not installed (missing).

[0089] Step 5) Check if the connectors and cable plugs in the wiring unit are mismatched. Please refer to [link / reference]. Figure 3This is a detailed flowchart for connector and cable plug mismatch detection. This step is used to verify that the cable plug is inserted into the correct connector. For each wiring unit under test in the electronics compartment:

[0090] Step 5.1) invokes the pre-trained PP-OCRv2 model to process the color image of the wiring unit to be detected. The model automatically detects and recognizes the text regions in the image, and outputs the label text on the cable plug and the label text on the connector.

[0091] Step 5.2) Based on the principle of spatial proximity, the label text of the identified cable plug is paired and compared with the label text of the nearest physically located connector. In practice, this can be achieved by calculating the Euclidean distance between the center points of each label text area. If the two paired label text strings are exactly the same (e.g., both are "CN1"), the connector and cable plug are considered a correct match. If they are different, the connector and cable plug are considered an incorrect match.

[0092] Step 6), check if the cable bending radius is too small. Please refer to [link / reference needed]. Figure 4 This is a detailed flowchart of the cable bending radius detection process described in this invention. This step is used to evaluate whether the wiring process of the flexible cable meets the requirements and to prevent cable damage due to excessive bending. For each cable to be tested in the electronic compartment:

[0093] Step 6.1) Reconstruct the 3D shape of the cable. First, use a depth threshold-based segmentation technique to process the depth image of the cable to be detected, segmenting the pixels located within a specific depth range (i.e., the area where the cable is located) to generate a binarized cable mask.

[0094] The cable mask is preprocessed. The specific steps are as follows: Using image processing libraries such as OpenCV, morphological closing operations are first used to fill any small holes that may exist inside the cable mask, and to smooth the edges of the cable mask's regions. Next, the `connectedComponentsWithStats` function is used to calculate the area of ​​all connected components, and target regions within the cable mask with areas smaller than a preset fifth threshold are removed to effectively eliminate isolated small noise points caused by sensor noise or background interference.

[0095] After preprocessing, a single-pixel-width cable skeleton is extracted from the clean cable mask using a thinning algorithm. To improve skeleton quality, the cable skeleton is optimized. The specific steps are as follows:

[0096] Step 6.1.1): First, remove redundant pixels. Traverse each pixel on the cable skeleton and check its 8-neighborhood. If the pixel satisfies one of the following four local neighborhood patterns: there is a pixel in the lower right and no pixel in the upper right to the right, or there is a pixel in the upper right and no pixel in the lower right to the right, or there is a pixel in the lower left and no pixel in the upper left to the left, or there is a pixel in the upper left and no pixel in the lower left to the left, then the pixel is determined to be a redundant pixel and is removed from the cable skeleton (value set to 0).

[0097] (Step 6.1.2) Next, disconnect the intersections of the cable skeleton and create a 3x3 convolution kernel, where the weight of the center point is much greater than the weight of the surrounding points. Perform a convolution operation between this kernel and the cable skeleton after removing redundant pixels. The points with larger values ​​in the convolution result image are the intersections. Compare the convolution value of each pixel in the cable skeleton after removing redundant pixels with a preset sixth threshold. For pixels with a convolution value greater than the sixth threshold, they are identified as intersections, and the values ​​of all pixels in the surrounding 3x3 area are set to 0, thereby effectively disconnecting the intersections of the cable skeleton and ensuring the independence of each cable skeleton.

[0098] Step 6.2) Calculate the radius of curvature based on the 3D morphological reconstruction of the cable. First, perform 2D point sampling on the optimized cable skeleton, sampling one point every certain pixel distance (e.g., 3 pixels) along the skeleton contour line. Based on the pixel coordinates (u, v) of the sampling point, find the corresponding depth value Z in the depth image to be detected. Combining the intrinsic parameter matrix of the industrial camera, calculate the X and Y coordinates of the sampling point in the camera coordinate system through back projection, obtaining a series of sampling points containing 3D information. Based on these 3D sampling points, perform smooth 3D morphological reconstruction of the flexible cable by fitting a space curve (e.g., a B-spline curve). Finally, calculate the radius of curvature at each sampling point according to the formula for calculating the curvature of the space curve. Compare the radius of curvature at the sampling point with a preset radius of curvature threshold. If it is less than the preset radius of curvature threshold, it is determined that the cable has a defect of too small a bending radius at that sampling point.

[0099] In summary, this invention, through a systematic and multi-step detection process, deeply integrates three-dimensional depth information with various two-dimensional vision algorithms such as correlation filtering, optical character recognition, image segmentation, and morphological analysis. This overcomes the shortcomings of existing two-dimensional vision detection technologies in being insensitive to assembly defects related to three-dimensional space, and realizes automated detection of various assembly defects of screws, wiring units, and cables in the electronic compartment, thereby improving the efficiency and reliability of assembly quality control.

[0100] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0101] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for inspecting the assembly quality of an electronic cabin based on three-dimensional vision, wherein a wiring unit within the electronic cabin includes a connector and a corresponding cable plug, wherein the connector and its corresponding cable plug in each wiring unit are provided with the same label, and the labels on the connectors in each wiring unit are different, characterized in that, Includes the following steps: Step 1), for the standard electronic chamber, generate data for comparison during testing; Step 1.1) Set and record the corresponding camera positions and shooting angles for all objects to be tested in the standard electronic cabin; the objects to be tested include screws, wiring units, and cables to be tested; Step 1.2): For each object to be inspected in the standard electronic cabin, an industrial camera is used to take pictures according to its corresponding camera position and shooting angle to obtain the depth image and color image of each object to be inspected, which are used as the template depth image and template color image of the object to be inspected, respectively. Step 1.3), for each screw to be inspected: Step 1.3.1), record its number and its corresponding template depth image and template color image; record the bounding rectangle of the pixels occupied by the screw to be detected in the template color image of the screw to be detected, and use it as the ROI region of the screw to be detected; Step 1.3.2) Extract HOG features from the ROI region of the screw to be detected in its template color image and apply a Hanning window to the HOG features to obtain the template features of the screw to be detected. Use the template features of the screw to be detected to train a discriminative correlation filter to obtain the correlation filter of the screw to be detected. Step 1.4), for each wiring unit to be tested: Step 1.4.1), record its connector number, its cable plug number, and the corresponding template depth image and template color image; record the pixel coordinates of the bounding rectangle of the occupied pixels of the cable plug in the template color image of the wiring unit as the ROI area of ​​the cable plug of the wiring unit. Step 1.4.2) Extract HOG features from the ROI region of the cable plug of the wiring unit in its template color image and apply Hanning window to the HOG features to obtain the template features of the cable plug of the wiring unit to be detected. Use the template features of the cable plug to train a discriminative correlation filter to obtain the correlation filter of the cable plug of the wiring unit to be detected. Step 2), for the electronic cabin to be inspected, according to the recorded camera positions and shooting angles of the objects to be inspected, use an industrial camera to take pictures of all the objects to be inspected inside, and obtain the depth image and color image of each object to be inspected, as its depth image and color image to be inspected. Step 3) Check whether the screws of the electronic compartment to be tested are missing or improperly installed. For each screw of the electronic compartment to be tested: Step 3.1) Extract the HOG features of the ROI region of the screw to be detected in the color image to be detected and apply the Hanning window to the HOG features to obtain the sample features of the screw. Substitute the template features and sample features of the screw to be detected into the Gaussian kernel correlation formula to calculate the kernel correlation matrix of the spatial domain of the screw to be detected. Substitute the kernel correlation matrix and the correlation filter of the screw to be detected into the correlation filter response formula to calculate the initial two-dimensional response map of the screw to be detected. Step 3.2): Calculate the depth deviation of each pixel in the ROI region of the screw to be detected in the template depth image and the depth image to be detected, and then calculate the average depth deviation. Substitute the average depth deviation into the exponential decay function to calculate the depth weight coefficient of the screw to be detected. Multiply the initial two-dimensional response map of the screw to be detected by the depth weight coefficient of the screw to be detected to obtain the depth information weighted three-dimensional enhanced response map of the screw to be detected. Compare the response peak of the three-dimensional enhanced response map of the screw to be detected with a preset first threshold and a preset second threshold, where the first threshold is greater than the second threshold. If the response peak of the three-dimensional enhanced response map of the screw to be detected is greater than or equal to the first threshold, the screw to be detected is determined to be correctly installed; if the response peak of the three-dimensional enhanced response map of the screw to be detected is less than the first threshold but greater than or equal to the second threshold, the screw to be detected is determined to be not installed properly; if the response peak of the three-dimensional enhanced response map of the screw to be detected is less than the second threshold, the screw to be detected is determined to be not installed. Step 4) Check whether the cable plugs are missing or improperly installed. For each wiring unit to be tested in the electronic compartment: Step 4.1) Extract the HOG features of the ROI region of the cable plug in the color image of the wiring unit to be tested, and apply the Hanning window to the HOG features to obtain the sample features of the cable plug. Substitute the template features and sample features of the cable plug into the Gaussian kernel correlation formula to calculate the kernel correlation matrix of the cable plug in the spatial domain. Substitute the kernel correlation matrix and the correlation filter of the cable plug of the wiring unit to be tested into the correlation filter response formula to calculate the initial two-dimensional response map of the cable plug of the wiring unit to be tested. Step 4.2) Calculate the depth deviation of each pixel in the ROI region of the cable plug of the wiring unit to be tested in its template depth image and the depth image to be tested, and then calculate the average depth deviation. Substitute it into the exponential decay function to calculate the depth weight coefficient of the cable plug. Multiply the initial two-dimensional response map of the cable plug of the wiring unit to be tested by the depth weight coefficient of the cable plug to obtain the depth information weighted three-dimensional enhanced response map of the cable plug. Compare the response peak of the three-dimensional enhanced response map of the cable plug with the preset third threshold and the preset fourth threshold, where the third threshold is greater than the fourth threshold. If the response peak of the three-dimensional enhanced response map of the cable plug is greater than or equal to the third threshold, the cable plug is determined to be correctly installed; if the response peak of the three-dimensional enhanced response map of the cable plug is less than the third threshold but greater than or equal to the fourth threshold, the cable plug is determined to be not installed properly; if the response peak of the three-dimensional enhanced response map of the cable plug is less than or equal to the fourth threshold, the cable plug is determined to be not installed. Step 5), check whether the connectors and cable plugs in the wiring unit are mismatched. For each wiring unit to be tested in the electronics compartment: Step 5.1) Use the PP-OCRv2 model to identify the label text of the cable plug and the label text of the connector in the color image of the wiring unit to be inspected; Step 5.2) Based on the principle of spatial proximity, compare the label text of the cable plug with the label text of the nearest connector. If they are the same, the connector and cable plug are matched correctly; if they are different, the connector and cable plug are not matched correctly. Step 6), check if the cable bending radius is too small. For each cable to be tested in the electronic compartment: Step 6.1) Use depth threshold-based segmentation technology to segment the cable region of the cable to be detected in the depth image to obtain the cable mask, preprocess the cable mask, and then extract the cable skeleton from the preprocessed cable mask through a thinning algorithm. Optimize the cable skeleton, remove redundant pixels on the cable skeleton and disconnect the intersections of the cable skeleton. Step 6.2) Two-dimensional point sampling is performed on the cable skeleton. The depth Z of the sampling point is obtained from the pixel coordinates of the sampling point in the depth image to be detected. Combined with the intrinsic parameters of the industrial camera, the X and Y coordinates of the sampling point in the camera coordinate system are calculated by back projection to obtain the three-dimensional information of the sampling point. Based on the sampling points containing three-dimensional information, the three-dimensional shape of the flexible cable is reconstructed by fitting a spatial curve. The radius of curvature at each sampling point is calculated and the minimum value is compared with the preset radius of curvature threshold. If the minimum value is less than the preset radius of curvature threshold, it is determined that the bending radius of the cable at that sampling point is too small.

2. The electronic cabin assembly quality inspection method based on three-dimensional vision according to claim 1, characterized in that, The Gaussian kernel correlation formulas mentioned in steps 3.1) and 4.1) are as follows: ； In the formula, The kernel correlation matrix represents the spatial domain. Indicates template features, Indicates sample features, Indicates the inverse Fourier transform. Representative template feature number The complex conjugate of each channel in the frequency domain. Indicates the sample feature number Frequency domain representation of each channel, This indicates element-wise multiplication. The total number of elements of the template feature in the spatial dimension; The standard deviation parameter of the Gaussian kernel function, i.e., the kernel function standard deviation, is used to control the response width of the kernel function.

3. The electronic cabin assembly quality inspection method based on three-dimensional vision according to claim 2, characterized in that, The correlation filter response formulas described in steps 3.1) and 4.1) are as follows: ； In the formula, This is the initial two-dimensional response diagram. For the pre-computed and stored frequency domain representation of the filter, Frequency domain representation of nuclear correlation results.

4. The electronic cabin assembly quality inspection method based on three-dimensional vision according to claim 1, characterized in that, The formulas for the exponential decay function mentioned in steps 3.2 and 4.2 are as follows: ； This refers to the depth weighting coefficient; This is a preset depth influence factor used to control the intensity of the influence of depth differences on the weight decay rate. This represents the average depth deviation.

5. The electronic cabin assembly quality inspection method based on three-dimensional vision according to claim 1, characterized in that, The detailed steps for preprocessing the cable mask in step 6.1) are as follows: Using the OpenCV library, morphological closing operations are used to remove holes inside the cable mask and smooth the edges of the cable mask area. Then, the connectedComponentsWithStats function is used to remove target areas within the cable mask whose area is smaller than a preset fifth threshold to eliminate noise.

6. The electronic cabin assembly quality inspection method based on three-dimensional vision according to claim 1, characterized in that, The specific steps for optimizing the cable skeleton in step 6.1) are as follows: Step 6.1.1): For each pixel on the cable skeleton, if there is a pixel in the lower right and no pixel in the upper right, or a pixel in the upper right and no pixel in the lower right, or a pixel in the lower left and no pixel in the upper left, or a pixel in the upper left and no pixel in the lower left, then the pixel is determined to be a redundant pixel and removed from the cable skeleton. Step 6.1.2) Create a 3x3 convolution kernel, where the weight of the center point is much greater than that of the surrounding points. Convolve the kernel with the cable skeleton after removing redundant pixels to obtain the convolution value of each pixel in the cable skeleton after removing redundant pixels. Compare each convolution value with the preset sixth threshold. Set the values ​​of the pixels in the 3x3 area around the pixel whose convolution value in the cable skeleton after removing redundant pixels is greater than the preset sixth threshold to 0, and disconnect the intersection of the cable skeleton.

Images