A method for assembling quality of parts in confined spaces based on three-dimensional enhanced correlation filtering

Abstract

This invention discloses a method for inspecting the assembly quality of parts in confined spaces based on 3D enhanced correlation filtering, relating to the field of machine vision. The method includes: acquiring template images of correctly assembled parts and marking the locations of the parts as regions of interest; extracting region features and training a correlation filter; obtaining the overall correlation filter response using features from the tested image; performing multi-feature adaptive fusion based on preset evaluation metrics; correcting the fused correlation filter response using a depth-sensing 3D enhancement method; determining whether a part is occluded based on the fused correlation filter response; dividing the identified occluded region into sub-regions, detecting each sub-region to obtain the fused correlation filter response, and weighting and correcting it; determining the unoccluded areas of the part based on the enhancement results; and determining whether the part is incorrectly installed based on the 3D enhanced correlation filter response. This invention effectively solves the problems of incorrect, missing, and incomplete installation of parts in confined spaces with occluded targets.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically to a method for inspecting the assembly quality of parts in confined spaces based on three-dimensional enhanced correlation filtering. Background Technology

[0002] In assembly tasks within confined spaces, traditional manual inspection methods suffer from limitations such as low accuracy, poor efficiency, and difficulty in covering all assembly details. This is especially true in complex assembly environments where various parts often obscure each other. Existing inspection methods are mainly divided into traditional image processing methods and deep learning image processing methods. While traditional image processing methods are highly adaptable and can complete tasks with simple configurations, they often employ complex feature extraction or modality fusion techniques, resulting in slow computation speeds and long detection times. This makes them unsuitable for the real-time requirements of assembly quality inspection and unable to identify and process occlusions within a single frame. Deep learning image processing methods, on the other hand, rely on large amounts of data for training, which is difficult to apply in assembly quality inspection tasks where datasets are scarce. Therefore, how to efficiently combine image features and depth information for occlusion analysis and processing to accurately detect assembly errors (such as misassembly, omissions, and improper installation) remains a major technological challenge. Summary of the Invention

[0003] 1. The technical problem to be solved:

[0004] To address the aforementioned technical problems, this invention provides a method for quality inspection of parts assembly in confined spaces based on three-dimensional enhanced correlation filtering. This solution uses an improved correlation filtering algorithm to provide an efficient and reliable solution for automated quality inspection of assembly in confined spaces, enabling the detection of mis-assembly, omission, and improper installation of parts, thereby improving product quality reliability and production efficiency.

[0005] 2. Technical Solution:

[0006] A method for inspecting the assembly quality of parts in confined spaces based on three-dimensional enhanced correlation filtering, characterized by comprising:

[0007] Step 1: Use a binocular vision device to acquire a template image containing correctly assembled parts; the template image includes a two-dimensional image and a depth image; in the template image, mark the location of the parts as the region of interest;

[0008] Step 2: Preset multiple feature types and assign a correlation filter to each feature type for each region of interest in the template image; use the corresponding feature extraction algorithm to extract the features of the corresponding type from the template image, and use the extracted features to train the corresponding correlation filter to obtain the trained correlation filter; define the trained correlation filter as the global correlation filter; input the binocular vision test image to be detected into its corresponding global correlation filter to obtain multiple initial global correlation filter responses for each region of interest;

[0009] Step 3: Preset multiple evaluation indicators for a single region of interest, and calculate multiple evaluation indicators for each initial response of each region of interest based on the initial overall correlation filter response; perform indicator fusion based on multiple evaluation indicators, and use a multi-feature adaptive fusion mechanism to fuse the initial correlation filter responses to obtain the fused correlation filter response of each region of interest;

[0010] Step 4: For each region of interest, the fusion correlation filter response is corrected using a depth-sensing 3D enhancement method to obtain a 3D enhanced correlation filter response. The depth-sensing 3D enhancement involves: calculating the depth difference between the depth map of the tested sample and the template depth map to obtain a depth difference map; obtaining a depth mask map based on the depth difference map; and using the depth mask map to perform weighted correction on the fusion correlation filter response.

[0011] Step 5: Preset the threshold range of the peak value of the filter response, and determine whether the part is blocked based on the fused correlation filter response. If it is blocked, proceed to step 6; otherwise, proceed to step 7.

[0012] Step 6: Divide the region of interest identified as occluded into multiple sub-regions, and assign multiple local correlation filters to each sub-region. Based on the local correlation filters trained in Steps 2-5, the detected fused correlation filter responses, and the enhancement results obtained after weighted correction, the component with the highest response peak value among the enhancement results is selected as the final response of the component.

[0013] Step 7: Preset the detection threshold for parts in the region of interest. If the peak value of the 3D enhanced correlation filter response of the part is less than the detection threshold, the part is classified as an installation error. The installation error includes incorrect installation, missing installation, or improper installation. Conversely, the part is classified as a correct installation.

[0014] Further, in step two, the feature extraction includes histogram of oriented gradients (HOR) features, grayscale features, and depth features generated by a convolutional neural network. Correspondingly, in this step, histogram of oriented gradients (HOR) features, grayscale features, and depth features generated by a convolutional neural network are extracted for each region of interest in the entire assembly space. The extracted features are used to train the corresponding global correlation filter. The corresponding feature extraction algorithm includes a histogram of oriented gradients extraction algorithm, grayscale values ​​of image pixels, and a convolutional neural network.

[0015] Further, in step three, the multiple evaluation metrics are peak response Rmax, peak-to-sidelobe ratio (PSR), average peak correlation energy (APCE), and response entropy; these multiple evaluation metrics are calculated for each individual region of interest; the multi-feature adaptive fusion mechanism calculates the adaptive fusion parameter S corresponding to each feature using the following formula. mix :

[0016] ;

[0017] In the above formula, R max S represents the peak value of the filtered response corresponding to the feature; PSR S represents the peak-to-sidelobe ratio of the corresponding feature; APCE S represents the average peak correlation energy of the corresponding feature; Entropy This represents the response entropy of the corresponding feature; k1, k2, k3, and k4 are the preset weight parameters for each evaluation index of this feature, used to balance the role of each index in the fusion evaluation, specifically:

[0018] ;

[0019] ;

[0020] ;

[0021] In the above formula, R max R represents the maximum value of the main response peak corresponding to the feature. min R represents the minimum response corresponding to the feature. w,h The response value at coordinates (w, h) in the response plot corresponding to the characteristic; μ sl It is the average response value of the sidelobe region of the characteristic-response map; σ sl is the standard deviation of the sidelobe region of the feature response map; W and H represent the width and height of the feature response map, respectively; p i The normalized probability at each position in the response graph represents the corresponding feature.

[0022] S for each feature mixAs fusion weights, the filtered responses of the three features of the region of interest are weighted and fused to generate the final fused correlation filtered response of the region.

[0023] Furthermore, step four specifically includes:

[0024] S41: Calculate the depth difference between the corresponding pixels in the region of interest of the depth map of the sample to be tested and the template depth map to obtain the depth difference map;

[0025] S42: Calculate the negative power weight of each pixel in the depth difference map; construct the depth mask map based on the negative power weight;

[0026] S43: Weight the fused correlation filter response R using the depth mask image, as shown in the following formula, which applies exponential weights to each coordinate position of the fused filter response:

[0027] ;

[0028] In the above formula, R D The weighted fused filter response for the corresponding region of interest is defined as the 3D enhanced correlation filter response; n is the total number of pixels in the corresponding region; d i β represents the depth difference at each pixel location in the depth difference map; β represents the sensitivity of the preset depth mask to the depth difference.

[0029] Furthermore, in step five, determining whether the target part is occluded based on the fusion correlation filter response is as follows: the low threshold of the fusion correlation filter response peak value is preset to T1, and the high threshold is T2; if the peak value of the fusion correlation filter response of the region of interest is outside the two thresholds T1 and T2, then the corresponding part is classified as unoccluded; otherwise, the part corresponding to the region is classified as occluded.

[0030] Furthermore, step six specifically includes:

[0031] S61: Divide the region of interest that is determined to be occluded into an n×n grid, where n is an integer greater than or equal to 1; each grid is defined as a sub-region of the region of interest; multiple local correlation filters corresponding to the feature types are deployed on each sub-region;

[0032] S62: Obtain the template image from step one, extract the regions of interest that are determined to be occluded, and divide them into corresponding sub-regions as training samples for the sub-regions; use the training samples of each sub-region to train based on step two and obtain the local correlation filter response of the image to be detected;

[0033] S63: Based on steps three and four, obtain the fusion correlation filter response and the three-dimensional enhanced correlation filter response of the sub-region; take n... 2The response with the highest peak value among the local three-dimensional enhanced correlation filter responses is taken as the final response.

[0034] 3. Beneficial effects:

[0035] (1) The method for detecting the assembly quality of parts in a narrow space based on three-dimensional enhanced correlation filtering provided by the present invention uses a depth-sensing three-dimensional enhancement method to generate a depth mask by the difference between the template and the sample depth map, and then realizes the three-dimensional space perception of the correlation filter through the weighted correlation filter response.

[0036] (2) The method for detecting the assembly quality of parts in a narrow space based on three-dimensional enhanced correlation filtering provided by the present invention adopts an adaptive multi-feature fusion strategy, which integrates the response map of multiple features based on the response map quantification index, thereby improving the robustness of the feature representation of the correlation filter and enabling it to adapt to a variety of parts in a complex narrow space.

[0037] (3) The assembly quality inspection method for parts in confined spaces based on three-dimensional enhanced correlation filtering provided by this invention designs an anti-occlusion strategy based on region division to address the common occlusion problem in confined spaces. This method divides the area that may be occluded into multiple local regions, and trains and applies local correlation filters independently in each sub-region. By effectively utilizing the features of the unoccluded sub-regions, these local filters can effectively compensate for the performance degradation of the overall filter under occlusion conditions. Attached Figure Description

[0038] Figure 1 This is an overall flowchart of the present invention;

[0039] Figure 2 This is a schematic diagram of the correlation filter training and detection in this invention;

[0040] Figure 3 This is a schematic diagram of the multi-feature adaptive fusion mechanism in this invention;

[0041] Figure 4 This is a schematic diagram of the depth-sensing 3D enhancement method in this invention;

[0042] Figure 5 This is a flowchart illustrating the process of region search for anti-occlusion of occluded parts in this invention.

[0043] Figure 6 To verify the comparison of detection accuracy of this method with 12 existing algorithms at different thresholds in Example 1;

[0044] Figure 7 To verify the response graph generated during the detection of a missing part in Example 2;

[0045] Figure 8To verify the actual testing effect deployed on the experimental assembly workbench in Example 2. Detailed Implementation

[0046] The present invention will now be described in detail with reference to the accompanying drawings.

[0047] As attached Figures 1 to 5 As shown, the method for inspecting the assembly quality of parts in confined spaces based on three-dimensional enhanced correlation filtering is characterized by including:

[0048] Step 1: Use a binocular vision device to acquire a template image containing correctly assembled parts; the template image includes a two-dimensional image and a depth image; in the template image, mark the location of the parts as the region of interest;

[0049] Step 2: Preset multiple feature types and assign a correlation filter to each feature type for each region of interest in the template image; use the corresponding feature extraction algorithm to extract the features of the corresponding type from the template image, and use the extracted features to train the corresponding correlation filter to obtain the trained correlation filter; define the trained correlation filter as the global correlation filter; input the binocular vision test image to be detected into its corresponding global correlation filter to obtain multiple initial global correlation filter responses for each region of interest;

[0050] Step 3: Preset multiple evaluation indicators for a single region of interest, and calculate multiple evaluation indicators for each initial response of each region of interest based on the initial overall correlation filter response; perform indicator fusion based on multiple evaluation indicators, and use a multi-feature adaptive fusion mechanism to fuse the initial correlation filter responses to obtain the fused correlation filter response of each region of interest;

[0051] Step 4: For each region of interest, the fusion correlation filter response is corrected using a depth-sensing 3D enhancement method to obtain a 3D enhanced correlation filter response. The depth-sensing 3D enhancement involves: calculating the depth difference between the depth map of the tested sample and the template depth map to obtain a depth difference map; obtaining a depth mask map based on the depth difference map; and using the depth mask map to perform weighted correction on the fusion correlation filter response.

[0052] Step 5: Preset the threshold range of the peak value of the filter response, and determine whether the part is blocked based on the fused correlation filter response. If it is blocked, proceed to step 6; otherwise, proceed to step 7.

[0053] Step 6: Divide the region of interest identified as occluded into multiple sub-regions, and assign multiple local correlation filters to each sub-region. Based on the local correlation filters trained in Steps 2-5, the detected fused correlation filter responses, and the enhancement results obtained after weighted correction, the component with the highest response peak value among the enhancement results is selected as the final response of the component.

[0054] Step 7: Preset the detection threshold for parts in the region of interest. If the peak value of the 3D enhanced correlation filter response of the part is less than the detection threshold, the part is classified as an installation error. The installation error includes incorrect installation, missing installation, or improper installation. Conversely, the part is classified as a correct installation.

[0055] Further, in step two, the feature extraction includes histogram of oriented gradients (HOR) features, grayscale features, and depth features generated by a convolutional neural network. Correspondingly, in this step, histogram of oriented gradients (HOR) features, grayscale features, and depth features generated by a convolutional neural network are extracted for each region of interest in the entire assembly space. The extracted features are used to train the corresponding global correlation filter. The corresponding feature extraction algorithm includes a histogram of oriented gradients extraction algorithm, grayscale values ​​of image pixels, and a convolutional neural network.

[0056] Furthermore, in step three, the multiple evaluation indicators are the peak response R. max The evaluation metrics include Peak Sidelobe Ratio (PSR), Average Peak Correlation Energy (APCE), and Entropy. These metrics are calculated for each individual region of interest. The multi-feature adaptive fusion mechanism calculates the adaptive fusion parameter S for each feature using the following formula. mix :

[0057] ;

[0058] In the above formula, R max S represents the peak value of the filtered response corresponding to the feature; PSR S represents the peak-to-sidelobe ratio of the corresponding feature; APCE S represents the average peak correlation energy of the corresponding feature; Entropy This represents the response entropy of the corresponding feature; k1, k2, k3, and k4 are the preset weight parameters for each evaluation index of this feature, used to balance the role of each index in the fusion evaluation, specifically:

[0059] ;

[0060] ;

[0061] ;

[0062] In the above formula, R max R represents the maximum value of the main response peak corresponding to the feature. min R represents the minimum response corresponding to the feature. w,h The response value at coordinates (w, h) in the response plot corresponding to the characteristic; μ sl It is the average response value of the sidelobe region of the characteristic-response map; σ slis the standard deviation of the sidelobe region of the feature response map; W and H represent the width and height of the feature response map, respectively; p i The normalized probability at each position in the response graph represents the corresponding feature.

[0063] S for each feature mix As fusion weights, the filtered responses of the three features of the region of interest are weighted and fused to generate the final fused correlation filtered response of the region.

[0064] Furthermore, step four specifically includes:

[0065] S41: Calculate the depth difference between the corresponding pixels in the region of interest of the depth map of the sample to be tested and the template depth map to obtain the depth difference map;

[0066] S42: Calculate the negative power weight of each pixel in the depth difference map; construct the depth mask map based on the negative power weight;

[0067] S43: Weight the fused correlation filter response R using the depth mask image, as shown in the following formula, which applies exponential weights to each coordinate position of the fused filter response:

[0068] ;

[0069] In the above formula, R D The weighted fused filter response for the corresponding region of interest is defined as the 3D enhanced correlation filter response; n is the total number of pixels in the corresponding region; d i β represents the depth difference at each pixel location in the depth difference map; β represents the sensitivity of the preset depth mask to the depth difference.

[0070] Furthermore, in step five, determining whether the target part is occluded based on the fusion correlation filter response is as follows: the low threshold of the fusion correlation filter response peak value is preset to T1, and the high threshold is T2; if the peak value of the fusion correlation filter response of the region of interest is outside the two thresholds T1 and T2, then the corresponding part is classified as unoccluded; otherwise, the part corresponding to the region is classified as occluded.

[0071] Furthermore, step six specifically includes:

[0072] S61: Divide the region of interest that is determined to be occluded into an n×n grid, where n is an integer greater than or equal to 1; each grid is defined as a sub-region of the region of interest; multiple local correlation filters corresponding to the feature types are deployed on each sub-region;

[0073] S62: Obtain the template image from step one, extract the regions of interest that are determined to be occluded, and divide them into corresponding sub-regions as training samples for the sub-regions; use the training samples of each sub-region to train based on step two and obtain the local correlation filter response of the image to be detected;

[0074] S63: Based on steps three and four, obtain the fusion correlation filter response and the three-dimensional enhanced correlation filter response of the sub-region; take n... 2 The response with the highest peak value among the local three-dimensional enhanced correlation filter responses is taken as the final response.

[0075] Example 1:

[0076] In this embodiment, the KCF algorithm, a target tracking algorithm based on kernel correlation filtering, is used as an example. The KCF algorithm first trains a correlation filter online in the region of interest by minimizing the least squares loss. Then, it uses a fast Fourier transform to convolve the trained filter with the target region, and locates the target by searching for response peaks. This algorithm is a well-known technique to those skilled in the art; this embodiment is merely an application, and the algorithm itself will not be described in detail here.

[0077] As attached Figure 1 The flowchart shown in this embodiment specifically includes the following steps:

[0078] 1. Obtain a two-dimensional image and depth map of the assembly space.

[0079] This embodiment uses a binocular structured light camera for image acquisition. The camera is fixed to the end of a robotic arm and driven by the robotic arm to a fixed position for taking pictures. A single shot simultaneously acquires a two-dimensional image and a depth map of the assembly space of correctly installed components; at the same time, the location of the components is marked as the region of interest.

[0080] 2. Feature extraction, online training and detection of the overall correlation filter.

[0081] The oriented gradient histogram features, grayscale features, and depth features generated by a convolutional neural network are extracted from the region of interest (ROI) in the template image, and a correlation filter is trained in real time. This correlation filter training uses the entire target part region; relative to the subsequent occlusion step, it is called a global correlation filter. The same three features are extracted from the corresponding ROI in the image to be detected, and the trained correlation filter is used for detection, yielding three filtered responses. This embodiment uses three features; in practice, other features, such as color statistical features, can also be used.

[0082] To avoid compromising the algorithm's real-time performance to some extent, this embodiment uses the first residual block of the ResNet-50 model (cnv2_x) to extract the depth features of each region of interest. Figure 2The diagram in the middle illustrates the training and detection of the relevant filter in this embodiment.

[0083] 3. Adaptive multi-feature fusion.

[0084] In this embodiment, multiple evaluation metrics are preset for each region of interest, including peak response, branch-sidelobe ratio, mean peak correlation energy, and response entropy. For each region of interest, the peak response, branch-sidelobe ratio, mean peak correlation energy, and response entropy of each filtered response are calculated. Then, their respective adaptive fusion parameters are calculated. Finally, the three filtered responses are adaptively fused to obtain the fused correlated filtered response. (See attached figure.) Figure 3 This is a schematic diagram of a multi-feature adaptive fusion mechanism provided in an embodiment of the present invention.

[0085] In the above process, the adaptive fusion parameter S mix The calculation method is as follows: ;

[0086] In the above formula, R max S represents the peak value of the filtered response corresponding to the feature; PSR Indicates the peak-to-sidelobe ratio; S APCE S represents the average peak correlation energy; Entropy ε represents the response entropy; k1, k2, k3, and k4 are preset weight parameters used to balance the roles of various indicators in the fusion evaluation. In this embodiment, they are set to 5, 0.07, 3, and 0.5, respectively; ε is a constant, set to 0.01 in this embodiment.

[0087] S mix As the fusion weights, the three filter responses are weighted and fused, and the final fused correlation filter response diagram is generated as follows:

[0088] ;

[0089] In the above formula, R fusion This represents the final fused correlation filter response; the subscripts HOG, Gray, and CNN correspond to the histogram of oriented gradients (HOG), grayscale features, and depth features generated by the convolutional neural network, respectively, such as S. mix,HOG R represents the adaptive fusion parameters of the directional gradient histogram features. HOG This represents the correlation filter response obtained based on the directional gradient histogram, and so on.

[0090] 4. Depth-sensing 3D enhancement.

[0091] The difference between the depth map to be detected and the template depth map is calculated to obtain a depth difference map. A depth mask is generated using the negative exponential weights of the depth difference map, and the fused correlation filter response is weighted and corrected to obtain the 3D enhanced correlation filter response. (See attached image) Figure 4 This is a schematic diagram of the depth-sensing 3D enhancement method provided in this embodiment.

[0092] In this embodiment, the effect of the depth-sensing 3D enhancement method on the correlation filter response can be represented as follows:

[0093] ;

[0094] In the above formula, R D The weighted fused filter response for the corresponding region of interest is defined as the 3D enhanced correlation filter response; n is the total number of pixels in the corresponding region; d i This represents the depth difference at each pixel location in the depth difference map; β represents the sensitivity of the preset depth mask to the depth difference, which is set to 1.1 in this example.

[0095] 5. Conduct an initial assessment of the obscured areas.

[0096] Based on the response intensity of the fused correlation filter response and the 3D enhanced correlation filter response, a response evaluation module is used to determine whether the target part is occluded. The preset low threshold for the peak value of the fused correlation filter response is T1, and the preset high threshold is T2; in this example, they are set to 0.2 and 0.9 respectively. If the peak value of the fused correlation filter response of the region of interest is outside the two thresholds T1 and T2, the corresponding part is classified as unoccluded; otherwise, the part in that region is classified as occluded.

[0097] 6. Perform area search to resist occlusion for obscured parts.

[0098] The target region is divided into several local sub-regions using an n×n grid. Local correlation filters are trained independently in each sub-region, and detection is performed on that local region. In this example, n is set to 3. Each local filter only focuses on the corresponding features of its corresponding region; therefore, even if some regions are occluded, the unoccluded sub-regions can still provide effective detection information, thus improving detection stability. Applying the above multi-aspect adaptive fusion mechanism, nine fused correlation filter responses are obtained. Applying the above depth-sensing 3D enhancement method, nine 3D enhancement correlation filter responses are obtained. Among all 3D enhancement correlation filter responses, the one with the highest peak value is selected as the final response of the part. Based on the position of the local region in the original region, its maximum response position is mapped back to the corresponding position in the original region, thereby determining the position of the part. Please refer to... Figure 5 This is a schematic diagram of the region search anti-occlusion strategy in this embodiment.

[0099] 7. Final assessment of response and judgment of test results.

[0100] A detection threshold T3 is defined and set to 0.7 in this embodiment of the invention. If the peak value of the three-dimensional enhanced correlation filter response of the part is less than this threshold, the part is classified as an installation error (misinstallation, omission, or improper installation); otherwise, the part is classified as a correct installation.

[0101] Verification Example 1:

[0102] To verify the effectiveness of the proposed method, it was compared with the original KCF algorithm, eight image similarity-based algorithms, and three template matching algorithms. The image similarity algorithms specifically included: Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), Cosine Similarity, Histogram Similarity, Mean Squared Error (MSE), Average Hash (aHash), Difference Hash (dHash), and Perceptual Hash (pHash). The image template matching algorithms specifically included Cross-Coeff, Cross-Corr, and Squared-Diff matching. To enable these two-dimensional algorithms to possess three-dimensional perception capabilities, each algorithm employed an improved bimodal configuration, processing the two-dimensional image and depth map separately. The final detection result was calculated by the arithmetic mean of the outputs from the two modalities.

[0103] Figure 6 The results show the detection accuracy of the proposed method and 12 comparative algorithms at different thresholds in this validation example. The results show that the proposed method significantly outperforms the other algorithms with an accuracy of 0.9421, while only MSE, pHash, and dHash among the comparative algorithms achieve an accuracy of approximately 0.75. These quantitative data fully demonstrate the superiority of the proposed method over traditional methods in assembly detection scenarios in confined spaces.

[0104] Verification Example 2:

[0105] Figure 7 This is the response map generated during the detection of a missing part in this verification example. It includes the initial correlation filter response generated by three features in the region of interest of the part, the fused correlation filter response, and the three-dimensional enhanced correlation filter response obtained by depth weighting correction.

[0106] Figure 8 This is the actual testing result of the setup deployed on the experimental assembly workbench for this verification example. A green box marked "Correct" indicates correct installation, and a red box marked "Incorrect" indicates incorrect installation.

[0107] Although the present invention has been disclosed above with reference to preferred embodiments, these are not intended to limit the invention. Any person skilled in the art can make various changes or modifications without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention should be defined by the scope of the claims of this application.

Claims

1. A method for inspecting the assembly quality of parts in confined spaces based on three-dimensional enhanced correlation filtering, characterized in that: include: Step 1: Use a binocular vision device to acquire a template image containing correctly assembled parts; the template image includes a two-dimensional image and a depth image; In the template image, the locations of the components are marked as regions of interest; Step 2: Preset multiple feature types and assign a correlation filter to each feature type for each region of interest in the template image; use the corresponding feature extraction algorithm to extract the features of the corresponding type from the template image, and use the extracted features to train the corresponding correlation filter to obtain the trained correlation filter; define the trained correlation filter as the global correlation filter; input the binocular vision test image to be detected into its corresponding global correlation filter to obtain multiple initial global correlation filter responses for each region of interest; Step 3: Preset multiple evaluation indicators for a single region of interest, and calculate multiple evaluation indicators for each initial response of each region of interest based on the initial overall correlation filter response; perform index fusion based on multiple evaluation indicators, and use a multi-feature adaptive fusion mechanism to fuse the initial correlation filter responses to obtain the fused correlation filter response of each region of interest; Step 4: For each region of interest, the fusion correlation filter response is corrected using a depth-sensing 3D enhancement method to obtain a 3D enhanced correlation filter response. The depth-sensing 3D enhancement involves: calculating the depth difference between the depth map of the tested sample and the template depth map to obtain a depth difference map; obtaining a depth mask map based on the depth difference map; and using the depth mask map to perform weighted correction on the fusion correlation filter response. Step 5: Preset the threshold range of the peak value of the filter response, and determine whether the part is blocked based on the fused correlation filter response. If it is blocked, proceed to step 6; otherwise, proceed to step 7. Step 6: Divide the region of interest identified as occluded into multiple sub-regions, assign multiple local correlation filters to each sub-region, and train the corresponding local correlation filters, the detected fused correlation filter response, and the enhancement result obtained by weighting correction based on steps 2 to 5. The component with the highest response peak in the enhancement results is taken as its final response. Step 7: Preset the detection threshold for parts in the region of interest. If the peak value of the 3D enhanced correlation filter response of the part is less than the detection threshold, the part is classified as an installation error. The installation error includes incorrect installation, missing installation, or improper installation. Conversely, the part is classified as a correct installation.

2. The method for inspecting the assembly quality of parts in confined spaces based on three-dimensional enhanced correlation filtering according to claim 1, characterized in that: In step two, the feature extraction includes histogram of oriented gradients (HOR) features, grayscale features, and depth features generated by a convolutional neural network. Correspondingly, in this step, histogram of oriented gradients (HOR) features, grayscale features, and depth features generated by a convolutional neural network are extracted for each region of interest in the entire assembly space. The extracted features are used to train the corresponding global correlation filter. The corresponding feature extraction algorithm includes an HOR extraction algorithm, a grayscale extraction algorithm for image pixel values, and a convolutional neural network.

3. The method for inspecting the assembly quality of parts in confined spaces based on three-dimensional enhanced correlation filtering according to claim 1, characterized in that: In step three, the multiple evaluation indicators are the peak response R. max The evaluation metrics include Peak Sidelobe Ratio (PSR), Average Peak Correlation Energy (APCE), and Entropy. These metrics are calculated for each individual region of interest. The multi-feature adaptive fusion mechanism calculates the adaptive fusion parameter S for each feature using the following formula. mix : ； In the above formula, R max S represents the peak value of the filtered response corresponding to the feature; PSR S represents the peak-to-sidelobe ratio of the corresponding feature; APCE S represents the average peak correlation energy of the corresponding feature; Entropy This represents the response entropy of the corresponding feature; k1, k2, k3, and k4 are the preset weight parameters for each evaluation index of this feature, used to balance the role of each index in the fusion evaluation, specifically: ； ； ； In the above formula, R max R represents the maximum value of the main response peak corresponding to the feature. min R represents the minimum response corresponding to the feature. w,h The response value at coordinates (w, h) in the response plot corresponding to the characteristic; μ sl It is the average response value of the sidelobe region of the characteristic-response map; σ sl is the standard deviation of the sidelobe region of the feature response map; W and H represent the width and height of the feature response map, respectively; p i The normalized probability at each position in the response graph represents the corresponding feature. S for each feature mix As fusion weights, the filtered responses of the three features of the region of interest are weighted and fused to generate the final fused correlation filtered response of the region.

4. The method for inspecting the assembly quality of parts in confined spaces based on three-dimensional enhanced correlation filtering according to claim 3, characterized in that: Step four specifically includes: S41: Calculate the depth difference between the corresponding pixels in the region of interest of the depth map of the sample to be tested and the template depth map to obtain the depth difference map; S42: Calculate the negative power weight of each pixel in the depth difference map; construct the depth mask map based on the negative power weight; S43: Weight the fused correlation filter response R using the depth mask image, as shown in the following formula, which applies exponential weights to each coordinate position of the fused filter response: ； In the above formula, R D The weighted fused filter response for the corresponding region of interest is defined as the 3D enhanced correlation filter response; n is the total number of pixels in the corresponding region; d i β represents the depth difference at each pixel location in the depth difference map; β represents the sensitivity of the preset depth mask to the depth difference.

5. The method for inspecting the assembly quality of parts in confined spaces based on three-dimensional enhanced correlation filtering according to claim 4, characterized in that: In step five, the determination of whether the target part is occluded based on the fusion correlation filter response is as follows: the low threshold of the fusion correlation filter response peak value is set as T1, and the high threshold is set as T2; if the peak value of the fusion correlation filter response of the region of interest is outside the two thresholds T1 and T2, the corresponding part is classified as unoccluded; otherwise, the part corresponding to the region is classified as occluded.

6. The method for inspecting the assembly quality of parts in confined spaces based on three-dimensional enhanced correlation filtering according to claim 5, characterized in that: Step six specifically includes: S61: Divide the region of interest that is determined to be occluded into an n×n grid, where n is an integer greater than or equal to 1; each grid is defined as a sub-region of the region of interest; multiple local correlation filters corresponding to the feature types are deployed on each sub-region; S62: Obtain the template image from step one, extract the regions of interest that are determined to be occluded, and divide them into corresponding sub-regions as training samples for the sub-regions; use the training samples of each sub-region to train based on step two and obtain the local correlation filter response of the image to be detected; S63: Based on steps three and four, obtain the fusion correlation filter response and the three-dimensional enhanced correlation filter response of the sub-region; take n... 2 The response with the highest peak value among the local three-dimensional enhanced correlation filter responses is taken as the final response.

Images