Narrow space part assembly quality detection method based on three-dimensional enhanced correlation filtering

Abstract

The invention discloses a narrow space part assembly quality detection method based on three-dimensional enhanced correlation filtering, and relates to the field of machine vision. Comprising the following steps: acquiring a template image of a correctly assembled part, and marking the position of the part as a region of interest; extracting regional features and training a correlation filter, and detecting the features of the detected image to obtain an overall correlation filter response; performing multi-feature adaptive fusion based on a preset evaluation index; correcting the fusion correlation filtering response by using a depth perception three-dimensional enhancement method; judging whether the part is shielded or not according to the fused correlation filtering response; dividing the area which is judged to be shielded into sub-areas, detecting each sub-area to obtain a fusion correlation filtering response, and carrying out weighted correction on the fusion correlation filtering response; judging an unshielded area of the part according to an enhancement result; and judging whether the parts are installed wrongly according to the three-dimensional enhanced correlation filtering response. The problems of wrong installation, neglected installation and not-in-place installation of parts under the condition of target shielding in a narrow space are effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a method for detecting the assembly quality of parts in a narrow space based on three-dimensional enhanced correlation filtering. Background Art

[0002] In assembly tasks within confined spaces, traditional manual inspection methods suffer from limitations such as low accuracy, poor efficiency, and difficulty covering all assembly details. This is particularly true in complex assembly environments, where multiple parts often occlude each other. Existing inspection methods are primarily categorized as traditional image processing and deep learning image processing. While traditional image processing methods are highly adaptable and can accomplish the task with simple configuration, they often employ complex feature extraction or modal fusion techniques, resulting in slow computation and long inspection times. These methods struggle to meet the real-time requirements of assembly quality inspection and are unable to detect and address occlusions within a single image frame. Deep learning image processing methods, on the other hand, rely on extensive data for training, making them difficult to apply to assembly quality inspection tasks where datasets are difficult to acquire. Therefore, efficiently combining image features and depth information to analyze and address occlusions and accurately detect assembly errors (such as misplaced, missing, or incomplete parts) remains a major technical challenge. Summary of the Invention

[0003] 1. Technical problems to be solved:

[0004] In response to the above technical problems, the present invention provides a method for quality inspection of parts assembly in confined spaces based on three-dimensional enhanced correlation filtering. This scheme uses an improved correlation filtering algorithm to provide an efficient and reliable solution for automated quality inspection of assembly in confined spaces, thereby realizing the detection of misinstalled, missing, and incomplete parts, thereby improving product quality reliability and production efficiency.

[0005] 2. Technical solution:

[0006] A method for detecting the quality of parts assembly in a narrow space based on three-dimensional enhanced correlation filtering is characterized by comprising:

[0007] Step 1: Using a binocular vision device to capture a template image containing correctly assembled parts; the template image includes a two-dimensional image and a depth image; in the template image, the location of the parts is marked as a region of interest;

[0008] Step 2: Preset multiple feature types and assign a correlation filter to each feature type for each region of interest in the template image; use the corresponding feature extraction algorithm to extract the corresponding type of features of the template image, use the extracted features to train the corresponding correlation filter to obtain a trained correlation filter; define the trained correlation filter as an overall correlation filter; input the binocular vision test image to be tested into its corresponding overall correlation filter to obtain multiple initial overall correlation filter responses for each region of interest;

[0009] Step 3: Preset multiple evaluation indicators for a single region of interest, and calculate multiple evaluation indicators of each initial response of each region of interest based on the initial overall correlation filter response; perform indicator fusion based on multiple evaluation indicators, and use a multi-feature adaptive fusion mechanism to fuse the initial correlation filter response to obtain a fused correlation filter response for each region of interest;

[0010] Step 4: For each region of interest, a depth-aware 3D enhancement method is used to correct the fused correlation filter response to obtain a 3D enhanced correlation filter response; the depth-aware 3D enhancement comprises: performing a depth difference calculation between the depth map of the sample under test and the template depth map to obtain a depth difference map; obtaining a depth mask map based on the depth difference map; and performing a weighted correction on the fused correlation filter response using the depth mask map;

[0011] Step 5: Preset the threshold range of the filter response peak value, and determine whether the part is blocked based on the fusion correlation filter response. If it is blocked, proceed to step 6, otherwise proceed to step 7;

[0012] Step 6: Divide the ROI determined to be occluded into multiple sub-regions. Multiple local correlation filters are assigned to each sub-region. The enhanced result is obtained by training the corresponding local correlation filters in steps 2-5, performing fusion correlation filter responses obtained through detection, and performing weighted correction. The enhanced result with the highest response peak is taken as the final response of the part.

[0013] Step 7: Preset a detection threshold for the part in the region of interest. If the peak value of the 3D enhanced correlation filter response of the part is less than the detection threshold, the part is classified as incorrectly installed. The installation error includes incorrect installation, missing installation, or incomplete installation. Otherwise, the part is classified as correctly installed.

[0014] Furthermore, in step 2, the feature extraction includes oriented gradient histogram features, grayscale features, and depth features generated by convolutional neural networks; correspondingly, in this step, oriented gradient histogram features, grayscale features, and depth features generated by convolutional neural networks are extracted for each region of interest in the entire assembly space, and the extracted features are used to train their corresponding overall correlation filters; the corresponding feature extraction algorithm includes a oriented gradient histogram extraction algorithm, the grayscale of image pixel values, and a convolutional neural network.

[0015] Furthermore, in step 3, the multiple evaluation indicators are response peak Rmax, peak sidelobe ratio PSR, average peak correlation energy APCE and response entropy Entropy; the multiple evaluation indicators are calculated for each single region of interest; the multi-feature adaptive fusion mechanism is to calculate the adaptive fusion parameter S corresponding to each feature by the following formula mix :

[0016] ;

[0017] In the above formula, R max Indicates the peak value of the filter response of the corresponding feature; S PSR represents the peak-to-sidelobe ratio of the corresponding feature; S APCE represents the average peak correlation energy of the corresponding feature; S Entropy represents the response entropy of the corresponding feature; k1, k2, k3, and k4 are the weight parameters of each evaluation index preset for this feature, which are used to balance the role of each index in the fusion evaluation. Specifically:

[0018] ;

[0019] ;

[0020] ;

[0021] In the above formula, R max Indicates the maximum value of the main response peak corresponding to the feature; R min Indicates the minimum response value corresponding to the feature; R w,h Represents the response value at the coordinate (w, h) in the response graph corresponding to the feature; μ sl is the average response value of the sidelobe area of ​​the feature corresponding response map; σ sl is the standard deviation of the sidelobe area of ​​the feature corresponding response map; W and H represent the width and height of the feature corresponding response map respectively; p i The normalized probability of each position in the corresponding response map of the representative feature;

[0022] The S of each feature mixThe filter responses of the three features of the region of interest are weightedly fused as fusion weights to generate the final fused correlation filter response of the region.

[0023] Furthermore, step four specifically includes:

[0024] S41: Calculate the depth difference between the depth map of the sample to be tested and the corresponding pixels of the corresponding region of interest in the template depth map to obtain a depth difference map;

[0025] S42: Calculate the negative exponential weight of each pixel in the depth difference map; construct a depth mask map of the map based on the negative exponential weight;

[0026] S43: Using the depth mask image, perform weighted correction on the fused correlation filter response R. The following formula weights each coordinate position of the fused filter response by exponential weight:

[0027] ;

[0028] In the above formula, R D represents the weighted fusion filter response of the corresponding region of interest, which is defined as the three-dimensional enhanced correlation filter response; n is the total number of pixels in the corresponding region; d i represents the depth difference of each pixel in the depth difference map; β represents the sensitivity of the preset depth mask to the depth difference.

[0029] Furthermore, in step five, whether the target part is occluded is determined based on the fusion correlation filter response: the low threshold of the fusion correlation filter response peak is preset to T1, and the high threshold is preset to T2; if the peak of the fusion correlation filter response of the area of ​​interest is outside the two thresholds T1 and T2, the corresponding component is classified as unoccluded; otherwise, the part corresponding to the area is judged to be occluded.

[0030] Furthermore, step six specifically includes:

[0031] S61: Divide the ROI determined to be blocked into n×n grids, where n is an integer greater than or equal to 1; each grid is defined as a sub-region of the ROI; and local correlation filters corresponding to multiple feature types are deployed on each sub-region;

[0032] S62: Obtain the template image in step 1, extract the region of interest determined to be blocked and divide it into corresponding sub-regions as training samples of the sub-regions; use the training samples of each sub-region to train based on step 2 and obtain the local correlation filter response of the image to be detected;

[0033] S63: Based on steps 3 and 4, the fusion correlation filter response and the three-dimensional enhanced correlation filter response of the sub-region are obtained; 2The one with the highest response peak among the local three-dimensional enhanced correlation filter responses is taken as the final response.

[0034] 3.Beneficial effects:

[0035] (1) The present invention provides a method for detecting the quality of assembly of parts in a narrow space based on three-dimensional enhanced correlation filtering. The method uses a three-dimensional enhancement method of depth perception to generate a depth mask by the difference between the template and the sample depth map, and then realizes the three-dimensional space perception of the correlation filter through the weighted correlation filter response.

[0036] (2) The method for detecting the quality of assembly of parts in a narrow space based on three-dimensional enhanced correlation filtering provided by the present invention adopts an adaptive multi-feature fusion strategy, which fuses the response graphs of multiple features based on the response graph quantification index, thereby improving the robustness of the correlation filter feature representation and enabling it to adapt to a variety of parts in complex and narrow spaces.

[0037] (3) The present invention provides a method for inspecting the quality of parts assembly in confined spaces based on three-dimensional enhanced correlation filtering. This method addresses the common occlusion problem in confined spaces and designs an anti-occlusion strategy based on region division. This method divides the area where occlusion may occur into multiple local regions, and independently trains and applies local correlation filters within each subregion. By effectively utilizing the features of the unoccluded subregions, these local filters can effectively compensate for the performance degradation of the overall filter under occlusion conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is the overall flow chart of the present invention;

[0039] Figure 2 Schematic diagram of correlation filter training and detection in the present invention;

[0040] Figure 3 Schematic diagram of the multi-feature adaptive fusion mechanism in the present invention;

[0041] Figure 4 Schematic diagram of the depth perception 3D enhancement method in the present invention;

[0042] Figure 5 This is a flow chart of implementing regional search and anti-occlusion for obscured parts in the present invention;

[0043] Figure 6 To verify the detection accuracy comparison of this method and the existing 12 algorithms at different thresholds in Example 1;

[0044] Figure 7 This is to verify the response diagram generated during the detection of a missing part in Example 2;

[0045] Figure 8To verify the actual detection effect deployed on the experimental assembly workbench in Example 2. DETAILED DESCRIPTION

[0046] The present invention will be described in detail below with reference to the accompanying drawings.

[0047] As attached Figures 1 to 5 As shown, the method for detecting the quality of parts assembly in a narrow space based on three-dimensional enhanced correlation filtering is characterized by comprising:

[0048] Step 1: Using a binocular vision device to capture a template image containing correctly assembled parts; the template image includes a two-dimensional image and a depth image; in the template image, the location of the parts is marked as a region of interest;

[0049] Step 2: Preset multiple feature types and assign a correlation filter to each feature type for each region of interest in the template image; use the corresponding feature extraction algorithm to extract the corresponding type of features of the template image, use the extracted features to train the corresponding correlation filter to obtain a trained correlation filter; define the trained correlation filter as an overall correlation filter; input the binocular vision test image to be tested into its corresponding overall correlation filter to obtain multiple initial overall correlation filter responses for each region of interest;

[0050] Step 3: Preset multiple evaluation indicators for a single region of interest, and calculate multiple evaluation indicators of each initial response of each region of interest based on the initial overall correlation filter response; perform indicator fusion based on multiple evaluation indicators, and use a multi-feature adaptive fusion mechanism to fuse the initial correlation filter response to obtain a fused correlation filter response for each region of interest;

[0051] Step 4: For each region of interest, a depth-aware 3D enhancement method is used to correct the fused correlation filter response to obtain a 3D enhanced correlation filter response; the depth-aware 3D enhancement comprises: performing a depth difference calculation between the depth map of the sample under test and the template depth map to obtain a depth difference map; obtaining a depth mask map based on the depth difference map; and performing a weighted correction on the fused correlation filter response using the depth mask map;

[0052] Step 5: Preset the threshold range of the filter response peak value, and determine whether the part is blocked based on the fusion correlation filter response. If it is blocked, proceed to step 6, otherwise proceed to step 7;

[0053] Step 6: Divide the ROI determined to be occluded into multiple sub-regions. Multiple local correlation filters are assigned to each sub-region. The enhanced result is obtained by training the corresponding local correlation filters in steps 2-5, performing fusion correlation filter responses obtained through detection, and performing weighted correction. The enhanced result with the highest response peak is taken as the final response of the part.

[0054] Step 7: Preset a detection threshold for the part in the region of interest. If the peak value of the 3D enhanced correlation filter response of the part is less than the detection threshold, the part is classified as incorrectly installed. The installation error includes incorrect installation, missing installation, or incomplete installation. Otherwise, the part is classified as correctly installed.

[0055] Furthermore, in step 2, the feature extraction includes oriented gradient histogram features, grayscale features, and depth features generated by convolutional neural networks; correspondingly, in this step, oriented gradient histogram features, grayscale features, and depth features generated by convolutional neural networks are extracted for each region of interest in the entire assembly space, and the extracted features are used to train their corresponding overall correlation filters; the corresponding feature extraction algorithm includes a oriented gradient histogram extraction algorithm, the grayscale of image pixel values, and a convolutional neural network.

[0056] Furthermore, in step 3, the multiple evaluation indicators are response peak R max , peak sidelobe ratio PSR, average peak correlation energy APCE and response entropy Entropy; the multiple evaluation indicators are calculated for a single region of interest; the multi-feature adaptive fusion mechanism is to calculate the adaptive fusion parameter S corresponding to each feature through the following formula mix :

[0057] ;

[0058] In the above formula, R max Indicates the peak value of the filter response of the corresponding feature; S PSR represents the peak-to-sidelobe ratio of the corresponding feature; S APCE represents the average peak correlation energy of the corresponding feature; S Entropy represents the response entropy of the corresponding feature; k1, k2, k3, and k4 are the weight parameters of each evaluation index preset for this feature, which are used to balance the role of each index in the fusion evaluation. Specifically:

[0059] ;

[0060] ;

[0061] ;

[0062] In the above formula, R max Indicates the maximum value of the main response peak corresponding to the feature; R min Indicates the minimum response value corresponding to the feature; R w,h Represents the response value at the coordinate (w, h) in the response graph corresponding to the feature; μ sl is the average response value of the sidelobe area of ​​the feature corresponding response map; σ slis the standard deviation of the sidelobe area of ​​the feature corresponding response map; W and H represent the width and height of the feature corresponding response map respectively; p i The normalized probability of each position in the corresponding response map of the representative feature;

[0063] The S of each feature mix As the fusion weight, the filter responses of the three features of the region of interest are weightedly fused to generate the final fused correlation filter response of the region.

[0064] Furthermore, step four specifically includes:

[0065] S41: Calculating the depth difference between the depth map of the sample to be tested and the corresponding pixels of the corresponding region of interest in the template depth map to obtain a depth difference map;

[0066] S42: Calculate the negative exponential weight of each pixel of the depth difference map; construct a depth mask map of the map based on the negative exponential weight;

[0067] S43: Using the depth mask image, perform weighted correction on the fused correlation filter response R. The following formula weights each coordinate position of the fused filter response by exponential weight:

[0068] ;

[0069] In the above formula, R D represents the weighted fusion filter response of the corresponding region of interest, which is defined as the three-dimensional enhanced correlation filter response; n is the total number of pixels in the corresponding region; d i represents the depth difference of each pixel in the depth difference map; β represents the sensitivity of the preset depth mask to the depth difference.

[0070] Furthermore, in step five, whether the target part is occluded is determined based on the fusion correlation filter response: the low threshold of the fusion correlation filter response peak is preset to T1, and the high threshold is preset to T2; if the peak of the fusion correlation filter response of the area of ​​interest is outside the two thresholds T1 and T2, the corresponding component is classified as unoccluded; otherwise, the part corresponding to the area is judged to be occluded.

[0071] Furthermore, step six specifically includes:

[0072] S61: Divide the ROI determined to be blocked into n×n grids, where n is an integer greater than or equal to 1; each grid is defined as a sub-region of the ROI; and local correlation filters corresponding to multiple feature types are deployed on each sub-region;

[0073] S62: Obtain the template image in step 1, extract the region of interest determined to be blocked and divide it into corresponding sub-regions as training samples of the sub-regions; use the training samples of each sub-region to train based on step 2 and obtain the local correlation filter response of the image to be detected;

[0074] S63: Based on steps 3 and 4, the fusion correlation filter response and the three-dimensional enhanced correlation filter response of the sub-region are obtained; 2 The one with the highest response peak among the local three-dimensional enhanced correlation filter responses is taken as the final response.

[0075] Example 1:

[0076] In this embodiment, the target tracking algorithm based on kernel correlation filtering, or KCF, is used as an example. The KCF algorithm first trains a correlation filter online on the region of interest by minimizing the least squares loss. The trained filter is then convolved with the target region using a fast Fourier transform, and the target is located by searching for the peak response. This algorithm is well known to those skilled in the art, and the embodiments of the present invention are merely examples of its application, so the details of the algorithm are not detailed here.

[0077] As attached Figure 1 As shown in the flowchart, this embodiment specifically includes the following steps:

[0078] 1. Obtain a 2D image and depth map of the assembly space.

[0079] This embodiment uses a binocular structured light camera for image acquisition. The camera is fixed to the end of a robotic arm and driven by the arm to a fixed position for shooting. A single shot simultaneously captures a 2D image and depth map of the assembly space where components are correctly installed. The component locations are also marked as regions of interest.

[0080] 2. Feature extraction, online training and detection of overall correlation filter.

[0081] The directional gradient histogram features, grayscale features and depth features generated by the convolutional neural network of the region of interest in the template image are extracted to perform real-time training of the correlation filter. The correlation filter training uses the complete target part area, which is called the overall correlation filter relative to the subsequent occlusion step. The above three features are also extracted in the corresponding region of interest of the image to be detected, and the trained correlation filter is used for detection to obtain three filter responses respectively. The three features used in this embodiment can also be used in practice with other features, such as color statistical features.

[0082] In order to avoid destroying the real-time performance of the algorithm to a certain extent, this embodiment uses the first residual block cnv2_x of the ResNet-50 model to extract the depth features of each region of interest. Figure 2] is a schematic diagram of correlation filter training and detection in this embodiment.

[0083] 3. Adaptive multi-feature fusion.

[0084] In this embodiment, multiple evaluation indicators are preset for each region of interest, namely, response peak value, branch sidelobe ratio, average peak correlation energy and response entropy. Taking each region of interest as a unit, the response peak value, branch sidelobe ratio, average peak correlation energy and response entropy of each filter response in the region are calculated, and then the respective adaptive fusion parameters are calculated. Then, the three filter responses are adaptively fused to obtain the fused correlation filter response. As shown in the attached figure, Figure 3 , which is a schematic diagram of a multi-feature adaptive fusion mechanism provided by an embodiment of the present invention.

[0085] In the above process, the adaptive fusion parameter S mix The calculation method is: ;

[0086] In the above formula, R max Indicates the peak value of the filter response of the corresponding feature; S PSR represents the peak sidelobe ratio; S APCE represents the average peak correlation energy; S Entropy represents the response entropy; k1, k2, k3, and k4 are preset weight parameters used to balance the effects of various indicators in the fusion evaluation. In this embodiment, they are set to 5, 0.07, 3, and 0.5, respectively; ε is a constant, which is set to 0.01 in this embodiment.

[0087] S mix As the fusion weight, the three filter responses are weightedly fused, and the final fusion correlation filter response graph is generated as follows:

[0088] ;

[0089] In the above formula, R fusion is the final fusion correlation filter response; the subscripts HOG, Gray, and CNN correspond to the directional gradient histogram feature, grayscale feature, and deep feature generated by the convolutional neural network, respectively. For example, S mix,HOG Represents the adaptive fusion parameter of the oriented gradient histogram feature, R HOG Represents the correlation filter response obtained based on the histogram of directional gradients, and so on.

[0090] 4. Depth perception 3D enhancement.

[0091] Calculate the difference between the depth map to be detected and the template depth map to obtain the depth difference map; generate a depth mask with the negative exponential weight of the depth difference map, perform weighted correction on the fusion correlation filter response, and obtain the three-dimensional enhanced correlation filter response. Figure 4 , which is a schematic diagram of the depth perception three-dimensional enhancement method provided in this embodiment.

[0092] In this embodiment, the process of the depth perception 3D enhancement method acting on the correlation filter response can be expressed as follows:

[0093] ;

[0094] In the above formula, R D represents the weighted fusion filter response of the corresponding region of interest, which is defined as the three-dimensional enhanced correlation filter response; n is the total number of pixels in the corresponding region; d i represents the depth difference of each pixel in the depth difference map; β represents the sensitivity of the preset depth mask to the depth difference, which is set to 1.1 in this example.

[0095] 5. Perform an initial assessment of the obscured area.

[0096] Based on the response strength of the fused correlation filter response and the 3D enhanced correlation filter response, the response evaluation module determines whether the target part is occluded. The lower threshold for the fused correlation filter response peak is preset as T1, and the upper threshold is preset as T2; in this example, they are set to 0.2 and 0.9, respectively. If the peak of the fused correlation filter response in the region of interest falls outside the T1 and T2 thresholds, the corresponding part is classified as unoccluded; otherwise, the part in that region is considered occluded.

[0097] 6. Perform regional search and anti-occlusion on the occluded parts.

[0098] Use an n×n grid to divide the target area into several local sub-areas, train local correlation filters independently on each sub-area, and detect the local area. In this example, n is set to 3. Each local filter only focuses on the corresponding features of its corresponding area, so even if part of the area is blocked, the unblocked sub-area can still provide effective detection information, thereby improving the stability of the detection. Applying the above-mentioned multi-specific adaptive fusion mechanism, 9 fused correlation filter responses can be obtained. Applying the above-mentioned depth-aware three-dimensional enhancement method, 9 three-dimensional enhanced correlation filter responses can be obtained. Among all the three-dimensional enhanced correlation filter responses, the one with the highest three-dimensional enhanced correlation filter response peak is selected as the final response of the part. According to the position of the local area in the original area, its maximum response position is mapped back to the corresponding position of the original area to determine the position of the part. Please refer to Figure 5 , which is a schematic diagram of the area search anti-occlusion strategy of this embodiment.

[0099] 7. Final response evaluation and test result judgment.

[0100] A detection threshold T3 is defined, which is set to 0.7 in this embodiment. If the peak value of the 3D enhanced correlation filter response of a part is less than this threshold, the part is classified as incorrectly installed (wrongly installed, missing, or not installed properly); otherwise, the part is classified as correctly installed.

[0101] Verification Example 1:

[0102] To verify the effectiveness of the proposed method, it was compared with the original KCF algorithm, eight image similarity-based algorithms, and three template matching algorithms. The image similarity algorithms include: structural similarity index (SSIM), peak signal-to-noise ratio (PSNR), cosine similarity, histogram similarity, mean square error (MSE), average hashing algorithm (aHash), difference hashing algorithm (dHash), and perceptual hashing algorithm (pHash); and the image template matching algorithms include cross-correlation coefficient matching (Cross-Coeff), cross-correlation matching (Cross-Corr), and squared difference matching (Squared-Diff). To enable these two-dimensional algorithms to have three-dimensional perception capabilities, each algorithm adopts an improved dual-modal configuration, processing the two-dimensional image and depth map separately. The final detection result is calculated as the arithmetic mean of the two modal outputs.

[0103] Figure 6 The results show the detection accuracy of the proposed method and 12 comparison algorithms at different thresholds in this verification example. The results show that the proposed method significantly outperforms the other algorithms with an accuracy of 0.9421, while only MSE, pHash, and dHash achieve an accuracy of approximately 0.75 among the comparison algorithms. These quantitative data fully demonstrate the superiority of the proposed method over traditional methods in the narrow space assembly inspection scenario.

[0104] Verification Example 2:

[0105] Figure 7 This is the response diagram generated in the process of detecting a missing part in this verification example, including the initial correlation filter response generated by three features in the part's region of interest, the fused correlation filter response obtained by fusion, and the three-dimensional enhanced correlation filter response obtained by depth weighted correction.

[0106] Figure 8 This is the actual test result of this verification example deployed on the experimental assembly workbench. The green box marked "Correct" indicates a correct installation, and the red box marked "Incorrect" indicates an incorrect installation.

[0107] Although the present invention has been disclosed above in terms of preferred embodiments, they are not intended to limit the present invention. Anyone skilled in the art can make various changes or modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection defined by the claims of this application.

Claims

1. A method for inspecting the quality of parts assembly in a narrow space based on three-dimensional enhanced correlation filtering, characterized by: include: Step 1: Using a binocular vision device to capture a template image containing correctly assembled parts; the template image includes a two-dimensional image and a depth image; In the template image, mark the location of the components as the region of interest; Step 2: Preset multiple feature types and assign a correlation filter to each feature type for each region of interest in the template image; use the corresponding feature extraction algorithm to extract the corresponding type of features of the template image, use the extracted features to train the corresponding correlation filter to obtain a trained correlation filter; define the trained correlation filter as an overall correlation filter; input the binocular vision test image to be tested into its corresponding overall correlation filter to obtain multiple initial overall correlation filter responses for each region of interest; Step 3: Preset multiple evaluation indicators for a single region of interest, and calculate multiple evaluation indicators of each initial response of each region of interest based on the initial overall correlation filter response; perform indicator fusion based on multiple evaluation indicators, and use a multi-feature adaptive fusion mechanism to fuse the initial correlation filter response to obtain a fused correlation filter response for each region of interest; Step 4: For each region of interest, a depth-aware 3D enhancement method is used to correct the fused correlation filter response to obtain a 3D enhanced correlation filter response; the depth-aware 3D enhancement comprises: performing a depth difference calculation between the depth map of the sample under test and the template depth map to obtain a depth difference map; obtaining a depth mask map based on the depth difference map; and performing a weighted correction on the fused correlation filter response using the depth mask map; Step 5: Preset the threshold range of the filter response peak value, and determine whether the part is blocked based on the fusion correlation filter response. If it is blocked, proceed to step 6, otherwise proceed to step 7; Step 6: Divide the ROI determined to be occluded into multiple sub-regions, assign multiple local correlation filters to each sub-region, and perform weighted correction based on the local correlation filters trained in steps 2 to 5, the fused correlation filter responses obtained by detection, and the enhanced results. The one with the highest response peak in the enhancement results is obtained as the final response of the part; Step 7: Preset a detection threshold for the part in the region of interest. If the peak value of the 3D enhanced correlation filter response of the part is less than the detection threshold, the part is classified as incorrectly installed. The installation error includes incorrect installation, missing installation, or incomplete installation. Otherwise, the part is classified as correctly installed.

2. The method for detecting the quality of parts assembly in a narrow space based on three-dimensional enhanced correlation filtering according to claim 1, characterized in that: In step 2, the feature extraction includes oriented gradient histogram features, grayscale features, and depth features generated by convolutional neural networks; correspondingly, in this step, oriented gradient histogram features, grayscale features, and depth features generated by convolutional neural networks are extracted for each region of interest in the entire assembly space, and the extracted features are used to train the corresponding overall correlation filter; the corresponding feature extraction algorithm includes oriented gradient histogram extraction algorithm, grayscale of image pixel values, and convolutional neural networks.

3. The method for detecting the quality of parts assembly in a narrow space based on three-dimensional enhanced correlation filtering according to claim 1 is characterized in that: In step 3, the multiple evaluation indicators are response peak R max , peak sidelobe ratio PSR, average peak correlation energy APCE and response entropy Entropy; the multiple evaluation indicators are calculated for a single region of interest; the multi-feature adaptive fusion mechanism is to calculate the adaptive fusion parameter S corresponding to each feature through the following formula mix : ； In the above formula, R max Indicates the peak value of the filter response of the corresponding feature; S PSR represents the peak-to-sidelobe ratio of the corresponding feature; S APCE represents the average peak correlation energy of the corresponding feature; S Entropy represents the response entropy of the corresponding feature; k1, k2, k3, and k4 are the weight parameters of each evaluation index preset for this feature, which are used to balance the role of each index in the fusion evaluation. Specifically: ； ； ； In the above formula, R max Indicates the maximum value of the main response peak corresponding to the feature; R min Indicates the minimum response value corresponding to the feature; R w,h Represents the response value at the coordinate (w, h) in the response graph corresponding to the feature; μ sl is the average response value of the sidelobe area of ​​the feature corresponding response map; σ sl is the standard deviation of the sidelobe area of ​​the feature corresponding response map; W and H represent the width and height of the feature corresponding response map respectively; p i The normalized probability of each position in the corresponding response map of the representative feature; The S of each feature mix The filter responses of the three features of the region of interest are weightedly fused as fusion weights to generate the final fused correlation filter response of the region.

4. The method for detecting the quality of parts assembly in a narrow space based on three-dimensional enhanced correlation filtering according to claim 3 is characterized in that: Step 4 specifically includes: S41: Calculate the depth difference between the depth map of the sample to be tested and the corresponding pixels of the corresponding region of interest in the template depth map to obtain a depth difference map; S42: Calculate the negative exponential weight of each pixel in the depth difference map; construct a depth mask map of the map based on the negative exponential weight; S43: Using the depth mask image, perform weighted correction on the fused correlation filter response R. The following formula weights each coordinate position of the fused filter response by exponential weight: ； In the above formula, R D represents the weighted fusion filter response of the corresponding region of interest, which is defined as the three-dimensional enhanced correlation filter response; n is the total number of pixels in the corresponding region; d i represents the depth difference of each pixel in the depth difference map; β represents the sensitivity of the preset depth mask to the depth difference.

5. The method for detecting the quality of parts assembly in a narrow space based on three-dimensional enhanced correlation filtering according to claim 4 is characterized in that: In step 5, whether the target part is occluded is determined based on the fusion correlation filter response: the low threshold of the fusion correlation filter response peak is preset to T1, and the high threshold is preset to T2; if the peak of the fusion correlation filter response in the area of ​​interest is outside the two thresholds T1 and T2, the corresponding component is classified as unoccluded; otherwise, the part corresponding to the area is judged to be occluded.

6. The method for detecting the quality of parts assembly in a narrow space based on three-dimensional enhanced correlation filtering according to claim 5, characterized in that: Step six specifically includes: S61: Divide the ROI determined to be blocked into n×n grids, where n is an integer greater than or equal to 1; each grid is defined as a sub-region of the ROI; and local correlation filters corresponding to multiple feature types are deployed on each sub-region; S62: Obtain the template image in step 1, extract the region of interest determined to be blocked and divide it into corresponding sub-regions as training samples of the sub-regions; use the training samples of each sub-region to train based on step 2 and obtain the local correlation filter response of the image to be detected; S63: Based on steps 3 and 4, the fusion correlation filter response and the three-dimensional enhanced correlation filter response of the sub-region are obtained; 2 The one with the highest response peak among the local three-dimensional enhanced correlation filter responses is taken as the final response.

Images