A Visual Feature Detection and Matching Method and System Based on Oriented Rotation

Abstract

This invention discloses a visual feature detection and matching method and system based on directional rotation, belonging to the field of image detection technology. The method performs image processing on the original image and template image to obtain corresponding BRIEF descriptors. Based on the BRIEF descriptors of the original image and the template image, a brute-force matching is performed on the original image and the template image. A homography matrix is ​​established based on the matching pairs to perform spatial position correction on the images, rotating the original image and the template image to the same angle and position. Feature matching is then performed on the corrected original image and the template image to evaluate the matching similarity. Based on the evaluation results, the target is detected. This method uses saliency detection, retaining only the foreground portion, which ensures that the detected key points are more meaningful and reduces computational complexity.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, specifically to a visual feature detection and matching method and system based on directional rotation. Background Technology

[0002] In industrial production, the quality of the electronic product assembly process directly affects the performance and reliability of the final product. Traditional manual inspection methods are not only time-consuming and labor-intensive, but also prone to false positives or false negatives due to subjective judgment differences. While deep learning models offer strong performance in widely used visual inspection systems, their practical deployment often faces high computational costs and lengthy development processes. Furthermore, they exhibit limited transferability when dealing with the diverse product range and rapid adjustments required on electronic product production lines. In contrast, traditional machine vision technology, due to its excellent versatility and stability, still holds an advantage in many industrial inspection tasks.

[0003] Image matching technology, as a traditional vision solution, has been widely used in industrial production scenarios. By finding similar parts between two or more images, it can determine the common content or structure between the images, enabling a series of problems such as target detection and localization. Common image matching methods include region-based matching and handcrafted feature-based matching, such as Scale Invariant Feature Transform (SIFT) and ORB. Among them, the ORB feature algorithm is widely used due to its speed, accuracy, and high robustness. By designing a visual inspection algorithm based on the ORB algorithm, it is possible to identify and locate electronic product parts in industrial production, ensuring that each component is installed in the correct order and position, or to perform product quality inspection in automated production lines.

[0004] Oriented Rotation (ORB) feature algorithm is a visual feature detection and matching technique. In the inspection of electronic product assembly processes, various incompatibilities can arise due to environmental factors. First, in complex backgrounds, background interference can significantly impact matching accuracy and speed, leading to more false matches or fewer correct matches. Second, the distribution of feature points in the image may be uneven, especially in areas with obvious textures, resulting in the extraction of many invalid features while ignoring important features in other areas. This local concentration of feature points leads to feature redundancy, resulting in the loss of overall image feature information and affecting the comprehensiveness and accuracy of the matching. Third, it is sensitive to spatial variations; although the ORB algorithm is optimized for rotation, it still performs poorly when handling large-angle rotations, and complex environments are easily affected by spatial position and angle. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a visual feature detection and matching method and system based on directional rotation. This method improves the accuracy of target key point detection: when detecting key points, if the image contains a lot of irrelevant background information, the ORB algorithm may result in many useless key points. By using saliency detection, only the foreground part is retained, which can ensure that the detected key points are more meaningful and reduce computational complexity.

[0006] This invention is achieved through the following technical solution:

[0007] Firstly, this application proposes a visual feature detection and matching method based on directional rotation, comprising the following steps:

[0008] Step 1: Perform image processing on the original image and template image to obtain the corresponding BRIEF descriptors. The image processing procedure is as follows:

[0009] The foreground image is extracted by saliency detection using multi-scale and edge information comparison. The frequency of each gray level in the foreground image is determined and a distribution mapping function is created to map the gray level distribution of the image to a new gray level distribution to obtain a salient image.

[0010] The salient image is downsampled and an image pyramid is constructed to obtain layers of different resolutions. The image of each layer is divided into grids. In each grid, the current grid threshold is calculated based on the gray density to extract Fast feature points. The feature distribution is managed using quadtree technology to achieve uniform sampling and obtain the optimal feature points with uniform distribution in the image.

[0011] Based on the optimal feature points of the original image and the optimal feature points of the template image, corresponding BRIEF descriptors are established respectively.

[0012] Step 2: Based on the BRIEF descriptors of the original image and the template image, perform brute-force matching on the original image and the template image. Based on the matching pairs, establish a homography matrix to perform spatial position correction on the images, so that the original image and the template image are rotated to the same angle and position. Perform feature matching on the corrected original image and the template image, evaluate the matching similarity between the original image and the template image, and detect the target based on the evaluation results.

[0013] Preferably, the foreground image extracted by saliency detection using multi-scale and edge information comparison includes:

[0014] The image is smoothed and downsampled multiple times using a Gaussian pyramid to generate an image enhancement space with decreasing resolution. The saliency map of each scale in the image enhancement space is determined. The saliency map is transformed into a color space. The difference between the channel mean and the Euclidean distance before the channel transformation is calculated and combined with the edge information of the image. The gradient magnitude and direction of each pixel are calculated and weighted and fused with the saliency value of the saliency map to obtain the saliency map with enhanced edge information. The saliency maps of different scales are fused to obtain the foreground map.

[0015] Preferably, an image saliency detection algorithm based on improved frequency domain analysis is used to calculate the saliency map for each scale of the image.

[0016] Preferably, the significant image is downsampled and an image pyramid is constructed to obtain layers of different resolutions. The image of each layer is divided into grids. In each grid, the current grid threshold is calculated based on the gray density to extract Fast feature points. The feature distribution is managed using quadtree technology to achieve uniform sampling and obtain the optimal feature points with uniform distribution in the image.

[0017] The determination of the optimal feature points uniformly distributed in the image includes:

[0018] Each layer of the image is divided into grids. The Fast point detection threshold of each grid is dynamically adjusted according to the sparsity of the feature points in each grid to obtain the feature points of the image. The image after obtaining the feature points is recursively divided using a quadtree management strategy. The optimal feature points are selected based on the divided image to obtain the optimal feature points with a uniform distribution.

[0019] Preferably, the brute-force matching method between the original image and the template image includes:

[0020] Centered on the feature point, several pixel blocks are randomly selected in the defined neighborhood. The average gray value within each pixel block is calculated, and the gray value difference between any two different pixels is calculated to obtain the pixel block descriptor. Two grid pairs are randomly selected as a set of comparison pixel block descriptors. When the similarity difference is less than a set value, a one-dimensional encoding is obtained. The pixel blocks are iterated through to obtain a 256-dimensional binary descriptor for the feature point. The Hamming distance of the comparison descriptors is used for matching to obtain a brute-force matching pair.

[0021] Preferably, the GMS algorithm is used to filter matching pairs based on motion statistics, eliminating incorrect matches and retaining stable matching pairs.

[0022] Preferably, the spatial position correction method includes:

[0023] Based on the selected matching pairs, a homography matrix is ​​established to describe the projection relationship between images. The homography matrix is ​​used to perform directional correction on the images, rotating the objects in the images to the same angle and coordinate system, thus completing the spatial position correction between the original image and the template image.

[0024] Secondly, this application proposes a visual feature detection and matching system based on directional rotation, comprising:

[0025] The image processing module is used to process the original image and the template image to obtain the corresponding BRIEF descriptors. The image processing process is as follows:

[0026] The foreground image is extracted by saliency detection using multi-scale and edge information comparison. The frequency of each gray level in the foreground image is determined and a distribution mapping function is created to map the gray level distribution of the image to a new gray level distribution to obtain a salient image.

[0027] The salient image is downsampled and an image pyramid is constructed to obtain layers of different resolutions. The image of each layer is divided into grids. In each grid, the current grid threshold is calculated based on the gray density to extract Fast feature points. The feature distribution is managed using quadtree technology to achieve uniform sampling and obtain the optimal feature points with uniform distribution in the image.

[0028] Based on the optimal feature points of the original image and the optimal feature points of the template image, corresponding BRIEF descriptors are established respectively.

[0029] The matching and detection module is used to perform brute-force matching on the original image and the template image based on the BRIEF descriptors of the original image and the template image. It then establishes a homography matrix based on the matching pairs to perform spatial position correction on the images, rotating the original image and the template image to the same angle and position. Finally, it performs feature matching on the corrected original image and the template image, evaluates the matching similarity between the original image and the template image, and detects the target based on the evaluation results.

[0030] Thirdly, this application proposes a visual inspection method for the assembly process of electronic products, including the following steps:

[0031] Collect standard product images corresponding to each key step in the electronic product assembly process, as well as the original images after assembly;

[0032] The aforementioned visual feature detection and matching method based on directional rotation is used to perform matching detection between the standard product image and the original image.

[0033] Fourthly, this application proposes a visual inspection system for the assembly process of electronic products, including a control unit and an image acquisition unit;

[0034] The control unit contains a computer program for a visual inspection method for electronic product assembly process. The control unit is connected to a host computer, and the image acquisition unit is used to acquire images of key steps.

[0035] When the program runs, it executes a visual inspection method for the electronic product assembly process to inspect the electronic product assembly process.

[0036] Compared with the prior art, the present invention has the following beneficial technical effects:

[0037] This application provides a visual feature detection and matching method based on directional rotation. When detecting keypoints, the ORB algorithm may result in many useless keypoints if the image contains a large amount of irrelevant background information. By using saliency detection and retaining only the foreground portion, the detected keypoints are ensured to be more meaningful, reducing computational complexity and improving the accuracy of target keypoint detection. Secondly, the adaptive threshold can automatically adjust the Fast detection threshold based on the local contrast of the image, thereby extracting more stable and reliable feature points under different brightness and contrast conditions. This method can adapt to various lighting changes and scene complexity. Furthermore, the application of the quadtree strategy in FAST feature point detection ensures that feature points are evenly distributed across the entire image, avoiding the concentration of feature points in a few high-contrast areas. An improved BRIEF algorithm is adopted, using pixel blocks to improve noise resistance, and the addition of grayscale difference descriptions further improves target detection accuracy. Additionally, in the absence of registration, the same object in the image may appear in different positions or angles due to differences in viewpoint, rotation, or scaling. After registration, identical objects in the image will be spatially aligned and have more similar geometric structures. The ORB algorithm can more effectively utilize this structural consistency for feature point matching, effectively reducing spatial coupling and thus improving the accuracy and stability of matching, and enhancing the anti-interference ability of target detection.

[0038] This application also proposes a visual feature detection and matching system based on directional rotation, an electronic device, and a computer storage medium, which possess all the advantages of the aforementioned visual feature detection and matching methods based on directional rotation. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of a visual inspection method according to the present invention;

[0041] Figure 2 The flowchart of the image processing of this invention

[0042] Figure 3 This is a flowchart of the feature extraction process of the present invention.

[0043] Figure 4 This is a flowchart of the feature matching process of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0045] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0046] A visual feature detection and matching method based on directional rotation includes the following steps:

[0047] Step 1: Perform image processing on the original image and template image to obtain the corresponding BRIEF descriptors. The image processing procedure is as follows:

[0048] S1.1. Use multi-scale and edge information comparison to extract the foreground image of the image for saliency detection, determine the frequency of each gray level of the foreground image, and create a distribution mapping function to map the gray level distribution of the image to a new gray level distribution to obtain the salient image.

[0049] The foreground is extracted from the original image and template image through multi-scale saliency detection with added edge information comparison, reducing background redundancy. The obtained foreground is used as input and weighted summation is performed on the RGB channels to obtain a grayscale image. Finally, the obtained image is further enhanced by grayscale histogram equalization and used as input for the next stage of processing.

[0050] like Figure 2 As shown, a saliency detection scheme with multi-scale and local information comparison is used, and Gaussian pyramids are used to smooth and downsample the original image multiple times to generate an image enhancement space with decreasing resolution.

[0051] For each layer of the image in the image enhancement space, the saliency map of each scale is calculated using the improved frequency domain analysis image saliency detection (FT) algorithm. The RGB space image of the saliency map is converted to the color-adverse (LAB) color space. The difference between the channel mean and the Euclidean distance before the channel conversion is calculated. Combined with edge information, the gradient magnitude and direction of each pixel are calculated by the Sobel operator and weighted and fused with the saliency value of the saliency map to further enhance the edge information of saliency detection and enhance the extraction of contours. Finally, the saliency maps of different scales are fused, binarized to extract the contours of salient regions, and finally a salient foreground map is generated.

[0052] The statistical saliency foreground image is plotted for each gray level frequency, and a distribution mapping function is created to map the gray level distribution of the original image to a new gray level distribution, resulting in a new salient image. This improves image contrast and enhances subsequent feature extraction capabilities.

[0053] S1.2. Downsample the significant image and construct an image pyramid to obtain layers of different resolutions. Divide the image of each layer into a grid. Calculate the current grid threshold based on the gray density in each grid to extract Fast feature points. Use quadtree technology to manage the feature distribution to achieve uniform sampling and obtain the optimal feature points with uniform distribution of the image.

[0054] like Figure 3 As shown, the detection process based on the ORB algorithm first constructs a scale pyramid, generating a series of layers with decreasing resolution from the salient image. An improved FAST is used to extract feature points. Each image layer is divided into a grid of 20*20 pixels. The gray-level density statistics of each grid reflect the local feature level of the image. The detection threshold of each grid is dynamically adjusted. In areas with drastic gray-level changes and dense features, the detection threshold is increased. In areas with sparse feature points and relatively sparse gray-level changes, the detection threshold is decreased to increase detection sensitivity and ensure that enough feature points are detected.

[0055] After obtaining the feature points, the image is further divided into four parts using a quadtree management strategy. The optimal feature points are selected from the N subspaces after the division to obtain the optimal feature points with a uniform distribution.

[0056] S1.3. Based on the optimal feature points of the original image and the optimal feature points of the template image, establish corresponding BRIEF descriptors respectively.

[0057] Step 2: Based on the BRIEF descriptors of the original image and the template image, perform brute-force matching on the original image and the template image. Use the Fast Robust Feature Matching (GMS) algorithm based on motion statistics to filter the obtained matching pairs. Based on the filtering results, establish a homography matrix to perform spatial position correction on the images, so that the original image and the template image are rotated to the same angle and position. Perform feature matching on the corrected original image and the template image.

[0058] like Figure 4 The diagram shows the feature matching flowchart. The obtained features are described using an improved BRIEF algorithm. Centered on the feature point, a neighborhood of a certain size is taken. Within this neighborhood, 512 2*2 pixel blocks are randomly selected. The gray-scale mean within each pixel block is calculated, and the gray-scale difference between any two different pixels is calculated, resulting in a 7-dimensional descriptor for the pixel block. Two grid pairs are randomly selected as a set of comparison pixel block descriptors. If the similarity difference is less than a set value, the result is 1; otherwise, it is 0, which is used as a 1-dimensional encoding. The pixel blocks are iterated through to obtain a 256-dimensional binary descriptor for the feature point. The Hamming distance of the comparison descriptors is used for matching to obtain preliminary matching pairs.

[0059] The GMS algorithm is used to filter matching pairs based on motion statistics, eliminating incorrect matches and retaining stable matching pairs;

[0060] Based on the selected matching pairs, a homography matrix is ​​established to describe the projection relationship between images. The homography matrix is ​​used to perform directional correction on the input image, rotating the objects in the image to the same angle and coordinate system. At this time, the template and the image to be detected have a high structural similarity. The above feature matching process is then performed on the corrected image to obtain more accurate and stable matching results.

[0061] Step 3: Evaluate the matching similarity between the original image and the template image, and detect the target based on the evaluation results.

[0062] To quantify the matching similarity, firstly, based on the difference in similarity scores between the template image and the correct and incorrect images, an appropriate threshold is set as the scoring distinction standard. If the similarity of the current node exceeds the similarity score threshold, the target detection is considered qualified.

[0063] The matching similarity is quantified and statistically analyzed by constructing an evaluation function. Let N be the number of good matching pairs remaining after screening, and D be the distance between matching pairs. Then, the similarity ratio between the two images is determined by setting an appropriate threshold as the scoring distinction standard. When the score is higher than the set threshold, the similarity ratio between the two images is determined by the score.

[0064] This application's visual feature detection and matching method based on directional rotation extracts the foreground through multi-scale saliency detection with the addition of edge gradient information comparison, reducing background redundancy. Secondly, based on the ORB algorithm, in feature extraction, an adaptive feature extraction threshold is calculated by statistically analyzing local grayscale information, replacing the original globally uniform threshold, and a uniform feature distribution is obtained through a quadtree method. In feature matching, an improved BRIEF algorithm is used to describe and calculate features, replacing the original two-point grayscale comparison with multi-level grayscale comparison of regions, reducing the influence of illumination and enhancing anti-interference ability. Furthermore, a two-stage cascaded detection scheme is adopted. A coarse match is obtained through primary detection, then matching pairs are selected, and the images are registered based on the matching results to obtain a more similar spatial structure. A second fine match is then performed on the registered images, resulting in more stable results.

[0065] The visual feature detection and matching method based on directional rotation proposed in this application has the following advantages:

[0066] 1. In image detection, when a complex background with rich texture is introduced, the ORB algorithm may excessively extract features concentrated in invalid background areas. This feature redundancy significantly impacts detection accuracy and precision when dealing with the rich information in electronic products. Although traditional manual background cropping can reduce background interference, this method cannot completely eliminate the introduction of background noise. This application introduces automatic detection technology to separate foreground and background regions, accurately identifying the target and extracting its contour to further reduce the influence of background factors and improve detection accuracy.

[0067] 2. Existing detection methods, such as the ORB algorithm, use a globally uniform threshold to extract FAST feature points during feature extraction. This makes the algorithm susceptible to changes in lighting conditions and results in poor feature extraction capabilities in areas with weak features, easily leading to the omission of local information. Furthermore, in areas with strong features, it extracts too many overlapping features, leading to over-representation of local information. Since electronic products typically consist of numerous components and materials, containing a wealth of feature elements, these factors contribute to feature clustering during feature extraction, reducing detection efficiency and affecting matching accuracy. This application replaces the original globally uniform threshold with a region-adaptive threshold to enhance feature extraction capabilities. Furthermore, a quadtree management scheme is employed to evenly distribute feature distribution, improving global feature representation.

[0068] 3. Existing ORB algorithms are susceptible to changes in spatial position and angle when handling large-angle rotations in complex environments. This limitation is particularly pronounced in practical applications, especially in scenarios requiring large-scale rotations. This application introduces an image registration step. After the initial coarse matching, feature matching is used to calculate the difference in rotation angle and coordinate position of the target in two comparison images, thereby registering the images to have similar spatial geometric structures and resulting in more stable representations.

[0069] Example 1

[0070] A visual feature detection and matching system based on directional rotation, comprising:

[0071] The image processing module is used to process the original image and the template image to obtain the corresponding BRIEF descriptors. The image processing process is as follows:

[0072] The foreground image is extracted by saliency detection using multi-scale and edge information comparison. The frequency of each gray level in the foreground image is determined, and a distribution mapping function is created to map the gray level distribution of the image to a new gray level distribution to obtain a salient image.

[0073] The salient image is downsampled and an image pyramid is constructed to obtain layers of different resolutions. The image of each layer is divided into grids. In each grid, the current grid threshold is calculated based on the gray density to extract Fast feature points. The feature distribution is managed using quadtree technology to achieve uniform sampling and obtain the optimal feature points with uniform distribution of the image.

[0074] Based on the optimal feature points of the original image and the optimal feature points of the template image, corresponding BRIEF descriptors are established respectively.

[0075] The image matching module performs brute-force matching on the original image and the template image based on the BRIEF descriptors of the original image and the template image. It then filters the obtained matching pairs using a fast and robust feature matching algorithm based on motion statistics. Based on the filtering results, it establishes a homography matrix to perform spatial position correction on the images, so that the original image and the template image are rotated to the same angle and position. Finally, it performs feature matching on the corrected original image and the template image.

[0076] The detection module is used to evaluate the matching similarity between the original image and the template image, and to detect the target based on the evaluation results.

[0077] Example 2

[0078] A visual inspection method for electronic product assembly process includes the following steps:

[0079] Step 1: Collect standard product images corresponding to each key step in the electronic product assembly process, and use them as template images;

[0080] Key steps refer to the important process nodes in the assembly of electronic products. It should be noted that important process nodes are one or more core processes in the product's production process.

[0081] Product image refers to the current image of the product after key steps have been completed;

[0082] Template images refer to standard product images corresponding to key steps.

[0083] During the visual inspection process, product images are captured after key steps are completed and then compared with template images of the key steps to determine whether the electronic products assembled in the key steps meet the requirements.

[0084] Step 2: Perform image processing on each standard product image to obtain the BRIEF descriptor of the standard product image.

[0085] Step 3: During the detection process, obtain the original image after the completion of the current key step, and perform image processing using the same method as in Step 2 to obtain the BRIEF descriptor of the original image.

[0086] Step 4: Based on the BRIEF descriptors of the original image and the template image, perform brute-force matching on the original image and the template image. Based on the matching pairs, establish a homography matrix to perform spatial position correction on the images, so that the original image and the template image are rotated to the same angle and position. Perform feature matching on the corrected original image and the template image.

[0087] Step 5: Evaluate the matching similarity between the original image and the template image. If the matching similarity is greater than the set threshold, it means that the product meets the requirements after this key step of assembly.

[0088] Step 6: Repeat steps 3-5 to test the next critical step.

[0089] Example 3

[0090] A visual inspection system for electronic product assembly process includes a control unit and an image acquisition unit;

[0091] The control unit is pre-programmed with the visual inspection method for the electronic product assembly process described in Example 1. The control unit is connected to the host computer, and the image acquisition unit is used to acquire images of key steps. When the control unit is running, it executes the visual inspection method for the electronic product assembly process to inspect the electronic product assembly process.

[0092] It should be noted that, in the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another device, or some features may be ignored or not executed. The modules described as separate components may or may not be physically separated. The components shown as modules may be one or more physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs.

[0093] Furthermore, in the various embodiments of the present invention, the modules can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional unit.

[0094] An electronic device provided in this application includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the visual feature detection and matching method based on directional rotation as described in any of the above embodiments.

[0095] Another electronic device provided in this application embodiment may further include: an input port connected to a processor for transmitting multimodal data collected by an external acquisition device to the processor; a display unit connected to the processor for displaying the processor's processing results to the outside world; and a communication module connected to the processor for enabling communication between the electronic device and the outside world. The display unit may be a display panel, a laser scanning display, etc.; the communication method adopted by the communication module includes, but is not limited to, Mobile High Definition Link (HML), Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), and wireless connection (including Wi-Fi, Bluetooth, Bluetooth Low Energy, and IEEE 802.11s-based communication technology).

[0096] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the visual feature detection and matching method based on directional rotation as described in any of the above embodiments.

[0097] For descriptions of relevant parts in the directional rotation-based visual feature detection and matching system, electronic device, and computer-readable storage medium provided in the embodiments of this application, please refer to the detailed descriptions of the corresponding parts in the directional rotation-based visual feature detection and matching method provided in the embodiments of this application, which will not be repeated here. Furthermore, parts of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0098] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A method for matching visual feature detection based on directed rotation, characterized in that, The method comprises the following steps: Step 1, image processing is performed on the original image and the template image to obtain corresponding BRIEF descriptors, and the image processing process is as follows: The foreground image of the image is extracted by using the multi-scale and edge information comparison based saliency detection, the frequency of each gray level of the foreground image is determined, and a distribution mapping function is created to map the gray distribution of the image to a new gray distribution to obtain a saliency image; The foreground image of the image is extracted by using the multi-scale and edge information comparison based saliency detection, and the foreground image of the image is extracted by using the multi-scale and edge information comparison based saliency detection. The image is smoothed and down-sampled multiple times using a Gaussian pyramid to generate an image enhancement space with decreasing resolution, the saliency image of each scale image in the image enhancement space is determined, the color space of the saliency image is converted, the difference between the channel mean value and the Euclidean distance before conversion is calculated, and the edge information of the image is combined, the gradient amplitude and direction of each pixel are calculated, and the saliency value of the saliency image is weighted and fused to obtain an edge information enhanced saliency image, and the saliency images of different scales are fused to obtain a foreground image; The saliency image is down-sampled and an image pyramid is constructed to obtain different resolution layers, the image of each layer is divided into grids, the Fast feature points are extracted in each grid according to the gray density, and the feature distribution is managed according to the quadtree technology to realize uniform sampling, and the optimal feature points of the image are obtained. The optimal feature points of the original image and the optimal feature points of the template image are used to establish corresponding BRIEF descriptors. Step 2, the original image and the template image are matched by using the BRIEF descriptors of the original image and the BRIEF descriptors of the template image, a homography matrix is established according to the matching pair to correct the spatial position of the image, the original image and the template image are rotated to the same angle and position, the corrected original image and the template image are matched, the matching similarity of the original image and the template image is evaluated, and the target is detected according to the evaluation result.

2. The method of claim 1, wherein, An image saliency detection algorithm based on improved frequency domain analysis is used to calculate the saliency image of each scale image.

3. The method of claim 1, wherein, The saliency image is down-sampled and an image pyramid is constructed to obtain different resolution layers, the image of each layer is divided into grids, the Fast feature points are extracted in each grid according to the gray density, and the feature distribution is managed according to the quadtree technology to realize uniform sampling, and the optimal feature points of the image are obtained. The determination of the optimal feature points of the image includes: The image is divided into grids, the Fast point detection threshold of each grid is dynamically adjusted according to the feature point sparsity of each grid, the feature points of the image are obtained, the quadtree management strategy is used for the image after the feature points are obtained, the image is recursively divided, the optimal feature points are selected according to the divided image, and the optimal feature points of the image are obtained.

4. The method of claim 1, wherein, The original image and the template image are matched by using the BRIEF descriptors of the original image and the BRIEF descriptors of the template image, a homography matrix is established according to the matching pair to correct the spatial position of the image, the original image and the template image are rotated to the same angle and position, the corrected original image and the template image are matched, the matching similarity of the original image and the template image is evaluated, and the target is detected according to the evaluation result. With the feature point as the center, a plurality of pixel blocks are randomly selected in a set field, the average gray value in the pixel block is calculated, and the gray value difference of any two different pixel points is calculated to obtain a descriptor of the pixel block. Two randomly selected grid pairs are taken as a group to compare the pixel block descriptors. When the similarity difference is less than a set value, a one-dimensional code is obtained. The pixel blocks are iterated to obtain a 256-dimensional binary descriptor of the feature point. The Hamming distance of the descriptors is iterated to obtain a matching pair of the brute force matching.

5. The method of claim 1, wherein, The GMS algorithm is used to screen the matching pairs according to motion statistics, to eliminate incorrect matching, and to retain stable matching pairs.

6. The method of claim 1, wherein, The method for correcting the spatial position comprises: A homography matrix is established according to the screened matching pairs, the projection relationship between the images is described, the homography matrix is used to correct the direction of the images, the objects in the images are rotated to the same angle and coordinate system, and the spatial position correction of the original image and the template image is completed.

7. A system for performing the method of directional rotation based visual feature detection matching of any of claims 1-6, characterized in that, It comprises: An image processing module is used to obtain corresponding BRIEF descriptors by processing the original image and the template image. The image processing process is as follows: A foreground image of the original image is extracted by using a multi-scale and edge information comparison significance detection, the frequency of each gray level of the foreground image is determined, and a distribution mapping function is created to map the gray distribution of the image to a new gray distribution to obtain a saliency image. The saliency image is down-sampled and a pyramid is constructed to obtain image layers with different resolutions. The image of each layer is divided into grids. Fast feature points are extracted in each grid according to the gray density and the threshold value of the current grid. The feature distribution is managed according to the quadtree technology to achieve uniform sampling, and the optimal feature points of the image are obtained. BRIEF descriptors are established according to the optimal feature points of the original image and the optimal feature points of the template image. A matching detection module is used to perform brute force matching on the original image and the template image according to the BRIEF descriptors of the original image and the template image. The spatial position of the images is corrected by establishing a homography matrix according to the matching pairs, so that the original image and the template image are rotated to the same angle and position. Feature matching is performed on the corrected original image and the template image, the matching similarity of the original image and the template image is evaluated, and the target is detected according to the evaluation result.

8. An electronic product assembly process vision inspection method, characterized in that, It comprises the following steps: Standard product images corresponding to each key step in the electronic product assembly process and an original image after assembly are collected. The standard product image and the original image are matched and detected by using the visual feature detection and matching method based on directional rotation according to any one of claims 1-6.

9. An electronic product assembly process vision inspection system, comprising: It comprises a control unit and an image acquisition unit. A computer program of the visual detection method for the electronic product assembly process is pre-stored in the control unit. The control unit is connected to the upper computer, and the image acquisition unit is used to collect the images of the key steps. When the program is running, the visual feature detection and matching method based on directional rotation according to any one of claims 1-6 is executed to detect the electronic product assembly process.

Images