Visual feature detection matching method and system based on directional rotation

Abstract

The invention discloses a visual feature detection matching method and system based on directional rotation, and belongs to the technical field of image detection.The method comprises the steps that an original image and a template image are subjected to image processing to obtain corresponding BRIEF descriptors, and the BRIEF descriptors of the original image and the BRIEF descriptors of the template image are extracted according to the BRIEF descriptors of the original image and the BRIEF descriptors of the template image; performing violent matching on the original image and the template image, establishing a homography matrix according to a matching pair, performing spatial position correction on the image, enabling the original image and the template image to rotate to the same angle and position, performing feature matching on the corrected original image and the template image, and evaluating the matching similarity of the original image and the template image; detecting the target according to the evaluation result; according to the method, through saliency detection, only a foreground part is reserved, it can be ensured that detected key points are more significant, and calculation complexity is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a visual feature detection and matching method and system based on directional rotation. Background Art

[0002] In industrial production, the quality of the electronic product assembly process directly affects the performance and reliability of the final product. Traditional manual inspection methods are not only time-consuming and labor-intensive, but also prone to false detection or missed detection due to subjective judgment differences. In widely used visual inspection systems, although deep learning models have strong performance, they often face high computing costs and long development processes in actual deployment. Faced with the characteristics of multiple varieties and rapid adjustments of electronic product production lines, they have low migration capabilities. In contrast, traditional machine vision technology still has an advantage in many industrial inspection tasks due to its excellent universality and stability.

[0003] Image matching technology has been widely used as a traditional visual solution in industrial production scenarios. By finding similar parts between two or more images to determine the same content or structure between the images, a series of problems such as target detection and positioning can be achieved. Common image matching methods include region-based matching and manual feature-based matching, such as scale-invariant feature transform (SIFT) and ORB. Among them, the ORB feature algorithm is widely used for its fast, accurate and high robustness. By designing a visual inspection algorithm based on the ORB algorithm, the identification and positioning of electronic product parts can be achieved in industrial production, ensuring that each component is installed in the correct order and position, or used for product quality inspection in automated production lines.

[0004] The oriented rotation (ORB) feature algorithm is a visual feature detection and matching technology. In the detection of electronic product assembly processes, various incompatibilities may occur due to environmental factors. First, in complex backgrounds, the background interference may have a greater impact on the matching accuracy and speed, resulting in more mismatches or fewer correct matches. Second, the distribution of feature points in the image may be uneven, especially in areas with obvious textures, which will extract a large number of invalid features and ignore important features in other areas. This local concentration of feature points will lead to feature redundancy, thereby losing the overall feature information of the image, affecting the comprehensiveness and accuracy of the match; third, the sensitivity to space. Although the ORB algorithm has optimized rotation, it still performs poorly when processing large-angle rotations, and complex environments are easily affected by spatial position and angle. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a visual feature detection and matching method and system based on directional rotation, which improves the accuracy of target key point detection: when the ORB algorithm detects key points, if the image contains a large amount of irrelevant background information, it may cause many useless key points. Through saliency detection, only the foreground part is retained, which can ensure that the detected key points are more meaningful and reduce the computational complexity.

[0006] The present invention is achieved through the following technical solutions: In a first aspect, the present application proposes a visual feature detection and matching method based on directional rotation, comprising the following steps: Step 1: Perform image processing on the original image and the template image to obtain the corresponding BRIEF descriptor. The image processing process is as follows: The foreground image of the image is extracted by using saliency detection based on multi-scale and edge information comparison, the frequency of each gray level of the foreground image is determined and a distribution mapping function is created, and the gray distribution of the image is mapped to a new gray distribution to obtain a salient image. Downsample the significant image and construct an image pyramid to obtain layers of different resolutions. Divide the image of each layer into grids, calculate the current grid threshold according to the grayscale density in each grid to extract Fast feature points, and manage the feature distribution according to the quadtree technology to achieve uniform sampling, and obtain the optimal feature points with uniform distribution of the image; According to the optimal feature points of the original image and the optimal feature points of the template image, the corresponding BRIEF descriptors are established respectively; Step 2: According to the BRIEF descriptor of the original image and the BRIEF descriptor of the template image, the original image and the template image are brute-force matched. According to the matching pairs, a homography matrix is ​​established to correct the spatial position of the image, so that the original image and the template image are rotated to the same angle and position, and the features of the corrected original image and template image are matched. The matching similarity between the original image and the template image is evaluated, and the target is detected according to the evaluation results.

[0007] Preferably, the method of extracting a foreground image of an image by saliency detection using multi-scale and edge information comparison includes: The Gaussian pyramid is used to perform multiple smoothing and downsampling of the image to generate an image enhancement space with decreasing resolution. The saliency map of each scale image in the image enhancement space is determined, and the saliency map is converted into color space. The difference between the channel mean and the Euclidean distance before the channel conversion is calculated and combined with the edge information of the image. The gradient amplitude and direction of each pixel are calculated and weightedly fused with the saliency value of the saliency map to obtain a saliency map with enhanced edge information. The saliency maps of different scales are fused to obtain the foreground map.

[0008] Preferably, an image saliency detection algorithm based on improved frequency domain analysis is used to calculate the saliency map of each scale image.

[0009] Preferably, the significant image is downsampled and an image pyramid is constructed to obtain layers of different resolutions, the image of each layer is divided into grids, the current grid threshold is calculated according to the grayscale density in each grid to extract Fast feature points, and the feature distribution is managed according to the quadtree technology to achieve uniform sampling, so as to obtain the optimal feature points of uniform distribution of the image; Determination of optimal feature points uniformly distributed in the image includes: Each layer of the image is divided into grids, and the Fast point detection threshold of each grid is dynamically adjusted according to the sparsity of the feature points of each grid to obtain the feature points of the image. The image after obtaining the feature points is managed using a quadtree strategy to recursively divide the image, and the optimal feature points are selected according to the divided image to obtain evenly distributed optimal feature points.

[0010] Preferably, the method for brute force matching of the original image and the template image includes: Taking the feature point as the center, several pixel blocks are randomly selected in the set field, the grayscale mean in the pixel block is calculated, and the grayscale difference between any two different pixels is calculated to obtain the descriptor of the pixel block. Two grid pairs are randomly selected as a group of pixel block descriptors for comparison. When the similarity difference is less than the set value, a 1-dimensional encoding is obtained, and the pixel blocks are traversed in a loop to obtain a 256-dimensional binary descriptor of the feature point. The Hamming distance of the comparison descriptor is traversed to match and obtain a matching pair of brute force matching.

[0011] Preferably, the GMS algorithm is used to screen the matching pairs according to the motion statistics, remove the wrong matches, and retain the stable matching pairs.

[0012] Preferably, the method for spatial position correction includes: A homography matrix is ​​established based on the screened matching pairs to describe the projection relationship between the images. The homography matrix is ​​used to perform direction correction on the image, and the objects in the image are rotated to the same angle and coordinate system to complete the spatial position correction between the original image and the template image.

[0013] In the second aspect, the present application proposes a visual feature detection and matching system based on directional rotation, comprising: The image processing module is used to process the original image and the template image to obtain the corresponding BRIEF descriptor. The image processing process is as follows: The foreground image of the image is extracted by using saliency detection based on multi-scale and edge information comparison, the frequency of each gray level of the foreground image is determined and a distribution mapping function is created, and the gray distribution of the image is mapped to a new gray distribution to obtain a salient image. Downsample the significant image and construct an image pyramid to obtain layers of different resolutions. Divide the image of each layer into grids, calculate the current grid threshold according to the grayscale density in each grid to extract Fast feature points, and manage the feature distribution according to the quadtree technology to achieve uniform sampling, and obtain the optimal feature points with uniform distribution of the image; According to the optimal feature points of the original image and the optimal feature points of the template image, the corresponding BRIEF descriptors are established respectively; The matching detection module is used to perform brute force matching between the original image and the template image based on the BRIEF descriptor of the original image and the BRIEF descriptor of the template image, establish a homography matrix based on the matching pair to correct the spatial position of the image, rotate the original image and the template image to the same angle and position, perform feature matching on the corrected original image and template image, evaluate the matching similarity between the original image and the template image, and detect the target based on the evaluation results.

[0014] In a third aspect, the present application proposes a visual inspection method for an electronic product assembly process, comprising the following steps: Collect standard product images corresponding to each key step in the electronic product assembly process, as well as the original images after assembly; The visual feature detection and matching method based on directional rotation is used to perform matching detection on the standard product image and the original image.

[0015] In a fourth aspect, the present application proposes a visual inspection system for an electronic product assembly process, comprising a control unit and an image acquisition unit; The control unit is pre-installed with a computer program of a visual inspection method for an electronic product assembly process, the control unit is connected to a host computer, and the image acquisition unit is used to acquire images of key steps; When the program is running, the electronic product organization process visual inspection method is executed to inspect the electronic product assembly process.

[0016] Compared with the prior art, the present invention has the following beneficial technical effects: The present application provides a visual feature detection and matching method based on directional rotation. When the ORB algorithm detects key points, if the image contains a large amount of irrelevant background information, it may lead to many useless key points. Through saliency detection, only the foreground part is retained, which can ensure that the detected key points are more meaningful, reduce the computational complexity, and improve the accuracy of target key point detection; secondly, the adaptive threshold can automatically adjust the Fast detection threshold according to the local contrast of the image, so as to extract more stable and reliable feature points under different brightness and contrast conditions. This method can adapt to various lighting changes and scene complexity. And the application of the quadtree strategy in FAST feature point detection can ensure that the feature points are evenly distributed over the entire image, avoiding the concentration of feature points in a few high-contrast areas. The improved BRIEF algorithm is adopted, and the pixel block form is used to improve the anti-noise interference ability, and the description of grayscale difference is added to further improve the target detection accuracy; in addition, in the case of unregistration, the same object in the image may appear in different positions or angles due to differences in viewing angle, rotation or scaling. After registration, the same objects in the image will be aligned in space and have more similar geometric structures. The ORB algorithm can more effectively use this structural consistency to match feature points, effectively reducing spatial coupling to improve matching accuracy and stability, and improve the anti-interference ability of target detection.

[0017] The present application also proposes a visual feature detection and matching system based on directional rotation, an electronic device and a computer storage medium, which have all the advantages of the above-mentioned visual feature detection and matching method based on directional rotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A flow chart of a visual inspection method of the present invention; Figure 2 Flow chart of image processing of the present invention Figure 3 Flowchart of feature extraction of the present invention Figure 4 This is a flow chart of feature matching of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0022] A visual feature detection and matching method based on directional rotation comprises the following steps: Step 1: Perform image processing on the original image and the template image to obtain the corresponding BRIEF descriptor. The image processing process is as follows: S1.1. Use multi-scale and edge information comparison saliency detection to extract the foreground image of the image, determine the frequency of each gray level of the foreground image, and create a distribution mapping function to map the gray distribution of the image to a new gray distribution to obtain a salient image.

[0023] The foreground is extracted from the original image and the template image through multi-scale and edge information comparison saliency detection to reduce background redundant information. The obtained foreground is used as input and weighted summed according to the RGB channels to obtain a grayscale image. Finally, the obtained image is further enhanced according to the grayscale histogram equalization as the input for the next level of processing.

[0024] like Figure 2 As shown in the figure, through a saliency detection scheme with multi-scale and local information comparison, the original image is smoothed and downsampled multiple times using a Gaussian pyramid to generate an image enhancement space with decreasing resolution; The image of each layer in the image enhancement space is respectively calculated using the improved frequency domain analysis image saliency detection (FT) algorithm to calculate the saliency map of each scale image, and the RGB space image of the saliency map is converted into the color-opponent (LAB) color space. The difference between the channel mean and the Euclidean distance before the channel conversion is calculated, and combined with the edge information, the Sobel operator is used to calculate the gradient amplitude and direction of each pixel and weightedly fused with the saliency value of the saliency map to further enhance the edge information of the saliency detection and enhance the extracted contour. Finally, the saliency maps of different scales are fused, binarized to extract the saliency area contour and finally generate a saliency foreground map.

[0025] The frequency of each gray level in the statistically significant foreground image is calculated, and a distribution mapping function is created to map the gray distribution of the original image to the new gray distribution to obtain a new significant image, thereby improving the image contrast and strengthening the subsequent feature extraction capabilities.

[0026] S1.2. Downsample the significant image and construct an image pyramid to obtain layers of different resolutions. Divide the image of each layer into grids, calculate the current grid threshold according to the grayscale density in each grid to extract Fast feature points, and manage the feature distribution according to the quadtree technology to achieve uniform sampling, and obtain the optimal feature points of uniform distribution of the image.

[0027] like Figure 3 As shown in the figure, the detection process scheme based on the ORB algorithm is used. First, a scale pyramid is constructed to generate a series of layers with decreasing resolution for the significant image. The improved FAST is used to extract feature points, and each layer of the image is divided into a grid. Each layer is divided into 20*20 grids. The grayscale density statistics of each grid are calculated to reflect the local feature level of the image. The Fast point detection threshold of each grid is dynamically adjusted. In the area where the grayscale changes drastically, the features are dense and the detection threshold is increased. In the area where the feature points are sparse, the grayscale changes and the feature points are sparse. The detection threshold is lowered to increase the detection sensitivity to ensure that enough feature points are detected.

[0028] After obtaining the feature points, the image is further managed using a quadtree strategy, the image is recursively divided into four parts, the optimal feature points are selected from the N subspaces after division, and the optimal feature points are evenly distributed.

[0029] S1.3. According to the optimal feature points of the original image and the optimal feature points of the template image, corresponding BRIEF descriptors are established respectively.

[0030] Step 2: According to the BRIEF descriptor of the original image and the BRIEF descriptor of the template image, the original image and the template image are brute-force matched, and the matching pairs obtained are screened by the fast robust feature matching (GMS) algorithm based on motion statistics. According to the screening results, a homography matrix is ​​established to correct the spatial position of the image, so that the original image and the template image are rotated to the same angle and position, and feature matching is performed on the corrected original image and template image.

[0031] like Figure 4The figure shows the feature matching flow chart. The improved BRIEF algorithm is used to describe the obtained features. Taking the feature point as the center, a neighborhood of a certain size is taken. 512 2*2 pixel blocks are randomly selected in the area, the grayscale mean in the pixel block is calculated, and the grayscale difference between any two different pixels is calculated to obtain a 7-dimensional descriptor for the pixel block. Two grid pairs are randomly selected as a group of compared pixel block descriptors. When the similarity difference is less than the set value, the result is 1, otherwise it is 0 as a 1-dimensional encoding. The pixel blocks are traversed in a loop to obtain a 256-dimensional binary descriptor for the feature points. The Hamming distance of the comparison descriptors is traversed to match and obtain a preliminary matching pair.

[0032] Use the GMS algorithm to filter matching pairs based on motion statistics, remove false matches, and retain stable matching pairs; According to the screened matching pairs, a homography matrix is ​​established to describe the projection relationship between the images. The homography matrix is ​​used to perform direction correction on the input image and rotate the objects in the image to the same angle and coordinate system. At this time, the template and the image to be detected have a high structural similarity. The above feature matching process is performed twice on the corrected image to obtain a more accurate and stable matching result.

[0033] Step 3: Evaluate the matching similarity between the original image and the template image, and detect the target based on the evaluation results.

[0034] The matching similarity is quantitatively evaluated. First, according to the difference in similarity scores between the template image and the correct image and the wrong image, an appropriate threshold is set as the scoring distinction standard. If the current node similarity exceeds the similarity scoring threshold, the target detection is qualified.

[0035] By constructing an evaluation function to quantify the matching similarity, let the number of remaining good matching pairs after screening be N, and the distance between the matching pairs be D. Then, the similarity ratio between the two images is calculated, and an appropriate threshold is set as the scoring distinction standard. When the score is higher than the set threshold,

[0036] The visual feature detection and matching method based on directional rotation of the present application extracts the foreground through multi-scale and saliency detection with edge gradient information comparison, and reduces the background redundant information. Secondly, on the basis of the ORB algorithm, in feature extraction, the adaptive feature extraction threshold is calculated by statistically analyzing the local grayscale information to replace the original global identical threshold, and a uniform feature distribution is obtained through the quadtree method. In feature matching, the improved BRIEF algorithm is used to describe and calculate the features, and the multi-level grayscale comparison of the region replaces the original grayscale comparison between the two points to reduce the influence of illumination and enhance the anti-interference ability. In addition, a two-level cascade detection scheme is adopted, a rough match is obtained through the first-level detection, and then the matching pairs are screened, and the images are aligned according to the matching results to obtain a more similar spatial structure, and then the aligned images are subjected to a second fine matching to obtain a more stable result.

[0037] The visual feature detection and matching method based on directional rotation in this application has the following advantages: 1. In the process of image detection, when a complex background with rich textures is introduced, the ORB algorithm may extract too many features concentrated in the invalid background area. When faced with the rich information of electronic products, this feature redundancy has a significant impact on the detection precision and accuracy. Although the background interference can be reduced by the traditional method of manually cutting out the background, this method still cannot completely avoid the introduction of background noise. This application introduces automatic detection technology to separate the foreground and background areas, and extracts the target contour by accurately identifying the target to reduce the influence of background factors to a greater extent, thereby improving the detection accuracy.

[0038] 2. In the feature extraction process of existing detection methods, the ORB algorithm uses a global unified threshold to extract FAST feature points, which makes the algorithm susceptible to changes in illumination, and for areas with weaker features, the algorithm's feature extraction capability is poor, which easily leads to the omission of local information. At the same time, for areas with stronger features, too many overlapping features will be extracted, resulting in excessive expression of local information. Since electronic products usually have more assembled components and materials and contain rich feature elements, the above factors lead to the phenomenon of feature clustering in the feature extraction process, which reduces detection efficiency and affects matching accuracy. This application adopts a regional adaptive threshold to replace the original global unified threshold to enhance feature extraction capabilities. On this basis, a quadtree management solution is used to evenly distribute feature distribution and improve global feature expression.

[0039] 3. When the existing ORB algorithm processes large-angle rotation in a complex environment, the ORB algorithm is easily affected by changes in spatial position and angle. This limitation is particularly prominent in practical applications, especially in scenes that require processing large-scale rotation. This application introduces an image registration link. After the first rough matching, the difference in rotation angle and coordinate position of the target in the two contrasting images is calculated through feature matching, and the images are registered to have similar spatial geometric structures, making the result expression more stable.

[0040] Example 1 A visual feature detection and matching system based on directional rotation, comprising: The image processing module is used to process the original image and the template image to obtain the corresponding BRIEF descriptor. The image processing process is as follows: The multi-scale and edge information comparison saliency detection is used to extract the foreground image of the image, determine the frequency of each gray level of the foreground image, and create a distribution mapping function to map the gray distribution of the image to a new gray distribution to obtain a salient image.

[0041] The significant image is downsampled and an image pyramid is constructed to obtain layers of different resolutions. The image of each layer is divided into grids. In each grid, the current grid threshold is calculated according to the grayscale density to extract Fast feature points. The feature distribution is managed according to the quadtree technology to achieve uniform sampling and obtain the optimal feature points of uniform distribution of the image.

[0042] According to the optimal feature points of the original image and the optimal feature points of the template image, the corresponding BRIEF descriptors are established respectively.

[0043] The image matching module performs brute force matching on the original image and the template image according to the BRIEF descriptor of the original image and the BRIEF descriptor of the template image, and screens the obtained matching pairs based on the fast and robust feature matching algorithm based on motion statistics. According to the screening results, the homography matrix is ​​established to correct the spatial position of the image, so that the original image and the template image are rotated to the same angle and position, and feature matching is performed on the corrected original image and template image.

[0044] The detection module is used to evaluate the matching similarity between the original image and the template image, and detect the target based on the evaluation results.

[0045] Example 2 A visual inspection method for an electronic product assembly process comprises the following steps: Step 1: Collect standard product images corresponding to each key step in the electronic product assembly process as template images; Key steps refer to the key process nodes in the assembly of electronic products. It should be noted that important process nodes are one or more core processes in the production process of the product.

[0046] Product image, which refers to the current image of the product after the key steps are completed; Template images refer to standard product images corresponding to key steps.

[0047] During the visual inspection process, a product image is captured after a key step is completed and then compared with the template image of the key step to determine whether the electronic product assembled at the key step meets the requirements.

[0048] Step 2: Perform image processing on each standard product image to obtain the BRIEF descriptor of the standard product image.

[0049] Step 3: During the detection process, obtain the original image after the current key step is completed, and use the same method as step 2 to process the image to obtain the BRIEF descriptor of the original image.

[0050] Step 4: According to the BRIEF descriptor of the original image and the BRIEF descriptor of the template image, the original image and the template image are brute-force matched, and the homography matrix is ​​established according to the matching pair to correct the spatial position of the image, so that the original image and the template image are rotated to the same angle and position, and the feature matching of the corrected original image and the template image is performed; Step 5: Evaluate the matching similarity between the original image and the template image. When the matching similarity is greater than the set threshold, it means that the product meets the requirements after the key step is assembled.

[0051] Step 6: Repeat steps 3-5 to test the next key step.

[0052] Example 3 A visual inspection system for an electronic product assembly process, comprising a control unit and an image acquisition unit; The program of the visual inspection method for the organization process of electronic products described in Example 1 is preset in the control unit. The control unit is connected to the host computer. The image acquisition unit is used to collect images of key steps. When the control unit is running, the visual inspection method for the organization process of electronic products is executed to inspect the assembly process of electronic products.

[0053] It should be noted that in the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each module is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another device, or some features can be ignored or not executed. The module described as a separate component may or may not be physically separated. The component displayed as a module may be a physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed in multiple different places. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0054] In addition, each module in each embodiment of the present invention may be integrated into a processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0055] An electronic device provided in an embodiment of the present application includes a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the steps of the visual feature detection and matching method based on directional rotation as described in any of the above embodiments are implemented.

[0056] Another electronic device provided in the embodiment of the present application may also include: an input port connected to the processor, used to transmit multimodal data collected by an external acquisition device to the processor; and a display unit connected to the processor, used to display the processing results of the processor to the outside world; a communication module connected to the processor, used to realize the communication between the electronic device and the outside world. The display unit can be a display panel, a laser scanning display, etc.; the communication mode adopted by the communication module includes but is not limited to mobile high-definition link technology (HML), universal serial bus (USB), high-definition multimedia interface (HDMI), wireless connection (including wireless fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, and communication technology based on IEEE802.11s).

[0057] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the visual feature detection and matching method based on directional rotation as described in any of the above embodiments are implemented.

[0058] For the description of the relevant parts of the visual feature detection and matching system based on directional rotation, the electronic device, and the computer-readable storage medium provided in the embodiments of the present application, please refer to the detailed description of the corresponding parts in the visual feature detection and matching method based on directional rotation provided in the embodiments of the present application, which will not be repeated here. In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0059] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A visual feature detection and matching method based on directional rotation, characterized in that: The following steps are involved: Step 1: Perform image processing on the original image and the template image to obtain the corresponding BRIEF descriptor. The image processing process is as follows: The foreground image of the image is extracted by using saliency detection based on multi-scale and edge information comparison, the frequency of each gray level of the foreground image is determined and a distribution mapping function is created, and the gray distribution of the image is mapped to a new gray distribution to obtain a salient image. Downsample the significant image and construct an image pyramid to obtain layers of different resolutions. Divide the image of each layer into grids, calculate the current grid threshold according to the grayscale density in each grid to extract Fast feature points, and manage the feature distribution according to the quadtree technology to achieve uniform sampling, and obtain the optimal feature points with uniform distribution of the image; According to the optimal feature points of the original image and the optimal feature points of the template image, the corresponding BRIEF descriptors are established respectively; Step 2: According to the BRIEF descriptor of the original image and the BRIEF descriptor of the template image, the original image and the template image are brute-force matched. According to the matching pairs, a homography matrix is ​​established to correct the spatial position of the image, so that the original image and the template image are rotated to the same angle and position, and the features of the corrected original image and template image are matched. The matching similarity between the original image and the template image is evaluated, and the target is detected according to the evaluation results.

2. The method for detecting and matching visual features based on directional rotation according to claim 1, characterized in that: The foreground image of the image extracted by saliency detection using multi-scale and edge information comparison includes: The Gaussian pyramid is used to perform multiple smoothing and downsampling of the image to generate an image enhancement space with decreasing resolution. The saliency map of each scale image in the image enhancement space is determined, and the saliency map is converted into color space. The difference between the channel mean and the Euclidean distance before the channel conversion is calculated and combined with the edge information of the image. The gradient amplitude and direction of each pixel are calculated and weightedly fused with the saliency value of the saliency map to obtain a saliency map with enhanced edge information. The saliency maps of different scales are fused to obtain the foreground map.

3. The method for detecting and matching visual features based on directional rotation according to claim 2, characterized in that: The image saliency detection algorithm based on improved frequency domain analysis is used to calculate the saliency map of each scale image.

4. The method for detecting and matching visual features based on directional rotation according to claim 1, characterized in that: Downsample the significant image and construct an image pyramid to obtain layers of different resolutions. Divide the image of each layer into grids, calculate the current grid threshold according to the grayscale density in each grid to extract Fast feature points, and manage the feature distribution according to the quadtree technology to achieve uniform sampling, and obtain the optimal feature points with uniform distribution of the image; Determining the optimal feature points uniformly distributed in the image includes: Each layer of the image is divided into grids, and the Fast point detection threshold of each grid is dynamically adjusted according to the sparsity of the feature points of each grid to obtain the feature points of the image. The image after obtaining the feature points is managed using a quadtree strategy to recursively divide the image, and the optimal feature points are selected according to the divided image to obtain evenly distributed optimal feature points.

5. The method for detecting and matching visual features based on directional rotation according to claim 1, characterized in that: The method for brute force matching of the original image and the template image includes: Taking the feature point as the center, several pixel blocks are randomly selected in the set field, the grayscale mean in the pixel block is calculated, and the grayscale difference between any two different pixels is calculated to obtain the descriptor of the pixel block. Two grid pairs are randomly selected as a group of pixel block descriptors for comparison. When the similarity difference is less than the set value, a 1-dimensional encoding is obtained, and the pixel blocks are traversed in a loop to obtain a 256-dimensional binary descriptor of the feature point. The Hamming distance of the comparison descriptor is traversed to match and obtain a matching pair of brute force matching.

6. The method for detecting and matching visual features based on directional rotation according to claim 1, characterized in that: The GMS algorithm is used to screen the matching pairs according to the motion statistics, eliminate the wrong matches, and retain the stable matching pairs.

7. The method for detecting and matching visual features based on directional rotation according to claim 1, characterized in that: The method for spatial position correction comprises: A homography matrix is ​​established based on the screened matching pairs to describe the projection relationship between the images. The homography matrix is ​​used to perform direction correction on the image, and the objects in the image are rotated to the same angle and coordinate system to complete the spatial position correction between the original image and the template image.

8. A visual feature detection and matching system based on directional rotation, characterized in that: include: The image processing module is used to process the original image and the template image to obtain the corresponding BRIEF descriptor. The image processing process is as follows: The foreground image of the image is extracted by using saliency detection based on multi-scale and edge information comparison, the frequency of each gray level of the foreground image is determined and a distribution mapping function is created, and the gray distribution of the image is mapped to a new gray distribution to obtain a salient image. Downsample the significant image and construct an image pyramid to obtain layers of different resolutions. Divide the image of each layer into grids, calculate the current grid threshold according to the grayscale density in each grid to extract Fast feature points, and manage the feature distribution according to the quadtree technology to achieve uniform sampling, and obtain the optimal feature points with uniform distribution of the image; According to the optimal feature points of the original image and the optimal feature points of the template image, the corresponding BRIEF descriptors are established respectively; The matching detection module is used to perform brute force matching between the original image and the template image based on the BRIEF descriptor of the original image and the BRIEF descriptor of the template image, establish a homography matrix based on the matching pair to correct the spatial position of the image, rotate the original image and the template image to the same angle and position, perform feature matching on the corrected original image and template image, evaluate the matching similarity between the original image and the template image, and detect the target based on the evaluation results.

9. A visual inspection method for an electronic product assembly process, characterized in that: The following steps are involved: Collect standard product images corresponding to each key step in the electronic product assembly process, as well as the original images after assembly; The standard product image and the original image are matched and detected by using the visual feature detection and matching method based on directional rotation as described in any one of claims 1 to 7.

10. A visual inspection system for electronic product assembly process, characterized in that: It includes a control unit and an image acquisition unit; The control unit is pre-installed with a computer program of a visual inspection method for an electronic product assembly process, the control unit is connected to a host computer, and the image acquisition unit is used to acquire images of key steps; When the program is running, the electronic product organization process visual inspection method is executed to inspect the electronic product assembly process.

Images