A target recognition method, system, device and medium based on binocular vision

Abstract

This invention discloses a target recognition method, system, device, and medium based on binocular vision, relating to the field of target recognition technology. The method includes: performing grayscale processing on preprocessed left and right views; extracting multiple corresponding feature points from the left and right grayscale images obtained through grayscale processing using the FAST corner detection method; obtaining descriptors for each feature point using the BEBLID algorithm, and calculating the Hamming distance between each feature point in the left grayscale image and each feature point in the right grayscale image using the descriptors; generating a feature point group using at least two feature points whose Hamming distance does not exceed a Hamming distance threshold; removing feature points from the feature point group that do not satisfy a cosine similarity threshold to obtain feature point sets for the left and right views; reducing mismatches of feature points to reduce computational time complexity, thereby improving recognition speed and real-time performance of object recognition.

Description

Technical Field

[0001] This invention relates to the field of target recognition technology, and more specifically, to a target recognition method, system, device, and medium based on binocular vision. Background Technology

[0002] In binocular stereo vision, disparity maps can be obtained by considering the relationship between corresponding pixels in the left and right images. Then, three-dimensional information can be obtained by triangulation. Through a series of calculations, disparity information can be calculated to obtain depth information. Compared with methods such as LiDAR, binocular vision obtains depth information from two-dimensional images, which has the advantages of simple implementation and low cost. At present, it is widely used in machine vision and intelligent driving.

[0003] In many application scenarios, there are high requirements for image real-time performance and hardware cost. Therefore, how to quickly obtain relatively accurate processed images is the current mainstream development direction and trend. In existing binocular target recognition, when using conventional binocular recognition (non-neural network), the focus is on feature matching and reconstruction. In existing technologies, when performing real-time recognition (using ORB (Oriented FAST and Rotated BRIEF algorithm for feature point matching), the accuracy of feature point recognition is relatively low while ensuring recognition speed.

[0004] Therefore, this application is hereby submitted. Summary of the Invention

[0005] The purpose of this invention is to provide a target recognition method, system, device, and medium based on binocular vision. By reducing the mismatch of feature points, the computational time complexity is reduced, thereby improving the recognition speed and the real-time performance of object recognition. At the same time, it has low hardware requirements, does not require the use of ASIC chips such as NPU, and reduces the cost of use.

[0006] The above-mentioned technical objective of the present invention is achieved through the following technical solution:

[0007] Firstly, this application provides a target recognition method based on binocular vision, comprising the following specific steps:

[0008] Get the left and right views of the current scene at the same moment, and then perform grayscale processing on the preprocessed left and right views;

[0009] The FAST corner detection method is used to extract multiple feature points from the left and right grayscale images obtained through grayscale processing, respectively.

[0010] The BEBLID algorithm is used to obtain the descriptors of each feature point, and the Hamming distance between each feature point in the left grayscale image and each feature point in the right grayscale image is calculated using the descriptors.

[0011] Generate a set of feature points using at least two feature points whose Hamming distance does not exceed the Hamming distance threshold;

[0012] Based on the calculated cosine similarity threshold, feature points that do not meet the cosine similarity threshold are removed from the feature point group, resulting in the feature point group sets for the left and right views.

[0013] Based on the above technical solution, the present invention can be further improved as follows.

[0014] Furthermore, the above grayscale processing specifically involves:

[0015] ;

[0016] In the formula, Indicates the coordinates in the left or right view. The grayscale value of a pixel after grayscale processing. , , These represent the coordinates in the left or right view, respectively. The values ​​of the R, G, and B components of the pixel. The coordinates are respectively The weights of the R, G, and B components corresponding to each pixel.

[0017] Furthermore, the aforementioned feature points are extracted using the following methods:

[0018] For either the left or right grayscale image, construct the corresponding image pyramid;

[0019] Key points of each layer of the image pyramid are extracted using the Fast algorithm;

[0020] Calculate the gray-level centroid of each key point, determine the direction of the vector of the gray-level centroid as the direction of the corresponding key point, and determine the key point with the obtained direction as the feature point.

[0021] Furthermore, the key points of each layer of the image pyramid extracted using the Fast algorithm are as follows:

[0022] For any pixel in each layer of the image pyramid, a detection circle is formed with that pixel as the center and a radius of 3 pixels, and the pixel values ​​of 16 detection pixels located on the detection circle are obtained in a clockwise or counterclockwise direction.

[0023] Calculate the pixel difference between each detected pixel and the pixel point. If the absolute value of the pixel difference of n consecutive detected pixels is greater than a preset threshold, then the pixel point is determined as a key point.

[0024] Furthermore, the aforementioned cosine similarity threshold is obtained in the following way:

[0025] Multiple feature point groups are selected for cosine similarity calculation;

[0026] For the multiple cosine similarities obtained, calculate the weight of the same cosine similarity.

[0027] Based on each cosine similarity and its weight, calculate the expected value of multiple cosine similarities;

[0028] The cosine similarity threshold is obtained by calculating the expected value.

[0029] Furthermore, the aforementioned cosine similarity threshold is specifically as follows:

[0030] ,in:

[0031] ;

[0032] In the formula, The cosine similarity threshold is... As expected, These represent the preset parameter values. The number of each weight. The weight of the nth element. Let be the cosine similarity corresponding to the weight of the nth element.

[0033] Secondly, this application provides a target recognition system based on binocular vision, applied to any of the target recognition methods based on binocular vision in the first aspect, comprising:

[0034] The view acquisition module is used to acquire the left and right views of the current scene at the same moment, and to perform grayscale processing on the preprocessed left and right views.

[0035] The feature point extraction module is used to extract multiple corresponding feature points from the left and right grayscale images obtained through grayscale processing using the FAST corner detection method.

[0036] The Hamming distance calculation module is used to obtain the descriptors of each feature point using the BEBLID algorithm, and to calculate the Hamming distance between each feature point in the left grayscale image and each feature point in the right grayscale image using the descriptors;

[0037] The feature point group generation module is used to generate a feature point group from at least two feature points whose Hamming distance does not exceed the Hamming distance threshold.

[0038] The feature point filtering module is used to remove feature points in the feature point group that do not meet the cosine similarity threshold based on the calculated cosine similarity threshold, thereby obtaining the feature point group sets for the left and right views.

[0039] Furthermore, the aforementioned cosine similarity threshold is obtained in the following way:

[0040] Multiple feature point groups are selected for cosine similarity calculation;

[0041] For the multiple cosine similarities obtained, calculate the weight of the same cosine similarity.

[0042] Based on each cosine similarity and its weight, calculate the expected value of multiple cosine similarities;

[0043] The cosine similarity threshold is obtained by calculating the expected value.

[0044] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a target recognition method based on binocular vision as described in any of the first aspects.

[0045] Fourthly, this application provides a non-transitory computer-readable storage medium that stores computer instructions that cause a computer to execute any of the methods for target recognition based on binocular vision in the first aspect.

[0046] Compared with the prior art, the present invention has at least the following beneficial effects:

[0047] Compared with traditional binocular vision, this solution improves feature point matching accuracy and reduces computation, thereby further improving the real-time performance of target recognition in binocular vision. Compared with neural network-based binocular vision, this solution uses traditional binocular vision and does not require ASIC (Application Specific Integrated Circuits), thus reducing the cost of use.

[0048] The feature point purification is based on an improved ORB algorithm. Compared with the commonly used brute-force matching algorithm, which has a high time complexity, this scheme uses Hamming distance and cosine similarity to impose dual constraints on the determination of feature points. While ensuring matching accuracy, it reduces the amount of matching computation, thereby improving the speed of feature point matching. Attached Figure Description

[0049] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0050] Figure 1 This is a flowchart of the identification method in Embodiment 1 of the present invention;

[0051] Figure 2 This is a schematic diagram of the identification system in Embodiment 2 of the present invention;

[0052] Figure 3 This is a schematic diagram of the image acquisition device in Embodiment 2 of the present invention;

[0053] Figure 4 This is a schematic diagram showing the view before and after the view transformation in Embodiment 2 of the present invention;

[0054] Figure 5 This is a schematic diagram of corner detection in Embodiment 2 of the present invention;

[0055] Figure 6 This is a schematic diagram showing the display screen in Embodiment 2 of the present invention;

[0056] Figure 7 This is a schematic diagram of the connection of the identification system in Embodiment 3 of the present invention.

[0057] The attached diagram shows the markings and corresponding component names:

[0058] 100, Image acquisition device; 200, Image preprocessing device; 300, Target processing device; 400, Output device; 101, First image acquisition device; 102, Second image acquisition device. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0060] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0061] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0062] In the description of the embodiments of the present invention, "multiple" means at least two.

[0063] Example 1: To address the low accuracy of feature point recognition in current binocular target recognition methods, this example provides a target recognition method based on binocular vision, such as... Figure 1 As shown, the specific steps include the following:

[0064] S1: Obtain the left and right views of the current scene at the same moment, and perform grayscale processing on the preprocessed left and right views.

[0065] Specifically, the grayscale processing mentioned above is as follows:

[0066] ;

[0067] In the formula, Indicates the coordinates in the left or right view. The grayscale value of a pixel after grayscale processing. , , These represent the coordinates in the left or right view, respectively. The values ​​of the R, G, and B components of the pixel. The coordinates are respectively The weights of the R, G, and B components corresponding to each pixel.

[0068] S2 uses the FAST corner detection method to extract multiple feature points from the left and right grayscale images obtained through grayscale processing.

[0069] Optionally, the above feature points are extracted in the following ways:

[0070] S21. For either the left or right grayscale image, construct the corresponding image pyramid.

[0071] S22 uses the Fast algorithm to extract key points from each layer of the image pyramid.

[0072] Specifically, the extraction of key points for each layer of the image pyramid using the Fast algorithm is as follows:

[0073] S221, for any pixel in each layer of the image pyramid, a detection circle is formed with the pixel as the center and a radius of 3 pixels, and the pixel values ​​of 16 detection pixels located on the detection circle are obtained in a clockwise or counterclockwise direction.

[0074] S222, calculate the pixel difference between each detected pixel and the pixel point. If the absolute value of the pixel difference of n consecutive detected pixels is greater than the preset threshold, then the pixel point is determined as a key point.

[0075] S23, calculate the gray-level centroid of each key point, determine the vector direction of the gray-level centroid as the direction of the corresponding key point, and determine the key point with the obtained direction as the feature point.

[0076] S3 uses the BEBLID algorithm to obtain descriptors for each feature point, and calculates the Hamming distance between each feature point in the left grayscale image and each feature point in the right grayscale image using the descriptors.

[0077] S4, generate a feature point group using at least two feature points whose Hamming distance does not exceed the Hamming distance threshold.

[0078] S5. Based on the calculated cosine similarity threshold, feature points in the feature point group that do not meet the cosine similarity threshold are removed to obtain the feature point group sets of the left view and the right view.

[0079] Optionally, the above cosine similarity threshold is obtained in the following way:

[0080] S501, select multiple feature point groups to calculate cosine similarity.

[0081] S502, for the multiple cosine similarities obtained, calculate the weight of the same cosine similarity.

[0082] S503, based on each cosine similarity and its weight, calculates the expected value of multiple cosine similarities.

[0083] S504, the cosine similarity threshold is calculated using the expected value; wherein, the aforementioned cosine similarity threshold is specifically:

[0084] ,in:

[0085] ;

[0086] In the formula, The cosine similarity threshold is... As expected, These represent the preset parameter values. The number of each weight. The weight of the nth element. Let be the cosine similarity corresponding to the weight of the nth element.

[0087] Example 2: This example provides a target recognition system based on binocular vision, which can be used to execute the target recognition method based on binocular vision in Example 1. The system structure diagram of this system is as follows. Figure 2 As shown, it consists of an image acquisition device 100, an image preprocessing device 200, a target detection device 300, and an output device 400. The functions of each device are as follows:

[0088] Image acquisition device 100: The device includes a first image acquisition device 101 and a second image acquisition device 102; the first image acquisition device 101 and the second image acquisition device 102 are devices capable of real-time image acquisition, and the first image acquisition device 101 and the second image acquisition device 102 can acquire the current scene image.

[0089] Furthermore, the first image acquisition device 101 and the second image acquisition device 102 can be two cameras; the two cameras are located in different positions (the two cameras are parallel or the angle between the lens of the two cameras and the object being photographed is acute), including but not limited to the following: Figure 3 The schematic diagram of the image acquisition device shows two cases, Example 1 and Example 2. Figure 3 In Example 1 (or Example 2), the first image acquisition device 101 is a left camera and the second image acquisition device 102 is a right camera. The current scene is captured simultaneously by the left and right cameras, and the left view captured by the left camera and the right view captured by the right camera are obtained respectively. This solution recommends that the left and right views be RGB color images, but is not limited to RGB color images. They can also be black and white images or other input images.

[0090] Image preprocessing device 200: This device receives a first real-time image (the left or right view in Embodiment 1) acquired by the first image acquisition device 101 in the image acquisition device 100, and receives a second real-time image (the right or left view in Embodiment 1) acquired by the second image acquisition device 102 in the image acquisition device 100; the first real-time image acquired by the first image acquisition device 101 in the image acquisition device 100 and the second real-time image acquired by the second image acquisition device 102 in the image acquisition device 100 are images from the same time; the image preprocessing device 200 converts the obtained first real-time image and second real-time image into a preprocessed image through an internally preset processing method, and outputs it to the target processing device 300.

[0091] The specific preprocessing steps of the image preprocessing device 200 for the first real-time image output by the first image acquisition device 101 and the second real-time image output by the second image acquisition device 102 are as follows:

[0092] Step 210: Input the first real-time image (i.e., left view) acquired by the first image acquisition device 101 and the second real-time image (i.e., right view) acquired by the second image acquisition device.

[0093] Step 220: Based on the first image acquisition device 101 and the second image acquisition device 102 respectively, using their respective intrinsic parameters (focal length, etc.) and distortion coefficients (radial distortion caused by the shape of the lens itself on the propagation of light and tangential distortion caused by the lens and the imaging plane not being completely parallel due to manufacturing technology), single-target calibration is performed according to the preset chessboard calibration method to eliminate barrel distortion or pincushion distortion.

[0094] Specifically, after completing single-target calibration, dual-target calibration is performed on the first image acquisition device 101 and the second image acquisition device 102 to determine the rotation and translation relationship between the two camera coordinate systems of the first image acquisition device 101 and the second image acquisition device 102; based on the homography matrix obtained after calibrating the left view acquired by the first image acquisition device 101 and the right view acquired by the second image acquisition device 102, stereo correction is performed, and the image planes of the left and right views are reprojected onto the same plane with mutually parallel optical axes. A schematic diagram before and after the view transformation is shown below. Figure 4 As shown, the corresponding pixel points p in the left and right views are ensured to be in the same plane (that is, the imaging planes of the first image acquisition device 101 and the second image acquisition device 102 after conversion are on the same plane and are vertically aligned), which reduces the computational load of subsequent feature point retrieval and matching, thereby improving the real-time performance of target recognition.

[0095] Step 230: Process the left and right images, corrected in Step 220, into grayscale images. The processed grayscale image of the left view is called the left grayscale image. Similarly, the right grayscale image is obtained after processing the right view. This method converts the left and right views into grayscale images using a weighted approach based on the R, G, and B components. The grayscale value calculation formula for each pixel satisfies the following formula:

[0096] ;

[0097] In the formula, Indicates the coordinates in the left or right view. The grayscale value of a pixel after grayscale processing. , , These represent the coordinates in the left or right view, respectively. The values ​​of the R, G, and B components of the pixel. The coordinates are respectively The weights of the R, G, and B components corresponding to each pixel.

[0098] Step 240: Output the left grayscale image and the right grayscale image obtained in step 230 to the target processing device 300.

[0099] Target processing device 300: This device receives the preprocessed image obtained after processing by the image preprocessing device 200, obtains feature point information by processing the preprocessed image through Gaussian pyramid, feature point detection, stereo matching and other methods, performs depth information calculation based on target recognition 3D reconstruction, and then transmits the feature point information and depth information to the output device 400.

[0100] Step 240: Output the left grayscale image and the right grayscale image obtained in step 230 to the target processing device 300.

[0101] Although the image preprocessing device 200 calibrates and corrects the image using a preset calibration method, the left and right cameras are not in the same position. Therefore, the position of the same object in the image differs between the left and right views (or between the left and right grayscale images). As a result, the image calibration and correction performed by the image preprocessing device 200 can only ensure that the corresponding pixel p is located on the same horizontal line in the vertical plane, rather than at the same position in the image coordinate system. Therefore, the left and right grayscale images need to be processed again to match feature points and ensure the accuracy of target recognition.

[0102] Furthermore, the detailed process flow of the target processing device 300 in this solution is as follows:

[0103] Step 310: Input the left and right grayscale images processed by the image preprocessing device 200, and extract feature points from the left and right grayscale images respectively; for this step, the feature point extraction method can be FAST (Features from accelerated segment test) corner detection to extract feature points in the grayscale image; the FAST algorithm has a faster calculation speed than SIFT (Scale-invariant feature transform) and SURF (Speed-Up Robust Features), reflecting the real-time function of target recognition mentioned in this solution.

[0104] Step 311: In order to ensure the scale invariance of the binocular image under the FAST algorithm and improve the accuracy and precision of feature point matching in feature matching, the left and right grayscale images output by the image preprocessing device 200 are processed by image pyramid processing in this step. Specifically, Gaussian pyramid processing is adopted (the image size is continuously reduced by Gaussian blur filtering and downsampling, so that a set of images with multiple resolutions are contained in the pyramid).

[0105] In this process, the downsampling factor (the ratio of the resolution of the i-th layer to that of the (i+1)-th layer) is recommended to be set to 2 (i.e., an image of size P*Q is downsampled once to obtain an image of size (P / 2)*(Q / 2)). In this way, an N-layer image pyramid is obtained (N is usually set to 3-6, depending on the accuracy requirements, and this scheme does not impose any restrictions).

[0106] Step 312: Calculate the key points of the left and right grayscale images. Following the image pyramid construction in Step 311, the Fast algorithm is used to quickly extract key points from each layer of the image pyramid. The key point extraction method for each layer is corner detection, as illustrated below. Figure 5 As shown: Select an arbitrary pixel point p(x,y) as the center and a circle with a radius of 3 pixels for pixel detection. Select 16 pixels located on this circle for detection; the detection is as follows. Figure 5 The differences between the 16 pixel values ​​numbered 1 to 16 and the center pixel p(x,y) are shown. If the absolute value of the difference between n consecutive pixels (the setting value of n is inversely related to the detection speed; the smaller the value of n, the faster the detection speed; the setting value of n is positively related to the detection accuracy; the larger the value of n, the higher the accuracy; in this embodiment, n=12) and the center pixel p(x,y) is greater than the preset threshold h (a specific value of h is not recommended here; the smaller the preset threshold h, the higher the accuracy), then the pixel p(x,y) is a feature point and is added to the feature point set; otherwise, it is not a feature point.

[0107] Step 313: Calculate the orientation of feature points in the image; based on the feature points obtained in step 312, calculate the gray centroid of the neighborhood of the feature point (the gray centroid is the ratio of the sum of the products of the gray values ​​of all pixels and their corresponding positions in a region to the total gray values ​​of the pixels in the region, i.e., the position of the average gray value in the region), and define the vector direction from the feature point to the gray centroid as the principal direction of the feature point to ensure the rotation invariance of the image in stereo matching.

[0108] Step 320: Perform feature point matching. Before feature point matching, to improve the accuracy of the identified feature points, the descriptors of the feature points are first calculated in step 320. Based on the feature points obtained in step 310, the feature points are numerically described to give them good discriminative ability, thus making them suitable for image matching. The BEBLID (Boosted Efficient Binary Local Image Descriptor) algorithm can be used to describe the feature points obtained in step 310, which can reduce execution time while ensuring image matching accuracy, achieving the real-time target recognition function mentioned in this scheme. After the descriptors of each feature point are calculated, the feature points are matched based on the feature point descriptors. To achieve the real-time target recognition in this scheme, during the feature point matching calculation, Hamming distance is calculated based on the feature points and descriptors obtained in steps 310 and 320, and then matching is performed based on the obtained Hamming distance.

[0109] Step 330: Feature point matching is performed in step 320, resulting in a series of feature point pairs. Step 330 refines these feature point pairs by discarding mismatched feature points, thus improving matching accuracy. Due to the singularity of Hamming distance in step 320, even with a set Hamming distance threshold D (the threshold D is set according to accuracy requirements; a smaller D value results in higher accuracy), a high error matching rate can still occur, even leading to multiple matching pixels for a single pixel. If a combination of Hamming distance matching and bidirectional brute-force matching is used for feature point matching, the algorithm has high time complexity due to the characteristics of bidirectional brute-force matching, and the processing time increases with the number of matching feature points. Therefore, the system proposed in this scheme uses a dual judgment method combining Hamming distance and cosine similarity to determine mismatches.

[0110] Step 331: Based on the Hamming distance obtained in step 320, select M pairs of representative single matching feature points.

[0111] Step 332: Based on step 331, calculate the feature point determination threshold; calculate the cosine similarity of the M pairs of matching feature points selected in step 331 to obtain M values. The cosine similarity is calculated from the M cosine similarity values ​​obtained through statistics. The same... Weight Calculate the expected value of M cosine similarities. The cosine similarity threshold is obtained. )-( ,),in and It is the preset value ( and The value is set according to the required precision; the smaller the value, the higher the precision, and vice versa.

[0112] Step 333: Refine and optimize the feature points, deleting feature points with low accuracy or mismatches. Based on the Hamming distance threshold calculated in step 320 and the cosine similarity threshold calculated in step 332, determine whether the feature points obtained in step 320 satisfy both ( )-( The threshold of ,) also meets the requirements of the Hamming distance threshold. If the requirements are not met, the feature point is discarded; otherwise, the feature point is temporarily retained.

[0113] Step 340: Image and feature point output. After the calculations in steps 310-330 above, an image containing a set of highly accurate feature points and information such as the coordinates and grayscale of the feature points can be obtained. The obtained information is then output to the output device 400.

[0114] Output device 400: After receiving a series of information including feature point coordinates and grayscale from the target processing device 300, it outputs different results according to the application scenario.

[0115] The following section further illustrates this solution through its application in vehicle-mounted intelligent driving.

[0116] In this scenario, the output device 400 can be a processing chip and a display screen inside the driver's cab. Feature points obtained from the target processing device 300 are input to the processing chip of the output device 400. The processing chip uses the first real-time image (left view) acquired by the first image acquisition device 101 as a basis, and renders the left view according to the feature point coordinates and other information in the left grayscale image processed by the target processing device 300. Objects in the left view are rendered in a manner similar to... Figure 6 As shown in the schematic diagram, the outer frame is marked, and the output is clearly displayed on the display screen of the output device 400 for easy observation by people inside the vehicle.

[0117] In this scenario, the output device 400 can also be a processing chip such as a CPU. The feature point coordinates of the left and right grayscale images obtained by the target processing device 300 are input to the output device 400. After obtaining the feature point pairs from the left and right grayscale images, the output device 400 calculates the depth information by transforming the corresponding feature point pairs from the image coordinate system to the world coordinate system, based on the triangle imaging principle shown in the triangle imaging principle diagram (this principle is common knowledge in the industry and will not be elaborated in detail in this solution). This yields the distance between the target object and the vehicle-mounted camera in reality. The processing chip of the output device 400 uses the distance between the object and the vehicle-mounted camera, the vehicle's own speed, and other information to determine a series of operations such as deceleration, braking, and turning, and outputs the determination results to various control terminals, thereby realizing intelligent driving.

[0118] This solution incorporates cosine similarity determination in the feature point purification step 330 of the target processing device 300. Currently used traditional stereo matching methods are based on the ORB algorithm. However, the conventional ORB algorithm typically only uses bidirectional brute-force matching based on Hamming distance when performing feature point matching. Simple Hamming distance matching has a high probability of mismatch, and brute-force matching increases the computational load, thus requiring higher chip computing power and, more importantly, increasing computation time. By adding cosine similarity and using a weighted method to calculate the value, the matching determination condition is increased, improving matching accuracy while reducing time complexity, thereby further demonstrating the real-time performance mentioned in this solution.

[0119] Example 3: This application provides a target recognition system based on binocular vision, applied to a target recognition method based on binocular vision in Example 1, such as... Figure 7 As shown, it includes:

[0120] The view acquisition module is used to acquire the left and right views of the current scene at the same moment, and to perform grayscale processing on the preprocessed left and right views.

[0121] The feature point extraction module is used to extract multiple corresponding feature points from the left and right grayscale images obtained through grayscale processing using the FAST corner detection method.

[0122] The Hamming distance calculation module is used to obtain the descriptors of each feature point using the BEBLID algorithm, and to calculate the Hamming distance between each feature point in the left grayscale image and each feature point in the right grayscale image using the descriptors;

[0123] The feature point group generation module is used to generate a feature point group from at least two feature points whose Hamming distance does not exceed the Hamming distance threshold.

[0124] The feature point filtering module is used to remove feature points in the feature point group that do not meet the cosine similarity threshold based on the calculated cosine similarity threshold, thereby obtaining the feature point group sets for the left and right views.

[0125] Optionally, the above cosine similarity threshold is obtained in the following way:

[0126] Multiple feature point groups are selected for cosine similarity calculation;

[0127] For the multiple cosine similarities obtained, calculate the weight of the same cosine similarity.

[0128] Based on each cosine similarity and its weight, calculate the expected value of multiple cosine similarities;

[0129] The cosine similarity threshold is obtained by calculating the expected value.

[0130] Example 4: This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a target recognition method based on binocular vision as described in Example 1.

[0131] Example 5: This application provides a non-transitory computer-readable storage medium that stores computer instructions, which cause the computer to execute a target recognition method based on binocular vision according to Example 1.

[0132] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0134] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0135] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0136] Those skilled in the art will understand that all or part of the steps in the above facts and methods can be implemented by a program instructing related hardware. The program or the program described therein can be stored in a computer-readable storage medium. When the program is executed, it includes the following steps: at this time, the corresponding method steps are introduced. The storage medium can be ROM / RAM, magnetic disk, optical disk, etc.

[0137] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for target recognition based on binocular vision, characterized in that, The specific steps include the following: Obtain the left and right views of the current scene at the same time, and then perform grayscale processing on the preprocessed left and right views; The FAST corner detection method is used to extract multiple feature points from the left and right grayscale images obtained through grayscale processing, respectively. The BEBLID algorithm is used to obtain the descriptors of each feature point, and the Hamming distance between each feature point in the left grayscale image and each feature point in the right grayscale image is calculated using the descriptors. Generate a set of feature points using at least two feature points whose Hamming distance does not exceed the Hamming distance threshold; Based on the calculated cosine similarity threshold, feature points in the feature point group that do not meet the cosine similarity threshold are removed, resulting in feature point sets for the left and right views; the cosine similarity threshold is obtained in the following way: Multiple feature point groups are selected for cosine similarity calculation; For the multiple cosine similarities obtained, calculate the weight of the same cosine similarity. Based on each of the cosine similarities and their respective weights, the expected values ​​of multiple cosine similarities are calculated. The cosine similarity threshold is calculated using the expected value.

2. The target recognition method based on binocular vision according to claim 1, characterized in that, The grayscale processing specifically involves: ； In the formula, Indicates the coordinates in the left or right view. The grayscale value of a pixel after grayscale processing. , , These represent the coordinates in the left or right view, respectively. The values ​​of the R, G, and B components of the pixel. The coordinates are respectively The weights of the R, G, and B components corresponding to each pixel.

3. The target recognition method based on binocular vision according to claim 1, characterized in that, Each of the aforementioned feature points is extracted in the following manner: For either the left or right grayscale image, construct the corresponding image pyramid; Key points of each layer of the image pyramid are extracted using the Fast algorithm; Calculate the gray-level centroid of each key point, determine the vector direction of the gray-level centroid as the direction of the corresponding key point, and determine the key point with the obtained direction as the feature point.

4. The target recognition method based on binocular vision according to claim 3, characterized in that, The extraction of key points for each layer of the image pyramid using the Fast algorithm specifically involves: For any pixel in each layer of the image pyramid, a detection circle is formed with that pixel as the center and a radius of 3 pixels, and the pixel values ​​of 16 detection pixels located on the detection circle are obtained in a clockwise or counterclockwise direction. Calculate the pixel difference between each detected pixel and the pixel point. If the absolute value of the pixel difference of n consecutive detected pixels is greater than a preset threshold, then the pixel point is determined as a key point.

5. The target recognition method based on binocular vision according to claim 1, characterized in that, The cosine similarity threshold is specifically: wherein: ； In the formula, The cosine similarity threshold is... As expected, These represent the preset parameter values. The number of each weight. The weight of the nth element. Let be the cosine similarity corresponding to the weight of the nth element.

6. A binocular vision-based target recognition system applied to the binocular vision-based target recognition method of any one of claims 1-5, characterized in that, include: The view acquisition module is used to acquire the left and right views of the current scene at the same time, and to perform grayscale processing on the preprocessed left and right views; The feature point extraction module is used to extract multiple corresponding feature points from the left and right grayscale images obtained through grayscale processing using the FAST corner detection method. The Hamming distance calculation module is used to obtain the descriptors of each feature point using the BEBLID algorithm, and to calculate the Hamming distance between each feature point in the left grayscale image and each feature point in the right grayscale image using the descriptors; The feature point group generation module is used to generate a feature point group from at least two feature points whose Hamming distance does not exceed the Hamming distance threshold. The feature point filtering module is used to remove feature points in the feature point group that do not meet the calculated cosine similarity threshold based on the calculated cosine similarity threshold, so as to obtain the feature point group sets of the left view and the right view.

7. The binocular vision-based target recognition system according to claim 6, wherein, The cosine similarity threshold is obtained in the following way: Multiple feature point groups are selected for cosine similarity calculation; For the multiple cosine similarities obtained, calculate the weight of the same cosine similarity. Based on each of the cosine similarities and their respective weights, the expected values ​​of multiple cosine similarities are calculated. The cosine similarity threshold is calculated using the expected value.

8. An electronic device, comprising: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a target recognition method based on binocular vision as described in any one of claims 1-5.

9. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that cause the computer to execute any one of claims 1-5, a target recognition method based on binocular vision.

Images