Binocular feature matching displacement measurement method and device

Abstract

The present invention relates to a binocular feature matching displacement measurement method and device, comprising obtaining historical frame images and a current frame image captured by a binocular camera, wherein the historical frame images are l0 images and r0 images respectively captured by the left and right cameras at a first moment of a target, and the current frame images are ln images and rn images respectively captured by the left and right cameras at a second moment of the target; extracting feature points on the historical frame images and the current frame images, wherein the feature points include sub-pixel coordinates of corner points and corresponding ORB descriptions; matching the feature points on the historical frame images and the current frame images according to the ORB descriptions to determine the sub-pixel coordinates of the same feature point on the l0, r0, ln, and rn images; performing a displacement measurement operation on each feature point to obtain the displacement of the feature point in the z, x, and y directions at the second moment relative to the first moment. The method and device provided by the present invention can effectively improve the accuracy of displacement measurement.

Description

Technical Field

[0001] The present invention relates to the field of displacement measurement technology, and in particular to a binocular feature matching displacement measurement method and device. Background Art

[0002] With the continuous development of measurement technology, many current measurement methods are limited in certain scenarios. Displacement measurement based on binocular stereo vision has begun to play a role. Binocular stereo vision is an important form of machine vision. It is based on the principle of parallax and uses imaging equipment to obtain two images of the object being measured from different positions. The method obtains the three-dimensional geometric information of the object by calculating the position deviation between corresponding points in the images.

[0003] Three-dimensional displacement measurement based on binocular stereo vision has higher measurement accuracy than monocular measurement, so it is widely used in target displacement detection in scenarios such as geological disaster monitoring, foundation pit monitoring, bridge monitoring, and tunnel monitoring.

[0004] However, in the existing technology, in terms of binocular vision, dense matching algorithms such as BM, SGBM, and GC are relatively coarse, resulting in low accuracy of displacement measurement results. Summary of the Invention

[0005] In order to solve the above problems, the present invention provides a binocular feature matching displacement measurement method and device.

[0006] The binocular feature matching displacement measurement method provided by the present invention includes:

[0007] Get the historical frame images and current frame images taken by the binocular cameras, where the historical frame images are the l0 image and r0 image taken by the left and right cameras at the first moment, respectively, and the current frame images are the ln image and rn image taken by the left and right cameras at the second moment, respectively.

[0008] Extract feature points on the historical frame image and the current frame image. The feature points include the sub-pixel coordinates of the corner points and the corresponding ORB description. Match the feature points of the historical frame image and the current frame image based on the ORB description to determine the sub-pixel coordinates of the same feature point on the l0, r0, ln, and rn images.

[0009] For each feature point, perform the following displacement measurement operations:

[0010] Based on the disparity corresponding to the feature point in the historical frame image and the disparity corresponding to the current frame image, the historical frame distance and the current frame distance corresponding to the feature point are calculated respectively, and the Z-direction displacement of the feature point at the second moment relative to the first moment is calculated based on the historical frame distance and the current frame distance;

[0011] Based on the difference in sub-pixel coordinates of the feature point in the l0 and ln images or in the r0 and rn images, the x-direction pixel change and y-direction pixel change are obtained. Based on the current frame distance corresponding to the feature point and the binocular camera parameters, the x-direction scale and y-direction scale corresponding to the feature point are calculated. Based on the x-direction pixel change, y-direction pixel change, x-direction scale, and y-direction scale corresponding to the feature point, the x-direction displacement and y-direction displacement of the feature point at the second moment relative to the first moment are calculated.

[0012] Furthermore, the method also includes: pre-calibrating the binocular camera to obtain the internal parameters, external parameters, and distortion correction parameters of the binocular camera.

[0013] Furthermore, before extracting feature points from the historical frame image and the current frame image, the method further includes: performing distortion correction on the historical frame image and the current frame image according to the distortion correction parameters.

[0014] Furthermore, based on the disparity of the feature point in the historical frame image and the disparity of the current frame image, the historical frame distance and the current frame distance corresponding to the feature point are calculated as follows:

[0015] Use the formula distance d = baseline * fx / disparity to calculate the historical frame distance and current frame distance corresponding to the feature point respectively;

[0016] Here, baseline refers to the distance between the optical centers of the left and right cameras, fx is the focal length in the pre-calibrated intrinsic parameter, and disparity is the parallax.

[0017] Furthermore, the Z-direction displacement of the feature point at the second moment relative to the first moment is calculated based on the historical frame distance and the current frame distance: Z-direction displacement=current frame distance-historical frame distance.

[0018] Furthermore, the binocular camera parameters include the camera horizontal viewing angle, camera vertical viewing angle, camera horizontal resolution, and camera vertical resolution. According to the current frame distance corresponding to the feature point and the binocular camera parameters, the x-direction scale and y-direction scale corresponding to the feature point are calculated as follows:

[0019] According to the formula: x-direction scale = tan(camera horizontal viewing angle)*dn*2 / camera horizontal resolution, calculate the x-direction scale corresponding to the feature point;

[0020] The y-direction scale corresponding to the feature point is calculated according to the formula: y-direction scale = tan(camera vertical viewing angle)*dn*2 / camera vertical resolution; where dn is the current frame distance.

[0021] Furthermore, based on the x-direction pixel change, y-direction pixel change, x-direction scale, and y-direction scale corresponding to the feature point, the x-direction displacement and y-direction displacement of the feature point at the second moment relative to the first moment are calculated as follows:

[0022] The x-direction displacement of the feature point is calculated according to x-direction displacement = x-direction pixel change * x-direction scale;

[0023] The y-direction displacement of the feature point is calculated according to y-direction displacement = y-direction pixel change * y-direction scale.

[0024] Furthermore, the displacement measurement operation also includes correction of the x-direction displacement and the y-direction displacement:

[0025] According to x-direction displacement = x-direction displacement - x-direction adjustment angle * dn and y-direction displacement = y-direction displacement - y-direction adjustment angle * dn, the x-direction displacement and the y-direction displacement are reassigned to complete the correction of the x-direction displacement and the y-direction displacement;

[0026] Among them, the x-direction adjustment angle is the median of the x-direction deflection angles of all feature points, and the y-direction adjustment angle is the median of the y-direction deflection angles of all feature points. The calculation formula for the x-direction deflection angle of each feature point is:

[0027] The x-direction deflection angle = atan (x-direction displacement / dn), and the calculation formula for the y-direction deflection angle of each feature point is: y-direction deflection angle = atan (y-direction displacement / dn), where dn is the current frame distance.

[0028] The present invention also provides a binocular feature matching displacement measurement device, which includes an acquisition module, a feature point extraction and matching module, and a displacement measurement module, wherein:

[0029] An acquisition module, connected to the feature point extraction and matching module, is used to obtain historical frame images and current frame images captured by the binocular cameras, where the historical frame images are the l0 image and r0 image captured by the left and right cameras at the first moment, respectively, and the current frame images are the ln image and rn image captured by the left and right cameras at the second moment, respectively.

[0030] The feature point extraction and matching module is connected to the displacement measurement module and is used to extract feature points on the historical frame image and the current frame image. The feature points include the sub-pixel coordinates of the corner points and the corresponding ORB description. The feature points of the historical frame image and the current frame image are matched according to the ORB description to determine the sub-pixel coordinates of the same feature point on the l0, r0, ln, and rn images.

[0031] The displacement measurement module is used to perform the following displacement measurement operations for each feature point:

[0032] Based on the disparity corresponding to the feature point in the historical frame image and the disparity corresponding to the current frame image, the historical frame distance and the current frame distance corresponding to the feature point are calculated respectively, and the Z-direction displacement of the feature point at the second moment relative to the first moment is calculated based on the historical frame distance and the current frame distance;

[0033] Based on the difference in sub-pixel coordinates of the feature point in the l0 and ln images or in the r0 and rn images, the x-direction pixel change and y-direction pixel change are obtained. Based on the current frame distance corresponding to the feature point and the binocular camera parameters, the x-direction scale and y-direction scale corresponding to the feature point are calculated. Based on the x-direction pixel change, y-direction pixel change, x-direction scale, and y-direction scale corresponding to the feature point, the x-direction displacement and y-direction displacement of the feature point at the second moment relative to the first moment are calculated.

[0034] Furthermore, the device also includes an image distortion correction module, which is respectively connected to the acquisition module and the feature point extraction and matching module, and is used to perform distortion correction on the historical frame image and the current frame image according to the distortion correction parameters, wherein the distortion correction parameters are obtained by calibrating the binocular camera.

[0035] The binocular feature matching displacement measurement method and device provided by the present invention have at least the following beneficial effects:

[0036] (1) The feature points extracted from the historical frame image and the current frame image include the sub-pixel coordinates of the corner points and the corresponding ORB descriptions. The use of sub-pixel coordinates improves the coordinate accuracy and obtains more accurate coordinates. At the same time, the ORB description can be used to accurately match the feature points. Subsequently, for each feature point, the sub-pixel coordinates of the feature point on the l0, r0, ln, and rn images obtained by matching are used to accurately identify the displacement changes and achieve accurate measurement of spatial displacement.

[0037] (2) After the displacement is measured, the x-direction displacement and the y-direction displacement will be corrected according to the x-direction deflection angle and the y-direction deflection angle to further improve the accuracy of the displacement measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 This is a schematic diagram of the structure of a displacement measurement system based on a binocular camera;

[0040] Figure 2 The process of binocular feature matching displacement measurement method in one embodiment of the present invention is as follows Figure 1 ;

[0041] Figure 3 Schematic diagram of the current frame and historical frame images;

[0042] Figure 4 This is a flow chart of a displacement measurement operation method in one embodiment of the present invention;

[0043] Figure 5 This is a process flow of a binocular feature matching displacement measurement method in another embodiment of the present invention. Figure 2 ;

[0044] Figure 6 This is a process flow of a binocular feature matching displacement measurement method in another embodiment of the present invention. Figure 3 ;

[0045] Figure 7 This is a flow chart of a displacement measurement operation method in yet another embodiment of the present invention;

[0046] Figure 8 Schematic diagram of a binocular feature matching displacement measurement device in one embodiment of the present invention;

[0047] Figure 9 Schematic diagram of a binocular feature matching displacement measurement device in yet another embodiment of the present invention;

[0048] 801-acquisition module, 802-feature point extraction and matching module, 803-displacement measurement module, 804-image distortion correction module. DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present invention.

[0050] In one embodiment of the present invention, a binocular feature matching displacement measurement method is provided. The method is applied in a displacement measurement system based on a binocular camera, such as Figure 1The figure shows a schematic diagram of a system including a binocular camera and an industrial control computer connected to the binocular camera. The execution subject of this method is the industrial control computer. The binocular camera further includes a left camera and a right camera.

[0051] When measuring the displacement of a target, a binocular camera captures the target to obtain a captured image. Specifically, each capture is performed by the left and right cameras of the binocular camera simultaneously capturing the target, thereby obtaining images captured simultaneously by the left and right cameras at multiple moments.

[0052] like Figure 2 As shown, the method includes the following steps:

[0053] Step S10: Acquire historical frame images and current frame images captured by the binocular camera.

[0054] The historical frame images are the l0 image (taken by the left camera) and the r0 image (taken by the right camera) obtained by the left and right cameras at the first moment. The current frame images are the ln image (taken by the left camera) and the rn image (taken by the right camera) obtained by the left and right cameras at the second moment. The second moment is after the first moment.

[0055] The present invention uses a historical frame image obtained by photographing the target at a first moment as a reference, and then determines the target's displacement at the second moment relative to the first moment based on a current frame image obtained by photographing the target at a second moment. It should be understood that the selection of the first and second moments can be determined by the technician based on actual testing needs, and the present invention does not impose any restrictions on this.

[0056] like Figure 3 The figure shows a schematic diagram of historical frame images and current frame images.

[0057] Step S20: Extract feature points on the historical frame image and the current frame image. The feature points include the sub-pixel coordinates of the corner points and the corresponding ORB descriptions. Match the feature points of the historical frame image and the current frame image according to the ORB descriptions to determine the sub-pixel coordinates of the same feature point on the l0, r0, ln, and rn images.

[0058] In this step, feature points are generated at the locations of corner points. The feature points include sub-pixel coordinates of the corner points and ORB (Oriented Fast and Rotated Brief) descriptions.

[0059] In this step, feature points are extracted from the historical frame image and the current frame image, which can be:

[0060] The Shi-Tomasi corner detection algorithm is used to detect and identify corner points (multiple corner points can be present) on the l0, r0, ln, and rn images. A sub-pixel regression algorithm is used to generate sub-pixel coordinates for each corner point. The ORB algorithm is used to generate an ORB description for each sub-pixel corner point, thereby generating feature points at the corner point locations. The generated feature points include the sub-pixel coordinates of the corner point and the ORB description. Specifically, the sub-pixel regression algorithm can adopt a sub-pixel corner detection algorithm known in the art, which is not limited in the present invention.

[0061] Furthermore, in this step, feature point matching is performed on the historical frame image and the current frame image according to the ORB description to determine the sub-pixel coordinates of the same feature point on the l0, r0, ln, and rn images, which can be:

[0062] Use BF matching to match the ORB description and find the sub-pixel coordinates of the same feature point on the l0, r0, ln, and rn images.

[0063] Step S30: performing a displacement measurement operation for each feature point.

[0064] Specifically, in one implementation of this embodiment, as Figure 4 As shown, the displacement measurement operation specifically includes the following steps:

[0065] Step S301: Calculate the historical frame distance and the current frame distance corresponding to the feature point based on the disparity corresponding to the feature point in the historical frame image and the disparity corresponding to the current frame image.

[0066] The distance in this step refers to the distance in the z direction from the measured point (the feature point) to the optical center of the camera.

[0067] Specifically, the formula distance d = baseline * fx / disparity is used to calculate the historical frame distance and the current frame distance corresponding to the feature point respectively;

[0068] Where baseline refers to the distance between the optical centers of the left and right cameras, fx is the pre-calibrated focal length, and disparity is the disparity. When calculating the distance to the historical frame, the disparity corresponding to the feature point in the historical frame image is substituted into disparity. When calculating the distance to the current frame, the disparity corresponding to the feature point in the current frame image is substituted into disparity.

[0069] Furthermore, in this step, spatial constraints can be used to filter distance data. Left-right parallax and spatial distance are constrained, and the neighborhood space has continuity constraints. It should be understood that if the historical frame distance or current frame distance corresponding to a feature point is filtered out by spatial constraints in this step, the feature point will be discarded during displacement measurement, and the corresponding z-, x-, and y-direction displacements will not be measured.

[0070] Step S302: Calculate the Z-direction displacement of the feature point at the second moment relative to the first moment based on the historical frame distance and the current frame distance.

[0071] Specifically, Z-direction displacement = current frame distance - historical frame distance.

[0072] Step S303: Obtain the pixel change amount in the x direction and the pixel change amount in the y direction according to the difference between the sub-pixel coordinates of the feature point in the l0 and ln images or in the r0 and rn images.

[0073] Specifically, assuming that the sub-pixel coordinates of the feature point on the l0 image are (x1, y1), and the sub-pixel coordinates on the ln image are (x2, y2), then the pixel change in the x direction = x2-x1, and the pixel change in the y direction is y2-y1.

[0074] Or assuming that the sub-pixel coordinates of the feature point on the r0 image are (x3, y3), and the sub-pixel coordinates on the rn image are (x4, y4), then the pixel change in the x direction = x4-x3, and the pixel change in the y direction is y4-y3.

[0075] Step S304: Calculate the x-direction scale and the y-direction scale corresponding to the feature point according to the current frame distance corresponding to the feature point and the binocular camera parameters.

[0076] Furthermore, the binocular camera parameters include the camera horizontal viewing angle, the camera vertical viewing angle, the camera horizontal resolution, and the camera vertical resolution, which are obtained by calibrating the binocular camera in advance.

[0077] According to the current frame distance corresponding to the feature point and the binocular camera parameters, the x-direction scale and y-direction scale corresponding to the feature point are calculated as follows:

[0078] The x-direction scale corresponding to the feature point is calculated according to x-direction scale = tan(camera horizontal viewing angle)*dn*2 / camera horizontal resolution;

[0079] The y-direction scale corresponding to the feature point is calculated according to y-direction scale = tan(camera vertical viewing angle)*dn*2 / camera vertical resolution; where dn is the current frame distance.

[0080] Specifically, since the viewing angle parameters of the left and right cameras are not completely consistent when calibrating the binocular camera, in this step, it is necessary to consider whether the left camera image or the right camera image is used to calculate the pixel change in the x direction and the pixel change in the y direction in step S303. If the left camera image is used, the camera horizontal viewing angle and the camera vertical viewing angle in this step are the camera horizontal viewing angle and the camera vertical viewing angle corresponding to the left camera. If the right camera image is used, the camera horizontal viewing angle and the camera vertical viewing angle in this step are the camera horizontal viewing angle and the camera vertical viewing angle corresponding to the right camera.

[0081] Step S305: Calculate the x-direction displacement and y-direction displacement of the feature point at the second moment relative to the first moment based on the x-direction pixel change, y-direction pixel change, x-direction scale, and y-direction scale corresponding to the feature point.

[0082] Specifically, in this step, the x-direction displacement of the feature point can be calculated according to x-direction displacement = x-direction pixel change * x-direction scale;

[0083] The y-direction displacement of the feature point is calculated according to y-direction displacement = y-direction pixel change * y-direction scale.

[0084] The binocular feature matching displacement measurement method provided in this embodiment measures the displacement of a target by using a binocular camera to capture historical frame images and current frame images. The feature points extracted from the historical frame images and the current frame images include sub-pixel coordinates of corner points and corresponding ORB descriptions. The use of sub-pixel coordinates improves the coordinate accuracy and obtains more accurate coordinates of each corner point. At the same time, the ORB description can be used to accurately match the feature points. Subsequently, for each feature point, the sub-pixel coordinates of the feature point obtained by matching on the l0, r0, ln, and rn images can be used to accurately identify the displacement change and achieve accurate measurement of spatial displacement.

[0085] In another embodiment of the present invention, Figure 5 As shown, the method further includes:

[0086] Step S40: pre-calibrate the binocular camera to obtain the intrinsic parameters, extrinsic parameters, and distortion correction parameters of the binocular camera.

[0087] Specifically, the method for calibrating the binocular camera in this step can adopt the calibration method in the prior art, and the present invention does not limit this.

[0088] In another embodiment of the present invention, Figure 6 As shown, before extracting feature points from the historical frame image and the current frame image, the method further includes:

[0089] Step S50: performing distortion correction on the historical frame image and the current frame image according to the distortion correction parameters.

[0090] In this embodiment, before performing displacement measurement, distortion correction is first performed on the historical frame image and the current frame image, thereby improving the accuracy of subsequent displacement measurement.

[0091] Further, such as Figure 7 As shown, the displacement measurement operation further includes step S306: correcting the displacement in the x-direction and the displacement in the y-direction.

[0092] According to x-direction displacement = x-direction displacement - x-direction adjustment angle * dn and y-direction displacement = y-direction displacement - y-direction adjustment angle * dn, the x-direction displacement and the y-direction displacement are reassigned to complete the correction of the x-direction displacement and the y-direction displacement.

[0093] It should be understood that the x-direction displacement before the equal sign in the above formula is the x-direction displacement of the feature point (feature point to be corrected) after correction, the x-direction displacement after the equal sign is the x-direction displacement of the feature point (feature point to be corrected) before correction, and dn is the current frame distance corresponding to the feature point (feature point to be corrected).

[0094] Furthermore, the x-direction adjustment angle is the median of the x-direction deflection angles of all feature points, and the y-direction adjustment angle is the median of the y-direction deflection angles of all feature points.

[0095] That is, it is necessary to first calculate the x-direction deflection angle and y-direction deflection angle of each feature point, and then calculate the median respectively as the x-direction adjustment angle and y-direction adjustment angle.

[0096] The calculation formula for the x-direction deflection angle of each feature point is:

[0097] x-direction deflection angle = atan (x-direction displacement / dn). Substituting the x-direction displacement corresponding to each feature point and the current frame distance dn into the above formula, the x-direction deflection angle of all feature points can be calculated.

[0098] The calculation formula for the y-direction deflection angle of each feature point is:

[0099] Y-direction deflection angle = atan (y-direction displacement / dn). Substituting the y-direction displacement corresponding to each feature point and the current frame distance dn into the above formula, the y-direction deflection angle of all feature points can be calculated. In this embodiment, based on the characteristics of the local displacement of the feature point before and after deformation, the calculated x-direction displacement and y-direction displacement are adjusted and corrected. In visual measurement, the synchronization error has the same deflection angle. The deflection angles in the x-direction and y-direction are first calculated, and then the deflection angle adjustment is calculated. The adjustment correction value is calculated, and the x-direction displacement and y-direction displacement are corrected using the correction value to eliminate the same-direction synchronization error caused by illumination or jitter.

[0100] The present invention also provides a binocular feature matching displacement measurement device, such as Figure 8 As shown, the device includes an acquisition module 801, a feature point extraction and matching module 802, and a displacement measurement module 803, wherein:

[0101] The acquisition module 801 is connected to the feature point extraction and matching module 802 and is used to obtain historical frame images and current frame images captured by the binocular cameras, wherein the historical frame images are the l0 image and r0 image captured by the left and right cameras at the first moment of the target, respectively, and the current frame images are the ln image and rn image captured by the left and right cameras at the second moment of the target, respectively;

[0102] Feature point extraction and matching module 802, connected to displacement measurement module 803, is used to extract feature points on the historical frame image and the current frame image. Feature points include sub-pixel coordinates of corner points and corresponding ORB descriptions. Feature points are matched on the historical frame image and the current frame image based on the ORB descriptions to determine the sub-pixel coordinates of the same feature point on the l0, r0, ln, and rn images.

[0103] The displacement measurement module 803 is configured to perform the following displacement measurement operations for each feature point:

[0104] Based on the disparity corresponding to the feature point in the historical frame image and the disparity corresponding to the current frame image, the historical frame distance and the current frame distance corresponding to the feature point are calculated respectively, and the Z-direction displacement of the feature point at the second moment relative to the first moment is calculated based on the historical frame distance and the current frame distance;

[0105] Based on the difference in sub-pixel coordinates of the feature point on the l0 and ln images or on the r0 and rn images, the x-direction pixel change and y-direction pixel change are obtained. Based on the current frame distance corresponding to the feature point and the binocular camera parameters, the x-direction scale and y-direction scale corresponding to the feature point are calculated. Based on the x-direction pixel change, y-direction pixel change, x-direction scale, and y-direction scale corresponding to the feature point, the x-direction displacement and y-direction image displacement of the feature point at the second moment relative to the first moment are calculated.

[0106] The binocular feature matching displacement measurement device provided by this embodiment uses a binocular camera to shoot historical frame images and current frame images when measuring the displacement of a target. The feature points extracted from the historical frame images and the current frame images include sub-pixel coordinates of corner points and corresponding ORB descriptions. The use of sub-pixel coordinates improves the coordinate accuracy and obtains more accurate coordinates of each corner point. At the same time, the ORB description can be used to accurately match the feature points. Subsequently, for each feature point, the sub-pixel coordinates of the feature point obtained by matching on the l0, r0, ln, and rn images are used to accurately identify the displacement change and achieve accurate measurement of spatial displacement.

[0107] like Figure 9 As shown, in another embodiment of the present invention, the binocular feature matching displacement measurement device provided by the present invention further includes an image distortion correction module 804, wherein the image distortion correction module 804 is respectively connected to the acquisition module 801 and the feature point extraction and matching module 802, and is used to perform distortion correction on the historical frame image and the current frame image according to the distortion correction parameters, wherein the distortion correction parameters are obtained by calibrating the binocular camera.

[0108] The binocular feature matching displacement measurement method and device provided by the present invention, when performing displacement measurement, extracts feature points from historical frame images and current frame images, including sub-pixel coordinates of corner points and corresponding ORB descriptions. The use of sub-pixel coordinates improves coordinate precision and obtains more accurate coordinates. At the same time, ORB descriptions enable precise matching of feature points. Subsequently, for each feature point, the sub-pixel coordinates of the feature point obtained by matching on the l0, r0, ln, and rn images are used to accurately identify displacement changes and achieve accurate measurement of spatial displacement. Furthermore, after the displacement is measured, the x- and y-direction displacements are corrected based on the x- and y-direction deflection angles, further improving the accuracy of the displacement measurement.

[0109] The terms and expressions used in the present invention specification are for illustration only and are not meant to constitute limitations. The "first" and "second" used in the claims and specification of the present invention are only for the purpose of convenience of distinction and have no special meaning and are not intended to limit the present invention. It should be understood by those skilled in the art that various changes may be made to the details in the above-mentioned embodiments without departing from the basic principles of the disclosed embodiments. Therefore, the scope of the present invention is determined only by the claims. In the claims, unless otherwise specified, all terms should be understood in the broadest and most reasonable sense.

Claims

1. A binocular feature matching displacement measurement method, characterized in that: The method comprises: Get the historical frame images and current frame images taken by the binocular cameras, where the historical frame images are the l0 image and r0 image taken by the left and right cameras at the first moment, respectively, and the current frame images are the ln image and rn image taken by the left and right cameras at the second moment, respectively. Extract feature points on the historical frame image and the current frame image, wherein the feature points are feature points generated at the locations of corner points, and the feature points include sub-pixel coordinates of the corner points and corresponding ORB descriptions. The ORB descriptions are ORB descriptions of each sub-pixel coordinate corner point generated by an ORB algorithm. Feature point matching is performed on the historical frame image and the current frame image based on the ORB descriptions to determine the sub-pixel coordinates of the same feature point on the l0, r0, ln, and rn images; For each feature point, perform the following displacement measurement operations: Based on the disparity corresponding to the feature point in the historical frame image and the disparity corresponding to the current frame image, the historical frame distance and the current frame distance corresponding to the feature point are calculated respectively, and the Z-direction displacement of the feature point at the second moment relative to the first moment is calculated based on the historical frame distance and the current frame distance; Based on the difference in sub-pixel coordinates of the feature point in the l0 and ln images or in the r0 and rn images, the x-direction pixel change and y-direction pixel change are obtained. Based on the current frame distance corresponding to the feature point and the binocular camera parameters, the x-direction scale and y-direction scale corresponding to the feature point are calculated. Based on the x-direction pixel change, y-direction pixel change, x-direction scale, and y-direction scale corresponding to the feature point, the x-direction displacement and y-direction displacement of the feature point at the second moment relative to the first moment are calculated.

2. The binocular feature matching displacement measurement method according to claim 1, characterized in that: The method further includes: pre-calibrating the binocular camera to obtain internal parameters, external parameters, and distortion correction parameters of the binocular camera.

3. The binocular feature matching displacement measurement method according to claim 2, characterized in that: Before extracting feature points from the historical frame images and the current frame image, the method further includes: Distortion correction is performed on the historical frame image and the current frame image according to the distortion correction parameters.

4. The binocular feature matching displacement measurement method according to claim 2, characterized in that: According to the disparity corresponding to the feature point in the historical frame image and the disparity corresponding to the current frame image, the historical frame distance and the current frame distance corresponding to the feature point are respectively calculated as follows: Use the formula: distance d = baseline * fx / disparity to calculate the historical frame distance and current frame distance corresponding to the feature point respectively; Here, baseline refers to the distance between the optical centers of the left and right cameras, fx is the focal length in the pre-calibrated intrinsic parameter, and disparity is the parallax.

5. The binocular feature matching displacement measurement method according to claim 1, characterized in that: The Z-direction displacement of the feature point at the second moment relative to the first moment calculated based on the historical frame distance and the current frame distance is: Z-direction displacement=current frame distance-historical frame distance.

6. The binocular feature matching displacement measurement method according to claim 1, characterized in that: The binocular camera parameters include the camera horizontal viewing angle, the camera vertical viewing angle, the camera horizontal resolution, and the camera vertical resolution. The x-direction scale and y-direction scale corresponding to the feature point are calculated based on the current frame distance corresponding to the feature point and the binocular camera parameters: According to the formula: x-direction scale = tan(camera horizontal viewing angle)*dn*2 / camera horizontal resolution, calculate the x-direction scale corresponding to the feature point; According to the formula: y-direction scale = tan(camera vertical viewing angle) * dn * 2 / camera vertical resolution, calculate the y-direction scale corresponding to the feature point; Where dn is the current frame distance.

7. The binocular feature matching displacement measurement method according to claim 1, characterized in that: Based on the x-direction pixel change, y-direction pixel change, x-direction scale, and y-direction scale corresponding to the feature point, the x-direction displacement and y-direction displacement of the feature point at the second moment relative to the first moment are calculated as follows: The x-direction displacement of the feature point is calculated according to x-direction displacement = x-direction pixel change * x-direction scale; The y-direction displacement of the feature point is calculated according to y-direction displacement = y-direction pixel change * y-direction scale.

8. The binocular feature matching displacement measurement method according to claim 1, characterized in that: The displacement measurement operation also includes correcting the x-direction displacement and the y-direction displacement: According to x-direction displacement = x-direction displacement - x-direction adjustment angle * dn and y-direction displacement = y-direction displacement - y-direction adjustment angle * dn, the x-direction displacement and the y-direction displacement are reassigned to complete the correction of the x-direction displacement and the y-direction displacement; Among them, the x-direction adjustment angle is the median of the x-direction deflection angles of all feature points, and the y-direction adjustment angle is the median of the y-direction deflection angles of all feature points. The calculation formula for the x-direction deflection angle of each feature point is: The x-direction deflection angle = atan (x-direction displacement / dn), and the calculation formula for the y-direction deflection angle of each feature point is: y-direction deflection angle = atan (y-direction displacement / dn), where dn is the current frame distance.

9. A binocular feature matching displacement measurement device, characterized in that: The device includes an acquisition module, a feature point extraction and matching module, and a displacement measurement module, wherein: The acquisition module is connected to the feature point extraction and matching module, and is used to obtain historical frame images and current frame images captured by the binocular cameras, wherein the historical frame images are the l0 image and r0 image respectively captured by the left and right cameras at the first moment of the target, and the current frame images are the ln image and rn image respectively captured by the left and right cameras at the second moment of the target; The feature point extraction and matching module is connected to the displacement measurement module and is used to extract feature points on the historical frame image and the current frame image. The feature points are feature points generated at the location of the corner points. The feature points include the sub-pixel coordinates of the corner points and the corresponding ORB descriptions. The ORB descriptions are ORB descriptions of each sub-pixel coordinate corner point generated by the ORB algorithm. Feature point matching is performed on the historical frame image and the current frame image based on the ORB descriptions to determine the sub-pixel coordinates of the same feature point on the l0, r0, ln, and rn images. The displacement measurement module is used to perform the following displacement measurement operations for each feature point: Based on the disparity corresponding to the feature point in the historical frame image and the disparity corresponding to the current frame image, the historical frame distance and the current frame distance corresponding to the feature point are calculated respectively, and the Z-direction displacement of the feature point at the second moment relative to the first moment is calculated based on the historical frame distance and the current frame distance; Based on the difference in sub-pixel coordinates of the feature point in the l0 and ln images or in the r0 and rn images, the x-direction pixel change and y-direction pixel change are obtained. Based on the current frame distance corresponding to the feature point and the binocular camera parameters, the x-direction scale and y-direction scale corresponding to the feature point are calculated. Based on the x-direction pixel change, y-direction pixel change, x-direction scale, and y-direction scale corresponding to the feature point, the x-direction displacement and y-direction displacement of the feature point at the second moment relative to the first moment are calculated.

10. The binocular feature matching displacement measurement device according to claim 9, characterized in that: The device also includes an image distortion correction module, which is respectively connected to the acquisition module and the feature point extraction and matching module, and is used to perform distortion correction on the historical frame image and the current frame image according to the distortion correction parameters, wherein the distortion correction parameters are obtained by calibrating the binocular camera.

Images