Monocular Vision Ranging Method and System Assisted by Carrier Phase Differential Technology

Abstract

The present invention proposes a monocular vision ranging method and system assisted by carrier phase differential technology, including: collecting multiple frames of images in the vehicle's forward direction and the vehicle position located in real time by carrier phase differential technology, obtaining the feature points and feature point coordinates of a single frame of image, calculating the distances between each pixel point in the image and the reference point through the vertical axis pixel difference and vehicle displacement difference of the same feature point in the front and rear two frames of images, and obtaining a reference distance vector table; collecting images containing the target object in real time, identifying the pixel points where the target object is located, searching for the target object pixel points from the reference distance vector table, and outputting the distance between the target object and the vehicle; this method integrates differential positioning technology into monocular vision ranging technology, effectively improving problems such as low accuracy existing in traditional monocular vision ranging algorithms and the need for preliminary calibration and preparation work in monocular vision ranging, and greatly enhancing the feasibility and reliability of monocular vision ranging in practical engineering applications.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and particularly relates to a monocular vision ranging method and system assisted by carrier phase differential technology. Background Art

[0002] The statements in this part merely provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] With the continuous improvement of the social economy and the acceleration of the informatization construction pace, the integration of various high-end technologies such as modern information technology, communication technology, computer vision technology, and navigation and positioning technology has increasingly become an important trend and direction of development. Among them, the technology of ranging the front object based on computer vision is an important branch of these high-end technologies and has great scientific research and application value.

[0004] Compared with the binocular vision system, the image ranging technology based on monocular vision has a simple structure, and the calibration of the camera is also relatively simple, thus avoiding the problem of stereo matching in binocular vision. Therefore, it has the advantages of simple equipment, low cost, and simple measurement process. However, the monocular vision ranging generally uses the corresponding point calibration method to obtain the depth information of the image, that is, to solve the transformation relationship of the coordinate system by the corresponding coordinates of the corresponding points in different coordinate systems. However, in the calibration process, due to equipment limitations, it is still impossible to record the corresponding coordinates of a point in the world coordinate system and the image coordinate system very accurately. If the coordinates are not accurate enough, the accuracy of the obtained transformation matrix will also be restricted, and the accuracy of the coordinate transformation result will fluctuate accordingly. Since the corresponding point calibration method calibrates the camera when the angles and heights of the camera are already determined, when any parameter of the camera changes, it is necessary to re-calibrate to obtain the transformation matrix in this specific situation. Therefore, this method is only applicable to the case of a fixed-position camera. Due to the existing technological and technical limitations of the current monocular vision equipment, problems such as low accuracy and pre-use calibration are caused, which are issues that need to be considered and solved in the actual application process.

[0005] For the Global Navigation Satellite System (GNSS), among which RTK (Real Time Kinematic), that is, the carrier phase differential technology, it can provide the three-dimensional positioning result of the measuring station in the specified coordinate system in real time and reach centimeter-level accuracy, and has been widely used in practical applications, which also brings hope for solving the calibration problem in the monocular vision image ranging technology. Summary of the Invention

[0006] To overcome the deficiencies of the above-mentioned existing technologies, the present invention provides a monocular vision ranging method and system assisted by carrier phase differential technology. By using carrier phase differential technology RTK to assist in positioning the vehicle's position, it solves the problems of low monocular vision ranging accuracy and cumbersome preparatory work such as pre-calibration, and effectively improves the feasibility and reliability of monocular vision ranging in practical engineering applications.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] In the first aspect, the present invention provides a monocular vision ranging method assisted by carrier phase differential technology, including the following steps:

[0009] Collect multiple frames of images in the vehicle's forward direction and the vehicle position located in real time by carrier phase differential technology. Calculate the distance between each pixel point in the image and the reference point through the vertical axis pixel difference and vehicle displacement difference of the same feature point in the front and rear frames of images, and obtain a reference distance vector table;

[0010] Collect an image containing the target object in real time, identify the pixel point where the target object is located, search for the target object pixel point in the reference distance vector table, and output the distance between the target object and the vehicle.

[0011] Use an in-vehicle monocular camera to capture images in the vehicle's forward direction, and use carrier phase differential technology to obtain the vehicle position at the time of each frame of image capture.

[0012] Construct a reference distance vector table, and the specific steps are as follows:

[0013] Extract the feature points and feature point coordinates of a single frame of image from the collected images, calculate the vertical axis pixel difference and vehicle displacement difference of the same feature point in the front and rear frames of images, and establish a pixel distance vector table of the vertical axis pixel and pixel distance difference;

[0014] Based on the pixel distance vector table, calculate the distance between each pixel point in the image and the reference point to obtain a reference distance vector table.

[0015] The method for obtaining the feature points and feature point coordinates: Use Gaussian filtering to perform denoising filtering on the image, use the Fast algorithm to extract feature points, use non-maximum suppression to eliminate redundant feature points, and identify the coordinates of the remaining feature points.

[0016] The vertical axis pixel difference of the same feature point in the front and rear frames of images is the difference in the vertical coordinates of the same feature point in the two frames of images, and the vehicle displacement difference is the distance difference between the vehicle positions when the two frames of images are captured respectively.

[0017] It is also possible to perform real-time update and correction on the established pixel distance vector table. Compare the newly calculated pixel distance difference with the pixel distance difference in the pixel distance vector table. If it is greater than the threshold, update the pixel distance vector table, otherwise do not update.

[0018] The reference point is selected as the pixel point closest to the bottom corner of the vehicle in the image. The reference distance vector table includes the distances from all pixel points in the image to the reference point.

[0019] Use the target detection system to perform target detection on the monocular vision system image collected, extract the pixel point coordinates of the bottom edge of the bounding box of the detected target, and obtain the distance from the target to the vehicle by looking up the reference distance vector table.

[0020] In a second aspect, the present invention provides a monocular vision ranging system assisted by carrier phase differential technology, including:

[0021] A reference distance vector table construction module, configured to: obtain multiple frames of images in the vehicle's forward direction and the vehicle position located in real time by carrier phase differential technology, obtain the feature points and feature point coordinates of a single frame of image, and calculate the distances from each pixel point in the image to the reference point through the vertical pixel difference and vehicle displacement difference of the same feature point in the front and rear two frames of images, so as to obtain the reference distance vector table;

[0022] A ranging module, configured to: obtain the image containing the target object in real time, identify the pixel points where the target object is located, look up the target object pixel points from the reference distance vector table, and output the distance between the target object and the vehicle.

[0023] In a third aspect, the present invention provides an electronic device, including a memory, a processor, and computer instructions stored on the memory and running on the processor. When the computer instructions are run by the processor, the steps of the monocular vision ranging method assisted by carrier phase differential technology are completed.

[0024] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by the processor, the steps of the monocular vision ranging method assisted by carrier phase differential technology are completed.

[0025] The above one or more technical solutions have the following beneficial effects:

[0026] The present invention is based on traditional computer vision processing, integrates differential positioning technology into monocular vision ranging technology, effectively improves problems such as low accuracy existing in traditional monocular vision ranging algorithms and the need for preliminary calibration and preparation work in monocular vision ranging, and greatly enhances the feasibility and reliability of monocular vision ranging in practical engineering applications. In addition, since the price of a monocular camera is relatively low and the computer vision algorithm is relatively simple, it has little impact on the cost and overhead of the original system.

[0027] Advantages of additional aspects of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings forming a part of this specification are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not unduly limit the present invention.

[0029] Figure 1 It is a general framework diagram of the monocular vision ranging method and system assisted by carrier phase differential technology according to the present invention.

[0030] Figure 2 It is a schematic diagram of reference point selection according to the present invention.

[0031] Figure 3 It is a flowchart of the initialization implementation of the monocular vision ranging method assisted by carrier phase differential technology according to the present invention.

[0032] Figure 4 It is a schematic diagram of pixel distance difference in an example of the present invention.

[0033] Figure 5 It is a schematic diagram of pixels in the pixel distance vector table in an example of the present invention.

[0034] Figure 6 It is a flowchart of the implementation for optimizing the pixel distance vector table in the monocular vision ranging method assisted by carrier phase differential technology according to the present invention.

[0035] Figure 7 It is a schematic diagram of target detection results in an example of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0037] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0038] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0039] The general idea proposed by the present invention:

[0040] The position of a moving vehicle is real-time located through carrier phase differential technology. A camera fixed in front of the vehicle captures the front image. The vertical-axis pixel difference of the same feature point in two consecutive frames of images and the displacement difference of the real-time position of the vehicle are used to linearly fit and calculate the unit distance difference between pixels. This process is repeated to construct a reference distance vector table composed of the distances from each pixel point in the image to a reference point. By looking up the distance corresponding to the lower-edge pixel of the target to be measured in the reference distance vector table, the distance from the target to be measured to the vehicle is obtained.

[0041] Embodiment 1

[0042] This embodiment discloses a monocular vision ranging method assisted by carrier phase differential technology

[0043] As Figure 1 shown, this embodiment provides a monocular vision ranging method assisted by carrier phase differential technology, including:

[0044] S1: Collect multiple frames of images in the vehicle's forward direction and the vehicle position real-time located by carrier phase differential technology, obtain the feature points and feature point coordinates of a single frame of image, and calculate the distances from each pixel point in the image to a reference point through the vertical-axis pixel difference of the same feature point in two consecutive frames of images and the vehicle displacement difference, so as to obtain a reference distance vector table;

[0045] S2: Real-time collect an image containing the target object, identify the pixel points where the target object is located, look up the target object pixel points in the reference distance vector table, and output the distance between the target object and the vehicle.

[0046] In the step S1, a vehicle-mounted monocular camera is used to capture images in the vehicle's forward direction, and the vehicle position at the time of shooting each frame of image is obtained by using carrier phase differential technology. The reference point, as Figure 2 shown, selects the pixel point at the lower right corner of the image close to the vehicle.

[0047] The specific steps for constructing the reference distance vector table are:

[0048] S-1-1: Extract the feature points and feature point coordinates of a single frame of image from the collected images, calculate the vertical-axis pixel difference of the same feature point in two consecutive frames of images and the vehicle displacement difference, and establish a pixel distance vector table of the vertical-axis pixels and pixel distance differences;

[0049] S-1-2: Based on the pixel distance vector table, calculate the distances from each pixel point in the image to a reference point, so as to obtain a reference distance vector table;

[0050] The pixel distance vector table of the vertical-axis pixels and pixel distance differences, as Figure 3 shown, its steps are:

[0051] Step a: Keep the vehicle moving straight ahead. Obtain the images during driving through a monocular vision camera, process the images, perform denoising filtering on the images using Gaussian filtering, extract feature points using the Fast algorithm, eliminate redundant feature points using non-maximum suppression, and form the remaining feature points into a set C p ={p0, p1, … p n-1 , p n}, and the corresponding set of pixel coordinate information S′ p ={(x0, y0), (x1, y1)…(x n-1 , y n-1 ), (x n , y n )}. At the same time, use RTK to locate the vehicle position at the moment of taking the image, denoted as p′=(lat′, lon′, h′);

[0052] Step b: Use the LK optical flow method to track the feature points, delete the feature points with lost tracking, and update the set C p and the corresponding set of pixel coordinate information S′ p , and at the same time use RTK to locate the vehicle position, denoted as p″=(lat″, lon″, h″);

[0053] Step c: Traverse the feature points in the set C p . Assume that the pixel information of the feature point p i in the front and back two frames of images are respectively where Then the pixel difference is expressed as:

[0054]

[0055] The displacement difference between two frames of images:

[0056]

[0057] where R is the radius of the earth;

[0058] Perform linear fitting on the pixels within the range of the vertical axis pixel to obtain the pixel distance difference Perform linear interpolation on the middle vertical axis pixel point within the range of the vertical axis pixel , iteratively record and update the vertical axis pixels and the corresponding pixel distance difference information within the range, and save them to the pixel distance vector table. The pixel distance difference is the actual distance corresponding to a unit pixel within the range;

[0059] Step d: Repeat steps a to c until the vertical axis pixels basically cover the core area of the image vertical axis, as shown in Figure 5 ;

[0060] The pixel distance vector table can also be updated and corrected in real time, such as Figure 6 shown, the specific method is as follows:

[0061] Step e: When it is determined that the vehicle has good satellite positioning, obtain the images during driving through a monocular vision camera, process the images, perform denoising filtering on the images using Gaussian filtering, extract feature points using the Fast algorithm, eliminate redundant feature points using non-maximum suppression, and form a set C p ={p0, p1,... p n-1 , p n}, and the corresponding set of pixel coordinate information S' p ={(x0, y0), (x1, y1)... (x n-1 , y n-1 ), (x n , y n )}, and at the same time, use RTK to locate the position at this key frame moment, denoted as p'=(lat', lon', h');

[0062] Step f: Use the LK optical flow method to track the feature points, delete the feature points that are lost in tracking and update the set C p and the corresponding set of pixel coordinate information S' p , and at the same time, use RTK to locate the position at this key frame moment, denoted as p''=(lat'', lon'', h'');

[0063] Step g: Traverse the feature points in the set C p , assuming that the pixel information of the feature point p i in the front and back two frames of images are respectively where then the pixel difference is expressed as:

[0064]

[0065] The displacement difference between two frames of images:

[0066]

[0067] where R is the radius of the earth;

[0068] Perform linear fitting on the pixels within the range of the vertical axis pixels to obtain the unit distance difference between pixels

[0069] Search for the pixel distance differences of the same vertical axis pixels in the pixel distance vector table, calculate the difference between the two distance differences, and compare it with the threshold. If it is greater than the threshold, update the pixel distance vector table, otherwise do not update;

[0070] Step h: Repeat the above steps e to g until the feature points in set C p are empty;

[0071] The threshold is set to 10 cm.

[0072] In step S-1-2, the calculation method for the distance from the pixel point to the reference point C o is as follows:

[0073] Traverse each pixel point (x, y) in the pixel distance vector table, add up the pixel distance differences of the vertical-axis pixels where each is less than or equal to y, and obtain the distance from the pixel point (x, y) to the reference point C o ;

[0074] The vertical-axis pixels of the pixel points in the pixel distance vector table cover the vertical-axis core area of the image, and the distance from each pixel in the image to the reference point C o can be calculated, so as to construct a reference distance vector table composed of each pixel point in the image and the distance to the reference point. After constructing the reference distance vector table, the distance measurement of the detected target in the image can be performed.

[0075] As Figure 7 shown, during the vehicle's traveling process, to measure the distance between the target object and the vehicle, the image of the target object is captured by a camera, then the target detection system is used for target detection, the coordinate information of the bottom edge of the bounding box of the detected target is extracted, the pixel point at the lower edge of the detection box is selected as the position where the target object is located, and by looking up the reference distance vector table, the distance from the target object to the vehicle is obtained.

[0076] Embodiment 2

[0077] The purpose of this embodiment is to provide a monocular vision ranging system assisted by carrier phase differential technology, including:

[0078] A reference distance vector table construction module, configured to: obtain multiple frames of images in the vehicle's forward direction and the vehicle position obtained by real-time positioning of the carrier phase differential technology, obtain the feature points and feature point coordinates of a single frame of image, calculate the distance between each pixel point in the image and the reference point through the vertical-axis pixel difference and vehicle displacement difference of the same feature point in the front and rear frames of images, and obtain the reference distance vector table;

[0079] A ranging module, configured to: obtain the image containing the target object in real time, identify the pixel point where the target object is located, look up the target object pixel point from the reference distance vector table, and output the distance between the target object and the vehicle.

[0080] Embodiment 3

[0081] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the above method are implemented.

[0082] Embodiment 4

[0083] The purpose of this embodiment is to provide a computer-readable storage medium.

[0084] A computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the above method are executed.

[0085] The steps involved in the devices of the above Embodiments 2, 3, and 4 correspond to those of Method Embodiment 1. For specific implementation manners, reference may be made to the relevant description part of Embodiment 1. The term "computer-readable storage medium" should be understood to include a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0086] Those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general-purpose computer device. Optionally, they can be implemented by program codes executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0087] Although the specific implementation manners of the present invention are described above in conjunction with the accompanying drawings, it is not a limitation to the protection scope of the present invention. Those skilled in the art should understand that, based on the technical solutions of the present invention, various modifications or deformations that can be made without creative efforts by those skilled in the art are still within the protection scope of the present invention.

Claims

1. A monocular vision ranging method assisted by carrier phase differential technology, characterized in that, Including: Collecting multiple frames of images in the vehicle's forward direction and the vehicle position located in real time by carrier phase differential technology, calculating the distance between each pixel point in the image and the reference point through the vertical pixel difference and vehicle displacement difference of the same feature point in the front and rear two frames of images, and obtaining a reference distance vector table; Collecting images containing the target object in real time, identifying the pixel points where the target object is located, searching for the target object pixel points in the reference distance vector table, and outputting the distance between the target object and the vehicle; Among them, to construct the reference distance vector table, the specific steps are as follows: S-1-1: Extract the feature points and feature point coordinates of a single frame of image from the collected images, calculate the vertical pixel difference and vehicle displacement difference of the same feature point in the front and rear two frames of images, and establish a pixel distance vector table of pixel coordinates and pixel distance differences; S-1-2: Based on the pixel distance vector table, calculate the distance between each pixel point in the image and the reference point to obtain a reference distance vector table.

2. The monocular vision ranging method assisted by carrier phase differential technology according to claim 1, wherein Using an in-vehicle monocular camera to capture images in the vehicle's forward direction, and using carrier phase differential technology to obtain the vehicle position at the time of each frame of image capture.

3. The monocular vision ranging method assisted by carrier phase differential technology according to claim 1, wherein The method for obtaining the feature points and feature point coordinates: Using Gaussian filtering to perform denoising filtering on the image, using the Fast algorithm to extract feature points, using non-maximum suppression to eliminate redundant feature points, and identifying the coordinates of the remaining feature points.

4. The monocular vision ranging method assisted by carrier phase differential technology according to claim 1, wherein It is also possible to perform real-time update and correction on the pixel distance vector table established in step S-1-1. Compare the newly calculated distance difference with the pixel distance difference in the pixel distance vector table. If it is greater than the threshold, update the pixel distance vector table, otherwise do not update.

5. The monocular vision ranging method assisted by carrier phase differential technology according to claim 1, wherein For the reference point, select the pixel point at the bottommost corner close to the vehicle in the image. The reference distance vector table includes the distances from all pixel points in the image to the reference point.

6. The monocular vision ranging method assisted by carrier phase differential technology according to claim 1, characterized in that Using a target detection system to perform target detection on the monocular vision system images collected, extracting the pixel point coordinates at the bottom edge of the bounding box of the detected target, and obtaining the distance from the target to the vehicle by searching the reference distance vector table.

7. A monocular vision ranging system assisted by carrier phase differential technology, characterized in that, Including: A reference distance vector table construction module, configured to: obtain multiple frames of images in the vehicle's forward direction and the vehicle position located in real time by carrier phase differential technology, obtain the feature points and feature point coordinates of a single frame of image, calculate the distance between each pixel point in the image and the reference point through the vertical pixel difference and vehicle displacement difference of the same feature point in the front and rear two frames of images, and obtain a reference distance vector table; A ranging module, configured to: obtain images containing the target object in real time, identify the pixel points where the target object is located, search for the target object pixel points from the reference distance vector table, and output the distance between the target object and the vehicle; Among them, to construct the reference distance vector table, the specific steps are as follows: S-1-1: Extract the feature points and feature point coordinates of a single frame of image from the collected images, calculate the vertical pixel difference and vehicle displacement difference of the same feature point in the front and rear two frames of images, and establish a pixel distance vector table of pixel coordinates and pixel distance differences; S-1-2: Based on the pixel distance vector table, calculate the distance between each pixel point in the image and the reference point to obtain a reference distance vector table.

8. A computing device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method described in any one of claims 1-6 above.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it performs the steps of the method according to any one of claims 1-6 above.

Images