An optical flow method, system, vehicle and storage medium for obstacle velocity measurement

Abstract

The application relates to the technical field of automatic driving, and discloses a light flow velocity measurement method and system for obstacles, a vehicle and a storage medium, the method comprising the following steps: acquiring two adjacent image frames containing a target obstacle around a vehicle; extracting lane line data and target obstacle grounding point data from the two adjacent image frames respectively; performing light flow estimation on the two adjacent image frames, and extracting a first light flow corresponding to the lane line data and a second light flow corresponding to the target obstacle grounding point data respectively; determining a homography matrix based on the lane line data and the first light flow; determining an absolute light flow corresponding to the target obstacle grounding point data based on the target obstacle grounding point data, the second light flow and the homography matrix, the absolute light flow being the displacement of the target obstacle on the two adjacent image frames; performing coordinate conversion on the absolute light flow, and determining an absolute speed of the target obstacle based on the sampling time of the two adjacent image frames. The application can improve the accuracy, interpretability and usability of the light flow velocity measurement result.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, specifically to a method, system, vehicle, and storage medium for optical flow velocity measurement of obstacles. Background Technology

[0002] In the field of autonomous driving, obstacle speed information is crucial for autonomous driving systems. By accurately measuring the speed of obstacles, autonomous driving systems can predict the future position and trajectory of obstacles, thereby avoiding collisions and accidents. Optical flow, a concept in computer vision used to describe the motion of pixels in an image sequence, can be used to estimate the speed of motion of objects or scenes.

[0003] However, existing optical flow velocimetry often calculates only the vehicle's own speed. It uses the vehicle's forward-facing camera to capture static targets and then uses the optical flow of these static targets to calculate the vehicle's displacement and velocity, without delving into the target vehicle's speed. Furthermore, even when optical flow velocimetry is used to measure the target vehicle's speed, the calculated speed is only relative to the vehicle. To obtain its absolute speed, this relative speed needs to be vector-sumped with the vehicle's speed, making the target vehicle's absolute speed calculation dependent on the vehicle's speed value. This results in the target vehicle's absolute speed being affected by the cumulative error of the vehicle's speed. Moreover, the calculated speed is pixel-level, which is not intuitive for subsequent modules using the target vehicle.

[0004] In summary, existing optical flow velocimetry suffers from calculation errors, cannot efficiently calculate the absolute speed of a target vehicle, and has poor interpretability and usability of the measurement results. Summary of the Invention

[0005] In view of this, the present invention provides an optical flow velocity measurement method, system, vehicle, and storage medium for obstacles, in order to solve the problems mentioned in the above-mentioned technical background, such as calculation errors, inability to efficiently calculate the absolute speed of the target vehicle, and poor interpretability and usability of the speed measurement results.

[0006] In a first aspect, the present invention provides a method for measuring the optical flow velocity of an obstacle, the method comprising:

[0007] Acquire two adjacent frames of images around the vehicle that contain the target obstacle;

[0008] Lane line data and target obstacle ground point data are extracted from two adjacent frames of images, respectively;

[0009] Optical flow estimation is performed on two adjacent frames of images, and the first optical flow corresponding to the lane line data and the second optical flow corresponding to the target obstacle grounding point data are extracted respectively.

[0010] The homography matrix is ​​determined based on lane line data and the first optical flow. The homography matrix represents the ground transformation relationship between two adjacent frames.

[0011] Based on the target obstacle grounding point data, the second optical flow, and the homography matrix, the absolute optical flow corresponding to the target obstacle grounding point data is determined. The absolute optical flow is the displacement of the target obstacle in two adjacent frames.

[0012] The absolute optical flow is transformed into coordinates, and the absolute velocity of the target obstacle is determined based on the sampling time of two adjacent frames.

[0013] This invention obtains the vehicle's pose by measuring the optical flow value of the vehicle's motion, then calculates the target pose, and subsequently solves for the velocity of the target obstacle. By combining the optical flow values ​​of effective feature points on the obstacle, the absolute optical flow value of the target obstacle is calculated, and finally, based on coordinate transformation relationships, the absolute velocity of the target obstacle is calculated. This method reduces the calculation error of optical flow velocimetry, achieves efficient calculation of the absolute velocity of the target obstacle, and ensures the accuracy, interpretability, and usability of the optical flow velocimetry results.

[0014] In one optional implementation, two adjacent frames include the current frame and the next frame; extracting target obstacle grounding point data from the two adjacent frames includes:

[0015] Target detection is performed on the current frame image and the next frame image respectively to obtain the detection box data corresponding to the target obstacle, which is denoted as the first detection box and the second detection box;

[0016] Data points corresponding to the road surface intersection edge are extracted in the first and second detection frames respectively, and the first grounding point set and the second grounding point set are obtained accordingly.

[0017] Determine whether each data point in the first grounding point set meets the first preset threshold range, and when the data point meets the first preset threshold range, determine the data point as the target obstacle grounding point data corresponding to the current frame image;

[0018] Determine whether each data point in the second grounding point set meets the second preset threshold range, and when the data point meets the second preset threshold range, determine the data point as the target obstacle grounding point data corresponding to the next frame image.

[0019] This invention obtains target obstacle ground point data by performing target detection on two adjacent frames of images, determining the corresponding ground point based on the detection result and the intersection edge of the road surface, and comparing the ground point with a preset threshold range. This can improve the accuracy and quality of target obstacle ground point data and help improve the calculation accuracy of optical flow velocity.

[0020] In one optional implementation, extracting lane line data from two adjacent image frames includes:

[0021] Target segmentation is performed on the current frame image and the next frame image respectively to obtain the first lane line point set and the second lane line point set;

[0022] First data corresponding to the intersection of lane lines is extracted from the first lane line point set, and lane line data corresponding to the current frame image is extracted from the first lane line point set based on the first data;

[0023] Extract the second data corresponding to the intersection of lane lines from the second lane line point set, and extract the lane line data corresponding to the next frame image from the second lane line point set based on the second data.

[0024] This invention performs target segmentation on two adjacent image frames and determines the corresponding lane line data based on the intersection of the segmentation results and lane lines, ensuring the accuracy and quality of the lane line data and improving the accuracy and efficiency of the absolute speed of the target obstacle to a certain extent.

[0025] In one optional implementation, two adjacent frames of images include the current frame image and the next frame image; optical flow estimation is performed on the two adjacent frames of images, and the first optical flow corresponding to the lane line data and the second optical flow corresponding to the target obstacle grounding point data are extracted respectively, including:

[0026] Corner detection is performed on the lane line data and target obstacle ground point data in the current frame image and the next frame image respectively to obtain the first corner point data and the second corner point data.

[0027] Optical flow calculation is performed on the first corner point data to obtain the first optical flow corresponding to the lane line data;

[0028] Optical flow calculations are performed on the second corner point data to obtain the second optical flow corresponding to the grounding point data of the target obstacle.

[0029] This invention uses corner detection on lane line data and target obstacle grounding point data to obtain more accurate sparse data, which helps to improve the efficiency of optical flow calculation and greatly ensures the accuracy of optical flow velocity measurement results.

[0030] In one alternative implementation, determining the homography matrix based on lane line data and a first optical flow includes:

[0031] Extract a preset number of data points from the first corner point data corresponding to the current frame image to obtain the first lane point set;

[0032] In the first corner point data corresponding to the next frame image, the data of each lane point set are matched to obtain the second lane point set.

[0033] The data from the first lane point set and the second lane point set are matched with the first optical flow to obtain the third optical flow and the fourth optical flow.

[0034] Based on the mapping relationship between the third and fourth optical flows, the homography matrix is ​​determined.

[0035] This invention determines the homography matrix based on the optical flow data corresponding to the lane line data, which can obtain the mapping relationship between two frames of images and provide a theoretical basis for the subsequent calculation of the absolute velocity of the target obstacle.

[0036] In one optional implementation, determining the absolute optical flow corresponding to the target obstacle grounding point data based on the target obstacle grounding point data, the second optical flow, and the homography matrix includes:

[0037] Extract any target grounding point corresponding to the current frame image from the target obstacle grounding point data;

[0038] Based on the target grounding point and homography matrix, the stationary optical flow corresponding to the target grounding point of the target obstacle in a stationary state is obtained;

[0039] Subtracting the stationary optical flow from the second optical flow yields the absolute optical flow corresponding to the target's grounding point when the target obstacle is in motion.

[0040] This invention obtains the absolute optical flow of the target obstacle by subtracting the stationary optical flow of the target obstacle from the total optical flow obtained from two frames of images, thus providing data support for the subsequent calculation of the absolute velocity of the target obstacle.

[0041] In one optional implementation, a coordinate transformation is performed on the absolute optical flow, and the absolute velocity of the target obstacle is determined based on the sampling time of two adjacent frames, including:

[0042] Obtain the coordinate transformation relationship from the image coordinate system to the camera coordinate system;

[0043] Determine the target optical flow corresponding to the absolute optical flow in the camera coordinate system based on coordinate transformation relationships;

[0044] The absolute velocity of the target obstacle is determined based on the sampling time and target optical flow of two adjacent frames.

[0045] This invention obtains the absolute velocity of the target obstacle through spatial transformation of coordinates and the absolute optical flow of the target obstacle, thus ensuring the interpretability and usability of the optical flow velocity measurement results.

[0046] In an optional implementation, the optical flow velocity measurement method for obstacles further includes:

[0047] Obtain multiple absolute velocities calculated from multiple sets of two adjacent frames containing the target obstacle;

[0048] The average of multiple absolute velocities is calculated to obtain the average absolute velocity, and this average absolute velocity is used as the final absolute velocity of the target obstacle.

[0049] This invention ensures the accuracy of absolute speed calculation by averaging the absolute speeds of multiple target obstacles and using the average absolute speed as the final absolute speed of the target obstacle.

[0050] In a second aspect, the present invention provides an optical flow velocity measurement system for obstacles, the system comprising:

[0051] The data acquisition module is used to acquire two adjacent frames of images around the vehicle that contain the target obstacle;

[0052] The feature extraction module is used to extract lane line data and target obstacle grounding point data from two adjacent frames of images, respectively;

[0053] The optical flow calculation module is used to estimate the optical flow of two adjacent frames of images and extract the first optical flow corresponding to the lane line data and the second optical flow corresponding to the target obstacle ground point data, respectively.

[0054] The first determining module is used to determine the homography matrix based on lane line data and the first optical flow. The homography matrix represents the ground transformation relationship between two adjacent frames of images.

[0055] The second determining module is used to determine the absolute optical flow corresponding to the target obstacle ground point data based on the target obstacle ground point data, the second optical flow, and the homography matrix. The absolute optical flow is the displacement of the target obstacle in two adjacent frames of images.

[0056] The third determination module is used to perform coordinate transformation on the absolute optical flow and determine the absolute velocity of the target obstacle based on the sampling time of two adjacent frames.

[0057] The optical flow velocity measurement system for obstacles of the present invention can reduce the calculation error of optical flow velocity measurement, realize the efficient calculation of the absolute velocity of obstacles, and to a certain extent ensure the accuracy, interpretability and usability of optical flow velocity measurement results.

[0058] Thirdly, the present invention provides a vehicle, the vehicle including a controller, the controller including a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform an optical flow velocity measurement method for obstacles as described in the first aspect or any corresponding embodiment.

[0059] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform an optical flow velocity measurement method for an obstacle according to the first aspect or any corresponding embodiment described above.

[0060] This invention provides an optical flow velocity measurement method for obstacles. It extracts lane line data and target obstacle grounding point data from two adjacent frames of images surrounding the vehicle containing the target obstacle. Optical flow estimation is then performed to obtain a first optical flow corresponding to the lane line data and a second optical flow corresponding to the target obstacle grounding point data. Based on the lane line data and the first optical flow, a homography matrix is ​​determined. This matrix, along with the target obstacle grounding point data and the second optical flow, determines the absolute optical flow corresponding to the target obstacle grounding point data. Coordinate transformation is then performed on the absolute optical flow, and the absolute velocity of the target obstacle is determined based on the sampling time of the two adjacent frames. This invention reduces calculation errors in optical flow velocity measurement, achieves efficient calculation of the absolute velocity of obstacles, and ensures the accuracy, interpretability, and usability of the optical flow velocity measurement results. Attached Figure Description

[0061] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0062] Figure 1 This is a schematic flowchart of an optical flow velocity measurement method for obstacles according to an embodiment of the present invention;

[0063] Figure 2 This is a flowchart illustrating another method for measuring optical flow velocity through an obstacle according to an embodiment of the present invention;

[0064] Figure 3 This is a flowchart illustrating another method for measuring optical flow velocity through obstacles according to an embodiment of the present invention;

[0065] Figure 4 This is a schematic diagram of the optical flow vector according to an embodiment of the present invention;

[0066] Figure 5 This is a schematic diagram of coordinate transformation according to an embodiment of the present invention;

[0067] Figure 6 This is another coordinate transformation diagram according to an embodiment of the present invention;

[0068] Figure 7 This is a structural block diagram of an optical flow velocity measurement system for obstacles according to an embodiment of the present invention;

[0069] Figure 8 This is a schematic diagram of the structure of the vehicle controller according to an embodiment of the present invention. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] This invention provides an embodiment of an optical flow velocity measurement method for obstacles. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0072] This embodiment provides a method for measuring optical flow velocity through obstacles. Figure 1 This is a schematic flowchart of an optical flow velocity measurement method for obstacles according to an embodiment of the present invention, as shown below. Figure 1 As shown, the process includes the following steps:

[0073] Step S101: Acquire two adjacent frames of images around the vehicle containing the target obstacle.

[0074] It should be noted that the target obstacle in this embodiment is any object other than the vehicle itself. The specific attributes of the object are not limited here, and it can be any static or dynamic object in a real driving scenario. For example, the target obstacle includes other vehicles, pedestrians, and public facilities on the road, etc., which are only illustrative examples and are not intended to be limiting. The obstacle is determined based on actual needs.

[0075] In this embodiment, images of the vehicle's surroundings are acquired using an onboard camera. The specific type, number, and installation location of the camera are not limited here, but are determined based on common setups in the field and actual application requirements. For example, a monocular camera mounted on the vehicle's windshield can be used to capture real-time images of the road conditions, collecting corresponding image data to identify and track objects such as roads, vehicles, and pedestrians. This is merely an example.

[0076] In this embodiment, two adjacent frames containing the target obstacle can be arbitrarily selected from multiple frames captured by the monocular camera mounted on the vehicle, requiring that the grounding point of the target obstacle be obtainable in the image.

[0077] Step S102: Extract lane line data and target obstacle grounding point data from two adjacent frames of images respectively.

[0078] It should be noted that the extraction methods for lane line data and target obstacle ground point data in this embodiment are not limited, and the corresponding data are obtained based on commonly used target detection algorithms and image segmentation algorithms in the field. For example, the SSD target detection algorithm is used to identify relevant information of target obstacles in the image, and the U-Net image segmentation algorithm is used to obtain lane line information in the image.

[0079] Step S103: Optical flow estimation is performed on two adjacent frames of images, and the first optical flow corresponding to the lane line data and the second optical flow corresponding to the target obstacle grounding point data are extracted respectively.

[0080] It should be noted that optical flow is the instantaneous velocity of pixels moving on the imaging plane of a spatially moving object. When the time interval between two consecutive frames of an image is very small, it is equivalent to the displacement of the target pixel. Optical flow expresses the changes in an image. By utilizing the changes of pixels in the image in the time domain and the correlation between adjacent frames, the correspondence between the previous frame and the current frame can be found, thereby calculating the motion information of the object between adjacent frames and realizing optical flow estimation. The specific implementation method of optical flow estimation in this embodiment is not limited here, and is determined according to the optical flow calculation methods commonly used in the art, such as calling the classic optical flow calculation function cv::calcOpticalFlowPyrLK() in the OpenCV library of computer vision.

[0081] In this embodiment, optical flow estimation is performed on all pixels in two adjacent frames of images to obtain the optical flow value for each pixel. Optical flow data corresponding to lane line data and target obstacle grounding point data are then extracted from these values. It is important to note that after the full-image optical flow calculation is completed, the optical flow points on the image need to be stored separately for subsequent use.

[0082] Step S104: Determine the homography matrix based on lane line data and the first optical flow. The homography matrix represents the ground transformation relationship between two adjacent frames of images.

[0083] It should be noted that homography matrices are commonly used in computer vision to represent the projection relationship between two planes, describing the mapping relationship from one plane to another. In this embodiment, optical flow velocimetry requires calculating the absolute velocity of the target obstacle. Here, absolute optical flow or absolute velocity refers to the optical flow or velocity of the target obstacle relative to the ground. Specifically, by treating the ground data of the current frame as a plane and the ground data of the next frame as the transformed plane, the obtained homography matrix represents the correspondence between the ground data in two adjacent frames. The obtained homography matrix is ​​then used to calculate the absolute optical flow at the obstacle's grounding point, assuming the target obstacle is stationary.

[0084] It is important to note that homography matrices are only applicable to stationary elements, such as lane line data; or for vehicles, assuming that when the vehicle is stationary, the tire contact points of the vehicle also satisfy this mapping relationship, and the homography matrix can be used to calculate the optical flow value corresponding to the tire contact points of the vehicle in the next frame.

[0085] Step S105: Based on the target obstacle grounding point data, the second optical flow, and the homography matrix, determine the absolute optical flow corresponding to the target obstacle grounding point data. The absolute optical flow is the displacement of the target obstacle in two adjacent frames.

[0086] It should be noted that, in this embodiment, the absolute optical flow corresponding to the grounding point data of the target obstacle is equivalent to the absolute optical flow of the target obstacle, and the absolute velocity of the target obstacle is calculated through this absolute optical flow.

[0087] Step S106: Perform coordinate transformation on the absolute optical flow and determine the absolute velocity of the target obstacle based on the sampling time of two adjacent frames.

[0088] In this embodiment, the absolute optical flow of the target obstacle, i.e., the displacement of the target obstacle in the image, is data in the 2D space of the image. It needs to be converted into displacement in 3D space, i.e., in the world coordinate system or camera coordinate system, in order to solve for the velocity of the target obstacle in 3D space. It should be noted that this embodiment performs a coordinate transformation operation from 2D to 3D space, and the corresponding transformation relationship can be directly obtained through theoretical derivation.

[0089] The optical flow velocity measurement method for obstacles in this invention, within a field of view that ensures sufficient optical flow accuracy, solves the pose of the vehicle camera based on the optical flow of ground feature points, solves the pose of the target obstacle, further solves the absolute optical flow of the target obstacle, and performs spatial transformation to obtain the absolute velocity of the target obstacle. This can reduce the calculation error of optical flow velocity measurement of the target obstacle, achieve efficient calculation of the absolute velocity of the target obstacle, and to a certain extent ensure the accuracy, interpretability, and usability of the optical flow velocity measurement results.

[0090] This embodiment provides a method for measuring optical flow velocity through obstacles. Figure 2 This is a flowchart illustrating another method for measuring optical flow velocity through an obstacle according to an embodiment of the present invention, as shown below. Figure 2 As shown, the process includes the following steps:

[0091] Step S201: Acquire two adjacent frames of images around the vehicle containing the target obstacle. See details below. Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0092] Step S202: Extract lane line data and target obstacle grounding point data from two adjacent frames of images respectively.

[0093] In this embodiment, two adjacent frames include the current frame image and the next frame image.

[0094] Specifically, step S202 above includes:

[0095] Step A1: Perform target detection on the current frame image and the next frame image respectively to obtain the detection box data corresponding to the target obstacle, which are denoted as the first detection box and the second detection box.

[0096] In this embodiment, the target obstacle is any target vehicle among other vehicles around the vehicle. The SSD target detection algorithm is used to detect the target vehicle and obtain the corresponding detection result containing the detection box.

[0097] Step A2: Extract the data points corresponding to the road surface intersection edges in the first and second detection frames respectively to obtain the first grounding point set and the second grounding point set.

[0098] In this embodiment, the grounding data of the target vehicle, i.e., the tire point data, is determined based on the data corresponding to the intersection of the detection frame of the target vehicle and the road surface.

[0099] Step A3: Determine whether each data point in the first grounding point set meets the first preset threshold range, and when the data point meets the first preset threshold range, determine the data point as the target obstacle grounding point data corresponding to the current frame image.

[0100] In this embodiment, the specific value of the first preset threshold range is not limited and can be set according to actual needs. For example, the first preset threshold range is 1, which is only used as an example.

[0101] Step A4: Determine whether each data point in the second grounding point set meets the second preset threshold range, and when the data point meets the second preset threshold range, determine the data point as the target obstacle grounding point data corresponding to the next frame image.

[0102] It should be noted that the specific value of the second preset threshold range in this embodiment is not limited here; it can be set to the same value as the first preset threshold range, and both are set according to actual application requirements. Specifically, this embodiment of the invention performs target detection on two adjacent frames of images, determines the corresponding grounding point based on the detection result and the intersection edge of the road surface, and compares the grounding point with the preset threshold range to obtain target obstacle grounding point data. This can improve the accuracy and quality of target obstacle grounding point data, and help improve the calculation accuracy of optical flow velocity.

[0103] Step A5: Perform target segmentation on the current frame image and the next frame image respectively to obtain the first lane line point set and the second lane line point set.

[0104] In this embodiment, the U-Net image segmentation algorithm is used to obtain the lane line information corresponding to the current frame image and the next frame image, respectively.

[0105] Step A6: Extract the first data corresponding to the intersection position of lane lines from the first lane line point set, and extract the lane line data corresponding to the current frame image from the first lane line point set based on the first data.

[0106] It should be noted that, considering the characteristic of lane lines appearing larger when closer and smaller when farther away in images acquired in actual driving scenarios, two lane lines at a greater distance in the image will intersect, and the intersection point is also called the vanishing line of the lane line data. This embodiment obtains valid lane line data by determining whether the lane line data is within the lane line outline area, that is, whether the lane line is located within the effective area of ​​the vanishing line. This provides effective data for subsequent optical flow calculation of target obstacles, helping to improve the accuracy of optical flow velocity measurement of target obstacles.

[0107] Step A7: Extract the second data corresponding to the intersection position of the lane lines from the second lane line point set, and extract the lane line data corresponding to the next frame image from the second lane line point set based on the second data.

[0108] This invention performs target segmentation on two adjacent image frames and determines the corresponding lane line data based on the intersection of the segmentation results and lane lines, ensuring the accuracy and quality of the lane line data and improving the accuracy and efficiency of the absolute speed of the target obstacle to a certain extent.

[0109] Step S203: Optical flow estimation is performed on two adjacent frames of images, and the first optical flow corresponding to the lane line data and the second optical flow corresponding to the target obstacle grounding point data are extracted respectively.

[0110] It should be noted that optical flow computation includes dense optical flow computation and sparse optical flow computation. Dense optical flow computation refers to optical flow estimation for every pixel in the image; sparse optical flow computation only tracks a subset of points in the image and performs optical flow estimation on the data points in the subset, which has the advantages of fast computation and reliability.

[0111] Specifically, step S203 above includes:

[0112] Step S2031: Perform corner detection on the lane line data and target obstacle ground point data in the current frame image and the next frame image respectively to obtain the first corner point data and the second corner point data.

[0113] This embodiment employs sparse optical flow calculation, using corner detection to filter the corresponding lane line corner data and target obstacle contact point corner data. It should be noted that corner data contains rich local information, referring to pixels that are relatively easy to identify in the image.

[0114] In this embodiment, the specific method of corner detection is not limited and can be adapted according to actual needs. For example, existing corner detection algorithms, such as Harris corner detection algorithm, SIFI algorithm and SURF, can all be used to obtain corners by calling the corresponding functions in the OpenCV library, such as the cv::goodFeaturesToTrack() function.

[0115] It should be noted that in this implementation, the cv::goodFeaturesToTrack() function is called to detect and obtain good corner data in the image, and then the cv::cornerSubPix() function is called to perform sub-pixel level fine correction on the detected corner data, which further improves the sparsity of the corner data and helps to provide higher quality data for subsequent optical flow calculations.

[0116] Step S2032: Perform optical flow calculation on the first corner point data to obtain the first optical flow corresponding to the lane line data.

[0117] Step S2033: Perform optical flow calculation on the second corner point data to obtain the second optical flow corresponding to the grounding point data of the target obstacle.

[0118] In this embodiment, the sparse optical flow of corresponding data from two frames of images is calculated by calling the cv::calcOpticalFlowPyrLK() function. Specifically, by performing corner detection on lane line data and target obstacle grounding point data, more accurate sparse data can be obtained, which helps to improve the efficiency of optical flow calculation and greatly ensures the accuracy of optical flow velocity measurement results.

[0119] Step S204: Determine the homography matrix based on lane line data and the first optical flow. The homography matrix represents the ground transformation relationship between two adjacent frames of images.

[0120] In this embodiment, the homography matrix is ​​applicable to stationary elements, so the homography matrix is ​​determined by using lane line data that is stationary on the ground.

[0121] Specifically, step S204 above includes:

[0122] Step B1: Extract a preset number of data points from the first corner point data corresponding to the current frame image to obtain the first lane point set.

[0123] In this embodiment, the specific setting of the preset quantity is not limited. For example, a preset quantity of 6 is only used as an example and can be adjusted according to actual needs.

[0124] Step B2: In the first corner point data corresponding to the next frame image, match the data based on each data in the first lane point set to obtain the corresponding second lane point set.

[0125] Step B3: Match each data point in the first lane point set and the second lane point set with the first optical flow to obtain the third optical flow and the fourth optical flow.

[0126] In this embodiment, the homography matrix is ​​solved based on the optical flow values ​​corresponding to the lane corner data of the current frame image and the optical flow values ​​corresponding to the lane corner data of the next frame image.

[0127] Step B4: Determine the homography matrix based on the mapping relationship between the third and fourth optical flows.

[0128] In one specific embodiment, the homography matrix can be calculated by calling the cv::findHomography() function in the OpenCV library. The specific calculation process includes: determining whether the first lane point set and the second lane point set are greater than 4 sets. If so, the cv::findHomography() function is called to calculate the ground transformation relationship between two adjacent frames, i.e., the homography matrix.

[0129] Step S205: Based on the target obstacle grounding point data, the second optical flow, and the homography matrix, determine the absolute optical flow corresponding to the target obstacle grounding point data. The absolute optical flow is the displacement of the target obstacle in two adjacent frames.

[0130] Specifically, step S205 above includes:

[0131] Step S2051: Extract any target grounding point corresponding to the current frame image from the target obstacle grounding point data.

[0132] In this embodiment, the method for selecting the target grounding point is not specifically limited and is determined according to actual needs. For example, selecting the average data corresponding to the target obstacle grounding point data is only an example.

[0133] Step S2052: Based on the target grounding point and homography matrix, obtain the stationary optical flow corresponding to the target grounding point of the target obstacle in a stationary state.

[0134] It should be noted that this embodiment obtains the actual total optical flow by calculating the optical flow of the target obstacle's grounding point data in two adjacent frames of images. The total optical flow is affected by the movement of the vehicle and the target obstacle. Specifically, assuming the vehicle is moving and the target obstacle is stationary, the total optical flow is equal to the optical flow caused by the vehicle's movement; assuming the vehicle is stationary and the target obstacle is moving, the total optical flow is equal to the optical flow caused by the obstacle's movement; if both are moving, the total optical flow is equal to the vector sum of the optical flows caused by both. For both the vehicle and the target obstacle, this embodiment decomposes the total optical flow into the sum of the vehicle's motion optical flow and the target obstacle's motion optical flow. Under the conditions of vehicle movement and target obstacle stationary states, the corresponding vehicle motion optical flow is solved using the homography matrix to obtain the target obstacle's motion optical flow.

[0135] In one specific embodiment, the static optical flow of the target obstacle grounding point, i.e., the grounding point of the target vehicle's tires, is solved using the homography matrix. This assumes that all target obstacles within the monocular camera's field of view are stationary. The static optical flow is then:

[0136] tire_v=H×tire_c

[0137] Where, `tire_c` represents the contact point position of the target vehicle's tires in the current frame image, `tire_v` represents the contact point position of the tires in the next frame image assuming the target vehicle is stationary, and `H` is the homography matrix. Specifically, by determining the homography matrix based on the optical flow data corresponding to the lane line data, the mapping relationship between the two frames of images can be obtained, providing a theoretical basis for the subsequent calculation of the absolute velocity of the target obstacle.

[0138] Step S2053: Subtract the stationary optical flow from the second optical flow to obtain the absolute optical flow corresponding to the target ground point when the target obstacle is in motion.

[0139] In this embodiment of the invention, the absolute optical flow of the target obstacle is obtained by subtracting the stationary optical flow of the target obstacle from the total optical flow obtained from two frames of images, which provides data support for the subsequent calculation of the absolute velocity of the target obstacle.

[0140] Step S206: Perform coordinate transformation on the absolute optical flow and determine the absolute velocity of the target obstacle based on the sampling time of two adjacent frames.

[0141] Specifically, step S206 above includes:

[0142] Step C1: Obtain the coordinate transformation relationship from the image coordinate system to the camera coordinate system.

[0143] Step C2: Determine the target optical flow corresponding to the absolute optical flow in the camera coordinate system based on the coordinate transformation relationship.

[0144] In this embodiment, a 2D to 3D conversion is performed, and the target optical flow corresponding to the absolute optical flow in the camera coordinate system 3D is determined based on the existing conversion relationship.

[0145] Step C3: Determine the absolute velocity of the target obstacle based on the sampling time and target optical flow of two adjacent frames.

[0146] In this embodiment, the sampling time between two adjacent frames is not limited and is determined based on actual needs and the camera's inherent properties. Specifically, the absolute velocity of the target obstacle is obtained through spatial coordinate transformation and the absolute optical flow of the obstacle, ensuring the interpretability and usability of the optical flow velocity measurement results.

[0147] It should be noted that the final absolute velocity of the target obstacle can be determined by performing optical flow velocity measurements on the obstacle multiple times in this embodiment, obtaining multiple velocity measurement results, and processing them accordingly to determine the final velocity measurement result. Therefore, in this embodiment, the optical flow velocity measurement method for obstacles further includes:

[0148] Step D1 involves acquiring multiple absolute velocities calculated from multiple sets of images consisting of two adjacent frames containing the target obstacle. It should be noted that the specific values ​​for these multiple sets are not limited here and are determined based on actual needs and the acquired image data.

[0149] Step D2 involves calculating the average of multiple absolute velocities to obtain the average absolute velocity, which is then used as the final absolute velocity of the target obstacle. Specifically, using the average absolute velocity as the final absolute velocity of the target obstacle significantly improves the accuracy of the absolute velocity calculation.

[0150] In one specific embodiment, a procedure for estimating the optical flow velocity of a target vehicle under monocular camera conditions is provided, see reference. Figure 3 Specifically, it includes:

[0151] Step E1 involves extracting feature points from two adjacent images captured by the monocular camera mounted on the vehicle in the autonomous driving scenario, then performing optical flow estimation to obtain the optical flow values ​​of lane line feature points and the tire contact points of the target vehicle.

[0152] In this embodiment, the process for obtaining the tire contact point within the vehicle detection frame and the lane line feature points located within the lane line contour area is as follows:

[0153] Step E11: Determine whether the lane line feature points are located within the lane line contour to obtain the lane line feature points.

[0154] In this embodiment, it is first determined whether the lane line feature point is located below the vanishing line of the image (the vanishing line can be obtained by calculating the intersection of two parallel lane lines). If not, it means that the feature point is not on the ground and the feature point is skipped directly; if so, it is further calculated whether the point is located within the lane line outline, which can save computing resources.

[0155] Specifically, for feature points within the lane line contour, also known as lane line corner points, non-corner points can be filtered out during feature point detection by reasonably controlling the threshold.

[0156] Step E12: Obtain the tire contact point.

[0157] For feature points within the target vehicle detection box, it is necessary to further determine which one is the tire point by judging the distance between the feature point and the edge of the detection box in the current frame image; at the same time, it is judged that the feature point and the corresponding edge of the detection box in the next frame image; if both are within the set distance threshold range, then the feature point is considered to be the tire contact point.

[0158] Step E2: Solve for the homography matrix.

[0159] In this embodiment, the homography matrix is ​​solved using lane corner data.

[0160] Step E3: Use the pose of the vehicle camera to solve the stationary optical flow at the tire contact point of the target vehicle.

[0161] Step E4: Calculate the absolute optical flow at the tire contact point of the target vehicle by combining the actual optical flow at the tire contact point.

[0162] Figure 4 This is a schematic diagram of optical flow vectors according to an embodiment of the present invention, wherein the solid-lined frame represents the vehicle, and the dashed-lined frame represents the dynamic obstacle. The motion directions of the vehicle and the dynamic obstacle are as follows: Figure 4 As shown. It should be noted that tire_c is the actual tire contact point position in the current frame; tire_r is the actual tire contact point position in the next frame; tire_v is the tire contact point position in the next frame assuming the dynamic obstacle is stationary; and tire_a is the tire contact point position in the next frame assuming the dynamic obstacle is moving.

[0163] In this embodiment, the optical flow vector CR (the optical flow of the obstacle's tire point in the actual situation), which is composed of tire_c to tire_r, is the total optical flow, which is obtained based on the optical flow estimation of two adjacent frames. Assuming that the optical flow corresponding to the obstacle being stationary and the vehicle moving is the optical flow vector CV composed of tire_c to tire_v, it is solved by the homography matrix. Then, tire_c to tire_a is the optical flow vector CA caused by the dynamic obstacle moving when the vehicle is stationary. It is a quantity to be determined. The optical flow vector CA caused by the obstacle moving is solved by the relationship CV+CA=CR. This optical flow represents the displacement of the dynamic obstacle on the image within the time interval Δt (the sampling time of two adjacent frames).

[0164] It should be noted that the velocity can be decomposed into lateral velocity and total velocity. In this embodiment, the velocity of the target obstacle in 3D space is solved by decomposing the optical flow displacement.

[0165] In this embodiment, the coordinate transformation operation from 2D to 3D space is implemented using a pinhole camera model, as follows:

[0166] The solution for the lateral velocity is as follows: Figure 5As shown, the coordinate system above the dashed line is the camera coordinate system, which is a 3D space, with the camera's optical center at its origin. The coordinate system below the dashed line is the image coordinate system, which has a translational and scaling relationship with the pixel coordinate system. The following relationship can be easily derived from similar triangles:

[0167]

[0168] Among them, f y Let f be the focal length of the camera along the horizontal axis, Δy be the optical flow along the horizontal axis (obtained by CA decomposition of the resulting optical flow vector), d be the distance from the target obstacle to the camera in the current frame image, and f be the focal length of the camera along the horizontal axis. y Δy and d are known quantities; ΔY is the displacement of the target obstacle in the horizontal direction during time Δt, which is the quantity to be solved. Then the horizontal velocity is ΔY / Δt.

[0169] The solution for longitudinal velocity is as follows: Figure 6 As shown, the left side of the long dashed line represents the image coordinate system, and the right side represents the camera coordinate system. Point A is the location of the target obstacle in the current frame, point B is the location of the target obstacle in the next frame, A' and B' are the corresponding points in the image coordinate system, point O is the camera's optical center, O' is its ground projection point, h is the camera height, f is the camera's physical focal length, d is the distance to the obstacle in the current frame, and ΔX is the displacement of the target obstacle between the two frames (this is the unknown quantity). AC is an auxiliary line. Since triangle OA'B' is similar to triangle OCA, we know that:

[0170]

[0171] Furthermore, since triangle ABC is similar to triangle O'BO, we know that:

[0172]

[0173] Combining equations (1) and (2) above, and transforming the image coordinate system to the pixel coordinate system, with the optical flow along the vertical axis taking Δx, the final result is as follows:

[0174]

[0175] The magnitude of the longitudinal velocity is ΔX / Δt. By combining the lateral velocity ΔY / Δt and the longitudinal velocity ΔX / Δt, the absolute velocity of the target obstacle can be obtained.

[0176] In summary, this invention obtains the vehicle's pose by measuring the optical flow value of the vehicle's motion, then calculates the target pose, and subsequently solves for the velocity of the target obstacle. By combining the optical flow values ​​of effective feature points on the obstacle, the absolute optical flow value of the target obstacle is calculated, and finally, based on coordinate transformation relationships, the absolute velocity of the target obstacle is calculated. This method reduces the calculation error of optical flow velocimetry, achieves efficient calculation of the absolute velocity of the target obstacle, and ensures the accuracy, interpretability, and usability of the optical flow velocimetry results.

[0177] This embodiment also provides an optical flow velocity measurement system for obstacles, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, a "module" can be a combination of software and / or hardware that performs a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0178] This invention provides an optical flow velocity measurement system for obstacles, such as... Figure 7 As shown, the system includes:

[0179] The data acquisition module 701 is used to acquire two adjacent frames of images around the vehicle that contain the target obstacle.

[0180] The feature extraction module 702 is used to extract lane line data and target obstacle ground point data from two adjacent frames of images, respectively.

[0181] The optical flow calculation module 703 is used to estimate the optical flow of two adjacent frames of images and extract the first optical flow corresponding to the lane line data and the second optical flow corresponding to the target obstacle ground point data, respectively.

[0182] The first determining module 704 is used to determine the homography matrix based on lane line data and the first optical flow. The homography matrix represents the ground transformation relationship between two adjacent frames of images.

[0183] The second determining module 705 is used to determine the absolute optical flow corresponding to the target obstacle ground point data based on the target obstacle ground point data, the second optical flow, and the homography matrix. The absolute optical flow is the displacement of the target obstacle in two adjacent frames.

[0184] The third determining module 706 is used to perform coordinate transformation on the absolute optical flow and determine the absolute velocity of the target obstacle based on the sampling time of two adjacent frames.

[0185] In some optional implementations, the feature extraction module 702 includes: a first extraction submodule, a second extraction submodule, a third extraction submodule, a fourth extraction submodule, a fifth extraction submodule, a sixth extraction submodule, and a seventh extraction submodule; wherein, the first extraction submodule is used to perform target detection on the current frame image and the next frame image respectively, to obtain detection box data corresponding to the target obstacle, denoted as the first detection box and the second detection box; the second extraction submodule is used to extract data points corresponding to the road surface intersection edge in the first detection box and the second detection box respectively, to obtain a first grounding point set and a second grounding point set accordingly; the third extraction submodule is used to determine whether each data point in the first grounding point set meets a first preset threshold range, and when the data point meets the first preset threshold range, the data point is determined as the target obstacle corresponding to the current frame image. The first extraction module is used to extract obstacle grounding point data; the second extraction module is used to determine whether each data point in the second grounding point set meets the second preset threshold range, and when the data point meets the second preset threshold range, the data point is determined as the target obstacle grounding point data corresponding to the next frame image; the third extraction module is used to perform target segmentation on the current frame image and the next frame image respectively, and obtain the first lane line point set and the second lane line point set accordingly; the fourth extraction module is used to extract the first data corresponding to the lane line intersection position from the first lane line point set, and extract the lane line data corresponding to the current frame image from the first lane line point set based on the first data; the fifth extraction module is used to extract the second data corresponding to the lane line intersection position from the second lane line point set, and extract the lane line data corresponding to the next frame image from the second lane line point set based on the second data.

[0186] In some optional implementations, the optical flow calculation module 703 includes: a first calculation submodule, a second calculation submodule, and a third calculation submodule; wherein, the first calculation submodule is used to perform corner detection on the lane line data and target obstacle ground point data in the current frame image and the next frame image respectively, to obtain first corner point data and second corner point data accordingly; the second calculation submodule is used to perform optical flow calculation on the first corner point data to obtain the first optical flow corresponding to the lane line data; the third calculation submodule is used to perform optical flow calculation on the second corner point data to obtain the second optical flow corresponding to the target obstacle ground point data.

[0187] In some optional implementations, the first determining module 704 includes: a first determining submodule, a second determining submodule, a third determining submodule, and a fourth determining submodule; wherein, the first determining submodule is used to extract a preset number of data from the first corner point data corresponding to the current frame image to obtain a first lane point set; the second determining submodule is used to match each data in the first lane point set in the first corner point data corresponding to the next frame image to obtain a second lane point set; the third determining submodule is used to match each data in the first lane point set and the second lane point set with a first optical flow respectively to obtain a third optical flow and a fourth optical flow; the fourth determining submodule is used to determine the homography matrix based on the mapping relationship between the third optical flow and the fourth optical flow.

[0188] In some optional implementations, the second determining module 705 includes: a first filtering submodule, a second filtering submodule, and a third filtering submodule; wherein, the first filtering submodule is used to extract any target ground point corresponding to the current frame image from the target obstacle ground point data; the second filtering submodule is used to obtain the stationary optical flow corresponding to the target ground point in the stationary state of the target obstacle based on the target ground point and the homography matrix; and the third filtering submodule is used to subtract the stationary optical flow from the second optical flow to obtain the absolute optical flow corresponding to the target ground point in the moving state of the target obstacle.

[0189] In some optional implementations, the third determining module 706 includes: a first acquisition submodule, a second acquisition submodule, and a third acquisition submodule; wherein, the first acquisition submodule is used to acquire the coordinate transformation relationship from the image coordinate system to the camera coordinate system; the second acquisition submodule is used to determine the target optical flow corresponding to the absolute optical flow in the camera coordinate system based on the coordinate transformation relationship; and the third acquisition submodule is used to determine the absolute velocity of the target obstacle based on the sampling time of two adjacent frames and the target optical flow.

[0190] In some optional implementations, the system further includes: an average calculation submodule, used to acquire multiple absolute velocities calculated from multiple sets of adjacent two-frame images containing the target obstacle; to perform an average calculation on the multiple absolute velocities to obtain an average absolute velocity, and to use the average absolute velocity as the final absolute velocity of the target obstacle.

[0191] Further functional descriptions of the above modules are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0192] In this embodiment, the optical flow velocity measurement system for obstacles is presented in the form of functional units. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0193] The optical flow velocity measurement system for obstacles in this invention can greatly reduce the calculation error of optical flow velocity measurement, realize the efficient calculation of the absolute velocity of obstacles, and to a certain extent ensure the accuracy, interpretability and usability of optical flow velocity measurement results.

[0194] This invention also provides a vehicle, which includes a controller. In this embodiment, the controller is a vehicle domain controller, used for powering on / off and waking up / hibernating its subordinate sub-controllers and network nodes. Simultaneously, each of its power supply interfaces can collect the real-time output current. Other controllers with the above functions are also applicable.

[0195] Figure 8 This is a schematic diagram of the structure of the controller provided in an optional embodiment of the present invention, as shown below. Figure 8 As shown, the controller includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the controller, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple controllers can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 Take a processor 10 as an example.

[0196] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0197] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0198] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the controller. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the controller via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0199] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0200] The controller also includes a communication interface 30 for the main control chip to communicate with other devices or communication networks.

[0201] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium after being downloaded via a network. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor main control chips, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the methods shown in the above embodiments are implemented.

[0202] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for measuring optical flow velocity through obstacles, characterized in that, The method includes: Acquire two adjacent frames of images around the vehicle that contain the target obstacle; Lane line data and target obstacle ground point data are extracted from the two adjacent image frames respectively; Optical flow estimation is performed on the two adjacent frames of images, and the first optical flow corresponding to the lane line data and the second optical flow corresponding to the target obstacle grounding point data are extracted respectively. The two adjacent frames of images include the current frame image and the next frame image. Based on the lane line data and the first optical flow, a homography matrix is ​​determined, wherein the homography matrix represents the ground transformation relationship between two adjacent frames of images; Based on the target obstacle grounding point data, the second optical flow, and the homography matrix, the absolute optical flow corresponding to the target obstacle grounding point data is determined, where the absolute optical flow is the displacement of the target obstacle in two adjacent frames. The determination of the absolute optical flow based on the target obstacle grounding point data, the second optical flow, and the homography matrix includes: extracting any target grounding point corresponding to the current frame image from the target obstacle grounding point data; obtaining the stationary optical flow corresponding to the target grounding point when the target obstacle is stationary, based on the target grounding point and the homography matrix; and subtracting the stationary optical flow from the second optical flow to obtain the absolute optical flow corresponding to the target grounding point when the target obstacle is in motion. The absolute optical flow is transformed into coordinates, and the absolute velocity of the target obstacle is determined based on the sampling time of the two adjacent frames.

2. The method for measuring optical flow velocity of obstacles according to claim 1, characterized in that, Extracting target obstacle grounding point data from the two adjacent image frames includes: Target detection is performed on the current frame image and the next frame image respectively to obtain the detection box data corresponding to the target obstacle, which is denoted as the first detection box and the second detection box; Data points corresponding to the road surface intersection edge are extracted in the first detection frame and the second detection frame respectively, and the first grounding point set and the second grounding point set are obtained accordingly. Determine whether each data point in the first grounding point set meets the first preset threshold range, and when the data point meets the first preset threshold range, determine the data point as the target obstacle grounding point data corresponding to the current frame image; Determine whether each data point in the second grounding point set meets the second preset threshold range, and when the data point meets the second preset threshold range, determine the data point as the target obstacle grounding point data corresponding to the next frame image.

3. The method for measuring optical flow velocity of obstacles according to claim 2, characterized in that, Extracting lane line data from two adjacent image frames includes: Target segmentation is performed on the current frame image and the next frame image respectively to obtain the first lane line point set and the second lane line point set; First data corresponding to the intersection of lane lines is extracted from the first set of lane line points, and lane line data corresponding to the current frame image is extracted from the first set of lane line points based on the first data. Second data corresponding to the intersection positions of lane lines are extracted from the second lane line point set, and lane line data corresponding to the next frame image is extracted from the second lane line point set based on the second data.

4. The method for measuring optical flow velocity of obstacles according to claim 1, characterized in that, The step of estimating optical flow for two adjacent image frames and extracting the first optical flow corresponding to the lane line data and the second optical flow corresponding to the target obstacle grounding point data includes: Corner detection is performed on the lane line data and target obstacle ground point data in the current frame image and the next frame image respectively to obtain the first corner point data and the second corner point data. Optical flow calculation is performed on the first corner point data to obtain the first optical flow corresponding to the lane line data; Optical flow calculation is performed on the second corner point data to obtain the second optical flow corresponding to the grounding point data of the target obstacle.

5. The method for measuring the optical flow velocity of an obstacle according to claim 4, characterized in that, The determination of the homography matrix based on the lane line data and the first optical flow includes: Extract a preset number of data points from the first corner point data corresponding to the current frame image to obtain the first lane point set; In the first corner point data corresponding to the next frame image, the data of the first lane point set are matched to obtain the second lane point set. The data in the first lane point set and the second lane point set are matched with the first optical flow to obtain the third optical flow and the fourth optical flow respectively. Based on the mapping relationship between the third optical flow and the fourth optical flow, the homography matrix is ​​determined.

6. The method for measuring optical flow velocity of obstacles according to claim 1, characterized in that, The step of performing coordinate transformation on the absolute optical flow and determining the absolute velocity of the target obstacle based on the sampling time of two adjacent frames includes: Obtain the coordinate transformation relationship from the image coordinate system to the camera coordinate system; Based on the coordinate transformation relationship, determine the target optical flow corresponding to the absolute optical flow in the camera coordinate system; The absolute velocity of the target obstacle is determined based on the sampling time of the two adjacent frames and the target optical flow.

7. The method for measuring the optical flow velocity of an obstacle according to any one of claims 1 to 6, characterized in that, The method further includes: Multiple absolute velocities are obtained by calculating from multiple sets of two adjacent frames containing the target obstacle; The average of the multiple absolute velocities is calculated to obtain the average absolute velocity, and the average absolute velocity is used as the final absolute velocity of the target obstacle.

8. An optical flow velocity measurement system for obstacles, characterized in that, The system includes: The data acquisition module is used to acquire two adjacent frames of images around the vehicle that contain the target obstacle; The feature extraction module is used to extract lane line data and target obstacle grounding point data from the two adjacent frames of images, respectively; The optical flow calculation module is used to estimate the optical flow of the two adjacent frames of images, and extract the first optical flow corresponding to the lane line data and the second optical flow corresponding to the target obstacle grounding point data, respectively. The two adjacent frames of images include the current frame image and the next frame image. The first determining module is used to determine a homography matrix based on the lane line data and the first optical flow, wherein the homography matrix represents the ground transformation relationship between two adjacent frames of images; The second determining module is used to determine the absolute optical flow corresponding to the target obstacle ground point data based on the target obstacle ground point data, the second optical flow, and the homography matrix, wherein the absolute optical flow is the displacement of the target obstacle in two adjacent frames; wherein, determining the absolute optical flow corresponding to the target obstacle ground point data based on the target obstacle ground point data, the second optical flow, and the homography matrix includes: extracting any target ground point corresponding to the current frame image from the target obstacle ground point data; obtaining the stationary optical flow corresponding to the target ground point when the target obstacle is stationary based on the target ground point and the homography matrix; and subtracting the stationary optical flow from the second optical flow to obtain the absolute optical flow corresponding to the target ground point when the target obstacle is in motion; The third determining module is used to perform coordinate transformation on the absolute optical flow and determine the absolute velocity of the target obstacle based on the sampling time of the two adjacent frames of images.

9. A vehicle, characterized in that, The vehicle includes a controller, which includes a memory and a processor, the memory and the processor being communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the optical flow velocity measurement method for obstacles according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the optical flow velocity measurement method for the obstacle according to any one of claims 1 to 7.

Images