Vehicle speed frame-by-frame detection method based on vehicle-mounted video image and application thereof

Abstract

The invention belongs to the technical field of vehicle speed detection, and particularly relates to a vehicle speed frame-by-frame detection method based on a vehicle-mounted video image and application of the vehicle speed frame-by-frame detection method. Due to the defects of poor mobility and high cost in speed measurement of traditional fixed equipment, although speed measurement based on video images rises, a traditional method depends on camera parameter calibration, vehicle body structure parameters and front vehicle reference, vehicle speed detection is large in limitation, frame-by-frame detection is difficult, and precision is poor. According to the vehicle speed frame-by-frame detection method based on the vehicle-mounted video image, camera calibration and taking a front vehicle as a target are not needed, and the average vehicle speed between any two adjacent frames can be accurately calculated only by means of the lane marking line with the known actual distance on the driving road surface under the conditions that camera parameters are unknown and no reference vehicle exists in front. The method breaks through the limitation of traditional vehicle-mounted video speed measurement, can adapt to various complex scenes, provides more reliable and efficient technical support for traffic flow acquisition and traffic accident identification, and greatly improves the accuracy of vehicle speed detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle speed detection, and particularly relates to a frame-by-frame vehicle speed detection method based on vehicle-mounted video images and its application. Background Art

[0002] Under the background of the rapid economic development and the rapid growth of the number of automobiles, problems such as traffic congestion and traffic accidents have increased day by day. By real-time detecting the vehicle speed, it can provide a data basis for obtaining traffic flow, and provide important theoretical support and factual basis in traffic accident identification. Traditional vehicle speed detection is carried out by fixed devices such as infrared rays, ground induction lines, radars, and lasers. These methods have disadvantages such as poor mobility, high installation and maintenance costs, etc.

[0003] With the gradual improvement of the road monitoring system in China and the continuous development of devices such as driving recorders, in recent years, the speed measurement method based on video images has been widely used in vehicle speed detection. The commonly used vehicle speed detection methods require camera parameters for camera calibration, or use vehicle body structure parameters combined with a video vehicle speed calculation model. Moreover, for vehicle-mounted video images, a reference vehicle in front is also required to calculate the vehicle speed, which makes such methods have great limitations and it is difficult to detect the vehicle speed frame by frame, or there are large errors in the accuracy of the vehicle speed. Therefore, the present invention uses relevant algorithms of image processing, without camera calibration and a vehicle in front as a target, and designs a vehicle speed calculation method based on vehicle-mounted video images to solve the problems existing in the existing methods. Summary of the Invention

[0004] The purpose of the present invention is to overcome the above-mentioned disadvantages of the prior art and provide a frame-by-frame vehicle speed detection method based on vehicle-mounted video images and its application.

[0005] To achieve the above purpose, the present invention provides the following technical solutions:

[0006] On the one hand, the present invention provides a frame-by-frame vehicle speed detection method based on vehicle-mounted video images, which specifically includes the following steps:

[0007] S1. Image preprocessing: Obtain the vehicle-mounted video to be measured and preprocess it into an image sequence; specifically:

[0008] S11. Frame rate detection: Detect the frame rate and complete the frame rate compensation of the vehicle-mounted video to be measured;

[0009] S12. Frame-by-frame division: Process the video into a frame-by-frame image sequence according to the video frame rate;

[0010] S2. Select a reference object: Select a lane marking on the road surface where the vehicle is driving in the vehicle-mounted video to be measured, and the actual distance L of which is known, as an environmental reference object;

[0011] S3. Inverse perspective transformation: Arbitrarily select two consecutive frames of images in the image sequence, and use the inverse perspective transformation algorithm to transform the road surface area where the vehicle is driving to obtain a bird's-eye view of the road surface with lane markings in two consecutive frames of images. Specifically:

[0012] S31. Select a first rectangular area on the road surface where the vehicle is driving in each frame of image, and determine the four vertices of the first rectangular area on each frame of image as the given source points (x n , y n );

[0013] S32. According to the known actual distances such as road width, lane marking length, marking interval, etc., and the size of the computer window, select a specific scaling ratio to determine the four vertices of the second rectangular area formed in the bird's-eye view corresponding to the first rectangular area as the target points (X n , Y n );

[0014] S33. Substitute the given source points (x n , y n ) and the target points (X n , Y n ) into the inverse perspective transformation relation formula 1, and then the inverse perspective transformation matrix H can be obtained:

[0015]

[0016] Among them, the inverse perspective transformation matrix H is a 3-order matrix as shown in formula 2:

[0017]

[0018] Since H is a transformation matrix for homogeneous coordinates, h 33 in the matrix can be set to 1, and

[0019] Formula 1 is simplified to obtain formula 3:

[0020]

[0021] S34. Each pixel coordinate (u, w) in the obtained bird's-eye view can be mapped back to the first rectangular area in reverse according to the inverse matrix H -1 of the inverse perspective transformation matrix, and the corresponding coordinates (x i , y i ) are calculated. Formula 4 is as follows:

[0022]

[0023] S35. According to the coordinates (x i , y i), extracting the color values ​​of the corresponding coordinates and performing interpolation processing, and filling the obtained color values ​​into the corresponding positions of the second rectangular area to generate a bird's-eye view of the road surface of the first rectangular area;

[0024] S4. Feature point selection: Select feature points at both ends of the lane markings in two consecutive frames of road bird's-eye view images, denoted as p1, p2, p3, ..., p n , record the pixel coordinates y of each feature point in the Y direction m value, and obtain the image distance l of the lane marking; specifically:

[0025] S41, establishing an image pixel coordinate system with the upper left corner of the image of the road bird's-eye view as the coordinate origin;

[0026] S42, select any lane marking, select its endpoint in the two frames of image as a feature point, and record its pixel coordinate y in the Y direction m value;

[0027] S43, calculating the image distance l of the lane marking in one frame according to Formula 5;

[0028] l=y m -y m-1 Formula 5

[0029] S5. Calculation of vehicle speed: The vehicle speed is calculated by the actual displacement distance of the feature point in two consecutive frames of images; specifically:

[0030] S51, find the proportional relationship K between the image distance l and the actual distance L of the lane marking in the road surface bird's-eye view, as shown in Formula 6:

[0031]

[0032] S52, calculating the pixel difference a in the Y direction of the same end point of the same lane marking in two consecutive frames of images, which is the displacement image distance s of the feature point;

[0033] S53, according to formula 7, the actual displacement distance S of the feature point is calculated.

[0034]

[0035] S54, the frame rate of the video to be tested is n, and the average speed v of the vehicle in the two frames before and after is calculated using Formula 8, in km / h, and the actual vehicle speed v is finally obtained:

[0036] v=3.6×S×n Formula 8.

[0037] On the other hand, the present invention provides an application of a method for detecting vehicle speed frame by frame based on vehicle-mounted video images, which is applied to detecting vehicle speed based on vehicle-mounted video images.

[0038] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:

[0039] Due to the disadvantages of poor mobility and high cost in the speed measurement of traditional fixed devices, although the speed measurement based on video images has emerged, the traditional methods rely on camera parameter calibration, vehicle body structure parameters and the reference of the vehicle in front. The limitations of vehicle speed detection are large, it is difficult to detect frame by frame and the accuracy is poor. The vehicle speed frame-by-frame detection method based on in-vehicle video images of the present invention does not require camera calibration and the vehicle in front as a target. Based on the lane markings with known actual distances on the driving road surface, it can accurately calculate the average vehicle speed between any two adjacent frames even when the camera parameters are unknown and there is no reference vehicle in front. This method breaks through the limitations of traditional in-vehicle video speed measurement, has a wide range of applications, can adapt to various complex scenarios, provides a more reliable and efficient technical support for traffic flow acquisition and traffic accident identification, and greatly improves the accuracy of vehicle speed detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings here are incorporated into the specification and form a part of this specification, and are used together with the specification to explain the principles of the present invention.

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0042] Figure 1 It is a flowchart of the vehicle speed frame-by-frame detection method of the present invention;

[0043] Figure 2 It is the 9th frame image of 2 minutes and 15 seconds in the in-vehicle video image of Embodiment 1 of the present invention;

[0044] Figure 3 It is the 10th frame image of 2 minutes and 15 seconds in the in-vehicle video image of Embodiment 1 of the present invention;

[0045] Figure 4 It is the bird's-eye view of the road surface obtained by inverse perspective transformation of the 9th and 10th frame images of Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present invention. On the contrary, they are only examples consistent with some aspects of the present invention detailed in the appended claims.

[0047] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

[0048] The present invention provides a method for detecting vehicle speed frame by frame based on in-vehicle video images. As shown in Figure 1 the following, the specific steps are as follows:

[0049] S1. Image preprocessing: Obtain the in-vehicle video to be measured and preprocess it into an image sequence; specifically:

[0050] S11. Frame rate detection: Detect the frame rate and complete the frame rate of the in-vehicle video to be measured;

[0051] S12. Frame-by-frame division: Process the video into a frame-by-frame image sequence according to the video frame rate;

[0052] S2. Select a reference object: Select a lane marking with a known actual distance L on the road surface where the vehicle is driving in the in-vehicle video to be measured as an environmental reference object;

[0053] S3. Inverse perspective transformation: Optionally select two consecutive frames of images in the image sequence, and use the inverse perspective transformation algorithm to transform the vehicle driving road surface area to obtain a bird's-eye view of the road surface containing lane markings in two consecutive frames of images; specifically:

[0054] S31. Select a first rectangular area on the road surface where the vehicle is driving in each frame of the image, and determine the four vertices of the first rectangular area on each frame of the image as the given source points (x n , y n );

[0055] S32. According to the known actual distances such as the road width, lane marking length, marking interval, etc., and the size of the computer window, select a specific scaling ratio to determine the four vertices of the second rectangular area formed in the bird's-eye view corresponding to the first rectangular area as the target points (X n , Y n );

[0056] S33. Substitute the given source points (x n , y n ) and the target points (X n , Y n ) into the inverse perspective transformation relation formula 1, and the inverse perspective transformation matrix H can be obtained:

[0057]

[0058] Among them, the inverse perspective transformation matrix H is a 3-order matrix as shown in formula 2:

[0059]

[0060] Since H is a transformation matrix for homogeneous coordinates, h in the matrix can be 33 Set to 1,

[0061] Formula 1 is simplified to get Formula 3:

[0062]

[0063] S34, each pixel coordinate (u, w) in the obtained bird's-eye view can be calculated according to the inverse matrix H of the inverse perspective transformation matrix -1 , and then the corresponding coordinates (x i ,y i ), Formula 4 is as follows:

[0064]

[0065] S35, the coordinates (x i ,y i ), extracting the color values ​​of the corresponding coordinates and performing interpolation processing, and filling the obtained color values ​​into the corresponding positions of the second rectangular area to generate a bird's-eye view of the road surface of the first rectangular area;

[0066] S4. Feature point selection: Select feature points at both ends of the lane markings in two consecutive frames of road bird's-eye view images, denoted as p1, p2, p3, ..., p n , record the pixel coordinates y of each feature point in the Y direction m value, and obtain the image distance l of the lane marking; specifically:

[0067] S41, establishing an image pixel coordinate system with the upper left corner of the image of the road bird's-eye view as the coordinate origin;

[0068] S42, select any lane marking, select its endpoint in the two frames of image as a feature point, and record its pixel coordinate y in the Y direction m value;

[0069] S43, according to Formula 5, the image distance l of the lane marking in one frame is calculated:

[0070] l=y m -y m-1 Formula 5

[0071] S5. Calculation of vehicle speed: The vehicle speed is calculated by the actual displacement distance of the feature point in two consecutive frames of images; specifically:

[0072] S51, find the proportional relationship K between the image distance l and the actual distance L of the lane marking in the road surface bird's-eye view, as shown in Formula 6:

[0073]

[0074] S52. Calculate the pixel difference a in the Y direction of the same end point of the same lane marking in two consecutive frames of images, which is the displacement image distance s of the feature point;

[0075] S53. Calculate the actual displacement distance S of the feature point according to Equation 7,

[0076]

[0077] S54. The frame rate of the video to be measured is n. Use Equation 8 to calculate the average speed v of the vehicle between two consecutive frames, with the unit of km / h, and finally obtain the actual vehicle speed v:

[0078] v = 3.6 × S × n Equation 8.

[0079] To prove the effect of the vehicle speed frame-by-frame detection method based on in-vehicle video images of the present invention, the following embodiments are provided for verification.

[0080] Embodiment 1

[0081] This embodiment provides a vehicle speed frame-by-frame detection method based on in-vehicle video images, specifically including the following steps:

[0082] S1. After performing frame rate detection and frame rate completion on the video to be measured for the vehicle, it can be obtained that the in-vehicle video image is a video of the vehicle driving on the highway, with a duration of 3 minutes, a frame rate n of 30 fps, and an image sequence of 5400 frames is obtained after frame-by-frame division;

[0083] S2. The lane markings on this road have an actual length of 6 m, a lane marking interval length of 9 m, and a lane width of 3.75 m; Select the area framed by two lane markings in the lane where the vehicle is located as the first rectangular area, where the lane marking is used as the environmental reference object, that is, L = 6 m;

[0084] S3. Select the 9th frame image and the 10th frame image. Refer to Figure 2 As shown, the pixel coordinates of the four vertices of the first rectangular area in the 9th frame image, that is, the given source points, are (898, 774), (1100, 775), (621, 955), and (1372, 956);

[0085] According to the known actual distances within this area and the computer window size, determine that the magnification ratio of the target point to the real-world object point is 30, that is, the coordinates of the target point are: (0, 0), (113, 0), (0, 630), (113, 630);

[0086] According to the inverse perspective transformation, the bird's-eye view of the road surface of the 9th frame image is obtained, as shown in Figure 4 (a), and the bird's-eye view of the road surface of the 10th frame image is obtained, as shown inFigure 4 as shown in (b);

[0087] S4. First, mark the feature points on the edges of the same lane marking in the bird's-eye view of the road surface. In the 9th frame image, p1 and p2 are the pixel values of the two endpoints of the same lane marking in the Y direction respectively. After calculation, p1 is 450 and p2 is 630; after p1 is displaced, it is marked as p3 in the 10th frame image, and after calculation, p3 is 475; according to Equation 5, the image distance l of the lane marking in the bird's-eye view of the road surface in the 9th frame image is obtained as l = p2 - p1 = 180 px.

[0088] S5. Calculate the proportional relationship between the image distance l of the lane marking in the bird's-eye view of the road surface and its actual distance L through Equation 6.

[0089] Since the feature points p1 and p3 are the same endpoint of the selected lane marking in two frames of images, first calculate the pixel difference a = p3 - p1 = 25 px between the two points in the two frames of images. The pixel difference a is equivalent to the displacement image distance s of the feature point, and then calculate the actual displacement distance of the feature point according to Equation 7. Finally, according to Equation 8, the average vehicle speed v between the 9th frame image and the 10th frame image is obtained as v = 3.6×S×n = 3.6×0.83×30 = 89.64 km / h, that is, at the 9th frame image and the 10th frame image, the vehicle speed v = 89.64 km / h is obtained by the vehicle speed frame-by-frame detection method of the present invention.

[0090] This in-vehicle video device has a built-in speed measurement function. Within the selected 9th frame image and 10th frame image, as Figure 2 、 3 shown, its vehicle speed display is 90 km / h, and the error is within the allowable range.

[0091] It can be seen from this that the vehicle speed frame-by-frame detection method based on in-vehicle video images of the present invention determines the vehicle speed by analyzing the accurate vehicle speed between adjacent two frames of in-vehicle video images, meeting the accuracy requirements.

[0092] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention.

[0093] It should be understood that the present invention is not limited to the content described above and can be modified and changed without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A vehicle speed frame-by-frame detection method based on vehicle-mounted video images, characterized in that: The specific steps include: S1. Image preprocessing: Obtain the vehicle video to be tested and preprocess it into an image sequence; S2. Selecting a reference object: selecting a lane marking whose actual distance L is known on the road surface where the vehicle is traveling in the vehicle-mounted video to be tested as an environmental reference object; S3, inverse perspective transformation: select two consecutive frames of images in the image sequence, and transform the vehicle driving road area using an inverse perspective transformation algorithm to obtain a bird's-eye view of the road surface containing lane markings in the two consecutive frames of images; S4. Feature point selection: Select feature points at both ends of the lane markings in two consecutive frames of road bird's-eye view images, denoted as p1, p2, p3, ..., p n , record the pixel coordinates y of each feature point in the Y direction m value, and obtain the image distance l of the lane marking; S5. Calculation of vehicle speed: The vehicle speed is calculated by the actual displacement distance of the feature point in two consecutive frames of images.

2. The vehicle speed frame-by-frame detection method according to claim 1, characterized in that: S1 specifically includes: S11, frame rate detection: performing frame rate detection and frame rate completion on the vehicle-mounted video to be tested; S12, frame-by-frame division: processing the video into a frame-by-frame image sequence according to the video frame rate.

3. The vehicle speed frame-by-frame detection method according to claim 1, characterized in that: S3 specifically includes: S31, selecting a first rectangular area on the road surface on which the vehicle is traveling in each frame image, and determining that the four vertices of the first rectangular area in each frame image are given source points (x n ,y n ); S32, according to the known road width, lane marking length, actual distance between markings, and size of the computer window, select a specific scaling ratio to determine the four vertices of the second rectangular area formed in the bird's-eye view corresponding to the first rectangular area as the target points (X n ,Y n ); S33, the given source point (x n ,y n ) and the target point (X n ,Y n ) is substituted into the inverse perspective transformation equation 1, and the inverse perspective transformation matrix H is obtained: Among them, the inverse perspective transformation matrix H is a 3rd-order matrix as shown in Formula 2: Since H is a transformation matrix for homogeneous coordinates, we replace h in the matrix 33 Set it to 1, simplify Formula 1 to get Formula 3: S34, each pixel coordinate (u, w) in the obtained bird's-eye view is calculated according to the inverse matrix H of the inverse perspective transformation matrix -1 , and then the corresponding coordinates (x i ,y i ), Formula 4 is as follows: S35, the coordinates (x i ,y i ), extract the color values ​​of the corresponding coordinates and perform interpolation processing, fill the obtained color values ​​into the corresponding positions of the second rectangular area, and generate a bird's-eye view of the road surface in the first rectangular area.

4. The vehicle speed frame-by-frame detection method according to claim 1, characterized in that: S4 specifically includes: S41, establishing an image pixel coordinate system with the upper left corner of the image of the road bird's-eye view as the coordinate origin; S42, select any lane marking, select its endpoint in the two frames of image as a feature point, and record its pixel coordinate y in the Y direction m value; S43, calculating the image distance l of the lane marking in one frame according to Formula 5; l = y m -y m-1 Equation 5.

5. The vehicle speed frame-by-frame detection method according to claim 1, characterized in that: S5 specifically includes: S51, find the proportional relationship K between the image distance l and the actual distance L of the lane marking in the road surface bird's-eye view, as shown in Formula 6: S52, calculating the pixel difference a in the Y direction of the same end point of the same lane marking in two consecutive frames of images, which is the displacement image distance s of the feature point; S53. Calculate the actual displacement distance S of the feature point according to Formula 7: S54, the frame rate of the video to be tested is n, and the average speed v of the vehicle in the two frames before and after is calculated using Formula 8, in km / h, and the actual vehicle speed v is finally obtained: v=3.6×S×n Formula 8.

6. Application of the vehicle speed frame-by-frame detection method according to any one of claims 1 to 5, characterized in that: Applied to the detection of vehicle speed based on vehicle-mounted video images.

Images