A real-time vehicle speed measurement system and method based on vehicle side images

Abstract

The application discloses a kind of real-time vehicle speed measurement system and method based on vehicle side image, including step 1, camera real-time obtains vehicle side image;Step 2, calibration board is placed on the lane in front of camera to calibrate, and perspective transformation matrix is obtained;Step 3, the pixel displacement of vehicle between two frames is obtained using phase correlation method and optical flow method to the vehicle side image in step 1;Step 4, the position of tire in vehicle image is identified using YOLOv8 model to the vehicle side image in step 1;Step 5, according to the pixel displacement in step 3 and the position of tire in step 4, the actual moving distance between two frames of vehicle is obtained using perspective transformation matrix, and the real-time speed of vehicle is obtained.Through the frame displacement detection of vehicle side image when vehicle passes, the real-time speed of vehicle can be obtained, and the device cost is low, easy to install and maintain.

Description

Technical Field

[0001] This invention belongs to the field of vehicle speed measurement technology, specifically to a real-time vehicle speed measurement system and method based on vehicle side images. Background Technology

[0002] Currently, the number of cars on the market is constantly increasing, which has brought considerable safety hazards to road traffic. Speeding is one of the important causes of traffic accidents, so speed measurement is particularly important.

[0003] Traditional vehicle speed measurement methods mainly include radar speed measurement, laser speed measurement, inductive loop speed measurement, and virtual loop video speed measurement. Radar speed measurement is susceptible to interference from radio waves, leading to measurement errors. Laser speed measurement equipment is expensive. Inductive loop speed measurement equipment is difficult to install and maintain. Virtual loop video speed measurement can only obtain the average speed of a vehicle passing through two loops, and cannot obtain real-time speed, and is prone to false detections. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a real-time vehicle speed measurement system and method based on vehicle side images. By detecting the inter-frame displacement of the vehicle side images as the vehicle passes by, the real-time speed of the vehicle can be obtained. The equipment is inexpensive and easy to install and maintain.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A real-time vehicle speed measurement method based on vehicle side images includes the following steps:

[0007] Step 1: The camera acquires real-time images of the vehicle's side.

[0008] Step 2: Place a calibration board on the lane directly in front of the camera for calibration and obtain the perspective transformation matrix;

[0009] Step 3: Use phase correlation and optical flow methods to obtain the pixel displacement of the vehicle between two frames in the vehicle side image from Step 1.

[0010] Step 4: Use the YOLOv8 model to identify the tire positions in the vehicle side image from Step 1;

[0011] Step 5: Based on the pixel displacement in Step 3 and the tire position in Step 4, use the perspective transformation matrix to obtain the actual moving distance of the vehicle between two frames, and obtain the real-time speed of the vehicle.

[0012] Preferably, step 2 specifically includes: placing the calibration board on the lane directly in front of the camera and saving the vehicle side image acquired by the camera at this time; recording the coordinates of the four vertices of the calibration board on the vehicle side image, and then selecting the target coordinates after perspective transformation corresponding to these four vertices. The four target coordinates should form a square of size 100*100 pixels; calculating the perspective transformation matrix from the original image to the target image based on the four pairs of coordinates on the original image and the target image, and saving the perspective transformation matrix at this time.

[0013] Preferably, step 3 specifically includes: using phase correlation and optical flow methods to detect the two consecutive frames of images, obtaining the phase correlation displacement and optical flow displacement; when the optical flow displacement has a value but the phase correlation displacement has no value, the phase correlation displacement of the current frame remains the same as the phase correlation displacement of the previous frame, and the final phase correlation displacement is the pixel displacement M of the vehicle between the two frames.

[0014] Preferably, in step 3, obtaining the pixel displacement of the vehicle using the phase correlation method specifically includes:

[0015] In step 1, a first region of interest and a second region of interest are selected on the vehicle-side image. The first region of interest is used to detect large vehicles, and the second region of interest is used to detect small vehicles.

[0016] For each frame of image, phase correlation is used to detect the first displacement between the current frame and the previous frame in the driving direction; phase correlation is used to detect the second displacement between the current frame and the previous frame in the driving direction in the second region of interest.

[0017] When the driving direction is set to left to right, the larger of the first displacement and the second displacement is selected as the phase correlation displacement of the current frame; when the driving direction is set to right to left, the first displacement and the second displacement are inverted and the larger of the two is selected as the phase correlation displacement of the current frame. The phase correlation displacement obtained in this way should be a positive value.

[0018] Preferably, in step 3, obtaining the pixel displacement of the vehicle using optical flow specifically includes:

[0019] In step 1, a third region of interest is selected on the vehicle side image. For each frame, the LK sparse optical flow method is used to detect whether there is optical flow between the current frame and the previous frame in the third region of interest. Non-stationary feature points with optical flow are retained. At the same time, the average value of the optical flow displacement in the driving direction is obtained, and the moving average filter is performed and rounded down to be used as the optical flow displacement of the current frame.

[0020] Preferably, step 4 specifically includes: when the phase correlation displacement in step 3 has a value, it indicates that the vehicle has started to enter. At this time, YOLOv8 is used to detect the vehicle image, identify the tire position in the image, retain the y-value of the bottom edge of the wheel detection box, and record the tire position as y0.

[0021] Preferably, step 5 specifically includes: based on the pixel displacement M of the vehicle between the two frames in step 3, taking two points (x1, y0) and (x2, y0) on the image, x1 and x2 should be symmetrical about the center line of the image and the distance between them should be M pixels. Performing perspective transformation on these two points according to the perspective transformation matrix obtained by calibration and calculating the Euclidean distance, the actual moving distance of the vehicle between the two frames is obtained, and dividing by the time interval between the two frames, the real-time speed of the vehicle is obtained.

[0022] A real-time vehicle speed measurement system based on vehicle side images includes an image acquisition module, a displacement detection module, and a wheel recognition and speed measurement module.

[0023] The image acquisition module is used to connect to a camera to acquire real-time images of the vehicle side, and to perform calibration using a calibration board to obtain a perspective transformation matrix.

[0024] The displacement detection module is used to obtain the pixel displacement of the vehicle between two frames using phase correlation and optical flow methods on the vehicle side image in the image acquisition module.

[0025] The wheel recognition and speed measurement module is used to identify the tire position in the vehicle image using the YOLOv8 model, and to obtain the actual moving distance between two frames of the vehicle using the perspective transformation matrix, thereby obtaining the real-time speed of the vehicle.

[0026] Preferably, the camera is placed at a fixed height on one side of the lane, the optical axis of the camera lens is perpendicular to the lane edge line, and the distance between the camera and the lane edge line is within 1 meter.

[0027] Compared with the prior art, the present invention has the following beneficial technical effects:

[0028] This invention provides a real-time vehicle speed measurement system based on vehicle side images. By detecting the inter-frame displacement of the vehicle side images as a vehicle passes by, the real-time speed of the vehicle can be obtained. The system designed by this invention is inexpensive and easy to deploy. It only requires placing a camera on one side of the lane, with the optical axis of the camera lens roughly perpendicular to the lane edge line. The distance between the camera and the lane edge line can be as close as 1 meter to detect the real-time speed of passing vehicles. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the overall structure of the real-time vehicle speed measurement system;

[0030] Figure 2A diagram showing the equipment installation;

[0031] Figure 3 A schematic diagram of the original image and the target image under perspective transformation;

[0032] Figure 4 This is a schematic diagram of the region of interest.

[0033] In the attached figures: 1 is the camera; 2 is the vehicle; 3 is the lane edge line; 4 is the original image after perspective transformation; 5 is the target image after perspective transformation; 6 is the calibration board on the original image; 7 is the calibration board on the target image; 11 is the first region of interest; 22 is the second region of interest; 33 is the third region of interest. Detailed Implementation

[0034] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0035] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0036] Example 1

[0037] This invention provides a real-time vehicle speed measurement method based on vehicle side images, comprising the following steps:

[0038] S01: A camera located on one side of the road acquires real-time images of the vehicle's side;

[0039] S02: Place a calibration board on the lane directly in front of the camera for calibration and obtain the perspective transformation matrix;

[0040] S03: Use phase correlation and optical flow methods to obtain the pixel displacement of the vehicle between two frames;

[0041] S04: When a vehicle is detected entering, use YOLOv8 to identify the tires in the vehicle image;

[0042] S05: Use perspective transformation to obtain the actual distance the vehicle moves between two frames, and get the vehicle's real-time speed.

[0043] Step S02 specifically includes: placing a 1m*1m square calibration board on the lane directly in front of the camera and saving the camera image at this time. Recording the coordinates of the four vertices of the calibration board 6 in the original image at this time, and then selecting the target coordinates after perspective transformation corresponding to these four vertices. The four target coordinates should form a 100*100 pixel square. Calculating the perspective transformation matrix from the original image to the target image based on the four pairs of coordinates on the perspective-transformed original image 4 and the perspective-transformed target image 5, and saving the perspective transformation matrix at this time. A schematic diagram of the perspective-transformed original image 4 and the perspective-transformed target image 5 is shown below. Figure 3 As shown.

[0044] Step S03 specifically includes: using phase correlation and optical flow methods to detect the two consecutive frames of images, obtaining the phase correlation displacement and optical flow displacement, maintaining the value of the phase correlation displacement when the optical flow displacement has a value but the phase correlation displacement has no value, and obtaining the final phase correlation displacement as the pixel displacement M of the vehicle between the two frames.

[0045] Step S04 specifically includes: when the phase correlation method displacement has a value, it indicates that the vehicle has begun to enter. At this time, YOLOv8 is used to detect the vehicle image, identify the tire position in the image, and retain the y value of the bottom edge of the wheel detection box, which is recorded as y0.

[0046] Step S05 specifically includes: based on the pixel displacement M of the vehicle between two frames, taking two points (x1, y0) and (x2, y0) on the image, x1 and x2 should be symmetrical about the center line of the image and the distance between them should be M pixels. Performing perspective transformation on these two points according to the perspective transformation matrix obtained by calibration and calculating the Euclidean distance, the actual moving distance of the vehicle between the two frames is obtained. Dividing the distance by the time interval between the two frames, the real-time speed of the vehicle is obtained.

[0047] Example 2

[0048] The hardware of the system of this invention consists of a camera, an embedded device, a fill light, and a housing support. The housing support is 80cm high, the camera is located inside the upper part of the housing support, and the fill light is located in the middle of the housing support for nighttime lighting.

[0049] The overall structure of the real-time vehicle speed measurement system based on vehicle side images designed in this invention is as follows: Figure 1 As shown, it mainly includes three modules: image acquisition module, displacement detection module, and wheel recognition and speed measurement module.

[0050] 1. Image acquisition module

[0051] (1) Acquisition of vehicle side images

[0052] The camera is placed at a fixed height on one side of the lane to acquire real-time images of the vehicle side. The optical axis of the camera lens is roughly perpendicular to the lane edge line, and the distance between the camera and the lane edge line can be as close as 1 meter. The equipment installation location is as follows: Figure 2 As shown.

[0053] (2) Perspective transformation calibration

[0054] Place a 1m x 1m square calibration board on the lane directly in front of the camera and save the camera image. Record the coordinates of the four vertices of the calibration board in this image. Then, select the target coordinates corresponding to these four vertices after perspective transformation. The four target coordinates should form a 100x100 pixel square. Figure 3 The calibration plate 7 on the target image is shown. Based on the four pairs of coordinates on the original image 4 and the target image 5, the perspective transformation matrix from the original image to the target image is calculated and saved. A schematic diagram of the original image 4 and the target image 5 is shown below. Figure 3 As shown.

[0055] 2. Displacement detection module

[0056] (1) Displacement detection by phase correlation method

[0057] A first region of interest (ROI) 11 and a second region of interest (ROI) 22 are selected. ROI 11 is primarily used for detecting large vehicles, and ROI 22 is primarily used for detecting small vehicles. The widths of ROI 11 and ROI 22 should be the same. The selection of ROI 11 and ROI 22 on the image is as follows: Figure 4 As shown.

[0058] For each frame of image, phase correlation is used to detect the first displacement between the current frame and the previous frame in the x-axis direction (driving direction) by using phase correlation; and the second displacement between the current frame and the previous frame in the x-axis direction (driving direction) is also detected by using phase correlation in the second region of interest 22.

[0059] When the driving direction is set to left to right, the larger of the first displacement and the second displacement is selected as the phase correlation displacement of the current frame; when the driving direction is set to right to left, the first displacement and the second displacement are inverted and the larger of the two is selected as the phase correlation displacement of the current frame. The phase correlation displacement obtained in this way should be a positive value.

[0060] (2) Optical flow displacement detection

[0061] The third region of interest 33 is selected, and the selection of the third region of interest 33 on the image is as follows: Figure 4As shown. For each frame of the image, the LK sparse optical flow method is used to detect whether there is optical flow between the current frame and the previous frame in the third region of interest 33. Non-stationary feature points with optical flow are retained for subsequent optical flow detection. At the same time, the average value of the optical flow displacement in the x-axis direction (driving direction) is obtained, and the moving average filter is applied and rounded down to be used as the optical flow displacement of the current frame.

[0062] (3) Phase correlation method for maintaining displacement

[0063] The displacement obtained by the phase correlation method is relatively accurate, but sometimes the displacement cannot be detected. The displacement obtained by the optical flow method is not very accurate, but it does not fail to detect the displacement when there is a vehicle. Therefore, the displacement obtained by the optical flow method can be maintained at the value of the displacement obtained by the phase correlation method.

[0064] If the phase correlation displacement has a value in the previous frame and the optical flow displacement has a value in the current frame, but the phase correlation displacement has no value in the current frame, the phase correlation displacement in the current frame remains the same as the phase correlation displacement in the previous frame. The final phase correlation displacement is the pixel displacement M of the vehicle between the two frames.

[0065] 3. Wheel recognition and speed measurement module

[0066] (1) Model training and inference

[0067] This invention uses cameras installed at highway intersections and test tracks to collect video recordings, which are then used to create a training dataset for wheel recognition. When using YOLOv8 for model training and directly deploying the trained weight files on embedded devices for wheel recognition, the detection frame rate is too low to achieve real-time performance. This invention optimizes inference using the TensorRT inference framework provided by NVIDIA, using an inference engine to accelerate the model and reduce its size to achieve faster inference speed, thus meeting real-time requirements.

[0068] (2) Wheel inspection

[0069] When a phase-correlation displacement is detected, it indicates that a vehicle has entered. YOLOv8 is used to detect the tires in the image. The y-value of the bottom edge of the wheel detection box when the wheel is detected in the first frame is retained as the y-value of this vehicle, denoted as y0. If no wheel is detected, it is considered as interference.

[0070] (3) Vehicle speed acquisition

[0071] The phase correlation displacement M between the two frames obtained after the above steps is the vehicle pixel displacement. Take two points (x1, y0) and (x2, y0) on the image. x1 and x2 should be symmetrical about the center line of the image and be M pixels apart. Perform perspective transformation on these two points according to the perspective transformation matrix obtained by calibration and calculate the Euclidean distance to obtain the actual distance the vehicle moves between the two frames. Divide the distance by the time interval between the two frames to obtain the real-time speed of the vehicle.

[0072] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the scope of the invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0073] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be appropriately combined to form other embodiments that can be understood by those skilled in the art. The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A method for real-time vehicle speed measurement based on vehicle side images, characterized in that, Includes the following steps, Step 1: The camera acquires real-time images of the vehicle's side. Step 2: Place a calibration board on the lane directly in front of the camera for calibration and obtain the perspective transformation matrix; Step 3: Use phase correlation and optical flow methods to obtain the pixel displacement of the vehicle between two frames in the vehicle side image of Step 1. Specifically, this includes: using phase correlation and optical flow methods to detect the two consecutive frames to obtain the phase correlation displacement and optical flow displacement. When the optical flow displacement has a value but the phase correlation displacement has no value, the phase correlation displacement of the current frame remains the same as the phase correlation displacement of the previous frame. The final phase correlation displacement is the pixel displacement M of the vehicle between the two frames. Step 4: Use the YOLOv8 model to identify the tire position in the vehicle image of the vehicle side image in Step 1. Specifically, when the phase correlation method displacement in Step 3 has a value, it means that the vehicle has started to enter. At this time, use YOLOv8 to detect the vehicle image, identify the tire position in the image, retain the y value of the bottom edge of the wheel detection box, and record the tire position as y0. Step 5: Based on the pixel displacement in Step 3 and the tire position in Step 4, use the perspective transformation matrix to obtain the actual moving distance of the vehicle between two frames, and obtain the real-time speed of the vehicle.

2. The real-time vehicle speed measurement method based on vehicle side images according to claim 1, characterized in that, Step 2 specifically includes: placing the calibration board on the lane directly in front of the camera and saving the vehicle side image acquired by the camera at this time; recording the coordinates of the four vertices of the calibration board on the vehicle side image, and then selecting the target coordinates after perspective transformation corresponding to these four vertices. The four target coordinates should form a 100° angle. A 100-pixel square; calculate the perspective transformation matrix from the original image to the target image based on 4 pairs of coordinates on the original and target images, and save the perspective transformation matrix at this time.

3. The real-time vehicle speed measurement method based on vehicle side images according to claim 1, characterized in that, Step 3, obtaining the vehicle's pixel displacement using the phase correlation method, specifically includes: In step 1, a first region of interest (11) and a second region of interest (22) are selected on the vehicle side image. The first region of interest (11) is used to detect large vehicles, and the second region of interest (22) is used to detect small vehicles. For each frame of image, the phase correlation method is used to detect the first displacement between the current frame and the previous frame in the driving direction in the first region of interest (11); the phase correlation method is used to detect the second displacement between the current frame and the previous frame in the driving direction in the second region of interest (22). When the driving direction is set to left to right, the larger of the first displacement and the second displacement is selected as the phase correlation displacement of the current frame; when the driving direction is set to right to left, the first displacement and the second displacement are inverted and the larger of the two is selected as the phase correlation displacement of the current frame. The phase correlation displacement obtained in this way should be a positive value.

4. The real-time vehicle speed measurement method based on vehicle side images according to claim 1, characterized in that, Step 3, obtaining the pixel displacement of the vehicle using optical flow specifically includes: In step 1, a third region of interest (33) is selected on the vehicle side image. For each frame of the image, the LK sparse optical flow method is used to detect whether there is optical flow between the current frame and the previous frame in the third region of interest (33). Non-stationary feature points with optical flow are retained. At the same time, the average value of the optical flow displacement in the driving direction is obtained, and the moving average is filtered and rounded to be used as the optical flow displacement of the current frame.

5. The real-time vehicle speed measurement method based on vehicle side images according to claim 1, characterized in that, Step 5 specifically includes: based on the pixel displacement M of the vehicle between the two frames in step 3, taking two points (x1, y0) and (x2, y0) on the image, x1 and x2 should be symmetrical about the center line of the image and the distance between them should be M pixels. Perform perspective transformation on these two points according to the perspective transformation matrix obtained by calibration and calculate the Euclidean distance to obtain the actual movement distance of the vehicle between the two frames. Divide the distance by the time interval between the two frames to obtain the real-time speed of the vehicle.

6. A real-time vehicle speed measurement system based on vehicle side images, characterized in that, It includes an image acquisition module, a displacement detection module, and a wheel recognition and speed measurement module; The image acquisition module is used to connect to a camera to acquire real-time images of the vehicle side, and to perform calibration using a calibration board to obtain a perspective transformation matrix. The displacement detection module is used to obtain the pixel displacement of the vehicle between two frames using the phase correlation method and the optical flow method on the vehicle side image in the image acquisition module. Specifically, it includes: using the phase correlation method and the optical flow method to detect the two consecutive frames of images, obtaining the phase correlation method displacement and the optical flow method displacement. When the optical flow method displacement has a value but the phase correlation method displacement has no value, the phase correlation method displacement of the current frame is kept as the phase correlation method displacement of the previous frame. The final phase correlation method displacement is the pixel displacement M of the vehicle between the two frames. The wheel recognition and speed measurement module is used to identify the tire position in a vehicle image using the YOLOv8 model. Specifically, when the phase correlation displacement has a value, it indicates that the vehicle has begun to enter. At this time, YOLOv8 is used to detect the vehicle image, identify the tire position in the image, retain the y-value of the bottom edge of the wheel detection box, record the tire position as y0, and use the perspective transformation matrix to obtain the actual moving distance between two frames of the vehicle, thus obtaining the real-time speed of the vehicle.

7. A real-time vehicle speed measurement system based on vehicle side images according to claim 6, characterized in that, The camera is placed at a fixed height on one side of the lane, with the optical axis of the camera lens perpendicular to the lane edge line, and the distance between the camera and the lane edge line is within 1 meter.

Images