Traffic detection method and detection system based on laser radar and video

Abstract

The invention discloses a traffic detection method and detection system based on a laser radar and a video. The method comprises the following steps: acquiring a shot video, vertical radar point cloud data and inclined radar point cloud data of a lane to be detected; according to the vertical radar point cloud data or the inclined radar point cloud data, frame-by-frame point cloud image restoration is carried out to obtain a vertical radar point cloud image sequence or an inclined radar point cloud image sequence, and then feature analysis is carried out to obtain vehicle data of each vehicle on the to-be-detected lane; according to the shot video, vehicle identification is carried out based on a trained identification model, and license plate data of each vehicle on the to-be-detected lane is obtained; and obtaining traffic volume data of the to-be-detected lane according to the vehicle data of all the vehicles and the corresponding license plate data so as to realize traffic detection of the to-be-detected lane. According to the method, the shot video and the radar point cloud data are combined, accurate detection of the speed and the vehicle type of the vehicle is achieved, recognition and recording of the license plate and the image information are achieved, and the method is suitable for complex traffic conditions.

Description

Technical Field

[0001] The invention relates to a traffic detection method and a detection system based on laser radar and video, belonging to the technical field of traffic detection. Background Art

[0002] Traffic survey refers to the observation and recording of the number of various types of traffic units passing through a certain section of a road within a certain time, a certain period or a continuous period. The purpose of traffic survey is to collect traffic volume data through long-term continuous observation or short-term intermittent and temporary observation, understand the changes and distribution patterns of traffic volume in time and space, and provide necessary data for traffic planning, road construction, traffic control and management, engineering economic analysis, etc.

[0003] Traffic survey stations are the main facilities for traffic surveys. They can be divided into manual survey stations and automated survey stations. Manual survey stations require a lot of manpower, have large detection errors and low efficiency, while automated survey stations have higher detection accuracy and efficiency, and lower overall costs. At present, the detection methods of automated survey stations are mainly based on geomagnetic coils, videos, lidar, ultrasound, infrared and other technologies, but generally only one of these technologies is used for detection. The method based on geomagnetic coils has low cost and high sensitivity for metal vehicle detection, but cannot identify vehicle models. The video-based method can identify information such as license plates, vehicle models, and colors, and is suitable for complex traffic conditions, but is greatly affected by light and weather. The lidar-based method can achieve high-precision distance and speed measurement, but high-performance multi-line lidar is expensive. The ultrasonic-based method is low-cost and easy to install, but the measurement distance is short and easily affected by noise. The infrared-based method is suitable for working in low-brightness environments, but is easily affected by temperature changes and direct sunlight. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a traffic detection method and detection system based on laser radar and video. Firstly, by processing the vertical radar point cloud data and the inclined radar point cloud data, the speed and model of the vehicle can be accurately detected. Secondly, the vehicle is identified by shooting the video based on the trained recognition model, and the license plate and image information can be recognized and recorded, which is suitable for complex traffic conditions.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions: In one aspect, the present invention discloses a traffic detection method based on laser radar and video, comprising the following steps: Obtain the captured video, vertical radar point cloud data, and tilted radar point cloud data of the lane to be tested; According to the vertical radar point cloud data, performing frame-by-frame point cloud image restoration to obtain a vertical radar point cloud image sequence; According to the tilted radar point cloud data, performing frame-by-frame point cloud image restoration to obtain a tilted radar point cloud image sequence; Performing feature analysis according to the vertical radar point cloud image sequence and the inclined radar point cloud image sequence to obtain vehicle data of each vehicle on the lane to be tested; According to the captured video, vehicle recognition is performed based on the trained recognition model to obtain the license plate data of each vehicle on the lane to be tested; According to the vehicle data of all vehicles and the corresponding license plate data, the traffic volume data of the lane to be tested is obtained to realize the traffic detection of the lane to be tested.

[0006] Furthermore, obtaining the vertical radar point cloud image sequence comprises the following steps: According to the vertical radar point cloud data, based on a preset vertical radar coordinate system, the coordinates of the obstacle are obtained by calculating the distance and angle from the obstacle to the origin, and the coordinates are converted into a two-dimensional point cloud image of the vertical plane; wherein each frame of the two-dimensional point cloud image of the vertical plane corresponds to a timestamp; For the two-dimensional point cloud image of any frame vertical plane, it is screened and cropped based on the preset lane parameters to obtain the two-dimensional point cloud image of the target lane; In response to the number of valid point clouds in the two-dimensional point cloud image of the target lane in any frame being greater than a preset point cloud threshold, it is determined that a vehicle has passed, and the corresponding two-dimensional point cloud image is used as a vertical radar point cloud image; A vertical radar point cloud image sequence is obtained according to all frames of vertical radar point cloud images, wherein each frame of the vertical radar point cloud image corresponds to a timestamp.

[0007] Furthermore, obtaining the tilted radar point cloud image sequence comprises the following steps: According to the tilted radar point cloud data, based on a preset tilted radar coordinate system, the coordinates of the obstacle are obtained by calculating the distance and angle from the obstacle to the origin, and the coordinates are converted into a two-dimensional point cloud image of the tilted plane; wherein each frame of the two-dimensional point cloud image of the tilted plane corresponds to a timestamp; For the two-dimensional point cloud image of the inclined plane of any frame, screening and cropping are performed based on the preset lane parameters to obtain the two-dimensional point cloud image of the target lane; In response to the number of valid point clouds in the two-dimensional point cloud image of the target lane in any frame being greater than a preset point cloud threshold, it is determined that a vehicle has passed, and the corresponding two-dimensional point cloud image is used as an inclined radar point cloud image; According to all frames of tilted radar point cloud images, a tilted radar point cloud image sequence is obtained; wherein each frame of the tilted radar point cloud image corresponds to a timestamp.

[0008] Furthermore, the angle between the inclined plane and the vertical plane is 0°-60°.

[0009] Furthermore, the vehicle data includes vehicle model data and vehicle speed data; the vehicle model data includes width, height and length of the vehicle.

[0010] Furthermore, obtaining the vehicle data of each vehicle on the lane to be tested includes the following steps: Take any vehicle as the target vehicle; Obtaining a width of the target vehicle according to a horizontal coordinate of the target vehicle in the vertical radar point cloud image sequence or the inclined radar point cloud image sequence; Obtaining the height of the target vehicle according to the vertical coordinate of the target vehicle in the vertical radar point cloud image sequence; According to the timestamp t0 of the first appearance of the oblique radar point cloud image of the target vehicle in the oblique radar point cloud image sequence, and the timestamp t1 of the first appearance of the vertical radar point cloud image of the target vehicle in the vertical radar point cloud image sequence, combined with the preset driving distance c, the speed v of the target vehicle is obtained, v=c / (t1-t0); According to the timestamp t2 of the vertical radar point cloud image in which the target vehicle appears last in the vertical radar point cloud image sequence, the length L of the target vehicle is calculated, where L=v*(t2-t1).

[0011] Furthermore, the training method of the recognition model includes the following steps: Obtaining a training set, the training set comprising multiple frames of captured video training images and corresponding license plate labels; According to the training set, the pre-constructed recognition model is iteratively trained based on the cross entropy loss function until a preset iteration termination condition is met, and the trained recognition model is output; Among them, the expression of the cross entropy loss function is as follows: ; In the formula, represents the cross entropy loss function; Indicates the license plate value output by the recognition model of the captured video training image; Indicates the license plate label corresponding to the captured video training image.

[0012] On the other hand, the present invention discloses a traffic detection system based on laser radar and video, which is applicable to the above-mentioned traffic detection method based on laser radar and video, and includes: A data acquisition module is used to acquire the captured video, vertical radar point cloud data and inclined radar point cloud data of the lane to be tested; A vertical radar module, used for performing frame-by-frame point cloud image restoration according to the vertical radar point cloud data to obtain a vertical radar point cloud image sequence; A tilted radar module, used for performing frame-by-frame point cloud image restoration according to the tilted radar point cloud data to obtain a tilted radar point cloud image sequence; A vehicle model data module, used for performing feature analysis according to the vertical radar point cloud image sequence and the inclined radar point cloud image sequence to obtain vehicle data of each vehicle on the lane to be tested; A vehicle data module is used to perform vehicle recognition based on the captured video and the trained recognition model to obtain the license plate data of each vehicle on the lane to be tested; The traffic detection module is used to obtain the traffic volume data of the lane to be tested based on the vehicle data of all vehicles and the corresponding license plate data, so as to realize the traffic detection of the lane to be tested.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The traffic detection method and detection system based on laser radar and video of the present invention firstly realizes accurate detection of vehicle speed and vehicle type by processing vertical radar point cloud data and inclined radar point cloud data, and secondly, recognizes vehicles in the captured video based on the trained recognition model, realizes recognition and recording of license plate and image information, and is applicable to complex traffic conditions. The method combines the captured video and radar point cloud data, has good robustness, and can take into account both economy and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of a traffic detection method based on laser radar and video provided in Example 1 of the present invention; Figure 2 is a schematic diagram of a vertical radar coordinate system and an inclined radar coordinate system provided by Example 1 of the present invention; Figure 3 It is a calculation diagram of the laser radar scanning surface provided in Example 1 of the present invention.

[0015] In the picture: 1. Tilt radar; 2. Vertical radar; 3. Camera. DETAILED DESCRIPTION

[0016] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention. Example

[0017] This embodiment 1 provides a traffic detection method based on laser radar and video, such as Figure 1 As shown, the following steps are included: Obtain the captured video, vertical radar point cloud data, and tilted radar point cloud data of the lane to be tested; According to the vertical radar point cloud data, point cloud image restoration is performed frame by frame to obtain a vertical radar point cloud image sequence; According to the tilted radar point cloud data, the point cloud image is restored frame by frame to obtain a tilted radar point cloud image sequence; Perform feature analysis based on the vertical radar point cloud image sequence and the inclined radar point cloud image sequence to obtain vehicle data of each vehicle on the lane to be tested; According to the video, the vehicle is identified based on the trained recognition model to obtain the license plate data of each vehicle on the lane to be tested; According to the vehicle data of all vehicles and the corresponding license plate data, the traffic volume data of the lane to be tested is obtained to realize the traffic detection of the lane to be tested.

[0018] The technical concept of the present invention is as follows: first, by processing the vertical radar point cloud data and the tilted radar point cloud data, the speed and model of the vehicle are accurately detected; second, based on the trained recognition model, the vehicle is recognized in the captured video, and the license plate and image information are recognized and recorded, which is suitable for complex traffic conditions. The method combines the captured video and the radar point cloud data, has good robustness, and can take into account both economy and accuracy.

[0019] like Figure 2 As shown, the equipment required for this method includes two single-line laser radars, a camera 3 and an industrial computer. The single-line laser radar includes a vertical radar 2 and an inclined radar 1. The two single-line laser radars are generally installed on a gantry in the middle of the road and are closely arranged in the vertical direction. In order to avoid blocking the light path, the inclined radar 1 is generally on the top and the vertical radar 2 is on the bottom. Figure 3 As shown, the scanning surface a of the vertical radar 2 is perpendicular to the ground c at an angle of 90°, and the scanning surface b of the inclined radar 1 is at an angle between 0° and 60° to the scanning surface a of the vertical radar. The camera 3 is installed near the radar, with the lens tilted downward, and the angle between the lens direction and the ground is between 0° and 30°. The industrial computer selects a small embedded industrial host with a network port function and is installed near the laser radar and camera. The laser radar, camera and industrial computer are all connected to the switch via a network cable, using LAN wired communication.

[0020] Detection principle Figure 2 As shown in the figure, the single-line laser radar emits a series of scanning lasers on a fixed plane at regular intervals. The lasers are reflected when they encounter objects. The single-line laser radar collects the reflected light, calculates the distance and organizes it into point cloud data. The industrial computer receives and analyzes the point cloud data from the two single-line laser radars, and restores the two-dimensional point cloud images of the vertical radar plane and the inclined radar plane respectively.

[0021] During the time when the vehicle passes through the laser radar scanning plane, the laser radar will scan several times, corresponding to several two-dimensional point cloud image frames, which form a two-dimensional point cloud image sequence. The industrial computer analyzes the two radar two-dimensional point cloud image sequences in real time, and can calculate the vehicle speed, length, height, width and other data. The vehicle type can be determined by analyzing the characteristics of the two-dimensional point cloud image sequence. At the same time, the camera can collect high-definition image information in real time, identify the vehicle and license plate information through the built-in image recognition model, capture the vehicle image, and then transmit these data to the industrial computer. The industrial computer matches the camera data with the laser radar data to form a single vehicle data record and store it in the local database. Finally, the industrial computer counts the traffic volume data and uploads it to the traffic data center regularly.

[0022] Specifically, the frequency of the single-line laser radar is 100Hz, that is, one frame is scanned every 10 milliseconds, the scanning range is -45° to 225°, and the resolution is 0.25°. The camera selects the traffic checkpoint camera, which can recognize vehicles and license plates. The CPU of the industrial computer is Inteli5-52000U, the running memory is 8GB, the operating system is Ubuntu18.04.6, and the C++ program runs on the industrial computer, which is responsible for data processing, calculation, storage and communication tasks.

[0023] The laser radar is installed on a gantry in the middle of the road. The two radars are fixed together by a customized bracket, with the tilted radar on the top and the vertical radar on the bottom. Figure 3 As shown, the angle θ between the two radar scanning surfaces a and b is fixed at 30°, and the vertical radar scanning surface is perpendicular to the ground at an angle of 90°, so that the scanning surfaces of the two radars form a right triangle with the ground, which is convenient for subsequent distance calculation. When installing the radar, a level is used for correction to ensure that the vertical radar is completely perpendicular to the ground. The traffic checkpoint camera is installed on the central gantry of the road close to the radar. In order to improve the license plate recognition effect, the angle between the camera's shooting direction and the ground is maintained at 0° to 30°. A waterproof junction box is set near the radar and camera. Inside the junction box are an industrial computer and a switch. The radar and camera are connected to the junction box via a network cable.

[0024] After the equipment is installed, it needs to be initialized and calibrated. Start the radar and view the real-time radar point cloud image through the debugging software. When no vehicle passes, the ground plane is a horizontal straight line. At this time, record the vertical coordinates of the ground plane as the initialization parameters of the software. The vertical coordinates of the ground plane of the vertical radar correspond to Figure 3 The length of a in the figure corresponds to the vertical coordinate of the tilted radar on the ground plane. Figure 3 The length of b in .

[0025] After the initialization calibration is completed, the lane parameters of the radar point cloud image need to be set. First, determine the lane number and lane direction. Then determine the left and right boundaries of the lane. The boundary coordinates of the lane can be roughly determined by analyzing the vehicle contour coordinates of the point cloud image when the vehicle passes through the radar. You can also place raised markers along the lane line, and then find the coordinates of the lane line on the radar point cloud image. Then set the left and right boundaries of the lane according to the coordinates of the lane line. Finally, determine the upper and lower boundaries of the lane based on the actual installation situation and detection results.

[0026] After the lane parameters are set, use the corresponding software to divide the lanes for the bayonet camera, directly divide them according to the lane lines, and then set the lane numbers and directions. They need to be consistent with the numbers set for the previous radar point cloud image.

[0027] After all initialization work is completed, the entire system is started. The internal program of the industrial computer will automatically process the radar data according to the steps described in the technical solution, match the camera data, count the traffic volume data, and upload it to the traffic data center regularly.

[0028] The specific steps of industrial computer processing are as follows: Step 1: Obtain the captured video, vertical radar point cloud data, and tilted radar point cloud data of the lane to be tested.

[0029] Step 2: Based on the vertical radar point cloud data, restore the point cloud image frame by frame to obtain a vertical radar point cloud image sequence.

[0030] The specific steps are as follows: 2.1. According to the vertical radar point cloud data, based on the preset vertical radar coordinate system, the coordinates of the obstacle are obtained by calculating the distance and angle from the obstacle to the origin, and then converted into a two-dimensional point cloud image of the vertical plane; wherein each frame of the two-dimensional point cloud image of the vertical plane corresponds to a timestamp.

[0031] First, mathematical modeling is performed on the scanning plane of the vertical radar. On the scanning plane, a rectangular coordinate system is constructed with the radar as the origin, the gantry beam where the vertical radar is located as the x-axis, and the vertical beam direction as the y-axis. Figure 3 As shown, the vertical radar coordinate system is Figure 3 On the a-plane.

[0032] The industrial computer parses the data packets transmitted by the laser radar and parses them into a series of distance value arrays according to the data protocol provided by the radar manufacturer. Each distance corresponds to an angle, and each distance value represents the distance from the radar to the obstacle at the corresponding angle, which corresponds to the distance from the obstacle to the origin in the above coordinate system. According to the angle and distance, the coordinates of the corresponding points are calculated in sequence with the help of trigonometric functions. These points form a frame of two-dimensional point cloud image, and each frame of image corresponds to a timestamp.

[0033] 2.2. For the two-dimensional point cloud image of the vertical plane of any frame, screening and cropping are performed based on the preset lane parameters to obtain the two-dimensional point cloud image of the target lane.

[0034] The area defined by the lane parameters is a rectangular area in the vertical radar coordinate system or the inclined radar coordinate system, including parameters such as the left boundary, right boundary, upper boundary, and lower boundary. The program will retain points that are exactly within this area and discard points outside the area. The lane parameters need to be adjusted according to the actual installation location and lane position. For the same lane, the left and right boundaries of the inclined radar and the vertical radar are consistent, and there will be some differences in the upper and lower boundaries.

[0035] 2.3. In response to the number of valid point clouds in the two-dimensional point cloud image of the target lane in any frame being greater than a preset point cloud threshold, it is determined that a vehicle has passed, and the corresponding two-dimensional point cloud image is used as a vertical radar point cloud image.

[0036] 2.4. A vertical radar point cloud image sequence is obtained based on all frames of vertical radar point cloud images; wherein each frame of vertical radar point cloud image corresponds to a timestamp.

[0037] Step 3: Based on the tilted radar point cloud data, restore the point cloud image frame by frame to obtain a tilted radar point cloud image sequence.

[0038] 3.1. According to the tilted radar point cloud data, based on the preset tilted radar coordinate system, the coordinates of the obstacle are obtained by calculating the distance and angle from the obstacle to the origin, and then converted into a two-dimensional point cloud image of the tilted plane; wherein each frame of the two-dimensional point cloud image of the tilted plane corresponds to a timestamp.

[0039] First, mathematical modeling is performed on the scanning plane of the tilted radar. On the scanning plane, the radar is taken as the origin, the gantry beam where the tilted radar is located is taken as the x-axis, and the direction of the tilted beam is taken as the y-axis to construct a rectangular coordinate system. Figure 3 As shown, the tilted radar coordinate system is Figure 3 on the b-plane.

[0040] The industrial computer parses the data packets transmitted by the laser radar and parses them into a series of distance value arrays according to the data protocol provided by the radar manufacturer. Each distance corresponds to an angle, and each distance value represents the distance from the radar to the obstacle at the corresponding angle, which corresponds to the distance from the obstacle to the origin in the above coordinate system. According to the angle and distance, the coordinates of the corresponding points are calculated in sequence with the help of trigonometric functions. These points form a frame of two-dimensional point cloud image, and each frame of image corresponds to a timestamp.

[0041] 3.2. For the two-dimensional point cloud image of the inclined plane of any frame, screening and cropping are performed based on the preset lane parameters to obtain the two-dimensional point cloud image of the target lane.

[0042] Similarly, the area defined by the lane parameters is a rectangular area in the vertical radar coordinate system or the inclined radar coordinate system, including parameters such as the left boundary, right boundary, upper boundary, and lower boundary. The program will retain the points that are exactly in this area and discard the points outside the area. The lane parameters need to be adjusted according to the actual installation position and lane position. For the same lane, the left and right boundaries of the inclined radar and the vertical radar are consistent, and there will be some differences in the upper and lower boundaries.

[0043] 3.3. In response to the number of valid point clouds in the two-dimensional point cloud image of the target lane in any frame being greater than a preset point cloud threshold, it is determined that a vehicle has passed, and the corresponding two-dimensional point cloud image is used as an inclined radar point cloud image; 3.4. Obtain a sequence of tilted radar point cloud images based on all tilted radar point cloud images, wherein each tilted radar point cloud image frame corresponds to a timestamp.

[0044] The included angle between the inclined plane and the vertical plane is 0°-60°.

[0045] Step 4: Perform feature analysis based on the vertical radar point cloud image sequence and the inclined radar point cloud image sequence to obtain vehicle data of each vehicle on the lane to be tested.

[0046] The vehicle data includes vehicle model data and vehicle speed data; the vehicle model data includes the width, height and length of the vehicle.

[0047] Obtaining the vehicle data of each vehicle on the lane to be tested includes the following steps: Take any vehicle as the target vehicle; The width of the target vehicle is obtained according to the horizontal coordinate, that is, the x-coordinate, of the target vehicle in the vertical radar point cloud image sequence or the inclined radar point cloud image sequence.

[0048] According to the vertical coordinate of the target vehicle in the vertical radar point cloud image sequence, that is, the y coordinate, the height of the target vehicle is obtained.

[0049] According to the timestamp t0 of the first appearance of the oblique radar point cloud image of the target vehicle in the oblique radar point cloud image sequence, and the timestamp t1 of the first appearance of the vertical radar point cloud image of the target vehicle in the vertical radar point cloud image sequence, combined with the preset driving distance c, the speed v of the target vehicle is obtained, v=c / (t1-t0).

[0050] According to the timestamp t2 of the vertical radar point cloud image in which the target vehicle appears last in the vertical radar point cloud image sequence, the length L of the target vehicle is calculated, L=v*(t2-t1).

[0051] Specifically, Figure 3 As shown, the target vehicle is moving from left to right. Figure 3Position 1 corresponds to the first appearance of the target vehicle in the tilted radar point cloud image sequence, with a timestamp of t0, and position 2 corresponds to the first appearance of the target vehicle in the vertical radar point cloud image sequence, with a timestamp of t1. In the time interval between these two frames (t1-t0), the distance traveled by the vehicle is the length of the base of the triangle c. Assuming that the vehicle maintains a constant speed during this period, the speed of the vehicle can be calculated according to the formula v=c / (t1-t0).

[0052] During the period from position 2 to position 3, the vehicle travels a distance exactly equal to the vehicle length. Position 3 corresponds to the vertical radar point cloud image where the target vehicle appears for the last time in the vertical radar point cloud image sequence. Its timestamp is t2, and the time interval is (t2-t1). The length L of the target vehicle can be calculated by the formula, L=v*(t2-t1).

[0053] The vehicle types are initially divided according to their length, and then accurately divided according to the data features of the vertical radar point cloud image sequence and the inclined radar point cloud image sequence. The main reference data features include width, height, width change rate and height change rate.

[0054] Step 5: According to the captured video, the vehicle is identified based on the trained recognition model to obtain the license plate data of each vehicle on the lane to be tested.

[0055] The training method of the recognition model includes the following steps: Obtain a training set, which includes multiple frames of video training images and corresponding license plate labels; According to the training set, the pre-built recognition model is iteratively trained based on the cross entropy loss function until the preset iteration termination condition is met, and the trained recognition model is output; Among them, the expression of the cross entropy loss function is as follows: ; In the formula, represents the cross entropy loss function; Indicates the license plate value output by the recognition model of the captured video training image; Indicates the license plate label corresponding to the captured video training image.

[0056] Step 6: According to the vehicle data of all vehicles and the corresponding license plate data, the traffic volume data of the lane to be tested is obtained to realize the traffic detection of the lane to be tested.

[0057] It should be noted that the lane to be tested is generally set as a single lane. If traffic detection is required for multiple lanes, it is only necessary to adjust the lane parameters of steps 2 and 3 and repeat the steps. In this way, it is possible to determine which lanes have vehicles passing during this time period, as well as data such as the vehicle speed and model.

[0058] On the one hand, the present invention uses a low-cost single-line laser radar to accurately measure distance and speed and determine the vehicle type based on the radar data. On the other hand, it uses a camera to recognize and record license plates and image information. Combining the advantages of the two technologies, the functional requirements of a multi-functional traffic investigation station are met. Example

[0059] This embodiment 2 provides a traffic detection system based on laser radar and video, which is applicable to the traffic detection method based on laser radar and video in embodiment 1, including: A data acquisition module is used to acquire the captured video, vertical radar point cloud data and inclined radar point cloud data of the lane to be tested; The vertical radar module is used to restore the point cloud image frame by frame according to the vertical radar point cloud data to obtain a vertical radar point cloud image sequence; The tilted radar module is used to restore the point cloud image frame by frame according to the tilted radar point cloud data to obtain a tilted radar point cloud image sequence; The vehicle model data module is used to perform feature analysis based on the vertical radar point cloud image sequence and the inclined radar point cloud image sequence to obtain the vehicle data of each vehicle on the lane to be tested; The vehicle data module is used to perform vehicle recognition based on the trained recognition model according to the captured video, and obtain the license plate data of each vehicle on the lane to be tested; The traffic detection module is used to obtain the traffic volume data of the lane to be tested based on the vehicle data of all vehicles and the corresponding license plate data, so as to realize the traffic detection of the lane to be tested.

[0060] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0061] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0062] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0063] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0064] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A traffic detection method based on laser radar and video, characterized in that: The following steps are involved: Obtain the captured video, vertical radar point cloud data, and tilted radar point cloud data of the lane to be tested; According to the vertical radar point cloud data, performing frame-by-frame point cloud image restoration to obtain a vertical radar point cloud image sequence; According to the tilted radar point cloud data, performing frame-by-frame point cloud image restoration to obtain a tilted radar point cloud image sequence; Performing feature analysis according to the vertical radar point cloud image sequence and the inclined radar point cloud image sequence to obtain vehicle data of each vehicle on the lane to be tested; According to the captured video, vehicle recognition is performed based on the trained recognition model to obtain the license plate data of each vehicle on the lane to be tested; According to the vehicle data of all vehicles and the corresponding license plate data, the traffic volume data of the lane to be tested is obtained to realize the traffic detection of the lane to be tested.

2. The traffic detection method based on laser radar and video according to claim 1 is characterized in that: The method of obtaining a vertical radar point cloud image sequence comprises the following steps: According to the vertical radar point cloud data, based on a preset vertical radar coordinate system, the coordinates of the obstacle are obtained by calculating the distance and angle from the obstacle to the origin, and the coordinates are converted into a two-dimensional point cloud image of the vertical plane; wherein each frame of the two-dimensional point cloud image of the vertical plane corresponds to a timestamp; For the two-dimensional point cloud image of any frame vertical plane, it is screened and cropped based on the preset lane parameters to obtain the two-dimensional point cloud image of the target lane; In response to the number of valid point clouds in the two-dimensional point cloud image of the target lane in any frame being greater than a preset point cloud threshold, it is determined that a vehicle has passed, and the corresponding two-dimensional point cloud image is used as a vertical radar point cloud image; A vertical radar point cloud image sequence is obtained according to all frames of vertical radar point cloud images, wherein each frame of the vertical radar point cloud image corresponds to a timestamp.

3. The traffic detection method based on laser radar and video according to claim 2 is characterized in that: The step of obtaining the tilted radar point cloud image sequence comprises the following steps: According to the tilted radar point cloud data, based on a preset tilted radar coordinate system, the coordinates of the obstacle are obtained by calculating the distance and angle from the obstacle to the origin, and the coordinates are converted into a two-dimensional point cloud image of the tilted plane; wherein each frame of the two-dimensional point cloud image of the tilted plane corresponds to a timestamp; For the two-dimensional point cloud image of the inclined plane of any frame, screening and cropping are performed based on the preset lane parameters to obtain the two-dimensional point cloud image of the target lane; In response to the number of valid point clouds in the two-dimensional point cloud image of the target lane in any frame being greater than a preset point cloud threshold, it is determined that a vehicle has passed, and the corresponding two-dimensional point cloud image is used as an inclined radar point cloud image; According to all frames of tilted radar point cloud images, a tilted radar point cloud image sequence is obtained; wherein each frame of the tilted radar point cloud image corresponds to a timestamp.

4. The traffic detection method based on laser radar and video according to claim 3 is characterized in that: The angle between the inclined plane and the vertical plane is 0°-60°.

5. The traffic detection method based on laser radar and video according to claim 1 is characterized in that: The vehicle data includes vehicle model data and vehicle speed data; the vehicle model data includes the width, height and length of the vehicle.

6. The traffic detection method based on laser radar and video according to claim 5 is characterized in that: The step of obtaining the vehicle data of each vehicle on the lane to be tested comprises the following steps: Take any vehicle as the target vehicle; Obtaining a width of the target vehicle according to a horizontal coordinate of the target vehicle in the vertical radar point cloud image sequence or the inclined radar point cloud image sequence; Obtaining the height of the target vehicle according to the vertical coordinate of the target vehicle in the vertical radar point cloud image sequence; According to the timestamp t0 of the first appearance of the oblique radar point cloud image of the target vehicle in the oblique radar point cloud image sequence, and the timestamp t1 of the first appearance of the vertical radar point cloud image of the target vehicle in the vertical radar point cloud image sequence, combined with the preset driving distance c, the speed v of the target vehicle is obtained, v=c / (t1-t0); According to the timestamp t2 of the vertical radar point cloud image in which the target vehicle appears last in the vertical radar point cloud image sequence, the length L of the target vehicle is calculated, where L=v*(t2-t1).

7. The traffic detection method based on laser radar and video according to claim 1 is characterized in that: The training method of the recognition model comprises the following steps: Obtaining a training set, the training set comprising multiple frames of captured video training images and corresponding license plate labels; According to the training set, the pre-constructed recognition model is iteratively trained based on the cross entropy loss function until a preset iteration termination condition is met, and the trained recognition model is output; Among them, the expression of the cross entropy loss function is as follows: ； In the formula, represents the cross entropy loss function; Indicates the license plate value output by the recognition model of the captured video training image; Indicates the license plate label corresponding to the captured video training image.

8. A traffic detection system based on laser radar and video, applicable to the traffic detection method based on laser radar and video according to any one of claims 1 to 7, characterized in that: include: A data acquisition module is used to acquire the captured video, vertical radar point cloud data and inclined radar point cloud data of the lane to be tested; A vertical radar module, used for performing frame-by-frame point cloud image restoration according to the vertical radar point cloud data to obtain a vertical radar point cloud image sequence; A tilted radar module, used for performing frame-by-frame point cloud image restoration according to the tilted radar point cloud data to obtain a tilted radar point cloud image sequence; A vehicle model data module, used for performing feature analysis according to the vertical radar point cloud image sequence and the inclined radar point cloud image sequence to obtain vehicle data of each vehicle on the lane to be tested; A vehicle data module is used to perform vehicle recognition based on the captured video and the trained recognition model to obtain the license plate data of each vehicle on the lane to be tested; The traffic detection module is used to obtain the traffic volume data of the lane to be tested based on the vehicle data of all vehicles and the corresponding license plate data, so as to realize the traffic detection of the lane to be tested.

Images