A video monitoring system for a highway tunnel

Abstract

The application discloses a video monitoring system for an expressway tunnel, and belongs to the technical field of video monitoring, which comprises a vehicle position coordinate interrelation coefficient construction module, a vehicle image acquisition module, a vehicle inclination calculation module, a vehicle coordinate acquisition module, a vehicle coordinate coincidence judgment module and a warning module. The application can realize vehicle collision detection, i.e. traffic accident detection, without detecting the whole vehicle, reduce the influence of light in the expressway tunnel on detection, improve the detection accuracy of traffic accidents, and further improve the effect and efficiency of traffic control.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of video monitoring, and particularly relates to a video monitoring system for a highway tunnel. BACKGROUND

[0002] The highway tunnel is a special closed space in the traffic network, and the accident risk is significantly higher than that of the ordinary road section due to the complex environment and difficult rescue.

[0003] Through video monitoring, accidents can be found in time, which plays an important role in ensuring traffic safety, improving emergency disposal efficiency and reducing disaster loss.

[0004] Chinese patent application No. 202310213617.0 discloses a smart highway tunnel traffic event rapid detection and processing system and method, which includes real-time monitoring of the state of the vehicle entering the tunnel entrance through the camera, detecting the time when the vehicle enters, the speed of the vehicle, and the judgment of whether the tunnel inside is congested; if it is judged that the congestion condition, the servo motor is started to drive the camera to move, the vehicle condition on the moving path is monitored and identified, and the rescue mechanism is simultaneously moved; S3, if a traffic accident occurs in the tunnel, the on-site rescuer can use the rescue mechanism to quickly and smoothly transport the injured person to the outside of the tunnel, and strive for the rescue opportunity; S4, the camera detects the traffic condition in the tunnel, and according to different states, the corresponding early warning measures are started.

[0005] The above-mentioned technology has the following problems: the above-mentioned existing highway tunnel traffic event rapid detection and processing system detects whether there is a traffic event in the highway tunnel through the movement of the camera. This method cannot detect traffic events comprehensively at all times due to the movement of the camera. At the same time, the light in the highway tunnel is relatively dark, and in some cases, such as a local small-area collision between two vehicles, the camera may not be able to detect the traffic event, which affects the effect and efficiency of traffic control.

[0006] Therefore, a video monitoring system for a highway tunnel is designed to solve the above-mentioned problems. SUMMARY

[0007] To solve the problems in the background art, the present application provides a video monitoring system for a highway tunnel, which can detect vehicle collisions, i.e. traffic accidents, without detecting the entire vehicle, reduces the impact of light in the highway tunnel on detection, improves the detection accuracy of traffic accidents, and further improves the effect and efficiency of traffic control.

[0008] To achieve the above-mentioned purpose, the present application provides the following technical solution: a video monitoring system for a highway tunnel, comprising:

[0009] The vehicle position coordinate correlation coefficient construction module obtains images of different perspectives of vehicles in horizontal state of various types through binocular cameras, calculates overall three-dimensional coordinates of different perspectives of vehicles in horizontal state of various types, and constructs correlation coefficients between license plate vertices and positions of vehicles in horizontal state of various types of vehicles;

[0010] The vehicle image acquisition module acquires vehicle images, including license plate images, through cameras in the highway tunnel.

[0011] The vehicle information acquisition module acquires vehicle information associated with the acquired vehicle license plate images, including license plate numbers and three-dimensional coordinates of license plates.

[0012] The vehicle inclination calculation module calculates the inclination angle of the acquired vehicle license plate images.

[0013] The vehicle coordinate acquisition module acquires vehicle model information through the acquired vehicle license plate numbers, acquires correlation coefficients between vehicle license plate vertices and positions of vehicles through the acquired vehicle model information, acquires vehicle vertex coordinates, and acquires three-dimensional coordinates of positions of vehicles through the vehicle vertex coordinates, correlation coefficients, and inclination angles.

[0014] The vehicle coordinate coincidence judgment module judges whether the coordinates of each vehicle in the highway tunnel coincide.

[0015] The early warning module determines that an accident has occurred if the coordinates of two vehicles in the highway tunnel coincide, and transmits an early warning message to a traffic control platform for traffic control.

[0016] Further, the specific steps of the vehicle position coordinate correlation coefficient construction module include:

[0017] Obtain images of different perspectives of a vehicle in horizontal state of a certain type through binocular cameras and perform preprocessing;

[0018] Extract feature points of left and right images of different perspectives of the vehicle in horizontal state of the certain type and perform matching;

[0019] Calculate the parallax of left and right images of different perspectives of the vehicle in horizontal state of the certain type;

[0020] Calculate the depth of left and right images of different perspectives of the vehicle in horizontal state of the certain type through the parallax;

[0021] Convert pixel coordinates of left and right images of different perspectives of the vehicle in horizontal state of the certain type to three-dimensional coordinates through the parallax and the depth;

[0022] Convert three-dimensional coordinates of different perspectives of the vehicle in horizontal state of the certain type to a unified coordinate system;

[0023] The unified coordinates are used for registration and fusion to form a complete three-dimensional coordinate of a vehicle in a horizontal state;

[0024] A correlation coefficient between a certain vertex of a license plate and three-dimensional coordinates of each position of a vehicle in a horizontal state of a certain vehicle type is constructed, and the expression is:

[0025]

[0026] In the formula, (X0, Y0, Z0) represents three-dimensional coordinates of a license plate vertex, (X1, Y1, Z1) represents three-dimensional coordinates of a position of a vehicle and a license plate vertex, and α, β and χ are correlation coefficients of the license plate vertex and the vehicle in X, Y and Z directions, respectively;

[0027] The correlation coefficients between the license plate vertex and the vehicle in a horizontal state of various vehicle types are constructed in the same steps.

[0028] Further, the specific steps of license plate number acquisition in the vehicle information acquisition module include:

[0029] The collected license plate image is converted into a gray-scale image and preprocessed;

[0030] The preprocessed gray-scale image is converted into a binary image;

[0031] The number is recognized based on the binary image to obtain the license plate number.

[0032] Further, the specific steps of license plate three-dimensional coordinate acquisition in the vehicle information acquisition module include:

[0033] The pixel coordinates of the collected vehicle license plate image are normalized;

[0034] The normalized vehicle license plate image pixel coordinates are converted into camera coordinates;

[0035] The camera coordinates are converted into three-dimensional coordinates, i.e., license plate three-dimensional coordinates.

[0036] Further, the specific steps of the vehicle inclination calculation module include:

[0037] The three-dimensional coordinates of two end points of a horizontal section of a license plate in a horizontal state of a vehicle are obtained, and the three-dimensional coordinates of the two end points of the horizontal section in a vehicle license plate image are obtained;

[0038] The direction vectors of the end points are calculated;

[0039] The vector dot product is calculated through the direction vectors of the end points;

[0040] The vector module length is calculated through the direction vectors of the end points;

[0041] The inclination angle of the two horizontal segments is calculated by the vector dot product and the vector module length of the end points, and the expression is:

[0042]

[0043] In the formula: represents the vector dot product, and represents the vector module length.

[0044] Further, the vehicle model information in the vehicle coordinate acquisition module is obtained through the data sharing vehicle management platform.

[0045] Further, the specific steps of obtaining the three-dimensional coordinates of each position of the vehicle in the vehicle coordinate acquisition module include:

[0046]

[0047] In the formula: (x0, y0, z0) represents the three-dimensional coordinates of the license plate vertex, (x1, y1, z1) represents the three-dimensional coordinates of the position of the vehicle and the license plate vertex, α, β and χ are the correlation coefficients of the license plate vertex and the vehicle at a certain position in the X, Y and Z directions, respectively, (x1', y1', z1') represents the three-dimensional coordinates of the position of the vehicle after tilting, and θ represents the inclination angle.

[0048] Compared with the prior art, the beneficial effects of the present application are:

[0049] The present application acquires the related vehicle model information based on the vehicle license plate image, and acquires the three-dimensional coordinates of the license plate based on the vehicle license plate image. The inclination angle of the license plate, i.e. the inclination angle of the vehicle, is calculated based on the three-dimensional coordinates of the license plate. The overall three-dimensional coordinates of the vehicle are calculated based on the vehicle model information, the correlation coefficient of the license plate vertex and the vehicle position coordinates of each vehicle model, and the license plate vertex coordinates. Whether there is a collision between vehicles is determined based on the coincidence degree between the overall three-dimensional coordinates of the vehicle. The detection of vehicle collision, i.e. traffic accident, can be realized without detecting the overall vehicle, the influence of light in the highway tunnel on detection is reduced, the detection accuracy of traffic accidents is improved, and the effect and efficiency of traffic control are improved. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 The system framework diagram of the present application is shown in the figure;

[0051] In the figure: 1, correlation coefficient construction module between vehicle position coordinates; 2, vehicle image acquisition module; 3, vehicle information acquisition module; 4, vehicle inclination calculation module; 5, vehicle coordinate acquisition module; 6, vehicle coordinate coincidence judgment module; 7, early warning module. DETAILED DESCRIPTION

[0052] Clearly, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present application.

[0053] The present application provides the following technical solutions: a video monitoring system for a highway tunnel, comprising:

[0054] A vehicle position coordinate correlation coefficient construction module 1 obtains images of different perspectives of vehicles in various vehicle types in a horizontal state through a binocular camera, calculates overall three-dimensional coordinates of the vehicles in various vehicle types in a horizontal state at different perspectives, and constructs correlation coefficients between license plate vertices and positions of the vehicles in various vehicle types in a horizontal state.

[0055] A vehicle image acquisition module 2 acquires vehicle images, including license plate images, through a camera in the highway tunnel.

[0056] A vehicle information acquisition module 3 acquires vehicle information associated with the acquired vehicle license plate images, including license plate numbers and three-dimensional coordinates of the license plates.

[0057] A vehicle inclination calculation module 4 calculates an inclination angle of the acquired vehicle license plate images.

[0058] A vehicle coordinate acquisition module 5 acquires vehicle type information through the acquired vehicle license plate numbers, acquires correlation coefficients between license plate vertices and positions of the vehicles through the acquired vehicle type information, acquires vehicle vertex coordinates, and acquires three-dimensional coordinates of positions of the vehicles through the vehicle vertex coordinates, the correlation coefficients, and the inclination angle.

[0059] A vehicle coordinate coincidence judgment module 6 judges whether the coordinates of each vehicle in the highway tunnel coincide.

[0060] An early warning module 7 determines that an accident has occurred if the coordinates of two vehicles in the highway tunnel coincide, and transmits an early warning message to a traffic control platform for traffic control.

[0061] Specifically, the specific steps of the vehicle position coordinate correlation coefficient construction module 1 include:

[0062] Obtaining images of different perspectives of a vehicle in a horizontal state through a binocular camera and performing preprocessing;

[0063] Extracting feature points of left and right images of different perspectives of the vehicle in a horizontal state and performing matching;

[0064] The disparity of the left and right images of different view angles of a vehicle in horizontal state is calculated, and the expression is:

[0065] d = u L - u R

[0066] In the formula, d represents the disparity, u L represents the pixel coordinates (u L , v) of the feature point in the left image, and u R represents the pixel coordinates (u R , v) of the feature point in the right image.

[0067] The depth of the left and right images of different view angles of a vehicle in horizontal state is calculated, and the expression is:

[0068]

[0069] In the formula, Z represents the depth of the vehicle to the camera, f represents the focal length of the camera, B represents the baseline distance, and d represents the disparity.

[0070] The pixel coordinates of the left and right images of different view angles of a vehicle in horizontal state are converted into three-dimensional coordinates, and the expression is:

[0071]

[0072] In the formula, (X, Y, Z) represents the three-dimensional coordinates, (u, v) represents the pixel coordinates, (u0, v0) represents the principal point coordinates of the camera, f represents the focal length of the camera, B represents the baseline distance, and Z represents the depth of the vehicle to the camera.

[0073] The three-dimensional coordinates of different view angles of a vehicle in horizontal state are converted into a unified coordinate system, and the expression is:

[0074]

[0075] In the formula, P w represents the unified coordinates, P c represents the three-dimensional coordinates, R c and T c represent the camera extrinsic parameters.

[0076] Based on the unified coordinates, registration and fusion are performed to form the complete three-dimensional coordinates of a vehicle in horizontal state.

[0077] The correlation between a certain vertex of a license plate and the three-dimensional coordinates of each position of a vehicle in horizontal state of a certain vehicle model is constructed, and the expression is:

[0078]

[0079] In the formula: (X0, Y0, Z0) represents the three-dimensional coordinates of the license plate vertex, (X1, Y1, Z1) represents the three-dimensional coordinates of the position of the vehicle and the license plate vertex, and α, β and χ are the correlation coefficients of the license plate vertex and the vehicle at a certain position in the X, Y and Z directions, respectively;

[0080] The correlation coefficients between the license plate vertex and the positions of the vehicle in the horizontal state of various vehicle models are constructed in the same steps.

[0081] Specifically, the specific steps of license plate number acquisition in the vehicle information acquisition module 3 include:

[0082] The collected license plate image is converted into a grayscale image and preprocessed;

[0083] The preprocessed grayscale image is converted into a binary image;

[0084] The number recognition is performed based on the binary image to obtain the license plate number;

[0085] The above number recognition is performed using a trained text recognition model.

[0086] Specifically, the specific steps of license plate three-dimensional coordinate acquisition in the vehicle information acquisition module 3 include:

[0087] The pixel coordinates of the collected vehicle license plate image are normalized, and the expression is:

[0088]

[0089] In the formula: (x, y) represents the normalized coordinates, (u, v) represents the pixel coordinates, (c x ,c y ) represents the principal point coordinates of the camera, and (f x ,f y ) represents the focal length of the camera;

[0090] The normalized vehicle license plate image pixel coordinates are converted into camera coordinates, and the expression is:

[0091]

[0092] In the formula: (x, y) represents the normalized coordinates, and Z c represents the depth;

[0093] The camera coordinates are converted into three-dimensional coordinates, i.e. the license plate three-dimensional coordinates, and the expression is:

[0094]

[0095] In the formula: P w represents the three-dimensional coordinates, P c represents the camera coordinates, and Rc and T c represents the camera extrinsic parameter.

[0096] Specifically, the specific steps of the vehicle inclination calculation module 4 include:

[0097] Obtain the three-dimensional coordinates of the two end points of the license plate horizontal segment in the vehicle horizontal state, and obtain the three-dimensional coordinates of the two end points of the horizontal segment in the vehicle license plate image;

[0098] The above license plate horizontal segment recognition uses the trained and converged image horizontal segment recognition model for recognition.

[0099] Calculate the direction vector, and the expression is:

[0100]

[0101] In the formula: (x2, y2, z2) and (x1, y1, z1) are the three-dimensional coordinates of the two end points of the license plate horizontal segment in the vehicle horizontal state, and (x4, y4, z4) and (x3, y3, z3) are the three-dimensional coordinates of the two end points of the horizontal segment in the vehicle license plate image.

[0102] Calculate the vector dot product, and the expression is:

[0103]

[0104] Calculate the vector module length, and the expression is:

[0105]

[0106] Calculate the inclination angle of the two horizontal segments, and the expression is:

[0107]

[0108] Specifically, the vehicle model information in the vehicle coordinate acquisition module 5 is obtained through the data sharing vehicle management platform.

[0109] Specifically, the specific steps of obtaining the three-dimensional coordinates of each position of the vehicle in the vehicle coordinate acquisition module 5 include:

[0110]

[0111] In the formula: (x0, y0, z0) represents the three-dimensional coordinates of the license plate vertex, (x1, y1, z1) represents the three-dimensional coordinates of the position of the vehicle and the license plate vertex, α, β and χ are the correlation coefficients of the license plate vertex and the vehicle at a certain position in the X, Y and Z directions, respectively, (x1', y1', z1') represents the three-dimensional coordinates of the position after the vehicle is inclined, and θ represents the inclination angle.

[0112] The recognition of the vehicle license plate vertexes is also performed by the image vertex recognition model trained to convergence.

[0113] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely divergences of the principles and spirit of the application and that numerous modifications, changes, substitutions, and alterations can be made thereto without departing from the spirit and scope of the application as defined by the appended claims and their equivalents.

Claims

1. A video monitoring system for a highway tunnel, characterized by, The application relates to a vehicle position coordinate correlation coefficient construction module (1) which obtains images of different visual angles of vehicles in various vehicle types in a horizontal state through binocular cameras, calculates overall three-dimensional coordinates of the images of different visual angles of the vehicles in various vehicle types in the horizontal state, and constructs correlation coefficients between license plate vertexes and positions of the vehicles in the horizontal state of the vehicles in various vehicle types. The specific steps of the vehicle position coordinate correlation coefficient construction module (1) include: obtaining images of different visual angles of a vehicle in a certain vehicle type in a horizontal state through binocular cameras and performing pretreatment; extracting feature points of left and right images of different visual angles of the vehicle in the horizontal state and performing matching; calculating parallax of the left and right images of different visual angles of the vehicle in the horizontal state; calculating depth of the left and right images of different visual angles of the vehicle in the horizontal state through the parallax; converting pixel coordinates of the left and right images of different visual angles of the vehicle in the horizontal state into three-dimensional coordinates through the parallax and the depth; converting the three-dimensional coordinates of different visual angles of the vehicle in the horizontal state into a unified coordinate system; performing registration and fusion based on the unified coordinate to form complete three-dimensional coordinates of the vehicle in the horizontal state of the vehicle in a certain vehicle type; constructing correlation coefficients between license plate vertexes and positions of the vehicles in the horizontal state of the vehicles in various vehicle types by using the same steps; The correlation between a certain vertex of a license plate and three-dimensional coordinates of each position of a vehicle in a horizontal state of a certain vehicle model is constructed, and the expression is: ; wherein: represents the three-dimensional coordinates of the license plate vertex, represents the three-dimensional coordinates of the position of the vehicle and the license plate vertex, , and are the correlation coefficients of the license plate vertex and the vehicle at a certain position in the , and directions, respectively. a vehicle image acquisition module (2) which acquires vehicle images including license plate images through cameras in a highway tunnel; a vehicle information acquisition module (3) which acquires vehicle information including license plate numbers and three-dimensional coordinates of the license plates associated with the acquired vehicle license plate images; a vehicle inclination calculation module (4) which calculates an inclination angle of the acquired vehicle license plate images; a vehicle coordinate acquisition module (5) which acquires vehicle type information through the acquired vehicle license plate numbers, acquires correlation coefficients between license plate vertexes and positions of the vehicles through the acquired vehicle type information, acquires vehicle vertex coordinates, and acquires three-dimensional coordinates of positions of the vehicles through the vehicle vertex coordinates, the correlation coefficients and the inclination angle; a vehicle coordinate coincidence judgment module (6) which judges whether the coordinates of all vehicles in the highway tunnel coincide; an early warning module (7) which judges that an accident occurs if the coordinates of two vehicles in the highway tunnel coincide, and transmits an early warning message to a traffic control platform so as to control traffic. The specific steps of acquiring the license plate numbers in the vehicle information acquisition module (3) include:

2. The video monitoring system for highway tunnels according to claim 1, characterized in that: converting the acquired license plate images into gray-scale images and performing pretreatment; converting the pretreated gray-scale images into binary images; performing number recognition based on the binary images to acquire the license plate numbers. The specific steps of acquiring the three-dimensional coordinates of the license plates in the vehicle information acquisition module (3) include:

3. The video monitoring system for highway tunnels according to claim 2, characterized in that: normalizing pixel coordinates of the acquired vehicle license plate images; converting the normalized pixel coordinates of the vehicle license plate images into camera coordinates; converting the camera coordinates into three-dimensional coordinates, i.e. the three-dimensional coordinates of the license plates. The specific steps of the vehicle inclination calculation module (4) include:

4. The video monitoring system for highway tunnels according to claim 3, characterized in that: acquiring three-dimensional coordinates of two end points of a horizontal section of a license plate in a horizontal state of a vehicle, and simultaneously acquiring three-dimensional coordinates of the two end points of the horizontal section in a vehicle license plate image; calculating a directional vector of the end points; ​ By the direction vector of the end point, a vector dot product is calculated; By the direction vector of the end point, a vector module length is calculated; The angle of inclination of the two horizontal segments is calculated by the vector dot product and the vector magnitude of the endpoints, expressed as: ; wherein: denotes the vector dot product, and denotes the vector length.

5. The video monitoring system for highway tunnels according to claim 4, characterized in that: The vehicle model information in the vehicle coordinate acquisition module (5) is acquired through the vehicle management platform of data sharing.

6. The video monitoring system for highway tunnels according to claim 5, characterized in that: The specific steps of acquiring the three-dimensional coordinates of each position of the vehicle in the vehicle coordinate acquisition module (5) include: ； ； In the formula: represents the three-dimensional coordinates of the license plate vertex, represents the three-dimensional coordinates of the position of the vehicle and the license plate vertex, , and respectively the correlation coefficient of the license plate vertex and the vehicle at a certain position in the , and direction; represents the three-dimensional coordinates of the position of the vehicle after tilting, represents the tilting angle.

Images