Vehicle information identification method, system, storage medium and electronic device
By collecting and splicing vertical side views of the vehicle by single cameras, using video correction and optical flow pyramid technology, the problem of insufficient matching accuracy of the front and body data in complex scenarios is solved, achieving 100% accurate matching of vehicle information and complete evidence links sex.
Patent Information
- Application Number
- CN202310740529.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-21
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2043-06-21
AI Technical Summary
In the prior art, in complex scenarios, such as congestion, plugging, reversing, etc., the accuracy of the data matching of the front and body of the vehicle is insufficient, making it difficult to achieve a 100% matching effect, especially when the vehicle color is the same or the capture position is inconsistent.
A single camera is used to identify vehicle information, collect driving videos through the camera, and splice the vertical side view of the vehicle using video correction and optical flow pyramid method to obtain information such as the license plate number, license plate color, brand, model and axle number of the vehicle to realize the vehicle data Accurate match.
Achieve 100% accuracy of vehicle data matching in complex scenarios, reduces the number of cameras and construction difficulty, provides a strong link of evidence, and supports post-event inspection and verification.
Smart Images

Figure CN116977894B_ABST
Abstract
Description
Background Art
[0002] According to the relevant standards for the abolition of provincial border toll stations in 2019, quick toll collection without stopping is realized. In order to help toll stations meet relevant technical standards, the toll vehicle classification technologies implemented mainly include pure video detection, external ultrasonic detectors, and external laser radar. However, in these three methods, the detected front image and body image are two separate images. The corresponding logic is needed to combine the front data and body data into the data of the entire vehicle and package them for upload.
[0003] Usually, the data packaging operation can be completed by sequentially matching the front and body of the vehicle, but the real scene is more complicated. There will be various problems such as missing the front of the vehicle, missing the body, not queuing according to regulations, etc. We cannot completely match according to the order, time, and characteristics.
[0004] The matching work is usually completed by assigning different weights and comprehensively considering these three methods. Through time matching, the front-end capture rate in the lane is about 99%. Once a car is missed, the subsequent data will be misplaced.
[0005] By feature matching, such as Figure 1 and Figure 2 As shown, there is no obvious intersection between the front and the body of the car, and it can only be judged by the color of the body. If there are consecutive vehicles of the same color queued in the lane, it is difficult to confirm how to match them, especially in the toll lane for sedans, where the number of white and black cars is very large.
[0006] There is still the problem of missing images through sequential matching. What's worse, some toll lanes have problems such as jamming and congestion. Because there is a distance of 8 to 10 meters between the capture position of the snapshot camera and the recognition position of the body camera, which can accommodate the next car, the order of the captured images is different from the order of the body recognition images, such as Figure 3 shown.
[0007] In such complex scenarios such as congestion, cutting in, and reversing, the current successful matching rate can reach more than 99% through methods such as time, features, sequence, and license plate correction queue at the rear of the vehicle, but it still cannot solve the problem 100%. Summary of the invention
[0008] The technical problem to be solved by the present invention is to provide a vehicle information recognition method, system, storage medium and electronic device in view of the deficiencies in the prior art.
[0009] The technical solution of a vehicle information recognition method of the present invention is as follows:
[0010] When a vehicle is identified in a preset detection area, a camera is used to capture a driving video of the vehicle in the preset detection area;
[0011] splicing the driving video to obtain a spliced side view of the vehicle at a vertical angle of the camera;
[0012] The spliced side view of the vehicle is identified to obtain vehicle information of the vehicle.
[0013] The beneficial effects of a vehicle information recognition method of the present invention are as follows:
[0014] Using a single camera to process data and simultaneously acquire images of the vehicle front and body, the vehicle data matching accuracy can reach 100% in a series of complex scenarios such as congestion, cutting in, and reversing.
[0015] Based on the above solution, a vehicle information identification method of the present invention can also be improved as follows.
[0016] Further, according to the driving video, a spliced side view of the vehicle at a vertical angle of the camera is obtained by splicing, including:
[0017] Using a video correction method, each frame of the driving video is corrected into a corrected image at a vertical angle of the camera;
[0018] Based on the multiple corrected images, a stitched side view of the vehicle at a vertical angle of the camera is obtained by stitching.
[0019] Further, based on the multiple corrected images, stitching is performed to obtain a stitched side view of the vehicle at a vertical angle of the camera, including:
[0020] The displacement width of the vehicle in each corrected image is calculated by using an optical flow pyramid method, and the displacement widths of the vehicle in the preset detection area are spliced to obtain the spliced side view.
[0021] Further, the vehicle information includes license plate number, license plate color, vehicle brand, vehicle model, axle number and axle type.
[0022] The technical solution of a vehicle information identification system of the present invention is as follows:
[0023] It includes a collection module, an acquisition module and a recognition module;
[0024] The acquisition module is used to: when a vehicle is identified in a preset detection area, use a camera to collect a driving video of the vehicle in the preset detection area;
[0025] The acquisition module is used to: splice the driving video to obtain a spliced side view of the vehicle at a vertical angle of the camera;
[0026] The recognition module is used to recognize the spliced side view of the vehicle to obtain the vehicle information of the vehicle.
[0027] The beneficial effects of a vehicle information recognition system of the present invention are as follows:
[0028] Using a single camera to process data and simultaneously acquire images of the vehicle front and body, the vehicle data matching accuracy can reach 100% in a series of complex scenarios such as congestion, cutting in, and reversing.
[0029] Based on the above solution, the vehicle information identification system of the present invention can also be improved as follows.
[0030] Further, the acquisition module includes a correction module and a splicing module;
[0031] The correction module is used to: use a video correction method to correct each frame of the driving video into a corrected image at a vertical angle of the camera;
[0032] The stitching module is used to stitch together a plurality of corrected images to obtain a stitched side view of the vehicle at a vertical angle of the camera.
[0033] Furthermore, the splicing module is specifically used for:
[0034] The displacement width of the vehicle in each corrected image is calculated by using an optical flow pyramid method, and the displacement widths of the vehicle in the preset detection area are spliced to obtain the spliced side view.
[0035] Further, the vehicle information includes license plate number, license plate color, vehicle brand, vehicle model, axle number and axle type.
[0036] A storage medium of the present invention stores instructions, and when a computer reads the instructions, the computer executes any one of the above-mentioned vehicle information recognition methods.
[0037] An electronic device of the present invention comprises a processor and the above-mentioned storage medium, wherein the processor executes instructions in the storage medium. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0039] Figure 1 It is one of the instance graphs of feature matching;
[0040] Figure 2 It is one of the instance graphs of feature matching;
[0041] Figure 3 This is an example diagram for explaining that the order of captured images is different from the order of vehicle body recognition images;
[0042] Figure 4 A schematic diagram of a vehicle information identification method according to an embodiment of the present invention;
[0043] Figure 5 A schematic diagram of a vehicle driving into a preset detection area;
[0044] Figure 6 This is one of the schematic diagrams of projection transformation;
[0045] Figure 7 This is the second schematic diagram of projection transformation;
[0046] Figure 8 This is one of the schematic diagrams of the splicing effect;
[0047] Fig. 9 This is the second schematic diagram of the splicing effect;
[0048] Fig.10 This is the third schematic diagram of the splicing effect;
[0049] Fig.11 The figure is a schematic diagram of the structure of a vehicle information identification system according to an embodiment of the present invention. DETAILED DESCRIPTION
[0050] like Figure 4 As shown, a vehicle information recognition method according to an embodiment of the present invention comprises the following steps:
[0051] S1. When a vehicle is identified in a preset detection area, a camera is used to collect a driving video of the vehicle in the preset detection area.
[0052] like Figure 5 As shown, when a vehicle enters a preset detection area, the front camera recognizes the license plate, starts video image transformation processing and vehicle tracking until the vehicle exits the preset detection area.
[0053] S2. splicing the driving video to obtain a spliced side view of the vehicle at a vertical angle of the camera;
[0054] S3. Identify the spliced side view of the vehicle to obtain vehicle information of the vehicle.
[0055] The vertical angle of the vehicle to the camera refers to the direction in which the camera is facing the side of the vehicle.
[0056] The present invention uses a single camera to process data and simultaneously obtain images of the vehicle front and vehicle body. In a series of complex scenarios such as congestion, cutting in, and reversing, the vehicle data matching accuracy can reach 100%.
[0057] Optionally, in the above technical solution, in S2, stitching the driving video to obtain a stitched side view of the vehicle at a vertical angle of the camera includes:
[0058] S21, using a video correction method, correcting each frame of the driving video into a corrected image at a vertical angle of the camera;
[0059] S22, based on the multiple corrected images, stitching to obtain a stitched side view of the vehicle at a vertical angle of the camera. The specific implementation process is as follows:
[0060] S200: First, a transformation matrix is calculated according to on-site manual markings or vehicle motion markings in a video image.
[0061] S201. When the equipment is installed, a single camera will be fixed in one position, and the area illuminated by its video image will not change. Therefore, four reference objects can be placed manually on site so that the four reference objects are parallel to the direction of the road and form a square. The pixel position in the camera can be found through the four reference objects. A quadrilateral can be formed in the camera image.
[0062] The manual marking method is more accurate. The second method is to obtain the upper and lower feature points of the same vehicle passing through the road at different positions through a single camera image to ensure that the feature points exist in the plane of the side of the vehicle body. Because the two feature points on the side of the vehicle do not change their ground height in the real environment when they are driving at different positions, they just move forward or backward along the direction of the road. Therefore, the punctuation pixel positions in the two images can be connected into a quadrilateral. This quadrilateral is the plane in the camera image parallel to the road driving direction.
[0063] Suppose that if our camera shoots the car body perpendicular to the direction of road travel, the quadrilateral obtained by the feature points will be a square or rectangle. When the vehicle moves, the y-axis of the feature points of the car body will basically not change. Because the movement of the vehicle needs to be tracked later. Therefore, the image of the measured car body is first transformed into an image of the car body shot vertically. The coordinates of the quadrilateral obtained by punctuation and the coordinates of the assumed vertically shot car body can be obtained through the principle of plane homography to obtain the homography matrix that converts the inclined plane matrix into horizontal and vertical plane matrices. Then, the image can be transformed into a 2D projection through the homography matrix to obtain the converted image, such as Figure 6 and Figure 7 Then the video vehicle is tracked, and finally a spliced side view of the tracked vehicle is generated.
[0064] Optionally, in the above technical solution, in S22, stitching together the images at the vertical angle of the camera to obtain a stitched side view of the vehicle at the vertical angle of the camera includes:
[0065] S220, using the optical flow pyramid method, calculate the displacement width of the vehicle in each corrected image, and splice the displacement width of the vehicle in the preset detection area to obtain a spliced side view, such as Figures 8 to 10 shown.
[0066] Optionally, in the above technical solution, the vehicle information includes license plate number, license plate color, vehicle brand, vehicle model, number of axles and axle type.
[0067] The specific process of identifying the spliced side view of the vehicle and obtaining the vehicle information of the vehicle is as follows:
[0068] The detection method for vehicle entry and exit adopts a hybrid detection of camera license plate detection and background modeling. When the camera uses yolov5 inference to obtain the license plate for each frame, it starts to detect the displacement width of each frame using the above method to stitch and obtain a large license plate snapshot.
[0069] When no moving objects are detected in a fixed area through background modeling, the vehicle is leaving. The images between entering and exiting are spliced to obtain the vehicle body image. The vehicle body image is inferred using yolov5 to obtain information such as the number of vehicle axles. The license plate image is inferred to obtain information such as whether it is a hazardous chemical. Because each captured license plate image corresponds to a spliced body image, the front of the vehicle and the body are one-to-one corresponding, and there will be no matching errors.
[0070] When the vehicle leaves the detected area, the video data processing is stopped, and the integrated vehicle side image and license plate capture image are inferred using a deep learning network to obtain data such as the number of vehicle axles, body position, toll model, vehicle brand, year, color, and whether it is a hazardous chemical vehicle.
[0071] Finally, the data and analysis results are uploaded to the FTP server for storage and sent to the web page for display.
[0072] The present invention uses a single camera that obliquely illuminates the lane to perform perspective transformation and complete the method of vehicle splicing and lateral fusion. Using a single video method, 3D visual measurement technology is performed at a fixed point to provide a digital video vehicle type detector for accurate classification of vehicle types. The image fusion data obtained by a single camera has vehicle head information (including but not limited to license plate number, license plate color, vehicle brand) and vehicle body information (including but not limited to vehicle model, number of axles, and axle type), and the beneficial effects are as follows:
[0073] 1) By using one video channel to simultaneously capture the license plate and the vehicle body, there will be no matching errors in vehicle segmentation and matching. In addition, the vehicle body splicing image can simultaneously show the license plate and the vehicle body to form a complete chain of evidence. Compared with other existing technologies, it reduces the number of cameras and reduces the cost of use, while also reducing the difficulty of construction caused by wiring and grooving.
[0074] 2) Through the detection of this system, a complete image of the vehicle front and body is finally provided, which provides a strong basis for subsequent inspection and result verification.
[0075] 3) The system can be integrated with any existing snapshot cameras, providing the possibility for upgrading toll stations. There is no need to purchase new snapshot cameras, and the system can be upgraded and used on the original cameras.
[0076] In the above embodiments, although the steps are numbered S1, S2, etc., these are only specific embodiments given in the present application. Those skilled in the art may adjust the execution order of S1, S2, etc. according to actual conditions, which is also within the scope of protection of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.
[0077] like Fig.11 As shown, a vehicle information recognition system 200 according to an embodiment of the present invention includes a collection module 210, an acquisition module 220 and a recognition module 230;
[0078] The acquisition module 210 is used to: when a vehicle is identified in a preset detection area, use a camera to collect a driving video of the vehicle in the preset detection area;
[0079] The acquisition module 220 is used to: splice the driving video to obtain a spliced side view of the vehicle at a vertical angle of the camera;
[0080] The recognition module 230 is used to recognize the spliced side view of the vehicle to obtain the vehicle information of the vehicle.
[0081] A vehicle information recognition system 200 of the present invention uses a single camera to process data and simultaneously obtain images of the front and body of the vehicle. In a series of complex scenarios such as congestion, cutting in, and reversing, the vehicle data matching accuracy can reach 100%.
[0082] Optionally, in the above technical solution, the acquisition module 220 includes a correction module and a splicing module;
[0083] The correction module is used to: use a video correction method to correct each frame of the driving video into a corrected image at a vertical angle of the camera;
[0084] The stitching module is used to stitch together a plurality of corrected images to obtain a stitched side view of the vehicle at a vertical angle of the camera.
[0085] Optionally, in the above technical solution, the splicing module is specifically used for:
[0086] The optical flow pyramid method is used to calculate the displacement width of the vehicle in each corrected image, and the displacement width of the vehicle in the preset detection area is spliced to obtain a spliced side view.
[0087] Optionally, in the above technical solution, the vehicle information includes license plate number, license plate color, vehicle brand, vehicle model, number of axles and axle type.
[0088] The above parameters and steps for each unit module to implement corresponding functions in a vehicle information identification system 200 of the present invention can refer to the parameters and steps in the embodiment of a vehicle information identification method mentioned above, and will not be repeated here.
[0089] A storage medium according to an embodiment of the present invention stores instructions, and when a computer reads the instructions, the computer executes any one of the above-mentioned vehicle information recognition methods.
[0090] An electronic device according to an embodiment of the present invention includes a processor and the above-mentioned storage medium, wherein the processor executes instructions in the storage medium. The electronic device may be a computer, a mobile phone, etc.
[0091] Those skilled in the art will appreciate that the present invention may be implemented as a system, method or computer program product.
[0092] Therefore, the present disclosure may be specifically implemented in the following forms, namely: it may be completely hardware, it may be completely software (including firmware, resident software, microcode, etc.), or it may be a combination of hardware and software, generally referred to herein as a "circuit", "module" or "system". In addition, in some embodiments, the present invention may also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.
[0093] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.
[0094] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A vehicle information recognition method, It is characterized in that include: When a vehicle is identified in a preset detection area, a camera is used to capture a driving video of the vehicle in the preset detection area; According to the driving video, a spliced side view of the vehicle at a vertical angle of the camera is obtained by splicing, where the vertical angle of the vehicle at the camera refers to a direction in which the camera is facing the side of the vehicle; Identify the spliced side view of the vehicle to obtain vehicle information of the vehicle; According to the driving video, a spliced side view of the vehicle at a vertical angle of the camera is obtained by splicing, including: Using a video correction method, each frame of the driving video is corrected into a corrected image at a vertical angle of the camera; Based on the multiple corrected images, stitching is performed to obtain a stitched side view of the vehicle at a vertical angle of the camera; Based on the multiple corrected images, stitching is performed to obtain a stitched side view of the vehicle at a vertical angle of the camera, including: A transformation matrix is calculated according to the vehicle motion punctuation points in the video image, and the image of the side-shot vehicle body is transformed into an image of the vertically shot vehicle body by plane transformation. The displacement width of the vehicle in each corrected image is calculated by using the optical flow pyramid method, and the displacement width of the vehicle in the preset detection area is spliced to obtain the spliced side view. A transformation matrix is calculated according to the vehicle motion punctuation points in the video image, including: The single camera image is used to obtain the upper and lower feature points of the same vehicle passing through the road at different positions, ensuring that the feature points exist in the plane of the side of the vehicle body. The two images are connected into a quadrilateral according to the pixel positions of the punctuation points. The quadrilateral is a plane parallel to the driving direction of the road in the camera image. The coordinates of the quadrilateral obtained by the punctuation points and the coordinates of the assumed vertically photographed vehicle body are used to obtain the homography matrix that converts the inclined plane matrix into the horizontal and vertical plane matrix through the principle of plane homography, and then the image is transformed into a 2D projection through the homography matrix to obtain the converted image; The spliced side view of the vehicle is identified to obtain vehicle information including: The detection method for vehicle entry and exit adopts a hybrid detection of camera license plate detection and background modeling. When the camera uses yolov5 inference to obtain the license plate for each frame of image, it starts to splice the displacement width of each frame detection and obtain a large license plate snapshot; When no moving objects are detected in the preset detection area through background modeling, the vehicle is leaving, and the images between entering and leaving are spliced to obtain the vehicle body image; When the vehicle leaves the preset detection area, the video data processing is stopped, and the integrated body image and license plate snapshot image are inferred using a deep learning network to obtain the vehicle's axle number, body position, toll model, vehicle brand, year, color, and whether it is a hazardous chemical vehicle; Finally, the data and analysis results are uploaded to the FTP server for storage and sent to the web page for display.
2. A vehicle information recognition system, It is characterized in that It includes a collection module, an acquisition module and a recognition module; The acquisition module is used to: when a vehicle is identified in a preset detection area, use a camera to collect a driving video of the vehicle in the preset detection area; The acquisition module is used to: splice the driving video to obtain a spliced side view of the vehicle at a vertical angle of the camera; The recognition module is used to: recognize the spliced side view of the vehicle to obtain vehicle information of the vehicle; The acquisition module includes a correction module and a splicing module; The correction module is used to: use a video correction method to correct each frame of the driving video into a corrected image at a vertical angle of the camera; The stitching module is used to stitch together a plurality of corrected images to obtain a stitched side view of the vehicle at a vertical angle of the camera, where the vertical angle of the vehicle at the camera refers to a direction in which the camera is facing the side of the vehicle; The splicing module is specifically used for: A transformation matrix is calculated according to the vehicle motion punctuation points in the video image, and the image of the side-shot vehicle body is transformed into an image of the vertically shot vehicle body by plane transformation. The displacement width of the vehicle in each corrected image is calculated by using the optical flow pyramid method, and the displacement width of the vehicle in the preset detection area is spliced to obtain the spliced side view. A transformation matrix is calculated according to the vehicle motion punctuation points in the video image, including: The single camera image is used to obtain the upper and lower feature points of the same vehicle passing through the road at different positions, ensuring that the feature points exist in the plane of the side of the vehicle body. The two images are connected into a quadrilateral according to the pixel positions of the punctuation points. The quadrilateral is a plane parallel to the driving direction of the road in the camera image. The coordinates of the quadrilateral obtained by the punctuation points and the coordinates of the assumed vertically photographed vehicle body are used to obtain the homography matrix that converts the inclined plane matrix into the horizontal and vertical plane matrix through the principle of plane homography, and then the image is transformed into a 2D projection through the homography matrix to obtain the converted image; The identification module is specifically used for: The detection method for vehicle entry and exit adopts a hybrid detection of camera license plate detection and background modeling. When the camera uses yolov5 inference to obtain the license plate for each frame of image, it starts to splice the displacement width of each frame detection and obtain a large license plate snapshot; When no moving objects are detected in the preset detection area through background modeling, the vehicle is leaving, and the images between entering and leaving are spliced to obtain the vehicle body image; When the vehicle leaves the preset detection area, the video data processing is stopped, and the integrated body image and license plate snapshot image are inferred using a deep learning network to obtain the vehicle's axle number, body position, toll model, vehicle brand, year, color, and whether it is a hazardous chemical vehicle; Finally, the data and analysis results are uploaded to the FTP server for storage and sent to the web page for display.
3. A storage medium, It is characterized in that The storage medium stores instructions, and when a computer reads the instructions, the computer executes the vehicle information identification method as claimed in claim 1.
4. An electronic device, It is characterized in that The invention comprises a processor and the storage medium as claimed in claim 3, wherein the processor executes instructions in the storage medium.
