A vehicle model identification method, device and system

Abstract

This invention relates to vehicle model recognition, and more particularly to a vehicle model recognition method, apparatus, and system. A vehicle model recognition method includes: acquiring images of the front, body, and rear of a vehicle; acquiring width, height, and length data of the vehicle; the width, height, and length data including first width and height data, second width and height data, and third width and height data; acquiring vehicle model data; the vehicle model data including model information and axle information; and integrating the width, height, and length data and the model data to output vehicle model recognition data. After obtaining the front, body, and rear images, license plate detection is performed, and a first proportional coefficient is obtained based on the license plate image size and the physical size of the license plate, thereby obtaining accurate first width and height data for the front, second width and height data for the rear, and third width and height data for the body. Simultaneously, accurate identification of the vehicle model and axle type is performed, enabling accurate acquisition of vehicle model recognition data for passing vehicles and improving recognition accuracy.

Description

Technical Field

[0001] This invention relates to vehicle model recognition, and more particularly to a vehicle model recognition method, apparatus, and system. Background Technology

[0002] Vehicle length, width, height, number of axles, axle type, license plate type, license plate color, license plate number, and vehicle appearance are important criteria for vehicle type classification. Vehicle model, as crucial information about motor vehicles, plays a vital role in applications such as autonomous driving, criminal investigation by public security authorities, traffic management, and road toll collection. Because vehicle type classification relies on numerous and complementary pieces of information, single sensor devices such as monocular cameras, binocular cameras, weighing systems, laser scanners, and infrared scanners cannot comprehensively and accurately acquire the relevant information. Therefore, achieving high accuracy in vehicle type classification is challenging.

[0003] Patent CN111783638A proposes a system and method for detecting the number of axles and identifying vehicle models. The system includes a ranging sensor, a high-frequency parallel signal acquisition device, and a computing and processing device. Through the ranging sensor, data sequences of wheel depth maps and vehicle body depth maps can be obtained to detect the number of axles. Based on the license plate information of the acquisition device, the vehicle model identification result is obtained. However, the information items based on the system are only the number of axles and the license plate information, which is insufficient to achieve accurate vehicle model identification.

[0004] Patent CN111523579A proposes a vehicle model recognition method and system based on improved deep learning. This method, based on deep learning, requires a large amount of traffic checkpoint vehicle image datasets to be segmented, sorted, classified, and trained to achieve vehicle model classification and recognition. However, it only utilizes the shape information of the vehicle images, making it difficult to guarantee the accuracy of vehicle model recognition.

[0005] Therefore, the existing vehicle model recognition technology still has shortcomings and needs to be improved and enhanced. Summary of the Invention

[0006] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a vehicle model recognition method, device and system that can improve the accuracy of vehicle length, width and height recognition, and at the same time perform comprehensive recognition of vehicle models, thereby comprehensively improving the accuracy of vehicle model recognition.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] On one hand, the present invention provides a vehicle model recognition method, comprising:

[0009] Acquire images of the vehicle's front, body, and rear.

[0010] Obtain the width, height, and length data of the vehicle; the width, height, and length data includes first width and height data, second width and height data, and third width and height data; wherein, license plate detection is performed on the front and rear images of the vehicle to obtain a first ratio coefficient between the license plate image size and the physical size of the license plate, and then the first width and height data of the front and the second width and height data of the rear are obtained according to the front image size and the rear image size, respectively; feature value extraction and matching are performed on the front image and / or the rear image and the vehicle body image, and the third width and height data of the vehicle body is obtained according to the first ratio coefficient and the vehicle body image size;

[0011] The vehicle model data is obtained; the vehicle model data includes model information and axle information; wherein, the model information is obtained by recognizing the front image of the vehicle using a first recognition model; and the axle information is obtained by recognizing the body image using a second recognition model.

[0012] Integrate the width, height, and length data with the vehicle model data to output vehicle model identification data.

[0013] Preferably, the steps for obtaining the first width and height data or the second width and height data include:

[0014] The license plate image is recognized by the third recognition model to obtain license plate location data and license plate type data; the first image is the front image of the vehicle or the rear image of the vehicle.

[0015] The aspect ratio of the license plate image is adjusted to be the same as the actual physical aspect ratio of the corresponding license plate type through affine transformation, and the aspect ratio of the first image is adjusted synchronously based on the same adjustment ratio to obtain the first adjusted image;

[0016] Obtain the first scaling factor between the image size of the license plate and the physical size of the license plate;

[0017] Obtain the image size of the first adjusted image, and obtain the first width and height data or the second width and height data based on the first scaling factor.

[0018] Preferably, obtaining the proportionality coefficient specifically includes:

[0019] Obtain the character image size of each character in the license plate image;

[0020] The first proportional coefficient is obtained based on the physical size of the characters on the license plate.

[0021] Preferably, the step of obtaining the third width and height data is as follows:

[0022] Obtain feature matching points between the first adjusted image and the vehicle body image, and then obtain the corresponding second scaling factor;

[0023] The vehicle image is normalized according to the second scaling factor to obtain a first normalized image;

[0024] Obtain the image size of the first normalized image, and obtain the third width and height data based on the first scaling factor.

[0025] Preferably, the first adjustment image includes a front adjustment image and a rear adjustment image;

[0026] The steps for obtaining the third width and height data are as follows:

[0027] The feature matching points of the front adjustment image and the body image are obtained, and then the corresponding third proportional coefficient is obtained.

[0028] The vehicle image is normalized according to the third scaling factor to obtain a second normalized image;

[0029] The feature matching points of the rear adjustment image and the body image are obtained, and then the corresponding fourth proportional coefficient is obtained.

[0030] The vehicle image is normalized according to the fourth scaling factor to obtain a third normalized image;

[0031] Obtain the image size of the second normalized image, and obtain the first vehicle body width and height data based on the first scaling factor;

[0032] Obtain the image size of the third normalized image, and obtain the second vehicle body width and height data based on the first scaling factor;

[0033] The third width and height data is obtained by weighted calculation based on the first vehicle width and height data and the second vehicle width and height data.

[0034] Preferably, the axle information includes the number of axles and the axle type;

[0035] The vehicle model recognition data also includes: the front image of the vehicle, the body image of the vehicle, the rear image of the vehicle, and license plate type data.

[0036] Preferably, the step of acquiring the front or rear image of the vehicle includes:

[0037] The second image is acquired, and the fourth recognition model is used to identify the front or rear of the vehicle in the second image to obtain rough rectangular box position information; the second image is an image with front or rear features.

[0038] Image edge detection is performed on the rough rectangular frame to obtain the front or rear image of the vehicle.

[0039] Preferably, the step of acquiring the vehicle body image includes:

[0040] Acquire multiple consecutively generated third images; the third images are images with vehicle body features.

[0041] Calculate the feature vector of the third image in each frame, and calculate the matching similarity with the third image in the previous frame, thereby obtaining the corresponding offset.

[0042] The vehicle body image is obtained by stitching together multiple frames of the third image according to the corresponding offset.

[0043] On the other hand, the present invention provides a vehicle model identification device, comprising:

[0044] The acquisition module is used to acquire images of the front, body, and rear of the vehicle.

[0045] The processing module is used to acquire the width, height, and length data of the vehicle; the width, height, and length data includes first width and height data, second width and height data, and third width and height data; wherein, license plate detection is performed on the front and rear images of the vehicle to obtain a first ratio coefficient between the license plate image size and the physical size of the license plate, and then the first width and height data of the front and the second width and height data of the rear are obtained according to the size of the front and rear images, respectively; feature value extraction and matching are performed on the front and / or rear images and the vehicle body image, and the third width and height data of the vehicle body is obtained according to the first ratio coefficient and the size of the vehicle body image; vehicle model data is acquired; the vehicle model data includes model information and axle information; wherein, the model information is obtained by recognizing the front image through a first recognition model; the axle information is obtained by recognizing the vehicle body image through a second recognition model; the width, height, and length data and the vehicle model data are integrated to output vehicle model recognition data.

[0046] On the other hand, the present invention provides a vehicle model recognition system, comprising:

[0047] Multi-view camera, which has multiple cameras facing different directions;

[0048] A vehicle model recognition device, connected to the multi-camera system, receives images transmitted from multiple cameras to obtain images of the vehicle's front, body, and rear. It also acquires the vehicle's width, height, and length data, including first, second, and third width and height data. Specifically, it performs license plate detection on the front and rear images to obtain a first proportionality coefficient between the license plate image size and the physical size of the license plate. Based on the front and rear image sizes, it obtains the first width and height data for the front and the second width and height data for the rear, respectively. It then performs feature value extraction and matching between the front and / or rear images and the body image, obtaining the third width and height data for the body based on the first proportionality coefficient and the body image size. Finally, it acquires vehicle model data, including model information and axle information. The model information is obtained by recognizing the front image using a first recognition model, and the axle information is obtained by recognizing the body image using a second recognition model. Finally, it integrates the width, height, and length data with the vehicle model data to output vehicle model recognition data.

[0049] Compared with existing technologies, the vehicle model recognition method, device, and system provided by this invention have the following advantages:

[0050] Using the vehicle model recognition method provided by this invention, after obtaining the front image, body image, and rear image of the vehicle, license plate detection is performed, and then a first proportional coefficient is obtained based on the size of the license plate image and the physical size of the license plate. This results in accurate first width and height data of the front of the vehicle, second width and height data of the rear of the vehicle, and third width and height data of the body of the vehicle. At the same time, the vehicle model and axle type are accurately identified, which can accurately obtain the vehicle model recognition data of the passing vehicle and improve the recognition accuracy. Attached Figure Description

[0051] Figure 1 This is a flowchart of the vehicle model recognition method provided by the present invention.

[0052] Figure 2 This is a schematic diagram of the installation of the multi-view camera provided by the present invention.

[0053] Figure 3 This is a flowchart of the steps for obtaining the width and height data of the front or rear of a vehicle provided by the present invention.

[0054] Figure 4 This is a flowchart of the steps for obtaining the first proportional coefficient provided by the present invention in another embodiment.

[0055] Figure 5 This is a flowchart of a method for obtaining the third width and height data provided by the present invention.

[0056] Figure 6 This is a flowchart illustrating another method for obtaining the third width and height data provided by the present invention.

[0057] Figure 7 This is a flowchart of the steps for acquiring vehicle body images provided by the present invention.

[0058] Figure 8 This is a schematic diagram of vehicle body image acquisition provided by the present invention.

[0059] Figure 9 This is a flowchart of an embodiment of the vehicle model recognition method provided by the present invention.

[0060] Figure 10 This is a structural block diagram of the vehicle model recognition device provided by the present invention.

[0061] Figure 11 This is a structural block diagram of the vehicle model recognition system provided by the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0063] Those skilled in the art will understand that the foregoing general description and the following detailed description are exemplary and illustrative embodiments of the present invention and are not intended to limit the invention.

[0064] The terms “comprising,” “including,” or any other variations thereof throughout this document are intended to cover non-exclusive inclusion, such that a process or method that includes a list of steps includes not only those steps but may also include other steps not expressly listed or inherent to such a process or method. Similarly, without further limitation, one or more devices or subsystems, elements, structures, or components beginning with “comprising…a” will not exclude the presence of other devices or other subsystems or other elements or other structures or components. Throughout the specification, the phrases “in one embodiment,” “in another embodiment,” and similar language may, but not necessarily, refer to the same embodiment.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0066] Please refer to the following: Figure 1 , Figure 2 and Figure 9 This invention provides a vehicle model recognition method, applied to a vehicle model recognition device, and connected to a multi-view camera. In this embodiment, depending on the actual application scenario, the multi-view camera is installed at position 1 or position 2 on the side of the road (e.g., ...). Figure 2As shown in the figure, the multi-view camera has at least a plurality of cameras, each facing a different direction, and acquires image data of the road in different directions.

[0067] In a further preferred embodiment, the number of cameras is three, each facing the road with the following orientations: facing the road directly (acquiring a second image for generating a vehicle body image), facing the direction the vehicle is approaching (acquiring an image for generating a front view image), and facing the direction the vehicle is leaving (acquiring an image for generating a rear view image). In a further preferred embodiment, the number of cameras exceeds three, with one or more cameras in each direction. This allows for the selection of preferred image data to generate corresponding front, rear, and vehicle body images for the same vehicle. The requirements for camera installation are shown in the figure: 1) The field of view angles of cameras 1 and 2, and cameras 2 and 3, overlap to a certain extent; 2) Each camera can adjust its angle vertically and horizontally; 3) When a vehicle passes the camera group, camera 1 can capture a complete front view image, camera 2 can capture a complete full-body height image of the vehicle with the image frame as horizontal and parallel to the road surface as possible, and camera 3 can capture a complete rear view image of the vehicle, thus facilitating feature point matching between images.

[0068] Furthermore, the preferred working process between the multiple cameras of the multi-view camera is as follows: When a vehicle enters area 1, camera 1 acquires a sequence of images of the vehicle's front end. The processing module detects and identifies the front end image based on the image sequence and simultaneously sends the image acquisition signal to camera 2. When a vehicle enters area 2, camera 2 acquires a sequence of images of the vehicle's body and simultaneously sends the image acquisition signal to camera 3. When a vehicle enters area 3, camera 3 acquires a sequence of images of the vehicle's rear end, thereby obtaining all image sequences of the vehicle passing through areas 1 to 3, and thus obtaining the front end image, body image, and rear end image. The cameras acquire images sequentially, which can effectively save power, storage resources, and computing resources.

[0069] The vehicle model identification method includes:

[0070] S1. Acquire the front image, body image, and rear image of the vehicle; In this embodiment, the front image, body image, and rear image are all generated by capturing image sequences by a camera. Of course, when using cameras with automatic recognition functions and capable of acquiring clean target images, the images transmitted by such cameras can be used directly.

[0071] S2. Obtain the width, height, and length data of the vehicle; the width, height, and length data includes first width and height data, second width and height data, and third width and height data; wherein, license plate detection is performed on the front and rear images of the vehicle to obtain a first ratio coefficient between the license plate image size and the physical size of the license plate, and then the first width and height data of the front and the second width and height data of the rear are obtained according to the front image size and the rear image size, respectively; feature value extraction and matching are performed on the front image and / or rear image and the vehicle body image, and the third width and height data of the vehicle body is obtained according to the first ratio coefficient and the vehicle body image size; in this embodiment, the first ratio coefficient includes a first width ratio coefficient and a first height ratio coefficient, which are calculated simply. For example, if the license plate image size is 400*150 pixels and the physical size of the license plate is 440mm×140mm, then the first width ratio coefficient is 0.91 pixels / mm and the first height ratio coefficient is 1.07 pixels / mm. After obtaining the front image size or the rear image size, the first width and height data and the second width and height data are quickly calculated, and the third width and height data can be calculated similarly.

[0072] S3. Obtain vehicle model data; the vehicle model data includes model information and axle information; wherein, the model information is obtained by recognizing the front image of the vehicle using a first recognition model; the axle information is obtained by recognizing the body image using a second recognition model; specifically, both the first and second recognition models are based on deep neural network models.

[0073] Specifically, the training process of the first recognition model includes:

[0074] Obtain a first training set, which includes several first training images. The first training images are images with vehicle type characteristics and are labeled with various vehicle types. The vehicle types include large cars, trailer cars, police cars, etc.

[0075] The first recognition model is obtained by training the initialized neural network model using the first training set.

[0076] The training process of the second recognition model includes:

[0077] Obtain a second training set, which includes several second training images. The second training images are images with axle features and are labeled with various axle types.

[0078] The second recognition model is obtained by training the initialized neural network model using the second training set.

[0079] S4. Integrate the width-height-length data and the vehicle model data, and output vehicle model recognition data. Specifically, the data integration process is to fuse the recognized data to obtain an accurate recognition of the vehicle model information for the vehicle.

[0080] Using the vehicle model recognition method provided by the present invention, after obtaining the front vehicle image, the vehicle body image, and the rear vehicle image, by performing license plate detection, and then obtaining the first proportionality coefficient based on the license plate image size and the license plate physical size, and further obtaining the accurate first width-height data of the front vehicle, the second width-height data of the rear vehicle, and the third width-height data of the vehicle body. At the same time, accurately recognize the vehicle model and the axle type, and can accurately obtain the vehicle model recognition data of the passing vehicle, improving the recognition accuracy.

[0081] Further, please refer to Figure 3 , as a preferred solution, in this embodiment, the obtaining steps of the first width-height data or the second width-height include:

[0082] S21. Perform license plate image recognition on the first image through the third recognition model to obtain license plate position data and license plate type data; the first image is the front vehicle image or the rear vehicle image; specifically, the third recognition model is trained based on a neural network model, can automatically recognize the first image, quickly obtain the corresponding recognition result, and ensure accuracy at the same time.

[0083] Specifically, the license plate types include: 1. Large vehicle license plate - front: 440mm×140mm - rear: 440mm×220mm - yellow background with black characters and black frame line; 2. Trailer license plate - 440mm×220mm - yellow background with black characters and black frame line; 3. Small vehicle license plate - 440mm×140mm - blue background with white characters and white frame line; 4. Diplomatic vehicle license plate - 440mm×140mm - black background with white characters, red "Diplomatic" and "Consular" characters with white frame line; 5. Consular vehicle license plate - 440mm×140mm - black background with white characters, red "Diplomatic" and "Consular" characters with white frame line; 6. Hong Kong and Macau entry-exit vehicle license plate - 440mm×140mm - black background with white characters, white "Hong Kong" and "Macau" characters with white frame line; 7. Training vehicle license plate - 440mm×140mm - yellow background with black characters, black "Training" character with black frame line; 8. Police vehicle license plate - 440mm×140mm - white background with black characters, red "Police" character with black frame line.

[0084] The training process of the third recognition model includes:

[0085] Obtain a third training set, the third training set includes a number of third training images, the third training images are images with license plate features and are labeled with tags of various license plate types;

[0086] Use the third training set to train the initialized neural network model to obtain the third recognition model.

[0087] S22. The aspect ratio of the license plate image is adjusted to be the same as the actual physical aspect ratio of the corresponding license plate type through affine transformation, and the aspect ratio of the first image is adjusted synchronously based on the same adjustment ratio to obtain the first adjusted image.

[0088] S23. Obtain the first proportionality coefficient between the image size of the license plate image and the physical size of the license plate;

[0089] S24. Obtain the image size of the first adjusted image, and obtain the first width and height data or the second width and height data based on the first scaling factor.

[0090] This embodiment mainly performs license plate detection, localization, correction, type classification, character segmentation, and recognition processing on vehicle front or rear images.

[0091] In a further embodiment, after obtaining the license plate location data, the positions of the four vertices of the license plate rectangle are obtained by target detection and feature point extraction methods. At the same time, the third recognition model can also obtain the license plate color and license plate type data. Then, combined with the license plate type information, an affine transformation is used to obtain a license plate image with the same aspect ratio as the actual physical license plate. Then, combined with semantic segmentation methods, the precise rectangle positions of each character are obtained. Finally, the license plate number information is identified, and the basic information of the license plate, including the number, color, and type, can be quickly obtained.

[0092] For further details, please refer to Figure 4 As a preferred embodiment, obtaining the proportionality coefficient specifically includes:

[0093] Obtain the character image size of each character in the license plate image;

[0094] The first proportional coefficient is obtained based on the physical dimensions of the license plate characters. Generating the first proportional coefficient using the physical dimensions of each character and the image dimensions results in greater accuracy, leading to more precise calculations of the width and height data of the vehicle's front or rear. This achieves accurate estimation of the physical dimensions of the license plate characters, thereby accurately estimating the vehicle's height, width, and length.

[0095] In the specific implementation process, after the affine transformation, the transformation parameters M (M is a 3x3 matrix) are first obtained, and combined with the precise position information of the vehicle's front, the frontal image of the vehicle's front is obtained through the affine transformation method. The resulting image size of the vehicle's front is W. img_h *H img_h .

[0096] The precise rectangular bounding box positions of each character are obtained, and the image size W of each character is obtained through affine transformation using the transformation parameter M. img_h_ch_i *H img_h_ch_i (where i is the character number).

[0097] Based on the image size of the characters and the imaging characteristics of each character, cluster analysis is used to remove characters such as "1" or other characters whose image size is too large or too small due to dirt or other reasons. Then, the average image size of the characters is calculated.

[0098] Where N1 is the number of valid characters in the width image size, and N2 is the number of valid characters in the height image size.

[0099] Based on the latest industry standard for motor vehicle license plates of the People's Republic of China, the physical size W of a single character is specified. phy_ch *H phy_h The ratio coefficient between the physical size of the character and the image size is calculated (i.e., the first ratio coefficient, which includes the first width ratio coefficient and the first height ratio coefficient):

[0100] The first width scaling factor is The first height ratio coefficient is

[0101] By combining the proportional imaging characteristics of camera images, the physical width and height of the vehicle's front can be calculated (i.e., the initial width and height data of the vehicle's front):

[0102]

[0103]

[0104] In a further embodiment, before obtaining the first scaling factor, distortion correction is performed on the first image to make the image closer to the actual shape of the car front.

[0105] For further details, please refer to Figure 5 As a preferred embodiment, the step of obtaining the third width and height data is as follows:

[0106] The feature matching points of the first adjusted image and the vehicle body image are obtained, and then the corresponding second scaling factor is obtained; the first adjusted image is either the first adjusted image of the front of the vehicle or the first adjusted image of the rear of the vehicle.

[0107] The vehicle image is normalized according to the second scaling factor to obtain a first normalized image;

[0108] The image size of the first normalized image is obtained, and the third width and height data are obtained based on the first scaling factor. In this embodiment, only the front or rear image of the vehicle body is used to normalize and adjust the vehicle body image, so that the image size of the vehicle body image is adaptively adjusted with the adjusted front or rear image, thereby obtaining a normalized image that adapts to the first adjusted image, which can further improve the accuracy of the width and height calculation of the vehicle body.

[0109] For further details, please refer to Figure 6 As a preferred embodiment, in this embodiment, the first adjustment image includes a front adjustment image and a rear adjustment image;

[0110] The steps for obtaining the third width and height data are as follows:

[0111] The feature matching points of the front adjustment image and the body image are obtained, and then the corresponding third proportional coefficient is obtained.

[0112] The vehicle image is normalized according to the third scaling factor to obtain a second normalized image;

[0113] The feature matching points of the rear adjustment image and the body image are obtained, and then the corresponding fourth proportional coefficient is obtained.

[0114] The vehicle image is normalized according to the fourth scaling factor to obtain a third normalized image;

[0115] Obtain the image size of the second normalized image, and obtain the first vehicle body width and height data based on the first scaling factor;

[0116] Obtain the image size of the third normalized image, and obtain the second vehicle body width and height data based on the first scaling factor;

[0117] The third width and height data is obtained by weighted calculation based on the first and second vehicle width and height data. In this embodiment, after feature point matching between the vehicle image and the front or rear adjusted image, a three-dimensional image of the vehicle can be obtained. Simultaneously using the front and rear images to match the vehicle image can effectively improve the calculation accuracy of the third width and height data.

[0118] In practice, feature matching points are calculated between the complete frontal image of the vehicle and the image of the vehicle body, and the third scaling factor f1 for the height transformation between the two images is determined.

[0119] Where H img_s H represents the height of the vehicle image. img_h F represents the height of the front view of the vehicle. s (y iF represents the y-coordinate of a feature point in the front image of the vehicle. h (y i ) represents the y-coordinate of a feature point in the vehicle image, and K represents the number of feature point pairs;

[0120] The second normalized image is calculated based on the third scaling factor to obtain the normalized length W of the complete side image of the vehicle body. img0_s and high H img0_s :

[0121] W img0_s =f1*W img_s W img_s The length of the vehicle body image.

[0122] H img0_s =f1*H img_s H img_s vehicle body image height

[0123] Based on the first scaling factor and the proportional imaging characteristics of the camera image, the length and height information of the vehicle body's first width and height data can be estimated:

[0124]

[0125]

[0126] Similarly, by using feature point matching between the rear adjustment image and the body image, and the rear height information, the length W of the second body width and height data can be obtained. phy_s and height H" phy_s information;

[0127] To more accurately estimate the third width and height data of the vehicle body, the first and second vehicle body width and height data are used to calculate a weighted value, which is then used as the length and height information in the third width and height data of the vehicle body:

[0128] W phy_s =W' phy_s *α+W" phy_s *β, where α∈(0,1) and β∈(0,1) are weights, and α+β=1;

[0129] where δ∈(0,1) and For weights,

[0130] Generally, we take α = 0.5, β = 0.5, δ = 0.5. That is, to calculate the average value.

[0131] For further details, please refer to Figure 7 and Figure 8As a preferred embodiment, in this embodiment, the axle information includes the number of axles and the axle type;

[0132] The vehicle model recognition data also includes: the front image, the body image, the rear image, and license plate type data. Specifically, after multi-data fusion processing, complete front view images of the front, body, and rear of the vehicle, as well as corresponding three-dimensional images, can be obtained. The fusion calculation yields information such as the vehicle's length, width, height, number of axles, axle type, license plate type, license plate color, license plate number, and vehicle shape, enabling accurate vehicle model recognition.

[0133] By combining the information obtained from all the above steps, a multi-data information fusion processing method is used to obtain a complete front view image of the front, body and rear of the vehicle. The fusion calculation yields information such as the vehicle's length, width and height, number of axles, axle type, license plate type, license plate color, license plate number and vehicle shape, thus achieving accurate vehicle model identification.

[0134] Furthermore, as a preferred embodiment, the step of acquiring the front or rear image of the vehicle includes:

[0135] A second image is acquired, and a fourth recognition model is used to identify the front or rear of the vehicle in the second image to obtain rough rectangular box position information; the second image is an image with front or rear vehicle features; preferably, the fourth recognition model is obtained by training an initialized neural network model. The image with the aforementioned front or rear vehicle features is captured by a preset first camera (e.g., camera 1 or camera 3), which is installed on one side of the road at a certain angle (not 90°) to the road edge. When a vehicle appears in the field of view of the camera, the captured photo has the aforementioned front or rear vehicle features.

[0136] Image edge detection is performed on the rough rectangular bounding box to obtain either a front or rear image of the vehicle. Using a neural network-based recognition model to identify the front or rear of the vehicle allows for rapid determination of whether a front or rear image exists in the second image, and quick annotation while ensuring accuracy.

[0137] A deep learning-based object detection method is used to obtain the approximate rectangular bounding box position information of the front or rear of the vehicle. This is combined with an image edge detection method to obtain the edge image of the front of the vehicle. Then, based on the edge information, the precise left-right and up-down positions of the front of the vehicle are determined, achieving accurate positioning of the front of the vehicle. Finally, by combining the detection confidence and the precise position information of the front of the vehicle, the complete image of the front of the vehicle can be captured.

[0138] The training process of the fourth recognition model includes:

[0139] Obtain a fourth training set, which includes several fourth training images. The fourth training images are images with front or rear features of a vehicle and are labeled with the front or rear of the vehicle.

[0140] The fourth recognition model is obtained by training the initialized neural network model using the fourth training set.

[0141] Furthermore, as a preferred embodiment, the step of acquiring the vehicle body image includes:

[0142] A series of continuously generated third images are acquired; the third images are images with vehicle body features; preferably, the series of continuously generated third images can be obtained by filtering images from a video segment within a unit of time. The images with vehicle body features are captured by a preset second camera (e.g., camera 2), the shooting direction of which is perpendicular to the edge of the road, and when a vehicle appears in the field of view, the captured photo has vehicle body features.

[0143] Calculate the feature vector of the third image in each frame, and calculate the matching similarity with the third image in the previous frame, thereby obtaining the corresponding offset.

[0144] The vehicle body image is obtained by stitching together multiple frames of the third image according to the corresponding offset. In this embodiment, a complete vehicle image can be obtained for both small and large vehicles (especially vehicles that are extra long or extra wide).

[0145] For specific implementation details, please refer to [link / reference]. Figure 8 Based on the multi-frame third image formed by the vehicle body image sequence, detect whether the vehicle (body) appears in the camera's field of view area 2. If so, record it as flag=1 and start stitching.

[0146] Calculate the feature point set of the third image in the current i-th frame. Where i is the sequence number of the third image frame, j is the sequence number of the feature point, and N is the number of valid feature points (x j ,y j Let be the image location corresponding to the feature point. Then, the set of matching feature points corresponding to the third image in the previous frame (i-1th frame) is:

[0147] Based on the feature point set of the third image of the i-th frame The feature point set of the third image in the previous frame i-1 Calculate the matching similarity of feature points, filter out points that fail to match using a threshold, and then calculate the average (x) of the matching feature pairs. i0 ,y i0 The displacement, i.e., the vehicle target in the third image of the i-th frame and the third image of the (i-1)-th frame are translated, has the following offset:

[0148] Where M is the number of effective feature pairs;

[0149] Based on the actual situation of the vehicle passing through area 2 and position 2 of camera 2 during normal driving, the vertical displacement y of the vehicle body image is... i0 The value is approximately zero and can be ignored. Therefore, the horizontal displacement x of the vehicle body image can be used as a reference. i0 Calculate the stitched image of the vehicle body target in the third image of the i-th frame and the third image of the (i-1)-th frame, as follows: Figure 8 As shown.

[0150] The same steps are performed on each third image until the vehicle (body) disappears from the camera's field of view area 2, i.e., flag=0. Once the last third image is processed, the complete vehicle body image stitching is complete, and the process moves on to the next vehicle stitching preparation stage.

[0151] Please see Figure 10 The present invention also provides a vehicle model recognition device, comprising:

[0152] The acquisition module is used to acquire images of the front, body, and rear of the vehicle.

[0153] The processing module is used to acquire the width, height, and length data of the vehicle; the width, height, and length data includes first width and height data, second width and height data, and third width and height data; wherein, license plate detection is performed on the front and rear images of the vehicle to obtain a first ratio coefficient between the license plate image size and the physical size of the license plate, and then the first width and height data of the front and the second width and height data of the rear are obtained according to the size of the front and rear images, respectively; feature value extraction and matching are performed on the front and / or rear images and the vehicle body image, and the third width and height data of the vehicle body is obtained according to the first ratio coefficient and the size of the vehicle body image; vehicle model data is acquired; the vehicle model data includes model information and axle information; wherein, the model information is obtained by recognizing the front image through a first recognition model; the axle information is obtained by recognizing the vehicle body image through a second recognition model; the width, height, and length data and the vehicle model data are integrated to output vehicle model recognition data.

[0154] Please see Figure 11 The present invention also provides a vehicle model recognition system, comprising:

[0155] Multi-view camera, which has multiple cameras facing different directions;

[0156] A vehicle model recognition device, connected to the multi-camera system, receives images transmitted from multiple cameras to obtain images of the vehicle's front, body, and rear. It also acquires the vehicle's width, height, and length data, including first, second, and third width and height data. Specifically, it performs license plate detection on the front and rear images to obtain a first proportionality coefficient between the license plate image size and the physical size of the license plate. Based on the front and rear image sizes, it obtains the first width and height data for the front and the second width and height data for the rear, respectively. It then performs feature value extraction and matching between the front and / or rear images and the body image, obtaining the third width and height data for the body based on the first proportionality coefficient and the body image size. Finally, it acquires vehicle model data, including model information and axle information. The model information is obtained by recognizing the front image using a first recognition model, and the axle information is obtained by recognizing the body image using a second recognition model. Finally, it integrates the width, height, and length data with the vehicle model data to output vehicle model recognition data.

[0157] It is understood that those skilled in the art can make equivalent substitutions or modifications to the technical solution and inventive concept of the present invention, and all such substitutions or modifications should fall within the protection scope of the appended claims.

Claims

1. A vehicle model identification method characterized by comprising: The method comprises the following steps: obtaining a front image, a body image and a rear image of a vehicle; obtaining width-height-length data of the vehicle; the width-height-length data comprises first width-height data, second width-height data and third width-height data; wherein license plate detection is performed on the front image and the rear image to obtain a first proportionality coefficient between the size of a license plate image and the physical size of the license plate, and then first width-height data of the front of the vehicle and second width-height data of the rear of the vehicle are obtained according to the size of the front image and the size of the rear image respectively; feature value extraction and matching are performed on the front image and / or the rear image and the body image, and third width-height data of the body of the vehicle is obtained according to the first proportionality coefficient and the size of the body image; obtaining vehicle model data; the vehicle model data comprises vehicle model information and axle information; wherein the vehicle model information is obtained by identifying the front image through a first identification model; and the axle information is obtained by identifying the body image through a second identification model; integrating the width-height-length data and the vehicle model data to output vehicle model recognition data; the step of obtaining the first width-height data or the second width-height data comprises: performing license plate image recognition on a first image through a third identification model to obtain license plate position data and license plate type data; the first image is the front image or the rear image; adjusting the width-height ratio of the license plate image to be the same as the actual physical width-height ratio of the corresponding license plate type through affine transformation, and synchronously adjusting the width-height ratio of the first image based on the same adjustment proportion to obtain a first adjusted image; obtaining a first proportionality coefficient between the size of the license plate image and the physical size of the license plate; obtaining the size of the first adjusted image, and obtaining the first width-height data or the second width-height data based on the first proportionality coefficient; the step of obtaining the third width-height data comprises:

2. The vehicle type identification method according to claim 1, characterized by, obtaining feature matching points of the first adjusted image and the body image, and then obtaining a corresponding second proportionality coefficient; performing normalization processing on the body image according to the second proportionality coefficient to obtain a first normalized image; obtaining the size of the first normalized image, and obtaining the third width-height data based on the first proportionality coefficient. The step of obtaining the proportionality coefficient comprises: obtaining the character image size of each character in the license plate image; 3. The vehicle type identification method according to claim 1, characterized by, obtaining the first proportionality coefficient according to the physical size of the characters of the license plate. The first adjusted image comprises a front adjusted image and a rear adjusted image; the step of obtaining the third width-height data comprises: obtaining feature matching points of the front adjusted image and the body image, and then obtaining a corresponding third proportionality coefficient; performing normalization processing on the body image according to the third proportionality coefficient to obtain a second normalized image; obtaining feature matching points of the rear adjusted image and the body image, and then obtaining a corresponding fourth proportionality coefficient; performing normalization processing on the body image according to the fourth proportionality coefficient to obtain a third normalized image; obtaining the size of the second normalized image, and obtaining first body width-height data based on the first proportionality coefficient; obtaining the size of the third normalized image, and obtaining second body width-height data based on the first proportionality coefficient; The third width-height data is obtained by performing a weighted calculation based on the first and second vehicle body width-height data.

4. The vehicle type identification method according to claim 1, characterized by, The axle information includes the number of axles and the type of axles. The vehicle type identification data further includes the vehicle head image, the vehicle body image, the vehicle tail image, and license plate type data.

5. The vehicle type identification method according to claim 1, characterized by, The vehicle head image or the vehicle tail image acquisition step includes: A second image is acquired, and a fourth identification model is used to identify the vehicle head or the vehicle tail from the second image to obtain rough rectangular frame position information; the second image is an image having a vehicle head feature or a vehicle tail feature; Image edge detection is performed on the rough rectangular frame to obtain a vehicle head image or a vehicle tail image.

6. The vehicle type identification method according to claim 1, characterized by The vehicle body image acquisition step includes: Multiple frames of third images are continuously generated; the third images are images having a vehicle body feature; A feature vector of each frame of the third images is calculated, and a matching similarity with a previous frame of the third images is calculated respectively, and then a corresponding offset is obtained; The multiple frames of the third images are translated and transformed according to the corresponding offset to splice the vehicle body image.

7. A vehicle type identification device characterized by comprising: It includes: An acquisition module is configured to acquire a vehicle head image, a vehicle body image, and a vehicle tail image of a vehicle; A processing module is configured to acquire width-height-length data of the vehicle; The width-height-length data includes first width-height data, second width-height data, and third width-height data; wherein license plate detection is performed on the vehicle head image and the vehicle tail image to obtain a first proportionality coefficient between the size of the license plate image and the physical size of the license plate, and then the first width-height data of the vehicle head and the second width-height data of the vehicle tail are obtained according to the size of the vehicle head image and the size of the vehicle tail image respectively; feature value extraction and matching are performed on the vehicle head image and / or the vehicle tail image and the vehicle body image, and the third width-height data of the vehicle body is obtained according to the first proportionality coefficient and the size of the vehicle body image; vehicle type data is acquired; the vehicle type data includes vehicle type information and axle information; wherein the vehicle type information is obtained by identifying the vehicle head image through a first identification model; the axle information is obtained by identifying the vehicle body image through a second identification model; the width-height-length data and the vehicle type data are integrated to output vehicle type identification data; The first width-height data or the second width-height data acquisition step includes: license plate image identification is performed on a first image through a third identification model to obtain license plate position data and license plate type data; the first image is the vehicle head image or the vehicle tail image; the width-height ratio of the license plate image is adjusted to be the same as the actual physical width-height ratio of the corresponding license plate type through affine transformation, and the width-height ratio of the first image is adjusted synchronously based on the same adjustment proportion to obtain a first adjusted image; a first proportionality coefficient between the size of the license plate image and the physical size of the license plate is obtained; the size of the first adjusted image is obtained, and the first width-height data or the second width-height data is obtained based on the first proportionality coefficient; The third width-height data acquisition step is: acquiring feature matching points of the first adjusted image and the vehicle body image, and then obtaining a corresponding second scale factor; performing normalization processing on the vehicle body image according to the second scale factor to obtain a first normalized image; acquiring an image size of the first normalized image, and obtaining third width-height data based on the first scale factor.

8. A vehicle type identification system characterized by comprising: Comprise: A multi-camera has multiple cameras with different orientations; A vehicle type recognition device is connected with the multi-camera, configured to receive images transmitted by multiple cameras, and then obtain a vehicle head image, a vehicle body image and a vehicle tail image; and acquire width-height-length data of the vehicle; the width-height-length data includes first width-height data, second width-height data and third width-height data; wherein, license plate detection is performed on the vehicle head image and the vehicle tail image to obtain a first scale factor between a license plate image size and a license plate physical size, and then the first width-height data of the vehicle head and the second width-height data of the vehicle tail are obtained according to the vehicle head image size and the vehicle tail image size respectively; feature value extraction and matching are performed on the vehicle head image and / or the vehicle tail image and the vehicle body image, and the third width-height data of the vehicle body is obtained according to the first scale factor and the vehicle body image size; vehicle type data is acquired; the vehicle type data includes vehicle type information and axle information; wherein, the vehicle type information is obtained by recognizing the vehicle head image through a first recognition model; the axle information is obtained by recognizing the vehicle body image through a second recognition model; the width-height-length data and the vehicle type data are integrated to output vehicle type recognition data; The first width-height data or the second width-height data acquisition step includes: performing license plate image recognition on the first image through a third recognition model to obtain license plate position data and license plate type data; the first image is the vehicle head image or the vehicle tail image; the width-height ratio of the license plate image is adjusted to be the same as the actual physical width-height ratio of the corresponding license plate type through affine transformation, and the width-height ratio of the first image is adjusted synchronously based on the same adjustment scale to obtain a first adjusted image; a first scale factor between an image size of the license plate image and a physical size of the license plate is acquired; an image size of the first adjusted image is acquired, and the first width-height data or the second width-height data is obtained based on the first scale factor; The third width-height data acquisition step is: acquiring feature matching points of the first adjusted image and the vehicle body image, and then obtaining a corresponding second scale factor; performing normalization processing on the vehicle body image according to the second scale factor to obtain a first normalized image; acquiring an image size of the first normalized image, and obtaining third width-height data based on the first scale factor.

Images