Vehicle image stitching method, device, computer equipment and storage medium

Abstract

The present application relates to a vehicle image stitching method, device, computer equipment, storage medium and computer program product. The method comprises: obtaining vehicle images obtained by taking target vehicles by cameras at different times; extracting image features in each vehicle image to obtain corresponding vehicle features; when it is determined based on each vehicle feature that there are inconsistent features in the overlapping area images between each vehicle image, adjusting the inconsistent features in each vehicle image to obtain a vehicle adjustment image; according to the distance between the camera and the target vehicle, cutting out image blocks of fixed size from the vehicle adjustment image respectively; stitching the image blocks corresponding to each vehicle image to obtain a vehicle stitching image. The use of this method can ensure that the pixels of the stitched image are uniform and restored completely without deformation, so that the vehicle stitching image has a high stitching accuracy.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a vehicle image stitching method, device, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of transportation economy, the behavior of highway drivers evading tolls by taking advantage of the inconsistency between the vehicle model / axle number in the passage medium and the actual vehicle model / axle number is becoming increasingly rampant.

[0003] In traditional technology, multiple cameras are calibrated and the images taken by multiple cameras are stitched together to obtain a seamless wide-angle image. Since the characteristics of a moving vehicle are dynamic variables, the vehicle may not travel in a straight line when driving along the lane, resulting in angle errors and more or less imaging differences between multiple cameras. This method has the defect of low stitching accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product that can achieve high-precision vehicle image recognition in order to address the above technical issues.

[0005] In a first aspect, the present application provides a method for stitching vehicle images, the method comprising:

[0006] Obtain vehicle images of the target vehicle captured by the camera at different times;

[0007] Extracting image features from each of the vehicle images to obtain corresponding vehicle features;

[0008] When it is determined based on the vehicle features that there are inconsistent features in the images of the overlapping regions between the vehicle images, the inconsistent features in the vehicle images are adjusted to obtain a vehicle adjusted image;

[0009] According to the distance between the camera and the target vehicle, image blocks of fixed sizes are respectively cut out from the vehicle image where the vehicle adjustment image is located;

[0010] The image blocks corresponding to the vehicle images are spliced ​​together to obtain a vehicle spliced ​​image.

[0011] In one embodiment, the step of acquiring vehicle images of the target vehicle captured by a camera at different times includes:

[0012] When the fisheye camera is located at one side of the lane where the target vehicle is traveling, obtaining vehicle images of the target vehicle captured by the fisheye camera at different times;

[0013] The extracting image features from each of the vehicle images to obtain corresponding vehicle features includes:

[0014] Correcting each of the vehicle images according to the image correction table to obtain each corrected vehicle image;

[0015] Image feature extraction is performed in each of the corrected vehicle images to obtain corresponding vehicle features.

[0016] In one embodiment, extracting image features from each of the corrected vehicle images to obtain corresponding vehicle features includes:

[0017] Normalizing each of the corrected vehicle images to obtain each normalized corrected vehicle image;

[0018] Image feature extraction is performed on each of the normalized and rectified vehicle images to obtain the corresponding vehicle features.

[0019] In one embodiment, the step of stitching the image blocks corresponding to the vehicle images to obtain a vehicle stitched image includes:

[0020] Calculating displacement information of feature points in each of the vehicle images;

[0021] Determining the stitching position information of each of the image blocks based on the length information of each of the image blocks, the feature point displacement information, and the estimated length of the vehicle stitching image;

[0022] The image blocks are spliced ​​based on the splicing position information to obtain a vehicle splicing image.

[0023] In one embodiment, the vehicle stitching image is obtained by stitching the vehicle images acquired at the corresponding moment in real time each time; stitching the image blocks based on the stitching position information to obtain the vehicle stitching image includes:

[0024] When stitching for the first time, stitching the image blocks corresponding to the vehicle images of the first two frames of the target vehicle based on the stitching position information, and performing pixel averaging on the overlapping area images;

[0025] When it is not the first splicing, the image blocks corresponding to the vehicle images of the subsequent frames of the target vehicle are superimposed on the intermediate spliced ​​image obtained by the previous splicing according to the splicing position information, and the overlapping area images are pixel-averaged to obtain the vehicle splicing image.

[0026] In one embodiment, the vehicle image of the target vehicle includes a front image, a middle image and a rear image of the vehicle;

[0027] When stitching for the first time, stitching the image blocks corresponding to the vehicle images of the first two frames of the target vehicle based on the stitching position information includes:

[0028] When stitching for the first time, stitching the image blocks corresponding to the front image of the target vehicle based on the stitching position information;

[0029] When it is not the first time to stitch, according to the stitching position information, superimposing the image block corresponding to the vehicle image of the subsequent frame of the target vehicle onto the intermediate stitching image obtained by the previous stitching, including:

[0030] When it is not the first splicing and there are more than two front images, the image blocks corresponding to the front image, the middle image and the rear image of the target vehicle are sequentially superimposed on the intermediate splicing image obtained in the previous splicing according to the splicing position information.

[0031] In one embodiment, the adjusting the inconsistent features in each of the vehicle images to obtain the vehicle adjusted image includes:

[0032] Obtain vehicle bottom information corresponding to the vehicle feature;

[0033] Acquire a correction coefficient based on the vehicle bottom information;

[0034] The inconsistent features are adjusted according to the correction coefficient to obtain a vehicle adjustment image.

[0035] In one embodiment, the camera is located on one side of the lane where the target vehicle is traveling, and the method further includes:

[0036] Extracting features from the vehicle stitching image to obtain vehicle side features;

[0037] Based on the vehicle side features, at least one of the vehicle model information and the axle number information of the vehicle is identified.

[0038] In a second aspect, the present application further provides a vehicle image stitching device, the device comprising:

[0039] The vehicle image acquisition module is used to acquire vehicle images obtained by the camera taking pictures of the target vehicle at different times;

[0040] A vehicle feature extraction module, used to extract image features from each of the vehicle images to obtain corresponding vehicle features;

[0041] A vehicle feature adjustment module, configured to adjust the inconsistent features in each of the vehicle images to obtain a vehicle adjusted image when it is determined based on each of the vehicle features that there are inconsistent features in the overlapping area images between the vehicle images;

[0042] A vehicle image segmentation module, used for segmenting image blocks of fixed sizes from the vehicle image where the vehicle adjustment image is located, according to the distance between the camera and the target vehicle;

[0043] The image stitching module is used to stitch the image blocks corresponding to the vehicle images to obtain a vehicle stitching image.

[0044] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of splicing vehicle images in any of the above embodiments are implemented.

[0045] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of splicing vehicle images in any of the above embodiments are implemented.

[0046] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of splicing the vehicle images in any of the above embodiments are implemented.

[0047] The above-mentioned vehicle image stitching method, device, computer equipment, storage medium and computer program product extract image features from each of the vehicle images to obtain corresponding vehicle features. When it is determined based on the vehicle features that there are inconsistent features in the overlapping area images between the vehicle images, the inconsistent features are adjusted to obtain a vehicle adjusted image, which can ensure that the pixels of the stitched image are uniform and restored completely without deformation. According to the distance between the camera and the target vehicle, fixed-size image blocks are cut out from the vehicle adjusted image, and image blocks with different correction requirements can be cut out. The image blocks corresponding to each of the vehicle images are stitched to obtain a vehicle stitched image, so that the vehicle stitched image has a higher stitching accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A diagram showing an application environment of a vehicle image stitching method in one embodiment;

[0049] Figure 2 is a schematic flow chart of a method for stitching vehicle images in one embodiment;

[0050] Figure 3 is a schematic flow chart of a method for stitching vehicle images in one embodiment;

[0051] Figure 4 A schematic diagram of a process of acquiring a vehicle image in one embodiment;

[0052] Figure 5 A schematic diagram of an application environment in a horizontal direction in an embodiment;

[0053] Figure 6 A schematic diagram of an application environment in a vertical direction in an embodiment;

[0054] Figure 7 is a structural block diagram of a vehicle image stitching device in one embodiment;

[0055] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] The vehicle image stitching method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 obtains vehicle images obtained by the camera shooting the target vehicle at different times; image features are extracted in each of the vehicle images to obtain corresponding vehicle features; when it is determined based on the vehicle features that there are inconsistent features in the overlapping area images between the vehicle images, the inconsistent features in each of the vehicle images are adjusted to obtain a vehicle adjustment image; according to the distance between the camera and the target vehicle, fixed-size image blocks are respectively cut out from the vehicle adjustment image; the image blocks corresponding to each of the vehicle images are spliced ​​to obtain a vehicle spliced ​​image.

[0058] The terminal 102 may be, but is not limited to, various panoramic stitching devices, panoramic stitching integrated machines, personal computers, laptops, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0059] In one embodiment, Figure 2 As shown, a vehicle image stitching method is provided, and the method is applied to Figure 1The terminal 102 in the example is used as an example to illustrate, and the following steps are included:

[0060] Step 202, obtaining vehicle images of the target vehicle captured by a camera at different times.

[0061] In one embodiment, the camera shoots the target vehicle at a preset shutter speed, and sequentially acquires vehicle images at different shooting times, and the sequentially acquired vehicle images are used to characterize the vehicle information of the target vehicle at different positions, so as to obtain a high-precision vehicle mosaic. For example: the front-end sensor captures the side image of the vehicle at a speed of 50 frames per second, and after being able to transmit the side image of the vehicle to the intelligent processing unit, the intelligent processing unit extracts features and mosaics the side image of the vehicle.

[0062] Step 204 , extracting image features from each vehicle image to obtain corresponding vehicle features.

[0063] In one embodiment, the process of extracting vehicle features from each vehicle image may be performed based on SIFT feature point extraction rules or SURF feature point extraction rules to obtain feature points in the vehicle image. Feature blocks in each vehicle image may also be determined based on a preset range of regions of each feature point in the vehicle image.

[0064] In one embodiment, after obtaining the vehicle features, it can be determined whether the corresponding vehicle images match based on the fast approximate nearest neighbor rule. When adjacent vehicle images are completely consistent, one of them can be retained to reduce the corresponding storage space. When there are overlapping area images of adjacent vehicle images, and the features in the overlapping area images are consistent, fixed-size image blocks are cut out from the vehicle adjustment image, and the image blocks in the adjacent vehicle images are spliced ​​to obtain a vehicle spliced ​​image.

[0065] Step 206 : when it is determined based on the features of each vehicle that there are inconsistent features in the images of the overlapping regions between the vehicle images, the inconsistent features in the vehicle images are adjusted to obtain a vehicle adjusted image.

[0066] Specifically, when inconsistent feature points or feature blocks exist in the overlapping area images of the vehicle images determined based on the features of each vehicle, the corresponding feature points or feature blocks will be adjusted to make the pixel values ​​of the spliced ​​image more uniform, which is conducive to restoring a complete and non-deformed vehicle spliced ​​image.

[0067] In one embodiment, inconsistent features in each of the vehicle images are adjusted to obtain a vehicle adjusted image, including: obtaining vehicle bottom information corresponding to the vehicle features; obtaining a correction coefficient based on the vehicle bottom information; and adjusting the inconsistent features according to the correction coefficient to obtain a vehicle adjusted image.

[0068] Optionally, different correction coefficients can be used to characterize the deformation degree or deformation type of corresponding vehicle characteristics, so as to make adjustments according to the data corresponding to the corresponding deformation degree or deformation type; different correction coefficients can be used to characterize the correction method of corresponding vehicle characteristics, so as to make adjustments using the corresponding correction method and corresponding correction data.

[0069] After obtaining the vehicle image, the corresponding lane is displayed in a pre-set area in the imaging screen, and the bottom information of the vehicle on the lane is collected to obtain the bottom information of the vehicle. The bottom information of the vehicle corresponds to a pre-set correction coefficient. Based on the corresponding correction coefficient, the features of the vehicle features that failed to match are adjusted to make the pixels of the spliced ​​image uniform and restored completely without deformation.

[0070] Step 208 , according to the distance between the camera and the target vehicle, image blocks of fixed sizes are cut out from the vehicle adjusted image.

[0071] During the driving process of the target vehicle, the distance between the camera and the target vehicle changes accordingly. The camera is used to capture vehicle images of the target vehicle at different positions, and the vehicle information in each vehicle image is carried in the corresponding multiple image blocks. The camera and the target vehicle are at different distances, and image blocks with different correction requirements can be segmented. When the distance between a certain image block in the vehicle image and the camera is at the corresponding threshold, the image block has less distortion in the original vehicle image, so the adjustment is smaller, which is conducive to improving the accuracy of stitching, and the total amount of data occupied by the image block used for stitching is less, which is conducive to improving the efficiency of stitching. The specific length, width and other information of the image block can be set based on the distance between the reserved area in the vehicle image and the camera. For example: after segmenting the vehicle image, the image block with an aspect ratio of 1:3 in the middle of the vehicle image is retained, so that the distance between the image block and the camera is close at all locations.

[0072] Step 210 , stitching the image blocks corresponding to each vehicle image to obtain a vehicle stitching image.

[0073] In one embodiment, the position information of each image block is obtained respectively, and the corresponding image blocks are spliced ​​according to the position information to obtain the corresponding vehicle spliced ​​image.

[0074] In one embodiment, image blocks corresponding to each vehicle image are stitched together to obtain a vehicle stitched image, including: calculating feature point displacement information in each vehicle image; determining stitching position information of each image block based on length information of each image block, feature point displacement information, and an estimated length of the vehicle stitched image; and stitching the image blocks based on the stitching position information to obtain a vehicle stitched image.

[0075] Each vehicle image includes multiple feature points, and each feature point generates corresponding feature point displacement information as the vehicle moves. Specifically, for the feature points of adjacent vehicle images, SIFT feature point detection is used respectively, and after the feature points are matched using the fast approximate nearest neighbor algorithm, the average displacement {dmx, dmy} of the matching points is calculated; where dmx represents the displacement vector on the X-axis of the corresponding coordinate system, and dmy represents the displacement vector on the Y-axis of the corresponding coordinate system.

[0076] After obtaining the feature point displacement information, calculations are performed based on the length information of the image block and the feature point displacement information to obtain an intermediate result for splicing the image blocks. The length information of the image block is the length in the preset direction for splicing, and the intermediate result for splicing the image blocks is the difference between the length information of each image block and the corresponding feature point displacement information.

[0077] After obtaining the intermediate result of each image block for stitching, the difference between the expected length of the vehicle stitching image and the intermediate result is calculated to obtain the stitching position information of each image block, so as to stitch the corresponding image blocks based on the stitching position information to obtain the vehicle stitching image. It can be understood that when the vehicle is not traveling in a straight line, causing an angle error, stitching can also be performed using the corresponding displacement information.

[0078] For example: when the length of the cut-out image block in the preset direction for stitching is w, the estimated length of the current vehicle stitching image in the preset direction is s, the mean displacement of the feature points in the corresponding coordinate system X-axis is dmx, and the mean displacement of the feature points in the corresponding coordinate system Y-axis is dmy, then the stitching starting point ssx in the vehicle stitching image is s-(w-dmx), and the current frame is superimposed on {ssx, dmy} in the vehicle stitching image to achieve image stitching.

[0079] In one embodiment, the vehicle stitching image is obtained by stitching the vehicle images acquired at the corresponding moment in real time each time. Starting from the acquisition of the second frame of vehicle image, each acquired vehicle image is used to stitch into the intermediate stitching image obtained by stitching the first frame of image or the previous frame of image.

[0080] The image blocks are stitched together based on the stitching position information to obtain a vehicle stitching image, including:

[0081] When stitching for the first time, the image blocks corresponding to the vehicle images of the first two frames of the target vehicle are stitched based on the stitching position information, and the pixels of the overlapping area images are averaged.

[0082] Specifically, during the first splicing, the image blocks corresponding to the first two frames of the target vehicle's vehicle images may be spliced, and the overlapping areas may be superimposed; and during the superposition process, the overlapping areas may be pixel averaged to make the superimposed and fused pixels of the first two frames of the vehicle images more uniform and the restoration accuracy higher. The first two frames of the vehicle images spliced ​​for the first time may be one or more of the vehicle head, the vehicle middle, and the vehicle tail.

[0083] When it is not the first stitching, the image blocks corresponding to the vehicle images of the subsequent frames of the target vehicle are superimposed on the intermediate stitching image obtained by the previous stitching according to the stitching position information, and the overlapping area images are pixel-averaged to obtain the vehicle stitching image.

[0084] Specifically, the vehicle images of the first two frames that are not stitched for the first time may be one or more of the vehicle head, the vehicle middle and the vehicle tail. When stitching for the first time, the image blocks corresponding to the vehicle images of the subsequent frames of the target vehicle may be superimposed on the intermediate stitched image obtained by the previous stitching; and in the process of superimposing, the pixels of the overlapping area are averaged to make the superimposed and fused pixels of the vehicle images of each frame more uniform and the restoration accuracy higher.

[0085] In one embodiment, the audit method based on vehicle front analysis has certain limitations, and the obtained detailed features are few, and the vehicle side restoration technology is used to obtain a panoramic image of the vehicle side, so as to obtain more detailed features from the vehicle stitching image. Specifically, the vehicle image of the target vehicle includes a vehicle front image, a vehicle middle image, and a vehicle rear image.

[0086] When stitching for the first time, the image blocks corresponding to the vehicle images of the first two frames of the target vehicle are stitched based on the stitching position information, including: when stitching for the first time, the image blocks corresponding to the front image of the target vehicle are stitched based on the stitching position information.

[0087] When it is not the first time stitching, the image blocks corresponding to the vehicle images of the subsequent frames of the target vehicle are superimposed on the intermediate stitching image obtained by the previous stitching according to the stitching position information, including: when it is not the first time stitching and there are more than two front images, the image blocks corresponding to the front image, the middle image and the rear image of the target vehicle are superimposed on the intermediate stitching image obtained by the previous stitching in sequence according to the stitching position information.

[0088] Specifically, Figure 3As shown in the figure, when the target vehicle enters the camera shooting range, the vehicle image is collected, and then the corresponding image blocks are cropped from the vehicle image and rectified. After the correction, after the front of the vehicle is identified, the image blocks of the front image, the middle image and the rear image of the vehicle side are extracted and spliced. During the splicing process, the vehicle axle is identified according to the extracted vehicle features until the rear of the vehicle is identified, that is, the splicing of the side image of the vehicle is completed.

[0089] Optionally, when the speed at which the camera captures the target vehicle is less than the sum of the speeds of feature extraction and stitching, feature extraction and stitching can be performed in real time. For example, when the front-end sensor captures the side image of the vehicle at a speed of 50 frames per second, and the intelligent processing unit processes more than 50 frames of vehicle side images per second, features can be extracted in real time and the vehicle images can be stitched together to obtain a real-time vehicle stitching image. Thus, the features of the vehicle images captured in real time are extracted in real time and stitched together into a complete vehicle side view; the vehicle features can also be accurately captured and identified, and the vehicle side image can be quickly stitched together.

[0090] Furthermore, in the process of stitching, thread parallel technology is used, and there is no synchronous waiting relationship between stitching and vehicle inspection. Vehicle inspection is performed while stitching, which is conducive to image stitching for high-speed vehicles on highways.

[0091] In the above-mentioned vehicle image stitching method, image feature extraction is performed in each vehicle image to obtain the corresponding vehicle feature. When it is determined based on each vehicle feature that there are inconsistent features in the overlapping area image between each vehicle image, the inconsistent features are adjusted to obtain a vehicle adjustment image, which can ensure that the pixels of the stitched image are uniform and restored completely without deformation. According to the distance between the camera and the target vehicle, fixed-size image blocks are cut out from the vehicle adjustment image respectively, and image blocks with different correction requirements can be cut out, so as to facilitate the stitching of the image blocks corresponding to each vehicle image to obtain a vehicle stitching image. When the vehicle is not traveling in a straight line, causing an angle error, the stitching accuracy of the vehicle stitching image can be improved by adjusting the corresponding features and stitching the corresponding image blocks. In addition, this embodiment can use a camera to obtain vehicle images to avoid imaging differences.

[0092] In one embodiment, a fisheye camera is used to expand the range of the vehicle image captured, and a vehicle image with more vehicle details is obtained. However, the vehicle image captured by the fisheye camera is greatly distorted, and it is difficult to obtain a high-precision vehicle image.

[0093] Based on this, Figure 4 As shown, the vehicle images obtained by the camera shooting the target vehicle at different times include:

[0094] Step 402: when the fisheye camera is located at one side of the lane where the target vehicle is traveling, vehicle images of the target vehicle captured by the fisheye camera at different times are obtained.

[0095] When the fisheye camera is located at one side of the driving lane, it can shoot from the side of the target vehicle to obtain at least part of the roof information and the bottom information of the target vehicle.

[0096] Correspondingly, image feature extraction is performed in each vehicle image to obtain corresponding vehicle features, including:

[0097] Step 404, correcting each vehicle image according to the image correction table to obtain each corrected vehicle image.

[0098] The image correction table is generated by the polynomial coordinate transformation algorithm before actual application. The generation process includes: in the on-site simulation environment built in the laboratory, the polynomial coordinate transformation algorithm is used to adjust the algorithm control points according to the location of the vehicle until the vehicle image is distortion-free, and the image correction table is obtained.

[0099] In actual application, the image correction table is pre-stored in the device, and the collected vehicle image is directly corrected to obtain the corresponding corrected vehicle image.

[0100] Step 406 , extracting image features from each corrected vehicle image to obtain corresponding vehicle features.

[0101] In the corrected vehicle image, image feature extraction is performed to obtain high-quality vehicle features. Dynamic correction can be performed during the stitching process to overcome the trapezoidal distortion problem caused by the different distances between the vehicle side and the camera due to the vehicle not driving in a straight line, and image stitching is achieved based on the dynamically corrected image. Among them, the dynamic correction technology uses block matching and calculation technology for the bottom area of ​​the vehicle, and selects different correction coefficients according to the position of different bottom areas in the image to achieve dynamic correction.

[0102] In one embodiment, performing image feature extraction on each corrected vehicle image to obtain corresponding vehicle features includes: normalizing each corrected vehicle image to obtain each normalized corrected vehicle image. Performing image feature extraction on each normalized corrected vehicle image to obtain each corresponding vehicle feature.

[0103] Normalizing the corrected vehicle image is to perform a series of standard processing transformations on the image to transform it into a preset standard corrected vehicle image, which belongs to the image preprocessing step. Therefore, when the vehicle images used are from the same side of the vehicle, and the distance between the side of the vehicle and the distance from the fisheye lens is not equal everywhere, the vehicle image can also be normalized to obtain the same features between the two frames of vehicle images, ensure that the pixels of the spliced ​​image are uniform, and restore the image completely without deformation.

[0104] In one embodiment, trucks are charged based on vehicle type / axle number. When the vehicle type / axle number in the driver's pass medium is inconsistent with the actual vehicle type / axle number, it is difficult to determine the corresponding fee through audit means mainly based on vehicle head analysis. Therefore, the vehicle side restoration technology is used to obtain a panoramic picture of the vehicle side to obtain more detailed features, so as to increase the audit means to solve the problem of inaccurate charging caused by the inconsistency between the vehicle type / axle number in the pass medium and the actual vehicle type / axle number.

[0105] Specifically, the camera is located on one side of the lane where the target vehicle is traveling, and the method further includes: extracting features from the vehicle splicing image to obtain vehicle side features; based on the vehicle side features, identifying at least one of the vehicle model information and axle number information. Thus, the corresponding vehicle model information and axle number information can be obtained through the vehicle splicing image, so as to determine whether the vehicle model and axle number have changed, so as to facilitate accurate charging.

[0106] In one embodiment, the camera installation process of the present application in a specific application scenario is described in detail. Figure 5 and Figure 6 In the application scenario shown, the stitching area supported by the camera is one lane, and a camera is set up. The camera is set in an all-in-one machine, which is perpendicular to the lane and 80 to 100 cm away from the lane. The height of the camera is 150 cm from the ground of the lane, the left and right field of view is 30 degrees, and the upper and lower field of view is 160 degrees. Correspondingly, in the imaging picture, the upper edge of the lane is between 50% and 60% of the height of the imaging picture, and the bottom edge of the lane is between 90% and 100% of the height of the imaging picture, so as to facilitate stitching and obtain the side information of the vehicle.

[0107] Among them, when the camera lens in the integrated machine is a fisheye camera, its focal length is 1mm to 2mm. The fisheye camera can capture a large range of vehicle images, not only the side image of the vehicle, but also the image of the top and bottom of the vehicle. Although the vehicle image captured by the fisheye camera is deformed, it can be corrected based on the pre-set image correction.

[0108] Optionally, a fill light is provided under the integrated machine; the fill light determines whether to fill light according to the brightness of the external environment. When a vehicle appears within the field of view of the integrated machine, the integrated machine automatically adjusts the camera parameters according to the brightness of the image, so that the side image of the vehicle taken in a darker environment can meet the requirements of stitching and recognition.

[0109] In one embodiment, the camera is located in the no-passing lane or toll station lane of a highway toll station. The center line of the camera's field of view is perpendicular to the direction of travel of vehicles in the lane. After the equipment is installed, a 1-meter ruler can be used to calibrate the scene. The ruler is placed at any three positions 20 to 100 cm away from the lane line, and the number of pixels corresponding to a length of 1 meter is marked in the system to calculate the correspondence between the actual distance of the target vehicle and the pixel value of the vehicle image. Specifically, after the camera is installed, it is necessary to use a 1-meter ruler to calibrate three different positions of the scene, measure the pixel value occupied by N pixels at the Y coordinate, and obtain and store the calibration data in a file of a computer system.

[0110] Therefore, in this embodiment, the center line of the integrated machine's field of view is set perpendicular to the lane, and it is installed very close to the lane, and the real-time vehicle image is extracted in real time and spliced ​​into a complete vehicle side view. It can not only accurately capture and identify vehicle features and quickly splice the vehicle side image, but also has the advantages of simple installation and convenient deployment.

[0111] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0112] Based on the same inventive concept, the embodiment of the present application also provides a vehicle image stitching device for implementing the vehicle image stitching method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiment of the vehicle image stitching device provided below can refer to the limitations of the vehicle image stitching method above, and will not be repeated here.

[0113] In one embodiment, Figure 7As shown, a vehicle image stitching device is provided, comprising: a vehicle image acquisition module 702, a vehicle feature extraction module 704, a vehicle feature adjustment module 706, a vehicle image segmentation module 708 and an image stitching module 710, wherein:

[0114] The vehicle image acquisition module 702 is used to acquire vehicle images obtained by the camera taking pictures of the target vehicle at different times;

[0115] The vehicle feature extraction module 704 is used to extract image features from each of the vehicle images to obtain corresponding vehicle features;

[0116] A vehicle feature adjustment module 706, configured to adjust the inconsistent features in the vehicle images to obtain a vehicle adjusted image when it is determined based on the vehicle features that there are inconsistent features in the overlapping area images between the vehicle images.

[0117] A vehicle image segmentation module 708, configured to segment image blocks of fixed sizes from the vehicle adjusted image according to the distance between the camera and the target vehicle;

[0118] The image stitching module 710 is used to stitch the image blocks corresponding to the vehicle images to obtain a vehicle stitching image.

[0119] In one embodiment, the vehicle image acquisition module 702 includes:

[0120] A fisheye shooting unit, used for obtaining vehicle images of the target vehicle captured by the fisheye camera at different times when the fisheye camera is located at one side of the lane where the target vehicle is traveling;

[0121] The vehicle feature extraction module 704 includes:

[0122] A correction table correction unit, used to correct each of the vehicle images according to the image correction table to obtain each corrected vehicle image;

[0123] The feature extraction unit is used to extract image features from each of the corrected vehicle images to obtain corresponding vehicle features.

[0124] In one embodiment, the feature extraction unit includes:

[0125] An image normalization subunit, used for normalizing each of the corrected vehicle images to obtain each normalized corrected vehicle image;

[0126] The image extraction subunit is used to extract image features from each of the normalized and rectified vehicle images to obtain the corresponding vehicle features.

[0127] In one embodiment, the image stitching module 710 includes:

[0128] A displacement information determination unit, used to calculate the displacement information of the feature points in each of the vehicle images;

[0129] A splicing position determining unit, configured to determine the splicing position information of each of the image blocks based on the length information of each of the image blocks, the feature point displacement information and the estimated length of the vehicle splicing image;

[0130] An image stitching unit is used to stitch the image blocks based on the stitching position information to obtain a vehicle stitching image.

[0131] In one embodiment, the vehicle stitching image is obtained by stitching the vehicle images acquired at the corresponding moment in real time each time; the image stitching unit includes:

[0132] A first splicing unit is used for splicing the image blocks corresponding to the vehicle images of the first two frames of the target vehicle based on the splicing position information during the first splicing, and performing pixel averaging on the overlapping area images;

[0133] The second stitching unit is used to, when it is not the first stitching, superimpose the image blocks corresponding to the vehicle images of the subsequent frames of the target vehicle on the intermediate stitching image obtained by the previous stitching according to the stitching position information, and perform pixel averaging on the overlapping area images to obtain the vehicle stitching image.

[0134] In one embodiment, the vehicle image of the target vehicle includes a front image, a middle image and a rear image of the vehicle;

[0135] The first splicing unit comprises:

[0136] A vehicle head splicing subunit, used for splicing the image blocks corresponding to the vehicle head image of the target vehicle based on the splicing position information during the first splicing;

[0137] The second splicing unit comprises:

[0138] The real-time stitching subunit is used to, when it is not the first stitching and there are more than two front images, sequentially superimpose the image blocks corresponding to the front image, the middle image and the rear image of the target vehicle on the intermediate stitching image obtained by the previous stitching according to the stitching position information.

[0139] In one embodiment, the vehicle feature adjustment module 706 includes:

[0140] A bottom information acquisition unit, used to acquire vehicle bottom information corresponding to the vehicle feature;

[0141] A correction coefficient acquisition unit, used to acquire a correction coefficient based on the vehicle bottom information;

[0142] The feature adjustment unit is used to adjust the inconsistent features according to the correction coefficient to obtain the vehicle adjustment image.

[0143] In one embodiment, the camera is located on one side of the lane where the target vehicle is traveling, and the method further includes a vehicle identification module; the vehicle identification module includes:

[0144] A side feature extraction unit, used to extract features from the vehicle stitching image to obtain vehicle side features;

[0145] A vehicle information identification unit is used to identify at least one of the vehicle model information and the axle number information of the vehicle based on the vehicle side features.

[0146] Each module in the above-mentioned vehicle image stitching device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0147] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for splicing vehicle images is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0148] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0149] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0150] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0151] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0152] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0154] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0155] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A vehicle image stitching method, characterized in that: The method comprises: When the fisheye camera is located at one side of the lane where the target vehicle is traveling, obtaining vehicle images of the target vehicle captured by the fisheye camera at different times; Correcting each of the vehicle images according to the image correction table to obtain each corrected vehicle image; Extracting image features from each of the corrected vehicle images to obtain corresponding vehicle features; When it is determined based on the vehicle features that there are inconsistent features in the images of the overlapping regions between the vehicle images, the inconsistent features in the vehicle images are adjusted to obtain a vehicle adjusted image; According to the distance between the camera and the target vehicle, image blocks of fixed sizes are respectively cut out from the vehicle adjusted image; The image blocks corresponding to the vehicle images are spliced ​​together to obtain a vehicle spliced ​​image.

2. The method according to claim 1, characterized in that The extracting image features from each of the corrected vehicle images to obtain corresponding vehicle features includes: Normalizing each of the corrected vehicle images to obtain each normalized corrected vehicle image; Image feature extraction is performed on each of the normalized and rectified vehicle images to obtain the corresponding vehicle features.

3. The method according to claim 1, characterized in that: The method further comprises: The fisheye camera shoots the target vehicle at a preset shutter speed, and sequentially obtains vehicle images at different shooting times; The vehicle images acquired sequentially at different moments are respectively used to represent the vehicle information of the target vehicle at different positions.

4. The method according to claim 1, characterized in that: The step of stitching the image blocks corresponding to the vehicle images to obtain a vehicle stitched image includes: Calculating displacement information of feature points in each of the vehicle images; Determining the stitching position information of each of the image blocks based on the length information of each of the image blocks, the feature point displacement information, and the estimated length of the vehicle stitching image; The image blocks are spliced ​​based on the splicing position information to obtain a vehicle splicing image.

5. The method according to claim 4, characterized in that The vehicle stitching image is obtained by stitching the vehicle images acquired at the corresponding moment in real time each time; the image blocks are stitched based on the stitching position information to obtain the vehicle stitching image, including: When stitching for the first time, stitching the image blocks corresponding to the vehicle images of the first two frames of the target vehicle based on the stitching position information, and performing pixel averaging on the overlapping area images; When it is not the first splicing, the image blocks corresponding to the vehicle images of the subsequent frames of the target vehicle are superimposed on the intermediate spliced ​​image obtained by the previous splicing according to the splicing position information, and the overlapping area images are pixel-averaged to obtain the vehicle splicing image.

6. The method according to claim 5, characterized in that The vehicle image of the target vehicle includes a front image, a middle image and a rear image of the vehicle; When stitching for the first time, stitching the image blocks corresponding to the vehicle images of the first two frames of the target vehicle based on the stitching position information includes: When stitching for the first time, stitching the image blocks corresponding to the front image of the target vehicle based on the stitching position information; When it is not the first time to stitch, according to the stitching position information, superimposing the image block corresponding to the vehicle image of the subsequent frame of the target vehicle onto the intermediate stitching image obtained by the previous stitching, including: When it is not the first splicing and there are more than two front images, the image blocks corresponding to the front image, the middle image and the rear image of the target vehicle are sequentially superimposed on the intermediate splicing image obtained in the previous splicing according to the splicing position information.

7. The method according to claim 1, characterized in that The step of adjusting the inconsistent features in the vehicle images to obtain the vehicle adjusted image includes: Obtain vehicle bottom information corresponding to the vehicle feature; Acquire a correction coefficient based on the vehicle bottom information; The inconsistent features are adjusted according to the correction coefficients to obtain the vehicle adjusted image.

8. The method according to claim 1, characterized in that: The method further comprises: Extracting features from the vehicle stitching image to obtain vehicle side features; Based on the vehicle side features, at least one of the vehicle model information and the axle number information of the vehicle is identified.

9. A vehicle image splicing device, characterized in that: The device comprises: The vehicle image acquisition module is used to acquire vehicle images of the target vehicle taken by the fisheye camera at different times when the fisheye camera is located at one side of the lane where the target vehicle is traveling; correct each of the vehicle images according to the image correction table to obtain each corrected vehicle image; and extract image features from each of the corrected vehicle images to obtain corresponding vehicle features; A vehicle feature extraction module, used to extract image features from each of the vehicle images to obtain corresponding vehicle features; A vehicle feature adjustment module, configured to adjust the inconsistent features in each of the vehicle images to obtain a vehicle adjusted image when it is determined based on each of the vehicle features that there are inconsistent features in the overlapping area images between the vehicle images; A vehicle image segmentation module, used for segmenting image blocks of fixed sizes from the vehicle adjusted image according to the distance between the camera and the target vehicle; The image stitching module is used to stitch the image blocks corresponding to the vehicle images to obtain a vehicle stitching image.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images