Vehicle side image acquisition method and device, system and computer equipment

Abstract

The present application relates to a method for acquiring a vehicle side image and its device, system and computer equipment. The vehicle side image acquisition method comprises: sequentially acquiring multiple consecutive frames of images of a target vehicle; performing stitching processing and detection processing on each acquired frame image at the same time to respectively acquire a stitching image of the target vehicle and the front position and rear position of the target vehicle in the stitching image, and in the same time period, the stitching processing and detection processing are used to process the same frame image of the target vehicle; and acquiring the vehicle side image of the target vehicle according to the stitching image, the front position and the rear position. By performing stitching processing and detection processing on the acquired multiple consecutive frames of images of the target vehicle at the same time, the vehicle side image of the target vehicle can be quickly acquired, and by detecting the front position and the rear position of the target vehicle, the complete vehicle side image can be accurately restored.

Description

Technical Field

[0001] The present application relates to the field of vehicle detection technology, and in particular to a method for acquiring a vehicle side image and a device, system and computer equipment thereof. Background Art

[0002] With the development of intelligent vehicle management, the vehicle management and charging systems need to judge the specific information of the vehicle based on the vehicle side view and use it as a forensic record. The vehicle side view can be used for structured recognition to further extract the vehicle model, axle type, and hazardous chemical identification. The highway toll collection method has been changed from weight-based to vehicle model-based charging. The vehicle side view is one of the most effective evidence for toll auditing; highways and service areas need to control hazardous chemical vehicles. Identifying the side of the vehicle based on the vehicle side view can more accurately identify hazardous chemical vehicles; highway overload control requires accurate vehicle axle type information. The vehicle side view can be used to identify the vehicle axle type to obtain the corresponding vehicle weight limit standard. However, due to the imaging perspective of the highway lane scene, the full vehicle side information can only be seen clearly under the camera installation angle of the high-position gantry or pole in a nearly unobstructed situation. Therefore, how to obtain the high-position vehicle side image has become an urgent need for the current intelligent and information-based vehicle management. Summary of the invention

[0003] Based on this, it is necessary to provide a vehicle side image acquisition method and its device, system and storage medium that can quickly acquire accurate vehicle side images in order to address the above technical problems.

[0004] A method for acquiring a vehicle side image, the method comprising:

[0005] Sequentially acquire multiple frames of continuous images of the target vehicle;

[0006] The acquired frames of images are respectively subjected to stitching processing and detection processing at the same time, so as to respectively obtain a stitching image of the target vehicle and the front position and the rear position of the target vehicle in the stitching image. In the same period of time, the stitching processing and the detection processing are used to process the same frame of the target vehicle;

[0007] According to the mosaic image, the front position and the rear position of the vehicle, the side image of the target vehicle is obtained.

[0008] In one embodiment, stitching the acquired frames of images includes:

[0009] Dividing the k-1th frame image into a plurality of first block images, and dividing the kth frame image into a plurality of second block images, wherein the plurality of first block images correspond to the plurality of second block images one by one;

[0010] Performing mobile matching on each first block image and each corresponding second block image, each first block image includes at least one first feature point, and each second block image includes at least one second feature point;

[0011] Respectively obtain the feature point displacement between each first feature point and the corresponding second feature point;

[0012] Obtaining preset correction coefficients corresponding to each first block image according to the displacements of the multiple feature points;

[0013] Performing correction processing on each first block image according to a preset correction coefficient;

[0014] The k-th frame image after correction is spliced ​​with the current splicing image according to the displacement of multiple feature points.

[0015] In one embodiment, before performing movement matching on each first block image and each corresponding second block image, the method further includes:

[0016] Placing the bottom of the same calibration plate at preset heights corresponding to the first block images;

[0017] Generating experimental correction coefficients corresponding to the first block images respectively according to the actual coordinates of the multiple calibration points in the calibration plate and the imaging coordinates of the multiple calibration points in the image;

[0018] Obtaining preset correction coefficients corresponding to each first block image according to the displacements of the multiple feature points respectively, including:

[0019] Get experimental correction factor;

[0020] According to the displacements of the plurality of feature points and the experimental correction coefficients, the preset correction coefficients corresponding to the first block images are respectively obtained.

[0021] In one embodiment, stitching the k-th frame image after correction processing with the current stitching image according to the displacement of multiple feature points includes:

[0022] Obtaining a displacement average value between at least one first feature point included in each first block image in the k-1th frame image and at least one second feature point included in each second block image in the kth frame image;

[0023] determining at least one of the plurality of displacement average values ​​as a first target displacement value;

[0024] splicing the k-th frame image after the correction process with the current splicing image according to the average value of the first target displacement value;

[0025] According to the displacements of the multiple feature points and the experimental correction coefficients, the preset correction coefficients corresponding to the first block images are respectively obtained, including:

[0026] Determining at least one of the plurality of first block images as a target block image according to the first target displacement value;

[0027] The experimental correction coefficient corresponding to the target block image is obtained as the preset correction coefficient.

[0028] In one embodiment, before determining at least one of the plurality of displacement average values ​​as the first target displacement value, the method further includes:

[0029] Determine at least one of the total mean values ​​of the plurality of displacement average values ​​that is greater than the first multiple and less than the second multiple as the second target displacement value, wherein the total mean value is the average value of the plurality of displacement average values, and the value of the first multiple is less than the value of the second multiple;

[0030] Determining at least one of the plurality of displacement averages as a first target displacement value, comprising:

[0031] When the average value of the second target displacement value is greater than a preset threshold, a displacement average value excluding the second target displacement value among the plurality of displacement average values ​​is determined as the first target displacement value.

[0032] In one embodiment, before the acquired frames of images are stitched and detected simultaneously, the method further includes:

[0033] Each acquired frame image is cropped according to a preset aspect ratio factor, and the cropped image is used as a cropped image of the target vehicle.

[0034] In one embodiment, performing vehicle detection on each acquired frame image includes:

[0035] When the front of the target vehicle is detected in the image of the kth frame, a spliced ​​image of the first width of the target vehicle from the 1st frame to the kth frame is obtained;

[0036] Acquire the front position of the target vehicle in the spliced ​​image according to the spliced ​​image of the first width;

[0037] When the rear end of the target vehicle is detected in the image of the k+nth frame, a spliced ​​image of the target vehicle of the first frame to the k+nth frame with a second width is obtained;

[0038] The rear position of the target vehicle in the spliced ​​image is obtained according to the spliced ​​image of the second width.

[0039] A vehicle side image acquisition device, comprising:

[0040] An image acquisition module, used to sequentially acquire multiple frames of continuous images of the target vehicle;

[0041] A stitching module is used to perform stitching processing and detection processing on each frame of the acquired image at the same time, so as to obtain a stitching image of the target vehicle and the front position and the rear position of the target vehicle in the stitching image respectively. In the same period of time, the stitching processing and the detection processing are used to process the same frame of the target vehicle;

[0042] The vehicle side image acquisition module is used to acquire the vehicle side image of the target vehicle according to the splicing image, the vehicle front position and the vehicle rear position.

[0043] A vehicle side image acquisition system, comprising:

[0044] A vehicle side image acquisition device, the vehicle side image acquisition device is used to sequentially acquire multiple consecutive frames of images of a target vehicle; the acquired frames of images are respectively and simultaneously subjected to stitching processing and detection processing to respectively acquire a stitching image of the target vehicle and the front position and the rear position of the target vehicle in the stitching image; within the same time period, the stitching processing and the detection processing are used to process the same frame of the target vehicle; the vehicle side image of the target vehicle is acquired based on the stitching image, the front position and the rear position; and a camera, the camera is connected to the vehicle side image acquisition device, and the camera is used to capture the image of the target vehicle.

[0045] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0046] The above-mentioned method for acquiring the side image of the vehicle includes: sequentially acquiring a plurality of continuous frames of images of the target vehicle; performing stitching processing and detection processing on each of the acquired frames of images respectively to respectively acquire a stitching diagram of the target vehicle, and the front position and the rear position of the target vehicle in the stitching diagram, and in the same period of time, the stitching processing and the detection processing are used to process the same frame image of the target vehicle; and acquiring the side image of the target vehicle according to the stitching diagram, the front position and the rear position. The present invention realizes rapid acquisition of the side image of the target vehicle by performing stitching processing and detection processing on the acquired plurality of continuous frames of images of the target vehicle respectively and simultaneously, and accurately restores the complete side image of the vehicle by detecting the front position and the rear position of the target vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 This is a schematic diagram of a method for acquiring a vehicle side image in one embodiment;

[0049] Figure 2 A schematic diagram of a splicing process flow in an embodiment;

[0050] Figure 3 A schematic diagram of a sub-process of step S202 and step S206 in one embodiment;

[0051] Figure 4 is a schematic diagram of image segmentation in one embodiment;

[0052] Figure 5 A schematic diagram of a sub-process of step S304 and step S308 in one embodiment;

[0053] Figure 6 is a schematic diagram of a sub-flow chart of step S312 in one embodiment;

[0054] Figure 7 A schematic diagram of a sub-process of step S508 in one embodiment;

[0055] Figure 8 is a schematic diagram of a sub-flow of step S702 in an embodiment;

[0056] Fig. 9 The second flowchart of a method for acquiring a vehicle side image in one embodiment;

[0057] Fig.10 The third flowchart of a method for acquiring a vehicle side image in one embodiment;

[0058] Fig.11 is a structural block diagram of a vehicle side image acquisition device in one embodiment;

[0059] Fig.12 Schematic diagram of camera positions in one embodiment. DETAILED DESCRIPTION

[0060] In order to facilitate understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. Embodiments of the present application are provided in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0062] It can be understood that the terms "first", "second", etc. used in this application can be used to describe various elements in this article, but these elements are not limited by these terms. These terms are only used to distinguish the first block image from another block image.

[0063] It should be noted that when an element is considered to be "connected" to another element, it can be directly connected to the other element, or connected to the other element through an intermediate element. In addition, the "connection" in the following embodiments should be understood as "electrical connection", "communication connection", etc. if there is transmission of electrical signals or data between the connected objects.

[0064] When used herein, the singular forms "a", "an", and "said / the" may also include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include / comprise" or "have" etc. specify the presence of stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof.

[0065] In the description of this specification, the description with reference to the terms "some embodiments", "other embodiments", "ideal embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example.

[0066] In one embodiment, if Figure 1 As shown, a method for acquiring a vehicle side image is provided, and the method includes steps S102 to S106.

[0067] Step S102, sequentially acquiring multiple frames of images of the target vehicle.

[0068] Step S104, stitching processing and detection processing are performed on each frame of the acquired image simultaneously to obtain a stitching image of the target vehicle and the front position and the rear position of the target vehicle in the stitching image. In the same time period, the stitching processing and the detection processing are used to process the same frame of the target vehicle.

[0069] Specifically, in the present embodiment, when processing each frame of image, two processing threads for processing the image are simultaneously created, namely, a stitching processing thread and a detection processing thread. After acquiring each frame of image, the image of the current frame is sent to the stitching processing thread and the detection processing thread respectively, so as to obtain a stitching image of the target vehicle after the stitching processing is completed. After the detection processing is completed, the positions of the front and rear of the target vehicle in the stitching image are obtained. Among them, the stitching processing thread and the detection processing thread only need to be created when processing the first frame of image, and do not need to be created again when processing non-first frame of image.

[0070] Step S106, acquiring a vehicle side image of the target vehicle according to the splicing image, the vehicle front position and the vehicle rear position.

[0071] In this embodiment, by simultaneously performing stitching processing and detection processing on the images of the same frame and the same target vehicle, the side image of the target vehicle can be quickly acquired. In addition, by performing detection processing in parallel with the stitching processing, the front position and the rear position of the vehicle in the stitched image are acquired, and by extracting the image of the middle area between the front position and the rear position in the stitched image, an accurate side image of the vehicle can be acquired.

[0072] In one embodiment, if Figure 2 As shown, a schematic diagram of a stitching process flow is provided. In the vehicle side image acquisition method in the above embodiment, the stitching process of each acquired frame image includes steps S202 to S206.

[0073] Step S202, performing mobile matching on the k-1th frame and the kth frame image to determine a plurality of first feature points and a plurality of second feature points that correspond one to one, wherein the first feature points are located in the k-1th frame image, and the second feature points are located in the kth frame image.

[0074] Specifically, the optical flow is used in this embodiment to extract feature points. That is, multiple corner points are extracted in the k-1th frame image, and the corresponding relationship on the kth frame image is found through the optical flow method. Among them, the corner point extracted in the k-1th frame image is the first feature point, and the point corresponding to the first feature point on the kth frame image is the second feature point. It can be understood that the mobile matching method in this embodiment is not limited to the optical flow matching method, but can also be other image matching methods such as Scale Invariant Feature Transform (SIFT), and according to different image matching methods, the feature points in the image can be straight lines and regions.

[0075] Step S204: acquiring the feature point displacement between each first feature point and the corresponding second feature point.

[0076] Step S206 , splicing the k-th frame image with the current spliced ​​image according to the displacement of the multiple feature points to update the spliced ​​image.

[0077] Specifically, in this embodiment, when stitching is performed based on the displacement of multiple feature points, it can be based on the mean of the multiple feature points or the median of the displacement of the multiple feature points. Those skilled in the art can make adjustments based on image processing requirements.

[0078] In this embodiment, the displacement of feature points in the image is obtained by mobile matching the k-1th frame image and the kth frame image, and then the kth frame image is spliced ​​with the current spliced ​​image according to the displacement of multiple feature points to update the spliced ​​image.

[0079] In one embodiment, if Figure 3 As shown, a schematic diagram of a sub-process of the above-mentioned step S202 and step S206 is provided, wherein the above-mentioned step S202 includes step S302 and step S304, and step S206 includes steps S306 to S310.

[0080] Step S302: dividing the k-1th frame image into a plurality of first block images, and dividing the kth frame image into a plurality of second block images, wherein the plurality of first block images correspond to the plurality of second block images one by one.

[0081] Since the distances between each area on the side of the target vehicle and the camera used for image acquisition are not the same, there is a problem that the near area is larger and the far area is smaller. Therefore, the block matching method is adopted in this embodiment to reduce the influence of image distortion on image stitching. Specifically, let the image width be w and the height be h, and the image is cut into n overlapping areas of equal height with a width of w, where the height of each area is hpatch, hpatch = h / m. Please refer to Figure 4, which is a schematic diagram of image block in this embodiment. For the convenience of representation, Figure 4 In the example, three block images are used. Each block image is distinguished by a solid line, a dotted line and a dashed line. The three block images are overlapped and the overlapping area is of equal height. The height of the three block images is h / m. Within this height, the distortion can be considered small, so the conventional matching method can be directly used for matching and splicing. Among them, m can be a value between 2 and 6. If m is too small, the image details will be insufficient. If it is too large, the upper and lower block images will be distorted. For example, n can be 8 and m can be 4.

[0082] Step S304: performing movement matching on each first block image and each corresponding second block image, each first block image includes at least one first feature point, and each second block image includes at least one second feature point.

[0083] Step S306 , obtaining preset correction coefficients corresponding to the first block images respectively according to the displacements of the plurality of feature points.

[0084] Since the distances of the feature points on the side of the target vehicle from the camera are different, there is a problem of large near and small far. At the same time, the degree of distortion of the vehicle is different when it is close or far. Therefore, in order to obtain a more accurate stitching image in the stitching process, the stitching of the first block images cannot be simplified to the stitching of horizontally moving image blocks. The correction coefficient is used to transform the pixel coordinates in each first block image to correct the distortion problem of large near and small far in the acquired image.

[0085] Step S308: performing correction processing on each first block image according to a preset correction coefficient.

[0086] Step S310 , splicing the k-th frame image after the correction process with the current spliced ​​image according to the displacement of multiple feature points to update the spliced ​​image.

[0087] Specifically, in this embodiment, when the corrected image is stitched, the width of the stitched superimposed image block is taken as d, where d can be empirically taken as half of w, or can be other values, which are not limited here. The wd to w columns of the k-th frame image are the stitched superimposed image blocks, and the stitching can be completed by superimposing the stitched superimposed image blocks on the stitching image, and the stitching image is updated. Specifically, if it is the first frame stitching, the first frame is taken as the stitching image; if it is not the first frame stitching, then the width of the stitching image is w0 at this time, and the stitching superimposed image block is superimposed on the w0-d+Vb column of the stitching image, and this process is repeated for a complete stitching process. Among them, a weighted fusion coefficient can be used when splicing and superimposing image blocks, and the spliced ​​and superimposed image blocks and the superimposed position pixels are weightedly fused. Specifically, the weight of each column of the new superimposed image increases from column w0-d+Vb to 1, and the weight of the original spliced ​​image decreases from 1 to 0, so that seamless fusion and splicing of images can be achieved, wherein Vb is the splicing value obtained according to the displacement of the feature points.

[0088] In this embodiment, the distortion of the divided images is corrected respectively, and a more accurate target vehicle image can be obtained by splicing the corrected images.

[0089] In one embodiment, if Figure 5 As shown, a schematic diagram of sub-flows of step S304 and step S308 in the above embodiment is provided. Before step S304, the above method further includes steps S502 to S504. Among them, step S308 obtains preset correction coefficients corresponding to each first block image according to the displacement of multiple feature points, including steps S506 to S508.

[0090] Step S502: placing the bottom of the same calibration plate at preset heights corresponding to the first block images.

[0091] Specifically, in this embodiment, a scene identical to the actual one is built in the laboratory, which includes a camera and a calibration plate. The calibration plate is facing the camera, and the side plane of the target vehicle is simulated to be perpendicular to the ground. The actual coordinates of multiple calibration points in the calibration plate in the image and the image coordinates of the multiple calibration points captured by the camera are perspective transformed to obtain a set of correction data S.

[0092] Since the distortion degree of each first block image in this embodiment is different, this embodiment places the calibration plate at a preset height corresponding to each first block image, specifically, aligns the bottom of the same calibration plate with the bottom of each first block image, repeats the perspective transformation of the actual coordinates and image coordinates of the calibration point, and obtains multiple sets of correction coefficients corresponding to each first block image, which are recorded as S(n). The calibration plate can be a square calibration plate with an area of ​​1 square meter, and the multiple calibration points can be four vertices in the square calibration plate.

[0093] Step S504 , generating experimental correction coefficients corresponding to the first block images respectively according to the actual coordinates of the multiple calibration points in the calibration plate and the imaging coordinates of the multiple calibration points in the image.

[0094] Specifically, as described in step S502, this embodiment adopts a perspective transformation method to transform multiple calibration points in the calibration plate placed in each first block image to obtain multiple groups of experimental correction coefficients.

[0095] Among them, according to the perspective transformation principle, u, v are the original image coordinates, and the transformed image coordinates x, y are given by the formula:

[0096]

[0097]

[0098] It can be seen that only four point coordinates are needed, namely the original image coordinates and the transformed image coordinates, to solve other transformation coefficients, and the correction method is I(x,y)=I(u,v). Specifically, this embodiment can obtain multiple sets of transformation coefficients S(n) by substituting the actual coordinates in the calibration plate and the image coordinates acquired by the camera in the laboratory into the above formula, and generate the experimental correction coefficients corresponding to each first block image.

[0099] Step S506, obtaining the experimental correction coefficient.

[0100] Step S508 : acquiring preset correction coefficients corresponding to the first block images respectively according to the displacements of the plurality of feature points and the experimental correction coefficients.

[0101] Specifically, as described above, in this embodiment, a plurality of experimental correction coefficients are obtained, and each experimental correction coefficient corresponds to each first segmented image. In practical applications, in order to simplify calculation, this embodiment can ignore the fact that the farther the target vehicle is from the camera, the less correction is required, and select a group of experimental correction coefficients as the preset correction coefficients to correct each segmented image.

[0102] Specifically, in this embodiment, by creating a corrected image, the coordinates of each pixel in each first block image are substituted into the perspective transformation formula as the original image coordinates to obtain the corresponding corrected image coordinates, that is, the transformed image coordinates, and then the correction process can be completed by assigning the pixel values ​​in each first block image to the corrected image.

[0103] In order to increase the correction effect, the coefficient of the bilinear interpolation algorithm is increased to reduce the aliasing effect of the final correction image. Specifically, by taking the decimal of the pixel coordinates in the calculated correction image as the weight, the weights sx and sy are obtained for the two components xy respectively, and the final pixel value is calculated according to the following formula: I(x,y)=I(x,y)*(1-sx)*(1-sy)+I(x+1,y)*(sx)*(1-sy)+I(x,y+1)* (1-sx)*(sy)+I(x+1,y+1)*(sx)*(sy), and the image correction processing is completed.

[0104] In this embodiment, experimental correction coefficients corresponding to each first block image are obtained in a laboratory, and at least one preset correction coefficient is determined from the experimental correction coefficients. The distorted first block image is corrected by the preset correction coefficient to obtain a block image with less distortion.

[0105] In one embodiment, if Figure 6 As shown, a sub-flow diagram of step S312 in the above embodiment is provided, and the above step S312 includes steps S602 to S608.

[0106] Step S602: Obtain the average displacement value between at least one first feature point included in each first block image in the k-1th frame image and at least one second feature point included in each second block image in the kth frame image.

[0107] Specifically, in this embodiment, the average displacement of the first feature point in the k-1th frame image to the second feature point in the kth frame image is obtained. That is, the number obtained is consistent with and corresponding to the number of multiple first block images (V1, V2, ..., Vn). Among them, V1 is the average displacement of multiple first feature points in the first block image to multiple second feature points; V2, ..., Vn and so on.

[0108] Step S604: determining at least one of the plurality of displacement average values ​​as a first target displacement value.

[0109] Determine at least one of (V1, V2, ..., Vn) as the first target displacement value, for example, V1 and Vn. It should be noted that the example of the first target displacement value in this embodiment is only for illustration and is not intended to limit the scope of protection of the present application. Those skilled in the art can reasonably determine it according to the actual image processing situation.

[0110] Step S608 , splicing the k-th frame image after the correction process with the current splicing image according to the average value of the first target displacement value.

[0111] Specifically, the average value of the target displacement values ​​in this embodiment is the Vb value in the above embodiment.

[0112] This embodiment determines at least one of a plurality of displacement average values ​​as a first target displacement value, and splices the corrected k-th frame image with the current spliced ​​image according to the average value of the first target displacement value, so as to quickly obtain an accurate spliced ​​image.

[0113] In one embodiment, if Figure 7 As shown, a sub-flow diagram of step S508 in the above embodiment is provided. Step S508 of the above method includes steps S702 to S704.

[0114] Step S702: Determine at least one of the plurality of first block images as a target block image according to the first target displacement value.

[0115] Specifically, the first target image selected in this embodiment is the first block image near the bottom of the image, which usually corresponds to the bottom of the target vehicle in the image. For example, in this embodiment, the first target displacement value is determined to be V1, and the first block image corresponding to V1 is the target block image.

[0116] Step S704: Acquire the experimental correction coefficient corresponding to the target block image as the preset correction coefficient.

[0117] In this embodiment, an experimental correction coefficient corresponding to the target segmented image is obtained as a preset correction coefficient, and correction processing is performed on each first segmented image according to the preset correction coefficient to obtain an accurate image mosaic.

[0118] In one embodiment, if Figure 8 As shown, a sub-process diagram of step S702 in the above embodiment is provided. Before step S702, the above method further includes step S802, wherein step S702 includes step S804.

[0119] Step S802, determine at least one of the total mean values ​​of multiple displacement average values ​​that is greater than the first multiple and less than the second multiple as the second target displacement value, wherein the total mean value is the average value of multiple displacement average values, and the value of the first multiple is less than the value of the second multiple.

[0120] Specifically, in the above embodiment, the average values ​​of the multiple displacements are (V1, V2, ..., Vn), and the total average value is the average value of V1, V2, ..., Vn, which is recorded as V set . From (V1, V2, ..., Vn), at least one of the total mean values ​​greater than the first multiple and less than the total mean value of the second multiple is selected as the second target displacement value. Among them, the value of the first multiple can be 0.5, and the value of the second multiple can be 1.5. It can be understood that the first multiple and the second multiple can be reasonably set by those skilled in the art, and are not limited to the examples of this embodiment.

[0121] Step S804: when the average value of the second target displacement value is greater than a preset threshold, determining the average displacement value among the plurality of average displacement values ​​that does not include the second target displacement value as the first target displacement value.

[0122] Specifically, the second target displacement value determined in the above embodiment is averaged and recorded as Vavg. When Vavg is greater than a preset threshold, it is considered that there is a vehicle in the currently detected image, and subsequent stitching processing is required for the currently detected image, and the displacement average value excluding the second target displacement value is determined from (V1, V2, ..., Vn) as the first target displacement value.

[0123] In this embodiment, non-vehicle body or mismatched areas are removed through the second target displacement value. When the average value of the second target displacement value meets the preset condition requirement, that is, is greater than the preset threshold, it is determined that there is a moving vehicle body in the currently processed image, and stitching processing can be performed according to the determined first target displacement value.

[0124] In one embodiment, before the above step S604, the following steps are also included:

[0125] When the average of the second target displacement values ​​is less than the preset threshold, the stitching process of the current frame image is terminated.

[0126] Specifically, when the mean of the second target displacement values ​​is less than the preset threshold, it indicates that there is no moving vehicle in the currently processed image, and therefore, the image of the current frame is not stitched, the image stitching process of the current frame is ended, and the image stitching process of the next frame is started. The preset threshold in this embodiment can be 0, or 1, etc., which is not limited here, and can be reasonably set by those skilled in the art according to actual processing conditions.

[0127] In one embodiment, the above method steps further include the following steps before sequentially acquiring multiple frames of images of the target vehicle:

[0128] Each acquired frame image is cropped according to a preset aspect ratio factor, and the cropped image is used as a cropped image of the target vehicle.

[0129] Specifically, the camera used to obtain multiple consecutive frames of target vehicle images in this embodiment has a high frame rate slice image acquisition output. It is applied to urban road scenes at least 50fps, and applied to highway scenes at a minimum frame rate of 100fps. In this embodiment, the acquired image is cropped into a cropped image with unchanged height and retaining the middle area of ​​the image using a preset aspect ratio coefficient. Among them, the preset aspect ratio coefficient can be 1:2. It can be understood that the preset aspect ratio coefficient in this embodiment is only used for example and is not used to limit the scope of protection of this application. Those skilled in the art can refer to the preset aspect ratio coefficient of this embodiment and make reasonable settings to reduce the distortion of the cropped image after cropping and the consumption in the image stitching processing operation.

[0130] In one embodiment, if Fig. 9 As shown, a method for acquiring a vehicle side image is provided, and vehicle detection is performed on each acquired frame image, including steps S902 to S908.

[0131] Step S902, when the front of the target vehicle is detected in the image of the kth frame, a spliced ​​image of the target vehicle of the first width from the 1st frame to the kth frame is obtained.

[0132] Specifically, in this embodiment, two flags are configured for the above-created image stitching processing thread: stitching in progress and stitching completed. When the image stitching thread is stitching the acquired images, the flag of the image stitching thread is stitching in progress; when the image stitching thread outputs an updated stitching image, or when it is determined that stitching is not required, the flag of the image stitching thread is stitching completed.

[0133] In this embodiment, when the image of the kth frame is detected, the image of the kth frame is also stitched. When the front of the target vehicle is detected in the image of the kth frame, the position of the front of the target vehicle in the current image is recorded as carH, and the flag of the image stitching thread is switched to stitching completion, and the first width stitching image of the target vehicle from the 1st frame to the kth frame can be copied from the image stitching thread.

[0134] Among them, in order to ensure the accuracy of vehicle head detection in this embodiment, a deep neural network yolo detection algorithm is adopted. However, those skilled in the art can also use other algorithms for detection to achieve the purpose of vehicle head detection, which will not be repeated here.

[0135] Step S904: acquiring the front position of the target vehicle in the spliced ​​image according to the spliced ​​image of the first width.

[0136] Specifically, as mentioned above, assuming that the width of the obtained stitching image of the first width is w0, the position of the car head in the stitching image is P_head=w0-(w-carH), wherein w is the width of each frame image. It should be noted that when the cropped image in the above embodiment is obtained, w is the width of the cropped image.

[0137] Step S906, when the rear end of the target vehicle is detected in the image of the k+nth frame, a spliced ​​image of the target vehicle of the second width from the first frame to the k+nth frame is obtained.

[0138] Similarly, when the side head position of the target vehicle is detected, this embodiment performs rear-end detection on subsequent continuous multiple frames of images to detect the rear end position of the same vehicle. When the rear end of the target vehicle is detected in the image of the k+nth frame, the position of the front of the target vehicle in the current image is recorded as carT, and the image stitching thread is waited for the stitching completion mark of the image of the k+nth frame to obtain the second width stitching image of the target vehicle from the 1st frame to the k+nth frame.

[0139] Among them, in order to ensure the accuracy of rear-end detection in this embodiment, a deep neural network yolo detection algorithm is adopted. However, those skilled in the art can also use other algorithms for detection to achieve the purpose of rear-end detection, which will not be repeated here.

[0140] Step S908, obtaining the rear position of the target vehicle in the spliced ​​image according to the spliced ​​image of the second width.

[0141] Specifically, as mentioned above, assuming that the width of the obtained stitching image of the first width is w1, the position of the car head in the stitching image is P_head=w1-(w-carT), wherein w is the width of each frame image. It should be noted that when the cropped image in the above embodiment is obtained, w is the width of the cropped image.

[0142] In this embodiment, the front position and the rear position of the target vehicle in the mosaic image are obtained by detecting and processing the same target vehicle, so as to calibrate the front position and the rear position of the same marked vehicle, so as to facilitate the subsequent extraction of the vehicle side image between the front position and the rear position to obtain an accurate vehicle side image.

[0143] In one embodiment, if Fig.10 As shown, a method for acquiring a vehicle side image is provided, and the method includes steps S1002 to S1036.

[0144] Step S1002, sequentially acquiring multiple frames of images of the target vehicle.

[0145] Step S1004: dividing the k-1th frame image into a plurality of first block images, and dividing the kth frame image into a plurality of second block images, wherein the plurality of first block images correspond to the plurality of second block images one by one.

[0146] Step S1006: placing the bottom of the same calibration plate at preset heights corresponding to the first block images.

[0147] Step S1008 , generating experimental correction coefficients corresponding to the first block images respectively according to the actual coordinates of the multiple calibration points in the calibration plate and the imaging coordinates of the multiple calibration points in the image.

[0148] Step S1010: performing movement matching on each first block image and each corresponding second block image, each first block image includes at least one first feature point, and each second block image includes at least one second feature point.

[0149] Step S1012: Obtain the feature point displacement between each first feature point and the corresponding second feature point.

[0150] Step S1014, obtaining the experimental correction coefficient.

[0151] Step S1016: Obtain preset correction coefficients corresponding to the first block images respectively according to the displacements of the plurality of feature points and the experimental correction coefficients.

[0152] Step S1018: performing correction processing on each first block image according to a preset correction coefficient.

[0153] Step S1020: Obtain the average displacement value between at least one first feature point included in each first block image in the k-1th frame image and at least one second feature point included in each second block image in the kth frame image.

[0154] Step S1022, determine at least one of the total mean values ​​of multiple displacement average values ​​that is greater than the first multiple and less than the second multiple as the second target displacement value, wherein the total mean value is the average value of multiple displacement average values, and the value of the first multiple is less than the value of the second multiple.

[0155] Step S1024: when the average value of the second target displacement value is greater than a preset threshold, determining the average displacement value among the plurality of average displacement values ​​that does not include the second target displacement value as the first target displacement value.

[0156] Step S1026 , splicing the k-th frame image after the correction process with the current splicing image according to the average value of the first target displacement value.

[0157] Step S1028, when the front of the target vehicle is detected in the image of the kth frame, a spliced ​​image of the first width of the target vehicle from the 1st frame to the kth frame is obtained.

[0158] Step S1030: acquiring the front position of the target vehicle in the spliced ​​image according to the spliced ​​image of the first width.

[0159] Step S1032, when the rear end of the target vehicle is detected in the image of the k+nth frame, a spliced ​​image of the second width of the target vehicle from the 1st frame to the k+nth frame is obtained.

[0160] Step S1034, obtaining the rear position of the target vehicle in the spliced ​​image according to the spliced ​​image of the second width.

[0161] Step S1036, acquiring a vehicle side image of the target vehicle according to the splicing image, the vehicle front position and the vehicle rear position.

[0162] Specifically, the limitations of each method step in this embodiment are the same as the method step limitations in the above-mentioned embodiment and will not be repeated here. This embodiment can quickly acquire the side view image of the target vehicle by performing detection processing of image stitching processing respectively and simultaneously, and in the image stitching process, the image distortion is processed by cropping, block matching and preset correction coefficients to achieve accurate image acquisition.

[0163] It should be understood that although Figure 1 to Figure 3 , Figures 5 to 10 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 to Figure 3 , Figures 5 to 10 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0164] In one embodiment, Fig.11As shown, a vehicle side image acquisition device 100 is provided, comprising: an image acquisition module 110, a splicing module 120 and a vehicle side image acquisition module 130. The image acquisition module 110 is used to sequentially acquire multiple frames of continuous images of a target vehicle; the splicing module 120 is used to simultaneously perform splicing processing and detection processing on each acquired frame image, so as to respectively acquire a splicing image of the target vehicle, and the front position and the rear position of the target vehicle in the splicing image. In the same period, the splicing processing and the detection processing are used to process the same frame image of the target vehicle; the vehicle side image acquisition module 130 is used to acquire the vehicle side image of the target vehicle according to the splicing image, the front position and the rear position.

[0165] For the specific limitations of the device for acquiring the vehicle side image, please refer to the limitations of the method for acquiring the vehicle side image above, which will not be repeated here. Each module in the above-mentioned device for acquiring the vehicle side image can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. It should be noted that the division of modules in the embodiment of the present application is schematic, which is only a logical function division, and there may be other division methods in actual implementation.

[0166] In one embodiment, a system for acquiring vehicle side images is provided, the system comprising a vehicle side image acquisition device and a camera, the vehicle side image acquisition device is used to sequentially acquire multiple consecutive frames of images of a target vehicle; stitching processing and detection processing are performed on each acquired frame of image respectively to acquire a stitching image of the target vehicle and the front position and the rear position of the target vehicle in the stitching image, and within the same time period, the stitching processing and the detection processing are used to process the same frame of image of the target vehicle; according to the stitching image, the front position and the rear position, the vehicle side image acquisition camera of the target vehicle is connected to the vehicle side image acquisition device, and the camera is used to capture the image of the target vehicle.

[0167] Specifically, the definition of the vehicle side image acquisition device in this embodiment, please refer to the above-mentioned definition of the vehicle side image acquisition device, which will not be repeated here. Among them, the camera in this embodiment is set at a high position above the ground, such as a highway gantry or a city road pole. Specifically, in order to obtain the vehicle side image of the vehicle on the highway, please refer to Fig.12 The figure shows the camera position diagram in this embodiment. The camera is set on the gantry or pole of the highway, and the center line of the camera field of view is perpendicular to the direction of the vehicle in the lane, and the downward angle is 45° to 55°. This method ensures that the vehicle moves horizontally in the picture. In practice, the center line of the camera field of view is not strictly vertical, and the deviation of about 10° is within the tolerance range.

[0168] The camera in this embodiment supports a splicing area of ​​1 lane, and the installation position is 3 meters to 4.5 meters away from the covered lane, and the height is 6 meters to 8 meters from the lane ground. Combined with the above-mentioned camera angle, the purpose of not blocking the lane covered by the camera due to the adjacent vehicle body or cargo can be achieved.

[0169] Among them, the camera of this embodiment selects to keep the upper edge of the lane between 30% and 60% of the height of the imaging screen, and the bottom edge of the lane between 70% and 100% of the height of the imaging screen in the imaging screen. Under this condition, it can be ensured that the vehicle is complete in the picture, that is, the side and roof of the large truck can be completely included in the field of view. In actual installation, the appropriate focal length can be selected to meet the above requirements.

[0170] In addition, in this embodiment, a fill light can be installed near the camera to fill light the vehicle body at night. The fill light direction is consistent with the camera viewing angle. The appropriate fill light power can be selected to make the vehicle body image clear at night.

[0171] In one of the embodiments, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above-mentioned method for acquiring the vehicle side image when executing the computer program.

[0172] In one of the embodiments, a computer program product is provided, including a computer program, which implements the above-mentioned method for acquiring the vehicle side image when executed by a processor.

[0173] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0174] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0175] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for acquiring a vehicle side image, characterized in that: The method comprises: Sequentially acquire multiple frames of continuous images of the target vehicle; Performing stitching processing and detection processing on each frame of the acquired image simultaneously, so as to respectively acquire a stitching image of the target vehicle and a front position and a rear position of the target vehicle in the stitching image, wherein the stitching processing and the detection processing are used to process the same frame of the target vehicle in the same time period; Acquire a vehicle side image of the target vehicle according to the spliced ​​image, the vehicle front position, and the vehicle rear position; The step of performing stitching processing on each frame of the acquired images includes: Dividing the k-1th frame image into a plurality of first block images, and dividing the kth frame image into a plurality of second block images, wherein the plurality of first block images correspond to the plurality of second block images one by one; Performing mobile matching on each of the first block images and the corresponding second block images, each of the first block images includes at least one first feature point, and each of the second block images includes at least one second feature point; Respectively obtaining a feature point displacement between each of the first feature points and the corresponding second feature point; Obtaining preset correction coefficients corresponding to the first block images respectively according to the displacements of the plurality of feature points; Performing correction processing on each of the first block images according to the preset correction coefficient; Obtaining a displacement average value between at least one first feature point included in each of the first block images in the k-1th frame image and at least one second feature point included in each of the second block images in the kth frame image; Determine at least one of the total mean values ​​of the plurality of displacement average values ​​that is greater than the first multiple and less than the second multiple as the second target displacement value, wherein the total mean value is the average value of the plurality of displacement average values, and the value of the first multiple is less than the value of the second multiple; When the average value of the second target displacement value is greater than a preset threshold, determining the displacement average value excluding the second target displacement value among the plurality of displacement average values ​​as the first target displacement value; The k-th frame image after correction is spliced ​​with the current spliced ​​image according to the average value of the first target displacement value.

2. The method according to claim 1, characterized in that Before performing mobile matching on each of the first block images and the corresponding second block images respectively, the method further includes: placing the bottom of the same calibration plate at preset heights corresponding to the first block images; Generating experimental correction coefficients corresponding to the first block images respectively according to the actual coordinates of the plurality of calibration points in the calibration plate and the imaging coordinates of the plurality of calibration points in the image; The obtaining of preset correction coefficients corresponding to the first block images respectively according to the displacements of the plurality of feature points comprises: Obtaining the experimental correction coefficient; According to the plurality of feature point displacements and the experimental correction coefficients, preset correction coefficients corresponding to the first block images are respectively obtained.

3. The method according to claim 2, characterized in that The obtaining, according to the plurality of feature point displacements and the experimental correction coefficients, respectively the preset correction coefficients corresponding to the first block images comprises: Determine at least one of the plurality of first block images as a target block image according to the first target displacement value; The experimental correction coefficient corresponding to the target block image is obtained as the preset correction coefficient.

4. The method according to claim 1, characterized in that: Before the acquired frames of images are stitched and detected simultaneously, the method further includes: Each acquired frame image is cropped according to a preset aspect ratio factor, and the cropped image is used as a cropped image of the target vehicle.

5. The method according to claim 1, characterized in that Vehicle detection for each acquired frame image includes: When the front of the target vehicle is detected in the image of the kth frame, obtaining the spliced ​​image of the first width of the target vehicle from the 1st frame to the kth frame; Acquire the front position of the target vehicle in the mosaic image according to the mosaic image of the first width; When the rear end of the target vehicle is detected in the image of the k+nth frame, obtaining the spliced ​​image of the second width of the target vehicle from the first frame to the k+nth frame; The rear position of the target vehicle in the mosaic image is acquired according to the mosaic image of the second width.

6. A vehicle side image acquisition device, characterized in that: include: An image acquisition module, used to sequentially acquire multiple frames of continuous images of the target vehicle; A stitching module, used for performing stitching processing and detection processing on each frame of the acquired image at the same time, so as to respectively obtain a stitching image of the target vehicle and the front position and the rear position of the target vehicle in the stitching image. In the same time period, the stitching processing and the detection processing are used to process the same frame of the target vehicle; A vehicle side image acquisition module, used to acquire a vehicle side image of the target vehicle according to the spliced ​​image, the vehicle front position and the vehicle rear position; Among them, the splicing module is used to divide the k-1th frame image into multiple first block images, and divide the kth frame image into multiple second block images, and the multiple first block images correspond to the multiple second block images one by one; respectively perform mobile matching on each of the first block images and the corresponding second block images, and each of the first block images includes at least one first feature point, and each of the second block images includes at least one second feature point; respectively obtain the feature point displacement between each of the first feature points and the corresponding second feature point; respectively obtain the preset correction coefficient corresponding to each of the first block images according to the multiple feature point displacements; respectively perform correction processing on each of the first block images according to the preset correction coefficients; obtain each of the first block images in the k-1th frame image. The method comprises the steps of: determining a displacement average value between at least one of the first feature points included in the first block image and at least one of the second feature points included in each of the second block images in the k-th frame image; determining a total mean value greater than a first multiple and at least one of the total means less than a second multiple among the plurality of displacement average values ​​as a second target displacement value, wherein the total mean value is an average value of the plurality of displacement average values, and the value of the first multiple is less than the value of the second multiple; when the average value of the second target displacement value is greater than a preset threshold value, determining a displacement average value among the plurality of displacement average values ​​that does not include the second target displacement value as a first target displacement value; and splicing the k-th frame image after correction with the current splicing image according to the average value of the first target displacement value.

7. A vehicle side image acquisition system, comprising: A vehicle side image acquisition device, the vehicle side image acquisition device is used to sequentially acquire multiple consecutive frames of images of a target vehicle; the acquired frames are respectively and simultaneously subjected to stitching processing and detection processing to respectively acquire a stitching diagram of the target vehicle and a front position and a rear position of the target vehicle in the stitching diagram, wherein the stitching processing and the detection processing are used to process the same frame image of the target vehicle in the same time period; the vehicle side image of the target vehicle is acquired according to the stitching diagram, the front position and the rear position; the acquired frames are specifically stitched by the following steps: the k-1th frame image is divided into a plurality of first block images, the kth frame image is divided into a plurality of second block images, and the plurality of the first block images correspond one to one with the plurality of the second block images respectively; Performing mobile matching on each of the first block images and the corresponding second block images respectively, wherein each of the first block images includes at least one first feature point and each of the second block images includes at least one second feature point; obtaining feature point displacements between each of the first feature points and the corresponding second feature points; obtaining preset correction coefficients corresponding to each of the first block images according to the plurality of feature point displacements; performing correction processing on each of the first block images according to the preset correction coefficients; obtaining at least one of the first feature points included in each of the first block images in the k-1th frame image and at least one of the first feature points included in each of the second block images in the kth frame image; including at least one of the second feature points; determining a total mean value greater than the first multiple and at least one of the total means less than the second multiple among the plurality of the displacement average values ​​as a second target displacement value, wherein the total mean value is an average value of the plurality of the displacement average values, and the value of the first multiple is less than the value of the second multiple; when the average value of the second target displacement value is greater than a preset threshold, determining a displacement average value excluding the second target displacement value among the plurality of the displacement average values ​​as a first target displacement value; splicing the corrected k-th frame image with the current splicing image according to the average value of the first target displacement value; and A camera is connected to the vehicle side image acquisition device, and the camera is used to capture the image of the target vehicle.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

Images