A method and apparatus for vehicle side panoramic stitching

Abstract

The application provides a vehicle side panoramic image splicing method and device, and relates to the technical field of vehicle image splicing. The method comprises the following steps: acquiring multiple continuous images of an image acquisition area; determining whether to splice a current frame image and a previous frame image according to the current frame image and the previous frame image; if yes, selecting a reference image in the previous frame image, the reference image being an image with a preset length from the rightmost side of the previous frame image to the left; dividing the current frame image into multiple column images, determining the number of pixel points in the reference image that match pixel points in a column image, determining the proportion of the pixel points in the column image that match successfully in all pixel points in the column image, and taking the leftmost side of the column image as a splicing position when the proportion is the largest; and splicing the current frame image with the previous frame image from the splicing position. In this way, the number of axles of a moving vehicle can be efficiently and quickly identified.

Description

Technical Field

[0001] This application relates to the field of vehicle image stitching technology, and in particular to a method and apparatus for stitching panoramic side views of a vehicle. Background Technology

[0002] Due to the reform of highway vehicle toll standards, highway vehicle tolls are no longer charged based on vehicle weight, but rather on vehicle type based on axle length, vehicle length, and passenger capacity.

[0003] Currently, the number of axles on vehicles is mostly confirmed manually, while some methods use ground sensors. Manual confirmation has high accuracy but low efficiency, and staff fatigue can lead to errors over long periods. Ground sensor confirmation has higher accuracy, but the sensors are easily damaged by the constant pressure of loaded trucks, and replacement is not easy. Summary of the Invention

[0004] To efficiently and quickly identify the number of axles of a moving vehicle, this application provides a method and apparatus for stitching together panoramic side views of a vehicle.

[0005] In a first aspect of this application, a method for stitching together a panoramic side view of a vehicle is provided. The method includes:

[0006] Acquire multiple consecutive images from the image acquisition area;

[0007] Based on the current frame image and the previous frame image, determine whether to stitch the current frame image and the previous frame image together;

[0008] If so, a reference image is selected from the previous frame image, and the reference image is an image of a preset length from the rightmost side of the previous frame image to the left.

[0009] The current frame image is divided into multiple columns of images, and the number of pixels in the comparison image that match the pixels in one column of images is determined.

[0010] Determine the percentage of successfully matched pixels in a column of images out of all pixels in that column of images;

[0011] When the ratio is at its maximum, the leftmost position of the column of images is taken as the stitching position;

[0012] The current frame image is stitched together with the previous frame image starting from the stitching position.

[0013] By adopting the above technical solution, before a vehicle enters a highway toll station, multiple vehicle photos are collected, and the stitching position between the current frame and the previous frame is quickly found by calculating the number of matching pixels in the two frames. The images containing vehicle information are then stitched together to obtain a complete panoramic view of the vehicle's side. The number of axles of the vehicle can then be accurately and quickly obtained from this panoramic view.

[0014] Preferably, selecting a reference image from the previous frame includes: obtaining the position of the moving vehicle in the previous frame; cutting out an initial stitched image of the moving vehicle from the position to the right; and using an image of a preset length from the rightmost side of the initial stitched image as the reference image.

[0015] Preferably, determining the number of pixels in the comparison image that match pixels in a column of images includes: obtaining the RGB values ​​of all pixels in the comparison image and the column of images; and determining whether the pixels match according to the following formula:

[0016] |R1-R2|+|G1-G2|+|B1-B2| <Thr1

[0017] Where R1, G1, and B1 are the RGB values ​​of pixels in the comparison image, R2, G2, and B2 are the RGB values ​​of pixels in a column of images, and Thr1 is the matching threshold; count the number of pixels that satisfy the above formula.

[0018] Preferably, determining the proportion of successfully matched pixels in a column of images out of all pixels in that column of images includes: determining the proportion according to the following formula:

[0019]

[0020] Where S is the percentage of successfully matched pixels in a column of images out of all pixels in the column of images, X is the number of successfully matched pixels, and P is the total number of pixels in the column of images.

[0021] Preferably, determining whether to stitch the current frame image and the previous frame image together based on the current frame image and the previous frame image includes:

[0022] Analyze the state of the current frame image and the state of the previous frame image. The state of the current frame image and the state of the previous frame image both include the start stitching state, the stitching state, the ignore state, and the end stitching state.

[0023] When the previous frame image is ignored and there is a moving vehicle in the current frame image, the current frame image is in the state of starting stitching.

[0024] When the previous frame image is in the start stitching state or stitching state and there is a vehicle in the current frame image, then the state of the current frame image is the stitching state;

[0025] When the previous frame image is in a stitched state or a skipped state and the current frame image does not contain moving vehicles, the output state of the current frame image is the skipped state;

[0026] When the previous frame image is in a stitched state and there are no moving vehicles in the current frame image, the output state of the current frame image is the "stitching ended" state.

[0027] Preferably, detecting whether a moving vehicle exists in the current frame image includes: calculating the sum of the number of pixels in three adjacent columns of the current frame image, and determining whether the sum of the number of pixels satisfies the following condition; if it does, then a moving vehicle exists in the current frame image:

[0028] SUM>H*R

[0029] Where SUM is the sum of the number of pixels in the three adjacent columns, H is the height of the current frame image, and R is the ratio threshold.

[0030] In a second aspect of this application, an apparatus for stitching together panoramic side views of a vehicle is provided. The apparatus includes:

[0031] The acquisition module is used to acquire multiple consecutive images from the image acquisition area;

[0032] The splicing discrimination module is used to determine whether to splice the current frame image and the previous frame image based on the current frame image and the previous frame image;

[0033] The stitching module is used to select a reference image from the previous frame image, wherein the reference image is an image of a preset length extending from the rightmost side of the previous frame image to the left; divide the current frame image into multiple columns of images; determine the number of pixels in the reference image that match pixels in a column of images; determine the percentage of successfully matched pixels in a column of images among all pixels in that column of images; when the percentage is the largest, take the leftmost side of the column of images as the stitching position; and stitch the current frame image with the previous frame image starting from the stitching position.

[0034] Preferably, the splicing discrimination module is specifically used to: analyze the state of the current frame image and the state of the previous frame image, wherein the state of the current frame image and the state of the previous frame image both include a start splicing state, a splicing state, an ignore state, and a end splicing state;

[0035] When the previous frame image is ignored and there is a moving vehicle in the current frame image, the current frame image is in the state of starting stitching.

[0036] When the previous frame image is in the start stitching state or stitching state and there is a vehicle in the current frame image, then the state of the current frame image is the stitching state;

[0037] When the previous frame image is in a stitched state or a skipped state and the current frame image does not contain moving vehicles, the output state of the current frame image is the skipped state;

[0038] When the previous frame image is in a stitched state and there are no moving vehicles in the current frame image, the output state of the current frame image is the "stitching ended" state.

[0039] Preferably, the device further includes a motion detection module, which is used to calculate the sum of the number of pixels in three adjacent columns of the current frame image, and determine whether the sum of the number of pixels satisfies the following condition: if it does, then there is a moving vehicle in the current frame image.

[0040] SUM>H*R

[0041] Where SUM is the sum of the number of pixels in the three adjacent columns, H is the height of the current frame image, and R is the ratio threshold.

[0042] In a third aspect of this application, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0043] It should be understood that the description in the Summary Section is not intended to limit the key or essential features of the embodiments of this application, nor is it intended to restrict the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0044] The above and other features, advantages, and aspects of the embodiments of this application will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0045] Figure 1 This is a flowchart of the method for stitching together panoramic side views of a vehicle in this application embodiment;

[0046] Figure 2 This is a schematic diagram of the initial splicing state in an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of the comparison images in the embodiments of this application;

[0048] Figure 4 This is a schematic diagram of the region of interest in the embodiments of this application;

[0049] Figure 5 This is a schematic diagram of the current frame image after division in an embodiment of this application;

[0050] Figure 6 This is a block diagram of the vehicle side panoramic image stitching device in the embodiments of this application;

[0051] Figure 7 This is a block diagram of the electronic device in the embodiments of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0053] To facilitate understanding of the embodiments of this application, some terms involved in the embodiments of this application will be explained first.

[0054] In the description of this application, the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0055] In a first aspect, embodiments of this application provide a method for stitching together a panoramic view of a vehicle's side profile, referring to... Figure 1 The method includes the following steps:

[0056] Step 110: Acquire multiple consecutive images of the image acquisition area.

[0057] Step 120: Based on the current frame image and the previous frame image, determine whether to stitch the current frame image and the previous frame image together.

[0058] Step 130: If yes, select a reference image from the previous frame image. The reference image is an image of a preset length from the rightmost side of the previous frame image to the left.

[0059] Step 140: Divide the current frame image into multiple columns of images and determine the number of pixels in the comparison image that match the pixels in one column of images.

[0060] Step 150: Determine the percentage of successfully matched pixels in a column of images out of all pixels in that column of images.

[0061] Step 160: When the ratio is the largest, take the leftmost position of a column of images as the stitching position.

[0062] Step 170: Start stitching the current frame image with the previous frame image from the stitching position.

[0063] It should be noted that the image acquisition area is located before the highway toll station entrance, and can be set to a distance of one vehicle length from the image acquisition area to the highway toll station. Furthermore, the height of the imaging unit is lower than the height of a regular vehicle. Additionally, the vehicles in the image are traveling from right to left.

[0064] Using the above method, before a vehicle enters the highway toll station, it first passes through the image acquisition area. At this time, images are taken from the side of the moving vehicle, obtaining multiple consecutive frames of images. Then, by analyzing and calculating the pixels in the current and previous frames, the stitching position between the two frames is quickly found and stitched together to obtain a complete panoramic image of the vehicle's side profile. Toll station staff can then quickly and accurately determine the number of axles of the vehicle based on this panoramic image.

[0065] The steps described above are explained in detail below:

[0066] In step 110, an image acquisition device is provided in the image acquisition area, which can be a camera or other device capable of continuous shooting. The shooting height of the image acquisition device is lower than the height of a normal vehicle, so that the captured image can show part of the vehicle body and wheels.

[0067] For example, an image acquisition device captures video of a moving vehicle passing through an image acquisition area. The image acquisition device then uploads the video to a computer, which can use video editing software to break the video down into multiple frames.

[0068] In step 120, after acquiring an image through the image acquisition device, the image acquisition device uploads the image to the storage device, and the storage device saves the image. Then, the saved current frame image and the previous frame image are compared and analyzed to determine whether to stitch the current frame image and the previous frame image together.

[0069] In some implementations, step 120 further includes the following steps:

[0070] Step 121: Detect whether there are moving vehicles in the current frame image.

[0071] The specific detection method is as follows: calculate the sum of the number of pixels in three adjacent columns of the current frame image, and determine whether the sum of the number of pixels satisfies the following condition. If it does, then there is a moving vehicle in the current frame image:

[0072] SUM>H*R

[0073] Where SUM is the sum of the number of pixels in three adjacent columns, H is the height of the current frame image, and R is the scale threshold.

[0074] It should be noted that before calculating the sum of the number of pixels in three adjacent columns of the current frame image, the number of pixels in each column of the current frame image needs to be calculated first. After calculating the number of pixels in each column, the image is traversed from the leftmost column to the right, and the sum of the number of pixels in three adjacent columns is counted in turn. Additionally, the ratio threshold can be set to 1.2.

[0075] Step 122: Analyze the state of the current frame image and the state of the previous frame image.

[0076] It should be noted that the state of the current frame image and the state of the previous frame image both include the start stitching state, stitching state, ignore state, and end stitching state.

[0077] Specifically, referring to Table 1, if the previous frame image is in an ignored state and there is a moving vehicle in the current frame image, then the current frame image is in the start stitching state; if the previous frame image is in the start stitching state or stitching state and there is a vehicle in the current frame image, then the current frame image is in the stitching state; if the previous frame image is in the stitching state or skipped state and there is no moving vehicle in the current frame image, then the current frame image is in the skipped state; if the previous frame image is in the stitching state and there is no moving vehicle in the current frame image, then the current frame image is in the end stitching state.

[0078] Table 1 Image Status Judgment Conditions

[0079]

[0080] For example, when a moving vehicle passes through the image acquisition area for the first time, the image acquisition device can sequentially capture the front, body, and rear of the vehicle. If the front of the moving vehicle is captured for the first time in the current frame image, and the previous frame image is in an ignored state, i.e., there is no moving vehicle, then the current frame image is in the stitching start state.

[0081] In step 130, if the current frame image is in a stitching state, it indicates that the previous frame image was in a stitching start state or a stitching state. Figure 2 This illustration shows a diagram of the image in the stitching start state in an embodiment of this application. See also: Figure 2As can be seen, the previous frame must contain part of the vehicle's location, and the vehicle's movement direction shown in the image is from right to left. Therefore, the image with a preset length from the rightmost side to the left in the previous frame is selected as the comparison image. Figure 3 Comparative images from embodiments of this application are shown; see [link / reference]. Figure 3 The reference image shows the area from the dotted line to the far right of the image.

[0082] It should be noted that the preset length is selected based on the image resolution; one-tenth of the image width can be selected as the preset length.

[0083] In some implementations, step 130 further includes the following steps:

[0084] Step 131: Obtain the position of the moving vehicle in the previous frame image.

[0085] Step 132: Cut out the image from the location to the right to obtain the initial stitched image of the moving vehicle.

[0086] Step 133: Use an image of a preset length from the rightmost side of the initial stitched image as a reference image.

[0087] It should be noted that steps 131, 132, and 133 are not shown in the figure. Furthermore, the vehicle position in the previous frame can be obtained simultaneously with the detection of the presence of a moving vehicle in the aforementioned image; that is, the image position that first meets the detection condition for the presence of a moving vehicle. See [link to relevant documentation]. Figure 4 The region from the detected location of a vehicle to the rightmost edge of the image is considered the region of interest (ROI). This region is the area that displays only vehicle information. Therefore, by cropping from the location of the moving vehicle in the previous frame to the rightmost edge of the image, we can obtain an initial stitched image that only displays vehicles.

[0088] In step 140, the current frame image is divided into multiple columns of images. The division criteria are based on the size of the reference image. The reference image is moved from left to right and matched with the pixels of each column of images in turn to calculate the number of pixels in the reference image that match the pixels in each column of images.

[0089] For example, see Figure 3 and 5 The current frame image is divided into 11 columns according to the length of the reference image. The pixel matching calculation is performed on the reference image from column 1 to column 11. Assuming that the number of matching pixels in column 6 is the largest, the leftmost position of column 6 is taken as the stitching position.

[0090] In some implementations, step 140 further includes the following steps:

[0091] Step 141: Obtain the RGB values ​​of all pixels in the comparison image and the column image.

[0092] Step 142: Determine whether the pixels in each column of images match the pixels in the reference image based on the matching criteria.

[0093] Step 143: Count the number of pixels in each column that meet the matching conditions.

[0094] Determine whether pixels match using the following formula:

[0095] |R1-R2|+|G1-G2|+|B1-B2| <Thr1

[0096] Where R1, G1, and B1 are the RGB values ​​of pixels in the comparison image, R2, G2, and B2 are the RGB values ​​of pixels in a series of images, and Thr1 is the matching threshold.

[0097] It should be noted that steps 141, 142, and 143 are not shown in the figure. Furthermore, RGB values ​​represent the brightness values ​​of the three primary colors (red, green, and blue) for each pixel, and different colors have different RGB values. In this embodiment, an RGB value database is provided, containing all colors and their corresponding RGB values. After identifying the colors of pixels in the comparison image and the pixels in each column of the image, the corresponding RGB values ​​can be retrieved from the RGB value database for matching calculations. It is worth mentioning that in this embodiment, the matching threshold is set to 20.

[0098] For example, assuming the vehicle being detected is a bright silver vehicle, the RGB values ​​of a pixel at a certain location in the comparison image are R1 = 192, G1 = 192, and B1 = 192, while the RGB values ​​of the corresponding pixel in a series of images compared to the comparison image are R2 = 190, G2 = 187, and B2 = 194. The Thr1 value is set to 20. Substituting these values ​​into the above formula yields:

[0099] |192-190|+|192-187|+|192-194|<20

[0100] If the inequality holds true, then the pixel is a successfully matched pixel. Calculate the successfully matched pixels in each column of images using the method described above, and then count the number of successfully matched pixels in each column of images.

[0101] After step 140 counts the number of pixels in each column of images that match the pixel in the reference image, the total number of pixels in each column of images is then counted. This pixel count can be performed using existing algorithms. After obtaining the total number of pixels in each column of images and the number of successfully matched pixels, the percentage of successfully matched pixels in a column of images relative to all pixels in that column is determined.

[0102] In some implementations, the above ratio is determined according to the following formula:

[0103]

[0104] Where S is the percentage of successfully matched pixels in a column of images out of all pixels in the column of images, X is the number of successfully matched pixels, and P is the total number of pixels in the column of images.

[0105] This formula can be used to calculate the matching degree between each column of images and the control image. The higher the ratio, the higher the matching degree.

[0106] After calculating the percentage of successfully matched pixels in each column of images, the column with the highest ratio is selected, representing the highest match with the reference image. The leftmost position of this selected image is then used as the stitching point to stitch the current frame image and the initial stitched image together. By finding the best match with the previous frame for stitching, the final vehicle stitched image will be more complete.

[0107] Secondly, embodiments of this application also provide a device for stitching together panoramic images of a vehicle's side profile.

[0108] Reference Figure 6 The device includes an acquisition module 210 for acquiring multiple consecutive images of the image acquisition area, a stitching discrimination module 220 for determining whether to stitch the current frame image and the previous frame image, and a stitching module 230 for stitching the current frame image and the previous frame image.

[0109] The following sections will introduce the acquisition module 210, the splicing discrimination module 220, and the splicing module 230 respectively.

[0110] The acquisition module 210 includes a shooting unit 221, which can be a camera or the like, located in the image acquisition area and capable of continuous shooting. The shooting unit 221 is positioned in front of the highway toll station entrance and on the left side of the direction of entry into the toll station, so that the captured images show the vehicle's direction of travel from right to left. In some embodiments, the device also includes a storage unit 240, connected to the shooting unit 221. The shooting unit 221 uploads the real-time captured images to the storage unit 240, and the storage unit 240 saves the images in real time.

[0111] The stitching discrimination module 220 is connected to the storage unit 240. While the image is being uploaded to the storage unit 240, the stitching discrimination module 220 retrieves the image in real time to determine whether stitching is required. Specifically, the stitching discrimination module 220 analyzes the state of the current frame image and the state of the previous frame image. It should be noted that the state of both the current and previous frame images includes a start stitching state, a stitching state, an ignore state, and a finish stitching state. After determining the state of the current frame image, the stitching discrimination module 220 saves the state of the current frame image as a condition for determining the state of the next frame image.

[0112] Specifically, when the previous frame was ignored and a moving vehicle exists in the current frame, the stitching discrimination module 220 outputs the current frame image as being in the "start stitching" state. When the previous frame was in the "start stitching" or "stitching" state and a moving vehicle exists in the current frame, the stitching discrimination module 220 outputs the current frame image as being in the "stitching" state. When the previous frame was in the "stitching" or "skipped" state and no moving vehicle exists in the current frame, the stitching discrimination module 220 outputs the current frame image as being in the "skipped" state. When the previous frame was in the "stitching" state and no moving vehicle exists in the current frame, the stitching discrimination module 220 outputs the current frame image as being in the "end stitching" state. The stitching discrimination module 220 is connected to the stitching module 230, and the stitching discrimination module 220 outputs the discrimination result to the stitching module 230.

[0113] In some embodiments, the device further includes a motion detection module 250, which is connected to the storage unit 240, the stitching discrimination module 220, and the stitching module 230. The motion detection module 250 is used to detect whether there is a moving vehicle in the image uploaded by the shooting unit 221 and upload the detection result to the stitching discrimination module 220.

[0114] Specifically, the motion detection module 250 includes an acquisition unit 251, a calculation unit 252, and a judgment unit 253. After the capturing unit 221 uploads an image to the storage unit 240, the acquisition unit 251 retrieves the image from the storage unit 240 and sequentially identifies and acquires the number of pixels in each column of the image from left to right. The acquisition unit 251 outputs the number of pixels in each column to the calculation unit 252. The calculation unit 252 also sequentially calculates the sum of the number of pixels in the three adjacent columns of the current frame image from left to right and outputs the result to the judgment unit 253. The judgment unit 253 determines whether there is a moving vehicle in the current frame image based on whether the received data meets the following judgment condition:

[0115] SUM>H*R

[0116] Where SUM is the sum of the number of pixels in three adjacent columns, H is the height of the current frame image, and R is the ratio threshold. In this embodiment, the ratio threshold can be 1.2. The judgment unit 253 outputs the judgment result to the stitching discrimination module 220, and simultaneously outputs the position of the first detected moving vehicle to the stitching module 230.

[0117] The stitching module 230 includes a start stitching unit 231, a stitching unit 232, and a stitched image display unit 233, all of which are interconnected. When the stitching discrimination module 220 determines that the current frame image is in the start stitching state, it outputs a start stitching command to the start stitching unit 231. Upon receiving the start stitching command, the start stitching unit 231 retrieves the current frame image from the storage unit 240. Simultaneously, the start stitching unit 231 receives the position of the first detected moving vehicle from the judgment unit 253. The start stitching unit 231 then cuts from this position to the right, thus obtaining the initial stitched image. The start stitching unit 231 then outputs the initial stitched image to the stitching unit 232 as the basis for stitching the final image.

[0118] When the stitching discrimination module 220 determines that the current frame image is in stitching mode, it outputs a stitching command to the stitching unit 232. Upon receiving the stitching command, the stitching unit 232 retrieves the current frame image from the storage unit 240. The stitching unit 232 first selects a reference image from the initial stitched image in the saved previous frame image as the selection condition for choosing the stitching position of the current frame image. The reference image is selected as the image with a preset length from right to left in the initial stitched image. It should be noted that the preset length is selected based on the image resolution; one-tenth of the image width can be used as the preset length. After selecting the reference image, the stitching unit 232 divides the current frame image into multiple columns, each column having the same length as the reference image. After division, the reference image is matched with the pixels of each column from left to right, and the number of matching pixels in the reference image with pixels in each column is calculated.

[0119] Specifically, in this embodiment, an RGB value database is provided, containing all colors and their corresponding RGB values. The splicing unit 232 retrieves the RGB values ​​of the reference image and the RGB values ​​of each column of images from the RGB value database, and calculates whether the pixels of the reference image match the pixels of a column of images. Specifically, the matching of pixels is determined according to the following formula:

[0120] |R1-R2|+|G1-G2|+|B1-B2| <Thr1

[0121] Wherein, R1, G1, and B1 are the RGB values ​​of pixels in the comparison image, R2, G2, and B2 are the RGB values ​​of pixels in a series of images, and Thr1 is the matching threshold. In this embodiment, the matching threshold is set to 20.

[0122] The stitching unit 232 calculates the number of pixels in each column of the image that match the pixels in the reference image. The stitching unit 232 also counts the total number of pixels in each column of the image; this counting can be performed using existing algorithms.

[0123] After obtaining the total number of pixels in each column of images and the number of successfully matched pixels, the stitching unit 232 determines the proportion of successfully matched pixels in a column of images out of all pixels in that column, and selects the stitching position for the current frame image. The proportion is determined using the following formula:

[0124]

[0125] Where S is the percentage of successfully matched pixels in a column of images out of all pixels in the column of images, X is the number of successfully matched pixels, and P is the total number of pixels in the column of images.

[0126] By calculation, the matching degree between each column of images and the reference image can be obtained. It should be noted that the larger the ratio, the higher the matching degree with the reference image. Therefore, the stitching unit 232 selects the leftmost position of the column of images with the largest ratio as the stitching position through comparison. After finding the stitching position, the stitching unit 232 stitches the current frame image with the initial stitched image.

[0127] The stitching unit 232 outputs the final stitched image to the stitched image display unit 233. The stitched image display unit 233 can be any electronic device capable of displaying images to display the final stitched vehicle image. Operators can visually see the number of axles of the vehicle through the stitched image display unit 233.

[0128] Thirdly, this application provides an electronic device, such as... Figure 7 As shown, Figure 7 The illustrated electronic device 300 includes a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302. Optionally, the electronic device 300 may also include a transceiver 304. It should be noted that in practical applications, the transceiver 304 is not limited to one type, and the structure of this electronic device 300 does not constitute a limitation on the embodiments of this application.

[0129] Processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0130] Bus 302 may include a pathway for transmitting information between the aforementioned components. Bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 302 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0131] The memory 303 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0132] The memory 303 is used to store application code that executes the solution of this application, and its execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the content shown in the foregoing method embodiments.

[0133] Among them, electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0134] The above description is merely a preferred example of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing application concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions claimed in this application.

Claims

1. A method for stitching together panoramic side views of a vehicle, characterized in that, include: Acquire multiple consecutive images from the image acquisition area; Based on the current frame image and the previous frame image, determine whether to stitch the current frame image and the previous frame image together; If so, a reference image is selected from the previous frame image, and the reference image is an image of a preset length from the rightmost side of the previous frame image to the left. The current frame image is divided into multiple columns of images, and the number of pixels in the comparison image that match the pixels in one column of images is determined. Determine the percentage of successfully matched pixels in a column of images out of all pixels in that column of images; When the ratio is at its maximum, the leftmost position of the column of images is taken as the stitching position; The current frame image is stitched together with the previous frame image starting from the stitching position; The step of determining whether to stitch the current frame image and the previous frame image together based on the current frame image includes: Analyze the state of the current frame image and the state of the previous frame image. The state of the current frame image and the state of the previous frame image both include the start stitching state, the stitching state, the ignore state, and the end stitching state. When the previous frame image is ignored and there is a moving vehicle in the current frame image, the current frame image is in the state of starting stitching. When the previous frame image is in the start stitching state or stitching state and there is a vehicle in the current frame image, then the state of the current frame image is the stitching state; When the previous frame image is in a stitched state or a skipped state and the current frame image does not contain moving vehicles, the output state of the current frame image is the skipped state; When the previous frame image is in a stitched state and there are no moving vehicles in the current frame image, the output state of the current frame image is the stitching end state; Detecting whether a moving vehicle exists in the current frame image includes: Calculate the sum of the number of pixels in three adjacent columns of the current frame image, and determine whether the sum of the number of pixels satisfies the following condition. If it does, then there is a moving vehicle in the current frame image: SUM>H*R Where SUM is the sum of the number of pixels in the three adjacent columns, H is the height of the current frame image, and R is the ratio threshold; The calculation of the sum of the number of pixels in the three adjacent columns of the current frame image includes: calculating the number of pixels in each column of the current frame image, traversing from the leftmost column of pixels in the current frame image to the right, and counting the sum of the number of pixels in the three adjacent columns in turn, with a ratio threshold of 1.

2.

2. The method according to claim 1, characterized in that, Selecting a comparison image from the previous frame includes: Obtain the position of the moving vehicle in the previous frame image; Cut out the image from the location to the right to obtain the initial stitched image of the moving vehicle; The image of a predetermined length, taken from the rightmost side of the initial stitched image, is used as the reference image.

3. The method according to claim 1, characterized in that, Determining the number of pixels in the comparison image that match pixels in a column of images includes: Obtain the RGB values ​​of all pixels in the comparison image and the column of images; Determine whether pixels match using the following formula: |R1-R2|+|G1-G2|+|B1-B2| <Thr1 Where R1, G1, and B1 are the RGB values ​​of pixels in the comparison image, R2, G2, and B2 are the RGB values ​​of pixels in a column of images, and Thr1 is the matching threshold. Count the number of pixels that satisfy the above formula.

4. The method according to claim 3, characterized in that, The determination of the proportion of successfully matched pixels in the column of images out of all pixels in the column of images includes: The ratio is determined according to the following formula: Where S is the percentage of successfully matched pixels in a column of images out of all pixels in the column of images, X is the number of successfully matched pixels, and P is the total number of pixels in the column of images.

5. A device for stitching together panoramic side views of a vehicle, used to implement the method of claim 1, characterized in that, The device includes: The acquisition module is used to acquire multiple consecutive images from the image acquisition area; The splicing discrimination module is used to determine whether to splice the current frame image and the previous frame image based on the current frame image and the previous frame image; The stitching module is used to select a reference image from the previous frame image, wherein the reference image is an image of a preset length extending from the rightmost side of the previous frame image to the left; divide the current frame image into multiple columns of images; determine the number of pixels in the reference image that match pixels in a column of images; determine the percentage of successfully matched pixels in a column of images among all pixels in that column; when the percentage is the largest, select the leftmost side of the column of images as the stitching position; and stitch the current frame image with the previous frame image starting from the stitching position. Determining the number of pixels in the comparison image that match pixels in a column of images includes: Obtain the RGB values ​​of all pixels in the comparison image and the column of images; Determine whether pixels match using the following formula: |R1-R2|+|G1-G2|+|B1-B2| <Thr1 Where R1, G1, and B1 are the RGB values ​​of pixels in the comparison image, R2, G2, and B2 are the RGB values ​​of pixels in a column of images, and Thr1 is the matching threshold. Count the number of pixels that satisfy the above formula; The determination of the proportion of successfully matched pixels in the column of images out of all pixels in the column of images includes: The ratio is determined according to the following formula: Where S is the percentage of successfully matched pixels in a column of images out of all pixels in the column of images, X is the number of successfully matched pixels, and P is the total number of pixels in the column of images.

6. The apparatus according to claim 5, characterized in that, The splicing discrimination module is specifically used for: Analyze the state of the current frame image and the state of the previous frame image. The state of the current frame image and the state of the previous frame image both include the start stitching state, the stitching state, the ignore state, and the end stitching state. When the previous frame image is ignored and there is a moving vehicle in the current frame image, the current frame image is in the state of starting stitching. When the previous frame image is in the start stitching state or stitching state and there is a vehicle in the current frame image, then the state of the current frame image is the stitching state; When the previous frame image is in a stitched state or a skipped state and the current frame image does not contain moving vehicles, the output state of the current frame image is the skipped state; When the previous frame image is in a stitched state and there are no moving vehicles in the current frame image, the output state of the current frame image is the "stitching ended" state.

7. The apparatus according to claim 5, characterized in that, The device also includes a motion detection module, which calculates the sum of the number of pixels in three adjacent columns of the current frame image and determines whether the sum of the number of pixels satisfies the following condition: If it does, then there is a moving vehicle in the current frame image. SUM>H*R Where SUM is the sum of the number of pixels in the three adjacent columns, H is the height of the current frame image, and R is the ratio threshold.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 4.

Images