Image processing method and device for remotely driving vehicle

Abstract

The invention discloses an image processing method and device for a remote driving vehicle, and the method comprises the following steps: correcting a to-be-spliced image, and restoring the real scene information; projecting the corrected image to a cylindrical surface with the same focal length; feature point matching is carried out by using an SIFT algorithm, a coordinate conversion relation between the two images is calculated according to a matching result, and the two images are projected into the same coordinate system based on the coordinate conversion relation; brightness error correction is carried out on the image, an energy function used for searching for a suture line is optimized, the optimal suture line is obtained in combination with a dynamic programming algorithm, and the processing delay of the dynamic programming algorithm is optimized; carrying out image splicing; and according to the brightness difference of the image on the suture line, adjusting the splicing result image, and eliminating the suture line. According to the technical scheme disclosed by the invention, splicing seams can be effectively eliminated, the color consistency is ensured, image ghosting is thoroughly eradicated, the processing speed and the image quality of multi-view visual splicing are improved, and reliable visual support is provided for remote driving.

Description

Technical Field

[0001] The present invention relates to the technical field of visual image splicing, and more specifically, to an image processing method and device for remote driving a vehicle. Background Art

[0002] With the rapid development of intelligent and unmanned driving technology, remote driving technology has become an important research direction as a safety solution for unmanned driving. Remote driving technology allows operators to operate vehicles in real time through a remote control platform, playing an important role in unmanned and automated road transportation. However, remote driving faces many technical challenges, one of which is how to ensure that operators can obtain sufficient environmental information to make accurate driving decisions. Traditional monocular cameras provide limited viewing angles and cannot meet the visual requirements in complex driving scenarios. Therefore, the demand for image stitching solutions based on multi-viewing is also increasing.

[0003] At present, multi-view visual stitching solutions generate a complete, wide-angle view of the driving environment by stitching image data collected from multiple cameras, providing operators with more visual information and enhancing their perception ability. This can not only effectively expand the operator's field of view, but also reduce visual blind spots, and improve the safety and efficiency of remote driving. In addition, multi-view visual stitching can also be fused with other sensor data to further improve the accuracy and real-time performance of driving decisions. There are many disadvantages in the current common image stitching solutions. Traditional image fusion methods directly perform weighted fusion of the pixel values ​​of two images, such as the maximum value method, weighted average method, and gradual weighted average method. When facing large parallax images or camera screens that are not synchronized, ghosting and other problems are prone to occur. The image stitching solution based on the best stitching line has become an important research direction in the field of image stitching in recent years because it can eliminate ghosting to a certain extent. For example: combining weighted average fusion on the basis of the best stitching line for graphic stitching, improving weighted average fusion method for image stitching, combining the best stitching line and improved fade-in and fade-out method for image stitching, etc. The above methods are essentially a combination of optimal seam line stitching and image weighted fusion. No matter how the weights are chosen, as long as the overlapping area is superimposed with the pixel values ​​of two parallax images, even if the ghosting can be reduced to a certain extent according to the above various improved schemes, it cannot be completely eradicated.

[0004] Therefore, how to provide an image processing method for remote driving vehicles that can effectively eliminate stitching seams and ensure color consistency, eradicate image ghosting, improve the processing speed and image quality of multi-view stitching, and provide reliable visual support for remote driving has become a technical problem that needs to be urgently solved by technicians in this field. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides an image processing method for remote driving vehicles, which can effectively eliminate stitching seams and ensure color consistency, eradicate image ghosting, improve the processing speed and image quality of multi-view stitching, and provide reliable visual support for remote driving.

[0006] The technical solution provided by the present invention is as follows:

[0007] The present invention provides an image processing method for a remote-controlled vehicle, in particular, a multi-view stitching method suitable for a remote-controlled vehicle, comprising the following steps: S1, distortion correction: correcting the image to be stitched and restoring the real scene information; S2, cylindrical projection: projecting the corrected image onto a cylinder with the same focal length; S3, image registration: using a SIFT algorithm to perform feature point matching, calculating the coordinate transformation relationship between two images according to the matching result, and projecting the two images into the same coordinate system based on the coordinate transformation relationship; S4, seam line search and stitching: performing brightness error correction on the image to be stitched, then optimizing an energy function for searching for seams, combining a dynamic programming algorithm to obtain an optimal seam line, optimizing the processing delay of the dynamic programming algorithm for searching for seams, and then performing image stitching according to the optimal seam line; S5, eliminating seam lines: finally, based on the brightness difference between the two stitched images on the seam line, adjusting the stitching result image to eliminate the seam lines.

[0008] Furthermore, in a preferred embodiment of the present invention, in step S1, the distortion correction adopts a checkerboard calibration algorithm, and the specific steps include:

[0009] S101, taking multiple groups of chessboard images at different angles, extracting corner points to calculate a homography matrix, and using the homography matrix to calculate camera internal and external parameters;

[0010] S102, then optimizing the camera internal and external parameters using maximum likelihood estimation;

[0011] S103, then calculate the distortion coefficient, and perform maximum likelihood estimation on all parameters again;

[0012] S104. Finally, the image is subjected to distortion correction using the required parameters.

[0013] Furthermore, in a preferred embodiment of the present invention, in step S2, the specific steps of the cylindrical projection include:

[0014] S201, mapping the images to be stitched into a plane coordinate system;

[0015] S202, obtaining the plane coordinates of each pixel in the image to be stitched, and determining the conversion relationship between the pixel coordinates before and after projection:

[0016]

[0017] Where W is the width of the original image, H is the height of the original image, f is the focal length, (x, y) and (x′, y′) are the coordinates before and after the cylindrical projection respectively;

[0018] S203: Based on the conversion relationship, map each pixel in the image from the plane coordinate system to a virtual cylinder.

[0019] Furthermore, in a preferred embodiment of the present invention, in step S3, the specific steps of image registration include:

[0020] S301, constructing a scale space, simulating multi-scale features of the image, and preliminarily detecting key points in the image;

[0021] S302, accurately locating the key points, removing points with low contrast and points with high response near the edge, and then assigning directions to the key points;

[0022] S303, in the area around each of the key points, calculating the direction and magnitude of the local gradient, and converting it to generate a feature descriptor;

[0023] S304, comparing the feature descriptors in the images to be stitched, selecting high-quality matching pairs through the RANSAC algorithm, and solving the homography matrix H representing the coordinate transformation relationship between the images based on the high-quality matching pairs;

[0024] S305 . According to the homography matrix H, project the image into the same coordinate system.

[0025] Furthermore, in a preferred embodiment of the present invention, in step S4, the step of correcting brightness errors of the images to be stitched includes:

[0026] Step 1: Obtain the overlapping area of ​​the two images to be stitched, and divide the overlapping area into N areas from top to bottom;

[0027] Step 2: respectively calculating the grayscale mean differences of the images to be stitched in corresponding areas, and using the grayscale mean differences to represent the brightness differences of the images to be stitched in corresponding areas;

[0028] Step 3: Based on the brightness difference, adjust the grayscale value of each point in the area of ​​the two images to be stitched to correct the brightness error of the image.

[0029] Furthermore, in a preferred embodiment of the present invention, the adjustment formula for the grayscale value of each point in the corresponding area of ​​the two images to be spliced ​​is specifically:

[0030]

[0031] In the formula, and is the grayscale value of the two images to be stitched after adjustment at any point in the region, I1(x, y) and I2(x, y) are the original grayscale values ​​corresponding to the point, I 1average and I 2average is the original grayscale mean of the two images to be stitched in this area.

[0032] Furthermore, in a preferred embodiment of the present invention, in step S4, the specific steps of optimizing the energy function include:

[0033] Step 1: Optimize the color difference intensity value: determine the gray value of the image to be stitched at point (x, y), and obtain the gray difference of 9 adjacent points centered at point (x, y);

[0034] According to the grayscale difference, the optimized energy function E is obtained. * The color difference intensity value E in (x,y) color (x,y);

[0035] in Where I1(x, y) and I2(x, y) are the grayscale values ​​of the image to be stitched at point (x, y);

[0036] Step 2: Optimize the structural difference intensity value: Use the Sobel operator to calculate the structural value and improve the convolution kernel of the Sobel operator in the x and y directions respectively. and in and They are:

[0037]

[0038] Based on the convolution kernel and Get the improved structural difference strength value E geometry (x,y), where

[0039] Step 3: Determine the color difference intensity value weight α and the structure difference intensity value weight β, and combine the color difference intensity value E color (x,y) and structural difference intensity value E geometry (x,y), get the optimized energy function E * (x,y), where E * (x,y)=αE color (x,y)+βE geometry (x,y).

[0040] Furthermore, in a preferred embodiment of the present invention, in step S4, the specific steps of obtaining the optimal suture line by combining the dynamic programming algorithm include:

[0041] Step 1: Initialization: The overlapping area of ​​the images to be stitched is regarded as an energy matrix, and the points in the first row of the energy matrix are taken as the starting minimum total energy value path;

[0042] Step 2: Dynamic programming: traverse row by row, trace back for each row, calculate the minimum total energy value to reach each point in the row, and record whether the source is from the left, middle, or right direction of the previous row;

[0043] Step 3: Backtrack the seam line: After traversing row by row, compare the minimum total energy value of each point in the last row, select the point with the smallest total value for backtracking, and obtain the best seam line by backtracking from the last row to the first row through the path matrix recording the source.

[0044] Further, in a preferred embodiment of the present invention, in step S4, the specific steps of optimizing the processing delay of the dynamic programming algorithm to find the suture line include:

[0045] Step 1, coarse sampling: downsample the two images to be stitched, find the stitching line after downsampling based on the dynamic programming algorithm, and obtain the coarse stitching line;

[0046] Step 2: Restoration: Restoring the coordinates of the thick seam line to the coordinate system of the original image to obtain the thick seam line relative to the original image;

[0047] Step 3: Fine sampling: Taking the coarse stitching line relative to the original image as the baseline, a dynamic programming algorithm is used to find the stitching line in the area adjacent to the baseline, and finally the best stitching line corresponding to the original image to be stitched is obtained.

[0048] Furthermore, in a preferred embodiment of the present invention, in step S5, the specific steps of eliminating the suture line include:

[0049] S501, calculating the BGR three-channel color difference of each point on the suture line;

[0050] S502, according to the calculation result of the color difference, keeping the pixel value on the left side of the stitching line on the spliced ​​image unchanged, and setting a transition area on the right side of the stitching line;

[0051] S503, determining adjustment data based on the three-channel color difference values ​​of a plurality of points on the stitching line adjacent to a point above and below a point in the transition zone, and obtaining an adjustment formula:

[0052]

[0053]

[0054] In the formula, the points on the 2k upper and lower adjacent suture lines corresponding to a point in the right transition zone are taken as point i, d i is the distance from point i to a point in the transition zone, w i is inversely proportional to the distance, W i is the weight of point i at a point in the transition zone after normalization, b idiff , g idiff 、r idiff are the BGR three-channel color differences of point i in the two images to be stitched, and d m is the transition zone width, d n is the distance between a point in the right transition zone and a point on the same suture line, B idiff , G idiff , R idiff The pixel values ​​that should be increased by the three channels at a certain point in the right transition area;

[0055] S504, according to the adjustment formula, obtain the pixel values ​​that should be increased in the three channels of a certain point in the right transition area, and adjust the pixel value of the point in the transition area accordingly, and then traverse each point in the transition area to eliminate the seam line.

[0056] In addition, the present invention also provides an image processing device for remote driving of a vehicle, comprising: a computer program, the computer program is used to execute the image processing method for remote driving of a vehicle as described above; a memory, the memory is used to store the computer program; and a processor, the processor is used to execute the computer program, which also has the above-mentioned technical effects.

[0057] In summary, the present invention provides an image processing method and device for remote driving vehicles, comprising the following steps: S1, distortion correction: correcting the image to be spliced ​​and restoring the real scene information; S2, cylindrical projection: projecting the corrected image onto a cylinder with the same focal length; S3, image registration: using the SIFT algorithm to match feature points, calculating the coordinate transformation relationship between the two images according to the matching results, and projecting the two images into the same coordinate system based on the coordinate transformation relationship; S4, seam line search and stitching: performing brightness error correction on the image to be spliced, then optimizing the energy function used to find the seam line, combining the dynamic programming algorithm to obtain the best seam line, and optimizing the processing delay of the dynamic programming algorithm to find the seam line, and then performing image stitching according to the best seam line; S5, eliminating the seam line: finally, based on the brightness difference between the two spliced ​​images on the seam line, adjusting the splicing result image to eliminate the seam line.The image processing method for remote driving vehicles is not based on weighted image fusion, but adjusts the image brightness through the brightness difference of the stitching line, thereby completely eradicating the image ghosting problem under the premise of effectively eliminating the stitching seam and ensuring color consistency; in the steps of the image processing method for remote driving vehicles, the image needs to be firstly subjected to distortion correction, the purpose of which is to eliminate the geometric distortion in the image so that the images taken by each camera can be better aligned, thereby achieving high-quality stitching; then, considering that the matched feature points are concentrated in the overlapping area of ​​the two images, the homography matrix obtained based on the feature point matching result is only for The overlapping area has a better effect. For the edge part of the non-overlapping area, obvious geometric deformation usually occurs in the area far away from the overlapping area. Therefore, in step S2, the image to be stitched needs to be pre-processed with cylindrical projection to reduce the edge deformation of the stitched image; after the cylindrical projection, the two images to be stitched need to be registered, and the images to be stitched obtained from different perspectives in the same scene are converted into the same coordinate system. In step S3, the corresponding feature point matching pairs in the two images are found by using the SIFT algorithm, and the coordinate transformation relationship is obtained from the feature point matching pairs, and the two images are projected into the same coordinate system for subsequent images. After the image is projected, the next step is to stitch the image. In this method, the best stitching line is used to stitch the image. The best stitching line is used to avoid the possible ghosting area. Therefore, in step S4, the best stitching line for image stitching is searched. The best stitching line is found in the overlapping area of ​​the two images. The alignment degree of a certain point is represented by setting an energy function. The path with the smallest total value of the energy function is selected as the best stitching line. In step S4, in order to solve the stitching line search error caused by brightness difference, the image is firstly brightness corrected. After processing, the energy function is optimized for the color difference intensity value and the structure difference intensity value, and the optimal seam line is obtained by combining the dynamic programming algorithm. Since the calculation amount of the dynamic programming algorithm is larger, the processing delay has increased significantly. In order to reduce the processing delay, step S4 improves the algorithm flow when searching for the seam line, which can greatly reduce the processing delay of searching for the optimal seam line while ensuring the quality of searching for the seam line, so that the image can be spliced ​​based on the optimal seam line; after the image is spliced ​​according to the seam line, the seam line can be eliminated by adjusting the splicing result image according to the brightness difference on the seam line, so that the color transition is more natural. It can be seen that the technical solution involved in the present invention can effectively eliminate the splicing seams and ensure color consistency compared to the prior art, completely eradicate image ghosting, improve the processing speed and image quality of multi-eye vision splicing, and provide reliable visual support for remote driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 A flowchart of the steps of the image processing method for remote driving a vehicle provided by an embodiment of the present invention;

[0060] Figure 2 The chessboard original image for distortion correction provided by an embodiment of the present invention;

[0061] Figure 3 The picture after distortion correction provided by the embodiment of the present invention;

[0062] Figure 4 A schematic top view of the cylindrical projection provided by an embodiment of the present invention;

[0063] Figure 5 The original image to be stitched provided by the embodiment of the present invention;

[0064] Figure 6 A mosaic of the non-cylinder projection provided by an embodiment of the present invention;

[0065] Figure 7 A cylindrical projection image corresponding to the original image to be stitched provided in an embodiment of the present invention;

[0066] Figure 8 A mosaic image after the cylindrical projection provided by an embodiment of the present invention;

[0067] Fig. 9 A matching result diagram of the SIFT algorithm provided in an embodiment of the present invention;

[0068] Fig.10 A schematic diagram of projecting the two images provided in an embodiment of the present invention onto the same coordinate system;

[0069] Fig.11 A schematic diagram of an image of a suture line directly searched according to an embodiment of the present invention;

[0070] Fig.12 A schematic diagram of an image of a stitching line found after simple brightness error correction provided by an embodiment of the present invention;

[0071] Fig.13 A schematic diagram of selecting stitching lines when the images to be stitched are at the same coordinate point A provided in an embodiment of the present invention;

[0072] Fig.14 A difference diagram of suture lines found before and after the energy function is improved provided in an embodiment of the present invention;

[0073] Fig.15 A schematic diagram of suture lines searched for in the control group 1 without an optimization process provided in an embodiment of the present invention;

[0074] Fig.16 A schematic diagram of suture lines searched under the optimization process of the control group 1 provided in an embodiment of the present invention;

[0075] Fig.17 A schematic diagram of a case where a stitching line in the stitched image provided by an embodiment of the present invention is not eliminated;

[0076] Fig.18 A schematic diagram of the stitched image after the seam lines are eliminated provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention.

[0078] It should be noted that when an element is referred to as being "fixed on" or "set on" another element, it can be directly on the other element or indirectly set on the other element; when an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.

[0079] It should be understood that the terms "length", "width", "up", "down", "front", "back", "first", "second", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0080] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" and "several" is two or more, unless otherwise clearly and specifically defined.

[0081] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with this technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantive technical significance. Any structural modification, change in proportion or adjustment of size, without affecting the effects and purposes that can be achieved by the present invention, should still fall within the scope of the technical contents disclosed by the present invention.

[0082] like Figures 1 to 18 As shown, the present invention provides an image processing method for remote driving vehicles, more specifically, a multi-eye vision stitching method suitable for remote driving vehicles, which specifically includes the following steps: S1, distortion correction: correct the image to be stitched and restore the real scene information; S2, cylindrical projection: project the corrected image onto a cylinder with the same focal length; S3, image registration: use the SIFT algorithm to match feature points, calculate the coordinate transformation relationship between the two images according to the matching results, and project the two images into the same coordinate system based on the coordinate transformation relationship; S4, seam line search and stitching: perform brightness error correction on the image to be stitched, then optimize the energy function used to find the seam line, combine the dynamic programming algorithm to obtain the best seam line, and optimize the processing delay of the dynamic programming algorithm to find the seam line, and then perform image stitching according to the best seam line;

[0083] S5. Eliminate stitching lines: Finally, based on the brightness difference between the two stitching images on the stitching lines, adjust the stitching result image to eliminate the stitching lines. The technical solution involved in the present invention can effectively eliminate stitching seams and ensure color consistency, completely eradicate image ghosting, improve the processing speed and image quality of multi-eye vision stitching, and provide reliable visual support for remote driving. In addition, the present invention also provides an image processing device for remote driving vehicles, including: a computer program, the computer program is used to execute the image processing method for remote driving vehicles as described above; a memory, the memory is used to store the computer program; a processor, the processor is used to execute the computer program, and also has the above-mentioned technical effects.

[0084] The image processing method for remote driving a vehicle disclosed in the present invention is specifically described below in conjunction with a specific embodiment. The image processing method for remote driving a vehicle includes (the following embodiment takes binocular vision stitching as an example):

[0085] S1. Distortion correction: Correct the image to be stitched and restore the real scene information.

[0086] Specifically, in a specific embodiment of the present invention, in step S1, the distortion correction adopts a checkerboard calibration algorithm, and its specific steps include: S101, shooting multiple groups of checkerboard images at different angles, extracting corner points to calculate the homography matrix, and using the homography matrix to calculate the camera internal and external parameters; S102, then using maximum likelihood estimation to optimize the camera internal and external parameters; S103, then calculating the distortion coefficient, and again performing maximum likelihood estimation on all parameters; S104, finally using the required parameters to perform distortion correction on the image.

[0087] Among them, in this embodiment, image distortion correction is a key preprocessing step for image stitching, especially for a wide-angle fisheye camera, the image captured by which is often distorted due to the physical properties of the lens; specifically, the image distortion includes: radial distortion and tangential distortion; the correction formula for the radial distortion is:

[0088] x dr =x(1+k1r 2 +k2r 4 +k3r 6 )

[0089] y dr =x(1+k1r 2 +k2r 4 +k3r 6 )

[0090] Where (x, y) is the coordinate of the undistorted image, (x dr ,y dr ) is the coordinate of the distorted image, and x 2 +y 2 =r 2 , k is the radial distortion coefficient; the correction formula for the tangential distortion is:

[0091] x dt =2p1xy+p2(r 2 +2x 2 )+1

[0092] y dt =2p2xy+2p1(r 2 +2y 2 )+1

[0093] Where (x, y) is the coordinate of the undistorted image, (x dt ,y dt ) is the coordinate of the distorted image, and x 2 +y 2 =r 2 , p is the tangential distortion coefficient. Radial distortion is the most common type of distortion, usually occurring in wide-angle lenses. Radial distortion causes straight lines in the image to bend at the edge of the lens, resulting in "barrel distortion" or "pincushion distortion"; tangential distortion is caused by the fact that the optical axis of the camera is not completely parallel to the image sensor, causing the points in the image to deviate from their theoretical positions. Tangential distortion causes objects in the image to be misaligned horizontally or vertically.

[0094] In multi-view stitching, a wide-angle fisheye camera is usually used to capture images. The main distortion form is radial distortion. Therefore, in step S1, a checkerboard calibration algorithm is used to correct the distortion, eliminate radial distortion, and restore the image. Figure 2 , 3 As shown, in the embodiment of the present invention, a checkerboard calibration algorithm is used to correct the camera distortion. Figure 2 These are the original images taken by some fisheye cameras for camera calibration. The main distortion form is barrel distortion. Figure 3 This is the picture after distortion correction. It can be seen that the image after eliminating barrel distortion is more in line with the real scene.

[0095] S2. Cylindrical projection: Project the corrected image onto a cylinder with the same focal length.

[0096] Specifically, in a specific embodiment of the present invention, in step S2, the specific steps of the cylindrical projection include: S201, mapping the image to be stitched into a plane coordinate system; S202, obtaining the plane coordinates of each pixel in the image to be stitched, and determining the conversion relationship between the pixel coordinates before and after projection:

[0097]

[0098] Wherein, W is the width of the original image, H is the height of the original image, f is the focal length, (x, y) and (x′, y′) are the coordinates before and after the cylindrical projection respectively; S203, based on the conversion relationship, map each pixel in the image from the plane coordinate system to the virtual cylinder.

[0099] In the embodiment of the present invention, the cylindrical projection in step S2 reduces the edge deformation of the spliced ​​image. The basic principle is to map each pixel in the image from a plane coordinate system to a virtual cylinder, such as Figure 4ABC is the image plane to be processed, and DBE is the surface obtained after projection. In the cylindrical projection step, the core is the projection transformation formula, that is, the conversion relationship. In this embodiment, the conversion relationship is obtained according to the original image width, height, focal length, coordinate value and other data, and then the image to be spliced ​​is mapped onto the cylinder.

[0100] In the embodiment of the present invention, Figure 5 , 6 , 7, and 8, Figure 5 is the original image to be spliced ​​in the embodiment, Figure 5 (a) is the basis, Figure 5 (b) Take the two images stitched together after perspective transformation as an example. Figure 6 This is a stitching image obtained without cylindrical projection. It can be seen that there is obvious stretching deformation on the right side of the image, and the transition of the stitching area is not smooth enough; Figure 7 (a) and Figure 7 (b) Figure 5 (a) and Figure 5 (b) The corresponding cylindrical projection image, at this time, image stitching is performed to obtain stitching Figure 8 , and there is no obvious deformation in the edge area. It can be seen that cylindrical projection has a significant improvement in the quality of the spliced ​​image.

[0101] S3. Image registration: Use SIFT algorithm to match feature points, calculate the coordinate transformation relationship between the two images according to the matching results, and project the two images into the same coordinate system based on the coordinate transformation relationship.

[0102] Step S3 is used to register the two images to be stitched, and convert the images to be stitched acquired from different viewing angles in the same scene into the same coordinate system.

[0103] Specifically, in a specific embodiment of the present invention, in step S3, the specific steps of the image registration include: S301, constructing a scale space, simulating the multi-scale features of the image, and preliminarily detecting the key points in the image; S302, accurately locating the key points, removing low-contrast points and points with higher responses near the edges, and then assigning directions to the key points; S303, calculating the direction and magnitude of the local gradient in the area around each of the key points, and converting it to generate a feature descriptor; S304, comparing the feature descriptors in the images to be spliced, screening out high-quality matching pairs through the RANSAC algorithm, and solving the homography matrix H representing the coordinate transformation relationship between the images based on the high-quality matching pairs; S305, projecting the images into the same coordinate system according to the homography matrix H.

[0104] Among them, in this embodiment, the first step of feature point extraction is to analyze the scale space of the image, because the appearance and shape of the object will change at different scales, and then feature point matching is performed to determine the positional relationship between them. Feature point matching is a key step in image registration. In step S3, the SIFT algorithm is used to match feature points. The SIFT algorithm has the advantages of scale invariance, rotation invariance, and strong noise resistance. High-quality matching pairs are screened out through the RANSAC algorithm, and the transformation matrix, that is, the homography matrix H, is obtained by using the high-quality matching pairs. The transformation matrix is ​​used to transform the image to be matched so that it coincides with the target image, that is, the image to be spliced ​​is projected into the same coordinate system.

[0105] Specifically, in a specific embodiment of the present invention, the solution formula of the homography matrix H is specifically:

[0106]

[0107] Where (x1, y1) and (x2, y2) are the corresponding matching points of the two images;

[0108] Solve to get the homography matrix

[0109] like Figure 7 As shown, in the embodiment of the present invention, the SIFT algorithm is used to Figure 7 (a) and Figure 7 (b) Perform feature point matching. The matching results are as follows: Fig. 9 As shown, the RANSAC algorithm is used to select high-quality matching pairs, and the two images are projected into the same coordinate system using the obtained homography matrix H. Fig.10 shown.

[0110] S4, seam line search and stitching: perform brightness error correction on the images to be stitched, then optimize the energy function used to find the seam line, combine the dynamic programming algorithm to obtain the best seam line, and optimize the processing delay of the dynamic programming algorithm to find the seam line, and then perform image stitching based on the best seam line.

[0111] In image stitching technology, the cause of ghosting is that when performing weighted fusion of images, the unaligned parts are fused using pixel values ​​of different images, resulting in ghosting. In an embodiment of the present invention, in order to eliminate ghosting, step S4 uses an optimal stitching line to perform image stitching, which optimizes the energy function and the dynamic programming algorithm to find the processing delay of the stitching line, so as to obtain the optimal stitching line, and use the optimal stitching line to avoid possible ghosting areas. Find the optimal stitching line, that is, find a stitching line with the highest degree of alignment in the overlapping area of ​​the two images. In an embodiment of the present invention, an energy function is set to represent the degree of alignment of a point, and the path with the smallest total value of the energy function is selected as the optimal stitching line. Finally, image stitching is performed based on the optimal stitching line.

[0112] Specifically, in a specific embodiment of the present invention, in step S4, the step of correcting the brightness error of the image to be stitched includes: step 1, obtaining the overlapping area in the two images to be stitched, and dividing the overlapping area into N areas from top to bottom; step 2, respectively calculating the difference in the grayscale means of the images to be stitched in the corresponding areas, and using the difference in the grayscale means to represent the brightness difference of the images to be stitched in the corresponding areas; step 3, adjusting the grayscale values ​​of each point in the area of ​​the two images to be stitched based on the brightness difference, and correcting the brightness error of the image.

[0113] Specifically, in a specific embodiment of the present invention, the adjustment formula for the grayscale value of each point in the corresponding area of ​​the two images to be spliced ​​is:

[0114]

[0115] In the formula, and is the grayscale value of the two images to be stitched after adjustment at any point in the region, I1(x, y) and I2(x, y) are the original grayscale values ​​corresponding to the point, I 1average and I 2average is the original grayscale mean of the two images to be stitched in this area.

[0116] In a specific embodiment of the present invention, due to the exposure difference of different cameras, different color difference intensity values ​​will appear in the same feature area of ​​the two images to be stitched, resulting in the error of avoiding the area when searching for the stitching line. In order to solve the stitching line search error caused by the brightness difference, the image needs to be corrected for brightness difference before optimizing the energy function. The brightness difference of the two images to be stitched in the area is represented by the difference in the grayscale mean of each point in the corresponding area of ​​the two images, and the grayscale value of each point in the area of ​​the two images to be stitched is adjusted based on the brightness difference, thereby completing the brightness correction. In the brightness error correction step, the image overlapping area must first be divided into N areas. Since the brightness difference between the two images in the overlapping area from top to bottom is not fixed, the brightness of the upper left image may be higher than that of the right image, and the brightness of the lower right image may be higher than that of the left image. Therefore, when selecting the parameter N, it is necessary to determine it according to the image parameters. If the value of N is too small, that is, the divided area is too large, then the grayscale mean difference in the area may not accurately reflect the brightness difference of each point in the area. If the value of N is too large, that is, the divided area is too small, due to the registration error between the two images, the overall grayscale mean of the area may be affected by the misalignment features, resulting in the difference in grayscale mean being unable to truly reflect the overall brightness difference between the two areas. After many experiments, it was found that for images to be spliced ​​with a height of 1080, the best effect is achieved by dividing using N=4. In an embodiment of the present invention, as shown in the attached Fig.11 This is a schematic diagram of the image of the suture line to be directly found, and its partial enlarged view, attached Fig.12 This is an image diagram of the stitching line found after a simple brightness error correction, and its local enlarged image. It can be clearly seen that the stitching line found directly avoids the sky area which is greatly affected by the brightness error but has a higher degree of alignment, and chooses to pass through the high-rise buildings with a lower degree of alignment. The stitching line found after a simple brightness error correction passes through the sky area correctly, obtaining a better stitching result.

[0117] Specifically, in a specific embodiment of the present invention, in step S4, the specific steps of optimizing the energy function include: Step 1, optimizing the color difference intensity value: determining the gray value of the image to be spliced ​​at point (x, y), and obtaining the gray difference of 9 adjacent points centered at point (x, y); according to the gray difference, obtaining the optimized energy function E * The color difference intensity value E in (x,y) color (x,y);

[0118] in

[0119] Where I1(x, y) and I2(x, y) are the grayscale values ​​of the image to be stitched at point (x, y);

[0120] Step 2: Optimize the structural difference intensity value: Use the Sobel operator to calculate the structural value and improve the convolution kernel of the Sobel operator in the x and y directions respectively. and in and They are:

[0121]

[0122] Based on the convolution kernel and Get the improved structural difference strength value E geometry (x,y), where

[0123] Step 3: Determine the color difference intensity value weight α and the structure difference intensity value weight β, and combine the color difference intensity value E color (x,y) and structural difference intensity value E geometry (x,y), get the optimized energy function E * (x,y), where E * (x,y)=αE color (x,y)+βE geometry (x,y).

[0124] In the embodiment of the present invention, the energy function E is composed of two parts: color difference and structure difference. The common E formula is: In the formula, represents the color difference intensity value of the pixel (x, y), α is the color difference intensity value weight, E g (x, y) represents the structural difference strength value of the pixel point (x, y), β is the structural difference strength value weight, and E g The solution formulas for (x,y) are: E g (x,y)=[S x *(I1(x,y)-I2(x,y))] 2 +[S y *(I1(x,y)-I2(x,y))] 2 , where I1(x, y) and I2(x, y) are the grayscale values ​​of the image to be stitched at point (x, y), S x and S y are the convolution kernels of the 3x3 Sobel operator in the x and y directions, S x and S y They are:

[0125]

[0126] Therefore, in the embodiment of the present invention, based on the common energy function E, the basic idea of ​​improving the energy function E is to optimize the color difference intensity value and the structure difference intensity value so that the calculated total intensity value can better represent the alignment degree of the image.

[0127] In the embodiment of the present invention, for the color difference intensity value, such as Fig.13 In the case of point A, Fig.13 Schematic diagram of selecting stitching lines when the images to be stitched are at the same coordinate point A. and E g (x, y) are both 0, and the total value of E is 0. In the original energy function E, the two images to be stitched are considered to be aligned at point A, resulting in an error in the selection of stitching lines. Therefore, the color difference intensity value needs to be adjusted, and the grayscale difference of the adjacent 9 points (including A itself) centered on point A is considered to obtain the color difference intensity value E. color (x, y) calculation formula, with the color difference intensity value E color (x, y) formula, the left and right upper corners and the left and right lower corners of point A all have large color differences, and the overall E color (x,y) is too large, and point A will be avoided when searching for the best seam line to achieve the optimal effect.

[0128] In the embodiment of the present invention, for the structural difference strength value, the original structural difference strength value E g (x, y) focuses on the positive weight, that is, the x direction focuses on the influence of the adjacent left and right points, and the y direction focuses on the influence of the adjacent upper and lower points. However, in a large number of image stitching experiments, it is found that the lateral weights of the four points in the upper and lower corners are equally important for the structural difference intensity value. Therefore, this algorithm increases the influence of the lateral weight and improves the convolution kernel in the x and y directions to obtain and Thus, the improved structural difference strength value E is obtained. geometry (x, y), combined with the color difference intensity value E color (x,y) and structural difference intensity value E geometry (x, y), and get the optimized energy function E * (x, y), and after a large number of experiments, α=0.5 and β=1 are taken.

[0129] In the embodiment of the present invention, Fig.14 As shown, Fig.14 (a) and Fig.14 (b) The energy function E based on the traditional energy function and the energy function E improved by this algorithm * It can be clearly seen that in practical applications, when faced with similar Fig.13In the case of * The obtained seams can avoid misaligned areas and achieve better stitching results.

[0130] Secondly, the algorithms used in the current visual field to find the best stitching line with the minimum total energy function value are mainly divided into two types: greedy algorithm and dynamic programming algorithm. The greedy algorithm is an algorithm that selects the minimum energy function by summing from top to bottom. Each point in the first row is used as the starting point of a stitching line, and it is extended in the left, middle, and right directions of the next row. The direction corresponding to the minimum energy value is obtained as the extension path. After going from top to bottom to the last row, the path with the minimum total energy value is the best stitching line. However, the greedy algorithm has a serious drawback. Since it only considers the minimum energy sum of the next step to the previous step from top to bottom, but does not consider the minimum total energy value of the whole, the greedy algorithm often loses the global optimal solution due to the current optimal solution. In this method, the dynamic programming algorithm is used to solve this problem.

[0131] Specifically, in a specific embodiment of the present invention, in step S4, the specific steps of combining the dynamic programming algorithm to obtain the best stitching line include: step one, initialization: regard the overlapping area of ​​the image to be stitched as an energy matrix, and take the points in the first row of the energy matrix as the starting minimum total energy value path; step two, dynamic programming: traverse row by row, trace back for each row, calculate the minimum total energy value reaching each point in the row, and record the source from the left, middle, or right direction of the previous row; step three, backtrack the stitching line: after the row-by-row traversal is completed, compare the minimum total energy values ​​of the points in the last row, select the point with the smallest total value for backtracking, and obtain the best stitching line by traversing the path matrix recording the source from the last row to the first row.

[0132] In the embodiment of the present invention, the dynamic programming algorithm ensures that the result is the global optimal solution by backtracking from bottom to top. However, compared with the greedy algorithm, the dynamic programming algorithm has a larger amount of calculation and a significantly increased processing delay. In order to ensure safety, remote driving has higher requirements for real-time processing of images. Therefore, in step S4, it is necessary to optimize the processing delay of the dynamic programming algorithm for finding the seam line.

[0133] Specifically, in a specific embodiment of the present invention, in step S4, the specific steps of optimizing the processing delay of the dynamic programming algorithm for finding the stitching line include: step one, coarse sampling: downsampling the two images to be stitched, and finding the stitching line after downsampling based on the dynamic programming algorithm to obtain a coarse stitching line; step two, restoration: restoring the coordinates of the coarse stitching line to the coordinate system of the original image to obtain a coarse stitching line relative to the original image; step three, fine sampling: taking the coarse stitching line relative to the original image as the baseline, performing a dynamic programming algorithm on the adjacent area of ​​the baseline to find the stitching line, and finally obtaining the best stitching line corresponding to the original image to be stitched.

[0134] Among them, the coarse seam line can avoid the ghosting part, but because the downsampled image lacks a lot of detail information compared to the original image, it is necessary to restore the coarse seam line to the original image, and use it as a reference to re-search the seam line in the adjacent area using the dynamic programming algorithm to ensure the quality of the final seam line.

[0135] In the embodiment of the present invention, ten sets of parallax images with an original size of 1920×1080 are stitched, and the processing delay and stitching line quality differences before and after the optimization process are compared. The optimization process uses 0.25 times downsampling in both the x-direction and the y-direction for coarse sampling, and the resulting data is shown in Table 1.

[0136]

[0137] Table 1 (a) Processing delay difference data before and after optimization process

[0138]

[0139] Table 1(b) Difference in total energy value before and after the optimization process

[0140]

[0141] Table 1(c) Energy peak difference data before and after the optimization process

[0142] Based on Table 1(a), it can be seen that after adopting the optimized process, the processing speed of finding sutures increased by an average of 363.012%, which is an excellent improvement in processing speed; based on Table 1(b), it can be seen that the average increase in total energy value is 14.628%, and the total energy value represents the overall quality of the sutures. The increase within 20% has almost no loss in the quality of the sutures; based on Table 1(c), it can be seen that the average increase in peak Emax is 6.749%, and peak Emax represents the largest abnormal point on the suture. Analyzing the data, it can be seen that after adopting the optimized process, the peak Emax has increased and decreased, but the overall change is not large and is within a reasonable range. Combined with the comparison results of control group 1 Fig.15 , 16,It can be seen that the algorithm using the ,optimization process can ensure the quality of finding the suture line while ,greatly reducing the processing delay of finding the best suture line.

[0143] S5. Eliminate seam lines: Finally, based on the brightness difference between the two stitched images on the seam line, adjust the stitched result image to eliminate the seam lines.

[0144] Specifically, in a specific embodiment of the present invention, in step S5, the specific steps of eliminating the stitching line include: S501, calculating the BGR three-channel color difference of each point on the stitching line; S502, according to the calculation result of the color difference, keeping the pixel value on the left side of the stitching line on the spliced ​​image unchanged, and setting a transition zone on the right side of the stitching line; S503, determining the adjustment data based on the three-channel color difference values ​​of several points on the stitching line adjacent to a point above and below a certain point in the transition zone, and obtaining the adjustment formula:

[0145]

[0146]

[0147] In the formula, the points on the 2k upper and lower adjacent suture lines corresponding to a point in the right transition zone are taken as point i, d i is the distance from point i to a point in the transition zone, w i is inversely proportional to the distance, W i is the weight of point i at a point in the transition zone after normalization, b idiff , g idiff 、r idiff are the BGR three-channel color differences of point i in the two images to be stitched, and d m is the transition zone width, d n is the distance between a point in the right transition zone and a point on the same suture line, B idiff , G idiff , R idiff The pixel values ​​that should be increased in the three channels at a certain point in the right transition zone respectively; S504, according to the adjustment formula, obtain the pixel values ​​that should be increased in the three channels at a certain point in the right transition zone respectively, and adjust the pixel value of the point in the transition zone accordingly, and then traverse each point in the transition zone to eliminate the stitching line.

[0148] In the embodiment of the present invention, the stitching line of the mosaic image in the parking lot scene is eliminated based on the above algorithm. Fig.17 , 18 As shown, the visible seams are successfully eliminated and the color transition is natural.

[0149] As described above, an embodiment of the present invention relates to an image processing method for remote driving vehicles, which is aimed at the problem that the existing visual stitching technology cannot completely eliminate ghosting and the image stitching quality is low. In the image processing method for remote driving vehicles, the image to be stitched is first processed, including distortion correction, cylindrical projection, and image registration based on feature points, and then the image is stitched, including using an improved energy function, brightness error correction, and optimizing the dynamic programming algorithm to find the image processing speed of the stitching line, and finally eliminating the stitching line based on the brightness difference of the stitching line and finally obtaining an excellent stitching result image to complete the multi-eye vision stitching operation. It can be seen that the technical solution involved in the present invention, compared with the prior art, can effectively eliminate the stitching seams and ensure color consistency, completely eradicate image ghosting, improve the processing speed and image quality of multi-eye vision stitching, and provide reliable visual support for remote driving.

[0150] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method for remote driving a vehicle, characterized in that: The following steps are involved: S1, distortion correction: correct the image to be stitched and restore the real scene information; S2, cylindrical projection: project the corrected image onto a cylindrical surface with the same focal length; S3, image registration: use SIFT algorithm to match feature points, calculate the coordinate transformation relationship between the two images according to the matching results, and project the two images into the same coordinate system based on the coordinate transformation relationship; S4, seam line search and stitching: perform brightness error correction on the image to be stitched, optimize the energy function used to find the seam line, combine the dynamic programming algorithm to obtain the best seam line, and optimize the processing delay of the dynamic programming algorithm to find the seam line, and stitch the images according to the best seam line; S5. Eliminate seam lines: Based on the brightness difference between the two stitched images on the seam line, adjust the stitched result image to eliminate the seam lines.

2. The image processing method for remote driving of a vehicle according to claim 1, characterized in that: In step S1, the distortion correction adopts a checkerboard calibration algorithm, and the specific steps include: S101, taking multiple groups of chessboard images at different angles, extracting corner points to calculate a homography matrix, and using the homography matrix to calculate camera internal and external parameters; S102, optimizing camera internal and external parameters using maximum likelihood estimation; S103, then calculate the distortion coefficient, and perform maximum likelihood estimation on all parameters again; S104: Use the required parameters to perform distortion correction on the image.

3. The image processing method for remote driving of a vehicle according to claim 1, characterized in that: In step S2, the specific steps of the cylindrical projection include: S201, mapping the images to be stitched into a plane coordinate system; S202, obtaining the plane coordinates of each pixel in the image to be stitched, and determining the conversion relationship between the pixel coordinates before and after projection: Where W is the width of the original image, H is the height of the original image, f is the focal length, (x, y) and (x′, y′) are the coordinates before and after the cylindrical projection respectively; S203: Based on the conversion relationship, map each pixel in the image from the plane coordinate system to a virtual cylinder.

4. The image processing method for remote driving of a vehicle according to claim 1, characterized in that: In step S3, the specific steps of image registration include: S301, constructing a scale space, simulating multi-scale features of the image, and preliminarily detecting key points in the image; S302, accurately locating the key points, removing points with low contrast and points with high response near the edge, and then assigning directions to the key points; S303, in the area around each of the key points, calculating the direction and magnitude of the local gradient, and converting it to generate a feature descriptor; S304, comparing the feature descriptors in the images to be stitched, selecting high-quality matching pairs through the RANSAC algorithm, and solving the homography matrix H representing the coordinate transformation relationship between the images based on the high-quality matching pairs; S305 . According to the homography matrix H, project the image into the same coordinate system.

5. The image processing method for remote driving of a vehicle according to claim 1, characterized in that: In step S4, the step of correcting brightness errors of the image to be stitched includes: Obtain an overlapping area in two images to be stitched, and divide the overlapping area into N areas from top to bottom; Calculating the grayscale mean differences of the images to be stitched in corresponding areas respectively, and using the grayscale mean differences to represent the brightness differences of the images to be stitched in corresponding areas; Based on the brightness difference, the grayscale value of each point in the area of ​​the two images to be stitched is adjusted to correct the brightness error of the image.

6. The image processing method for remote driving of a vehicle according to claim 5, characterized in that: The specific formula for adjusting the grayscale value of each point in the corresponding area of ​​the two images to be stitched is: In the formula, and is the grayscale value of the two images to be stitched after adjustment at any point in the region, I1(x, y) and I2(x, y) are the original grayscale values ​​corresponding to the point, I 1average and I 2average is the original grayscale mean of the two images to be stitched in this area.

7. The image processing method for remote driving of a vehicle according to claim 6, characterized in that: In step S4, the specific steps of optimizing the energy function include: Optimize the color difference intensity value: determine the gray value of the image to be stitched at point (x, y), and obtain the gray difference of the adjacent 9 points centered at point (x, y); According to the grayscale difference, the optimized energy function E is obtained. * The color difference intensity value E in (x,y) color (x,y); in Where I1(x, y) and I2(x, y) are the grayscale values ​​of the image to be stitched at point (x, y); Optimize the structural difference intensity value: Use the Sobel operator to calculate the structural value, and improve the convolution kernel of the Sobel operator in the x and y directions respectively and in and They are: Based on the convolution kernel and Get the improved structural difference strength value E geometry (x,y), where Determine the color difference intensity value weight α and the structure difference intensity value weight β, combined with the color difference intensity value E color (x,y) and structural difference intensity value E geometry (x,y), get the optimized energy function E * (x,y), where E * (x,y)=αE color (x,y)+βE geometry (x,y).

8. The image processing method for remote driving of a vehicle according to claim 7, characterized in that: In step S4, the specific steps of obtaining the optimal suture line by combining the dynamic programming algorithm include: Initialization: The overlapping area of ​​the images to be stitched is regarded as an energy matrix, and the points in the first row of the energy matrix are taken as the starting path with the minimum total energy value; Dynamic programming: traverse row by row, trace back for each row, calculate the minimum total energy value to reach each point in the row, and record whether the source is from the left, middle, or right direction of the previous row; Backtracking the suture line: after the row-by-row traversal is completed, the minimum total energy value of each point in the last row is compared, and the point with the smallest total value is selected for backtracking, and the best suture line is obtained by backtracking from the last row to the first row through the path matrix of the recorded source; and / or in step S4, the specific steps of optimizing the processing delay of the dynamic programming algorithm to find the suture line include: Coarse sampling: downsample the two images to be stitched, find the stitching line after downsampling based on the dynamic programming algorithm, and obtain the coarse stitching line; Restoration: restore the coordinates of the thick seam line to the coordinate system of the original image to obtain the thick seam line relative to the original image; Fine sampling: Taking the coarse seam line relative to the original image as the baseline, a dynamic programming algorithm is used to find the seam line in the area adjacent to the baseline, and finally the best seam line corresponding to the original image to be stitched is obtained.

9. The image processing method for remote driving of a vehicle according to any one of claims 1 to 8, characterized in that: In step S5, the specific steps of eliminating the suture line include: S501, calculating the BGR three-channel color difference of each point on the suture line; S502, according to the calculation result of the color difference, keeping the pixel value on the left side of the stitching line on the spliced ​​image unchanged, and setting a transition area on the right side of the stitching line; S503, determining adjustment data based on the three-channel color difference values ​​of a plurality of points on the stitching line adjacent to a point above and below a point in the transition zone, and obtaining an adjustment formula: In the formula, the points on the 2k upper and lower suture lines corresponding to a point in the right transition zone are taken as point i, d i is the distance from point i to a point in the transition zone, w i is inversely proportional to the distance, W i is the weight of point i at a point in the transition zone after normalization, b idiff , g idiff 、r idiff are the BGR three-channel color differences of point i in the two images to be stitched, and d m is the transition zone width, d n is the distance between a point in the right transition zone and a point on the same suture line, B idiff , G idiff , R idiff The pixel values ​​that should be increased by the three channels at a certain point in the right transition area; S504, according to the adjustment formula, obtain the pixel values ​​that should be increased in the three channels of a certain point in the right transition area, and adjust the pixel value of the point in the transition area accordingly, and then traverse each point in the transition area to eliminate the seam line.

10. An image processing device for remote driving of a vehicle, characterized in that: include: A computer program for executing the image processing method for remote driving a vehicle according to any one of claims 1 to 9; A memory, the memory being used to store the computer program; A processor is used to execute the computer program.

Images