Vehicle panoramic image generation method, electronic equipment and storage medium

Abstract

The invention relates to a vehicle panoramic image generation method, electronic equipment and a storage medium, and is applied to the technical field of vehicles, and the method comprises the steps: obtaining a chassis image and surrounding images of a vehicle at the current moment in real time; pixel coordinates of all pixel points of the chassis image are converted, a converted image is obtained, and the image view angle of the converted image is the same as that of the peripheral image; and fusing the converted image with the surrounding image to obtain a panoramic image of the vehicle. The problem that in the prior art, information lagging and inaccuracy exist in the actual condition of the vehicle bottom is solved.

Description

Technical Field

[0001] This application relates to the technical field of vehicles, and in particular, to a method for generating a panoramic image of a vehicle, an electronic device, and a storage medium. Background Art

[0002] With the development of science and technology, vehicles are used more and more commonly, and automotive consumers also pay more and more attention to the safety functions of the Advanced Driving Assistance System (ADAS). Among them, the transparent chassis function can enable the driver to intuitively perceive the position of the tires and obstacles such as potholes, stones, small animals or manhole covers on the road surface, which plays a huge role in eliminating the driver's vision blind area, avoiding potential safety hazards during driving, and improving the safety performance of the vehicle.

[0003] In the related art, the image of the vehicle bottom uses the previously stored image, which cannot reflect the actual condition of the vehicle bottom in real time, and there are problems of information lag and inaccuracy. Summary of the Invention

[0004] This application provides a method for generating a panoramic image of a vehicle, an electronic device, and a storage medium, so as to solve the problems of information lag and inaccuracy in the actual condition of the vehicle bottom in the prior art.

[0005] According to the first aspect of the embodiments of this application, a method for generating a panoramic image of a vehicle is provided, including:

[0006] Obtaining the chassis image and the surrounding images of the vehicle at the current moment in real time;

[0007] Converting the pixel coordinates of each pixel point of the chassis image to obtain a converted image, and the image view angle of the converted image is the same as that of the surrounding images;

[0008] Fusing the converted image with the surrounding images to obtain the panoramic image of the vehicle.

[0009] Optionally, converting the pixel coordinates of each pixel point of the chassis image to obtain a converted image includes:

[0010] Performing distortion correction on each pixel coordinate of the chassis image to obtain a corrected image;

[0011] Projecting the corrected image onto the ground coordinate system to obtain the converted image.

[0012] Optionally, performing distortion correction on each pixel coordinate of the chassis image to obtain a corrected image includes:

[0013] Obtaining the distortion coefficient and camera parameters of the camera that captures the chassis image;

[0014] Obtain the correction relationship between the undistorted image coordinates and the distorted image coordinates;

[0015] Substitute the pixel coordinates, distortion coefficients, and camera parameters of each pixel of the chassis image into the correction relationship to determine the undistorted coordinates of the pixel and obtain the corrected image.

[0016] Optionally, the correction relationship between the undistorted image coordinates and the distorted image coordinates includes:

[0017]

[0018] Among them,

[0019]

[0020] r d = θ(1 + k1θ 2 + k2θ 4 + k3θ 6 + k4θ 8 );

[0021] θ = arctan(r u / f);

[0022] Among them, x dst represents the abscissa in the undistorted coordinates, y dst represents the ordinate in the undistorted coordinates, x src represents the abscissa in the pixel coordinates, y src represents the ordinate in the pixel coordinates, k1, k2, k3, k4 represent different distortion coefficients, f represents the equivalent focal length, c x , c y respectively represent the abscissa and ordinate in the pixel coordinates of the camera optical center, f x , f y respectively represent the horizontal focal length and vertical focal length of the camera.

[0023] Optionally, the step of projecting the corrected image onto the ground coordinate system to obtain the converted image includes:

[0024] Obtain the height between the vehicle chassis and the ground and the tilt angle with the horizontal ground;

[0025] Obtain the internal parameter matrix of the camera that captured the chassis image;

[0026] Based on the height, tilt angle, and internal parameter matrix, determine the homography matrix;

[0027] Based on the homography matrix and the pixel coordinates of the rectified image, determine the homogeneous coordinates after projection, and determine the converted image based on the homogeneous coordinates.

[0028] Optionally, determining the homography matrix based on the height, tilt angle, and intrinsic matrix includes:

[0029]

[0030] Wherein, t = [0, 0, -h] T ;

[0031] Wherein, H represents the homography matrix, K represents the intrinsic matrix, θ represents the tilt angle, and h represents the height.

[0032] Optionally, determining the homogeneous coordinates after projection based on the homography matrix and the pixel coordinates of the rectified image includes:

[0033]

[0034] Wherein, ω represents the scaling factor of the homogeneous coordinates, H represents the homography matrix, X g and Y g respectively represent the abscissa and ordinate of the homogeneous coordinates, and x src and y src respectively represent the abscissa and ordinate of the pixel points of the rectified image.

[0035] Optionally, the fusing the converted image with the surrounding images to obtain the panoramic image of the vehicle includes:

[0036] Determine the chassis area of the vehicle;

[0037] Crop a cropped image including the chassis area from the converted image,

[0038] Stitch the positions with the same edge features of the cropped image and the surrounding images to obtain the panoramic image.

[0039] According to the second aspect of the embodiments of the present application, there is provided a device for generating a panoramic image of a vehicle, including:

[0040] An acquisition unit for real-time acquiring the chassis image and the surrounding images of the vehicle at the current moment;

[0041] A conversion unit for converting the pixel coordinates of each pixel point of the chassis image to obtain a converted image, and the image view angle of the converted image is the same as that of the surrounding images;

[0042] A fusion unit for fusing the converted image with the surrounding images to obtain the panoramic image of the vehicle.

[0043] According to a third aspect of the embodiments of the present application, a vehicle is provided, including a memory and a processor;

[0044] The memory is connected to the processor and is used for storing programs;

[0045] The processor is configured to implement the method for generating a panoramic image of a vehicle as described in the first aspect by running the program in the memory.

[0046] According to a fourth aspect of the embodiments of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is run by a processor, the method for generating a panoramic image of a vehicle as described in the first aspect is implemented.

[0047] According to a fifth aspect of the embodiments of the present application, a computer program product is provided, including computer program instructions. When the computer program instructions are run by a processor, the processor is caused to execute the method for generating a panoramic image of a vehicle as described in the first aspect.

[0048] The above technical solutions provided by the embodiments of the present application have the following advantages compared with the prior art: In the method provided by the embodiments of the present application, the chassis image and the surrounding images of the vehicle at the current moment are obtained in real time; the pixel coordinates of each pixel point of the chassis image are converted to obtain a converted image, and the image perspective of the converted image is the same as that of the surrounding images; the converted image is fused with the surrounding images to obtain the panoramic image of the vehicle. In this way, the obtained chassis image and the surrounding images are real-time images at the same moment. After image processing and fusion, a panoramic image including the vehicle chassis at the current moment can be obtained, which can reflect the actual condition of the vehicle bottom in real time, so that the information of the vehicle bottom and the surrounding of the vehicle can be fed back to the user in a timely and accurate manner, improving the safety during the driving process. Description of the Drawings

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0050] Figure 1 It is a schematic structural diagram among the functional modules of the vehicle provided by an embodiment of the present application;

[0051] Figure 2 It is a flowchart of the method for generating a panoramic image of a vehicle provided by an embodiment of the present application;

[0052] Figure 3Structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0053] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0054] Exemplary implementation environment

[0055] According to the method for generating a vehicle panoramic image in an embodiment of the present application, it can be executed by a vehicle or a server for controlling the vehicle. In addition to the components constituting the vehicle body, referring to Figure 1 The vehicle further includes a chassis camera 100, a panoramic camera 200, a transparent chassis controller 300, a vehicle-mounted display screen 400, and a vehicle status information acquisition module 500.

[0056] Among them, the chassis camera 100: is used to collect images of the bottom of the vehicle and transmit the video stream to the transparent chassis controller through a network bus. To make the chassis image more complete, a chassis camera 100 can be set at the front and rear of the chassis.

[0057] A panoramic camera 200 is installed on each of the front, rear, left, and right of the vehicle: is used to collect images in four directions of the front, rear, left, and right and transmit the video stream to the transparent chassis controller through a network bus.

[0058] The transparent chassis controller 300: is used to process the images of the chassis camera 100 and the four panoramic cameras 200 in the front, rear, left, and right directions to generate a panoramic image including the 360° surrounding image of the vehicle in the front, rear, left, and right and the 180° chassis image of the ground;

[0059] Further, the transparent chassis controller includes an image decoding module 301, an image fusion processing module 302, and a control module 303.

[0060] The image decoding module 301 is used to preprocess the images of the chassis camera and the panoramic cameras in the four directions of the front, rear, left, and right. The preprocessing includes distortion correction and projection transformation;

[0061] The image fusion processing module 302 is used to crop and fuse the preprocessed images to generate a panoramic image including the 360° surrounding image of the vehicle in the front, rear, left, and right and the 180° chassis image of the ground;

[0062] The control module 303 is configured to receive the current gear position, vehicle speed, steering wheel angle, turn signal status, and on / off status. Among them, the gear position is used to obtain the direction in which the vehicle is about to move forward; the vehicle speed, steering wheel angle, turn signal status, and on / off status are used to control the activation and inhibition of functions.

[0063] The in-vehicle display screen 400: is used to display the 360° panoramic view of the front, rear, left, and right of the vehicle and the 180° chassis perspective image of the ground.

[0064] The vehicle status information acquisition module 500: is used to provide the on / off status, current gear position, vehicle speed, steering wheel angle, and turn signal status. Through the above information, the system can determine when to actively activate or deactivate the generation of the panoramic image and other matters of the vehicle.

[0065] Exemplary method

[0066] Please refer to Figure 2 , in an exemplary embodiment, a method for generating a vehicle panoramic image is provided, including:

[0067] Step 201, obtain the chassis image and the surrounding images of the vehicle at the current moment in real time.

[0068] In some embodiments, the chassis image and the surrounding images are obtained by simultaneously shooting at the same moment.

[0069] The chassis image can be obtained after being shot and transmitted in real time by the chassis camera described in the above embodiment. Since the chassis position is relatively low, if a large number of panoramic cameras are set, the chassis camera can be, but is not limited to, a fish-eye camera. The surrounding images can be obtained after being shot and transmitted in real time by four panoramic cameras arranged at the four front, rear, left, and right positions of the vehicle.

[0070] It can be understood that the surrounding images can be obtained by splicing after being shot by four panoramic cameras.

[0071] It can be understood that, considering the display effect and convenience, a fish-eye camera is used for chassis image acquisition, and other new type of vision sensors can also be used for fusion and supplementation, such as lidar, far and near infrared cameras, millimeter wave radars, etc.

[0072] Step 202, convert the pixel coordinates of each pixel point of the chassis image to obtain a converted image, and the image perspective of the converted image is the same as that of the surrounding images.

[0073] In some embodiments, since the image shot by the fish-eye camera will cause image distortion, and it will also be unable to complete the fusion due to the different shooting angles from the surrounding images. Therefore, it is necessary to convert the shot chassis image so that the image perspective of the converted image is the same as that of the surrounding images.

[0074] In an alternative embodiment, converting the pixel coordinates of each pixel point of the chassis image to obtain a converted image includes:

[0075] Performing distortion correction on each pixel coordinate of the chassis image to obtain a corrected image;

[0076] Projecting the corrected image onto the ground coordinate system to obtain the converted image.

[0077] In some embodiments, distortion correction is performed on the image of the chassis fisheye camera. The goal of distortion correction is to map each pixel point in the distorted image to the undistorted image coordinate system. The image after distortion correction is projected onto the ground coordinate system to generate a bird's-eye view corresponding to the chassis image, so that it can be fused with the surrounding images.

[0078] In an alternative embodiment, performing distortion correction on each pixel coordinate of the chassis image to obtain a corrected image includes:

[0079] Obtaining the distortion coefficient and camera parameters of the camera that captured the chassis image;

[0080] Obtaining the correction relationship between the undistorted image coordinates and the distorted image coordinates;

[0081] Substituting the pixel coordinates, distortion coefficient, and camera parameters of each pixel point of the chassis image into the correction relationship to determine the undistorted coordinates of the pixel point and obtain the corrected image.

[0082] In some embodiments, both the distortion coefficient and camera parameters of the camera can be obtained directly or indirectly through camera calibration. The implementation process can refer to related technologies and is not limited here. Among them, the camera parameters include the pixel coordinates of the camera optical center, the equivalent focal length, and the camera focal length.

[0083] Among them, the correction relationship can be derived through the following inverse mapping distortion correction method:

[0084] First, normalized coordinate conversion: The purpose is to convert the pixel coordinates of the target image to the normalized plane (an undistorted coordinate system centered on the camera optical center).

[0085]

[0086] Among them, c x , c y respectively represent the abscissa and ordinate in the pixel coordinates of the camera optical center, f x , f y respectively represent the horizontal focal length and vertical focal length of the camera, x dst represents the abscissa of the pixel point in the undistorted image, ydst Denote the ordinate of the pixel point in the undistorted image.

[0087] Second, calculate the undistorted radius: calculate the undistorted radius r from the normalized coordinates u .

[0088]

[0089] Third, inversely solve the distortion model, using the following distortion model:

[0090] r d = θ(1 + k1θ 2 + k2θ 4 + k3θ 6 + k4θ 8 );

[0091] where θ = arctan(r u / f), r d is the normalized image radius after distortion, k1, k2, k3, k4 are radial distortion coefficients, and f is the equivalent focal length. Solve for θ by the Newton iteration method, and then obtain r u .

[0092] Fourth, calculate the normalized coordinates after distortion: calculate the normalized coordinates after distortion according to the distorted radius r d and the undistorted radius r u .

[0093]

[0094] Fifth,; convert the distorted normalized coordinates to the pixel coordinates of the distorted image.

[0095] x src = u distorted · f x + c x

[0096] y src = v distorted · f y + c y

[0097] where, x src denotes the abscissa of the pixel point in the distorted image, and y src denotes the ordinate of the pixel point in the distorted image.

[0098] Through the above first to fifth steps, the correction relationship between the undistorted image coordinates and the distorted image coordinates can be determined as:

[0099]

[0100] Substitute the pixel coordinates, distortion coefficients, and camera parameters of each pixel point in the obtained chassis image into the above correction relationship to determine the undistorted coordinates of the pixel point, and then correct the chassis image based on the undistorted coordinates to obtain a corrected image.

[0101] In an alternative embodiment, the step of projecting the corrected image onto the ground coordinate system to obtain the converted image includes:

[0102] Obtain the height between the vehicle chassis and the ground and the tilt angle with respect to the horizontal ground;

[0103] Obtain the internal parameter matrix of the camera that captured the chassis image;

[0104] Based on the height, tilt angle, and internal parameter matrix, determine the homography matrix;

[0105] Based on the homography matrix and the pixel coordinates of the corrected image, determine the projected homogeneous coordinates, and determine the converted image based on the homogeneous coordinates.

[0106] In some embodiments, the camera height can be used as the height between the vehicle chassis and the ground. In the case where the vehicle load changes or the ground is uneven, the chassis height and tilt angle will change. This height can be measured by a distance sensor. The tilt angle can be measured by an angle sensor.

[0107] The essence of the homography matrix is the affine transformation of the image. It is the core mathematical tool in computer vision to describe the projection relationship between two planes. Its essence is a transformation matrix that maps points on one plane to another plane through homogeneous coordinates. Homogeneous coordinates are the core tools in mathematics and computer graphics to describe geometric transformations and projection relationships. By introducing an additional dimension, it maps points in n-dimensional space to n + 1-dimensional vectors, thereby uniformly handling transformations such as translation, rotation, and scaling, and solving the problem of infinite points that cannot be expressed in Euclidean geometry.

[0108] In this application, the projected homogeneous coordinates are determined through the following projection formula.

[0109]

[0110] where

[0111] t = [0, 0, -h] T ;

[0112] where ω represents the scaling factor of the homogeneous coordinates, H represents the homography matrix, X g and Y grespectively represent the abscissa and ordinate of its secondary coordinate, x src and y src respectively represent the abscissa and ordinate of the pixel points of the corrected image.

[0113] During vehicle driving, when the chassis height changes (such as due to load changes) or the ground is uneven (such as having slopes or bumps), by obtaining the height and tilt angle in real time, dynamic correction is performed in a timely manner.

[0114] Step 203: Fuse the converted image with the surrounding images to obtain the panoramic image of the vehicle.

[0115] In some embodiments, after obtaining the converted image, its image angle is the same as that of the surrounding images. Therefore, the two can be fused. To improve the fusion efficiency, the converted image can be initially cropped to cut off the redundant parts, so that when comparing features with the panoramic image, the number of features to be compared can be reduced.

[0116] In an alternative embodiment, the fusing the converted image with the surrounding images to obtain the panoramic image of the vehicle includes:

[0117] Determine the chassis area of the vehicle;

[0118] Crop a cropped image including the chassis area from the converted image,

[0119] Stitch the positions where the edge features of the cropped image and the surrounding images are the same to obtain the panoramic image.

[0120] In some embodiments, according to the vehicle body parameters such as the length, width, wheelbase, and track width of the vehicle body, the range where the chassis area is located is calculated, and according to the usage requirements, the image of the transparent chassis area is intercepted. The intercepted cropped image and the panoramic image retain a certain overlapping area, so that when fusing with the panoramic image later, the geometric transformation relationship between the two images can be found through feature points to align the overlapping areas.

[0121] Among them, the vehicle body parameters of the actual vehicle can be scaled according to a fixed ratio, and then the converted image can be cropped according to the scaled vehicle body parameters.

[0122] After the above steps, both the converted image and the surrounding images are bird's-eye views. Related technologies can be further used for denoising, balancing the exposure differences of different cameras, mask generation, etc., to eliminate the illumination and color differences at the stitching boundary and achieve smooth transition between images.

[0123] This application collects the vehicle bottom image through the chassis camera installed on the vehicle. After processing, converting, and cropping the obtained chassis image, it is fused with the traditional panoramic image to realize the function of real-time transparent chassis.

[0124] The present invention provides a method for generating a vehicle bottom image. This method expands the range of panoramic stitching display, enabling real-time display of image information in the area of the vehicle body bottom, enhancing driving safety, enriching the functions of panoramic display, and improving the user experience. The specific technical problems to be solved are as follows:

[0125] Eliminating delay: To avoid the situation in the prior art where the vehicle bottom image uses a previously stored image instead of real-time acquisition. Especially when the vehicle is just started, since there is no previously stored vehicle bottom image, the transparent chassis is unavailable. The present invention solves the problem of the invisible area of the vehicle bottom camera in the prior art, enabling it to also display image information, thereby eliminating the delay problem;

[0126] Improving accuracy: In the prior art, basically the previous moment's image stored by the camera is used. By calculating the relative mapping relationship between the vehicle and the image at the current moment through vehicle speed and steering wheel angle, the panoramic image obtained at the previous moment is filled into the vehicle bottom at the current moment to achieve the transparent chassis effect. However, the accuracy of this filling method often depends on multiple factors, such as ① the accuracy of vehicle speed and steering wheel angle measurement. Especially on off-road outdoor roads, wheel slippage and bumps often greatly affect the filling effect; ② the advancement of parameter conversion and filling algorithms, and the overall implementation technology is difficult.

[0127] The present invention can avoid the above problems by using the real-time vehicle bottom image, calculating and filling not through the previous moment's image but using the image at the current moment.

[0128] Exemplary device

[0129] Correspondingly, an embodiment of the present application further provides a device for generating a vehicle panoramic image, including:

[0130] An acquisition unit for real-time acquisition of the chassis image and the surrounding images of the vehicle at the current moment;

[0131] A conversion unit for converting the pixel coordinates of each pixel point of the chassis image to obtain a converted image, and the image perspective of the converted image is the same as that of the surrounding images;

[0132] A fusion unit for fusing the converted image with the surrounding images to obtain the panoramic image of the vehicle.

[0133] The vehicle panoramic image generation device provided in this embodiment belongs to the same inventive concept as the vehicle panoramic image generation method provided in the above embodiments of the present application. It can execute the methods provided in any of the above embodiments of the present application and has the corresponding functional modules and beneficial effects for executing the methods. For the technical details not described in detail in this embodiment, reference may be made to the specific processing content of the vehicle panoramic image generation method provided in the above embodiments of the present application, which will not be elaborated here.

[0134] In the above vehicle panoramic image generation device, the functions implemented by each unit can be respectively implemented by the same or different processors, which is not limited in the embodiments of the present application.

[0135] It should be understood that each functional unit in the above device can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or the functions of each unit of the device. The processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory inside the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of a hardware circuit. By designing the hardware circuit, some or all of the unit functions can be implemented. This hardware circuit can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and through the design of the logical relationship of the components in the circuit, some or all of the above unit functions are implemented. Another example is that in another implementation, the hardware circuit can be implemented by a PLD. Taking an FPGA as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured through a configuration file to implement some or all of the above unit functions. All units of the above device can be all implemented in the form of a processor calling software, or all implemented in the form of a hardware circuit, or some are implemented in the form of a processor calling software, and the remaining part is implemented in the form of a hardware circuit.

[0136] In the embodiments of the present application, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and running capabilities, such as a CPU, a microprocessor, a GPU, or a DSP, etc. In another implementation, the processor can implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of this hardware circuit is fixed or can be reconstructed. For example, the processor is a hardware circuit implemented by an ASIC or a PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the configuration of the hardware circuit can be understood as the process of the processor loading instructions to implement some or all of the above unit functions. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as a type of ASIC, such as an NPU, a TPU, a DPU, etc.

[0137] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method. For example: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0138] In addition, each unit in the above device can be integrated in whole or in part, or can be independently implemented. In one implementation, these units are integrated together and implemented in the form of an SOC. The SOC can include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The types of the at least one processor can be different. For example, it includes CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0139] Exemplary electronic device

[0140] Another embodiment of the present application also proposes an electronic device. Refer to Figure 3 As shown, the device includes:

[0141] A memory 300 and a processor 310;

[0142] Wherein, the memory 300 is connected to the processor 310 and is used for storing programs;

[0143] The processor 310 is used to implement the method for generating a panoramic image of a vehicle disclosed in any of the above embodiments by running the program stored in the memory 300.

[0144] Specifically, the above device for generating a panoramic image of a vehicle may further include: a bus, a communication interface 320, an input device 330, and an output device 340.

[0145] The processor 310, the memory 300, the communication interface 320, the input device 330, and the output device 340 are interconnected through the bus. Among them:

[0146] The bus may include a path for transmitting information between various components of the computer system.

[0147] The processor 310 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0148] The processor 310 may include a main processor, and may also include a baseband chip, a modem, etc.

[0149] The memory 300 stores a program for implementing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include program code, and the program code includes computer operation instructions. More specifically, the memory 300 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk memory, a flash memory, and so on.

[0150] The input device 330 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor, etc.

[0151] The output device 340 may include a device for allowing information to be output to a user, such as a display screen, a printer, a speaker, etc.

[0152] The communication interface 320 may include a device using any transceiver type to communicate with other devices or communication networks, such as an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0153] The processor 310 executes the program stored in the memory 300, and calls other devices, which can be used to implement each step of any one of the vehicle panoramic image generation methods provided in the above embodiments of the present application.

[0154] Exemplary computer program product and storage medium

[0155] In addition to the above methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, and when the computer program instructions are run by a processor, the processor is caused to execute the steps in the vehicle panoramic image generation method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0156] The computer program product can be written in any combination of one or more programming languages for programming code to perform the operations of the embodiments of the present application. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The programming code can be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0157] In addition, an embodiment of the present application can also be a storage medium on which a computer program is stored. The computer program is executed by a processor to perform the steps in the method for generating a vehicle panoramic image according to various embodiments of the present application described in any of the above embodiments of this specification. Specifically, the following steps can be implemented:

[0158] Obtain the chassis image and the surrounding images of the vehicle at the current moment in real time;

[0159] Convert the pixel coordinates of each pixel point of the chassis image to obtain a converted image, and the image perspective of the converted image is the same as that of the surrounding images;

[0160] Fuse the converted image with the surrounding images to obtain the panoramic image of the vehicle.

[0161] For the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0162] It should be noted that the embodiments in this specification are all described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0163] The steps in the methods of the embodiments of the present application can be adjusted, combined, and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0164] The modules and sub-modules in the devices and terminals in the embodiments of the present application can be combined, divided, and deleted according to actual needs.

[0165] In several embodiments provided in this application, it should be understood that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the terminal embodiments described above are merely illustrative. For example, the division of modules or sub-modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple sub-modules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be an indirect coupling or communication connection through some interfaces, devices, or modules, and can be in electrical, mechanical, or other forms.

[0166] The modules or sub-modules described as separate components may or may not be physically separated. The components as modules or sub-modules may or may not be physical modules or sub-modules, that is, they can be located in one place, or they can be distributed to multiple network modules or sub-modules. Some or all of the modules or sub-modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0167] In addition, in each embodiment of this application, the functional modules or sub-modules can be integrated in a processing module, or each module or sub-module can exist physically alone, or two or more modules or sub-modules can be integrated in one module. The above-mentioned integrated modules or sub-modules can be implemented in the form of hardware or in the form of software functional modules or sub-modules.

[0168] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0169] The steps of the methods or algorithms described in combination with the embodiments disclosed in this article can be directly implemented by hardware, software units executed by a processor, or a combination of the two. The software units can be placed in a random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0170] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising said element.

[0171] The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating a panoramic image of a vehicle, characterized in that, Including: Obtaining the chassis image and the surrounding images of the vehicle at the current moment in real time; Converting the pixel coordinates of each pixel point of the chassis image to obtain a converted image, and the image perspective of the converted image is the same as that of the surrounding images; Fusing the converted image with the surrounding images to obtain the panoramic image of the vehicle.

2. The method according to claim 1, wherein Converting the pixel coordinates of each pixel point of the chassis image to obtain a converted image, including: Performing distortion correction on each pixel coordinate of the chassis image to obtain a corrected image; Projecting the corrected image onto the ground coordinate system to obtain the converted image.

3. The method according to claim 2, wherein Performing distortion correction on each pixel coordinate of the chassis image to obtain a corrected image, including: Obtaining the distortion coefficient and camera parameters of the camera that captures the chassis image; Obtaining the correction relationship between the undistorted image coordinates and the distorted image coordinates; Substituting the pixel coordinates, distortion coefficient, and camera parameters of each pixel point of the chassis image into the correction relationship to determine the undistorted coordinates of the pixel point and obtain the corrected image.

4. The method according to claim 3, wherein The correction relationship between the undistorted image coordinates and the distorted image coordinates includes: Among them, r d = θ(1 + k1θ 2 + k2θ 4 + k3θ 6 + k4θ 8 ); θ = arctan(r u / f); where x dst represents the abscissa in the undistorted coordinates, y dst represents the ordinate in the undistorted coordinates, x src represents the abscissa in the pixel coordinates, y src represents the ordinate in the pixel coordinates, k1, k2, k3, k4 represent different distortion coefficients, f represents the equivalent focal length, c x , c y respectively represent the abscissa and ordinate in the pixel coordinates of the camera optical center, f x , f y respectively represent the horizontal focal length and vertical focal length of the camera.

5. The method according to claim 2, wherein The projecting the corrected image onto the ground coordinate system to obtain the converted image includes: Obtaining the height between the vehicle chassis and the ground and the tilt angle with the horizontal ground; Obtaining the internal parameter matrix of the camera that captures the chassis image; Determining the homography matrix based on the height, tilt angle, and internal parameter matrix; Determining the projected homogeneous coordinates based on the homography matrix and the pixel point coordinates of the corrected image, and determining the converted image based on the homogeneous coordinates.

6. The method according to claim 5, wherein Determining the homography matrix based on the height, tilt angle, and internal parameter matrix, including: Among them, t = [0, 0, -h] T ; Wherein, H represents the homography matrix, K represents the internal parameter matrix, θ represents the tilt angle, and h represents the height.

7. The method according to claim 5 or 6, characterized in that, Determining the projected homogeneous coordinates based on the homography matrix and the pixel point coordinates of the corrected image, including: where ω represents the scaling factor of homogeneous coordinates, H represents the homography matrix, and X g and Y g represent the abscissa and ordinate of homogeneous coordinates respectively, and x src and y src represent the abscissa and ordinate of the pixel points of the rectified image respectively.

8. The method according to claim 1, characterized in that The fusing the converted image with the surrounding images to obtain the panoramic image of the vehicle includes: Determining the chassis area of the vehicle; Cropping from the converted image to obtain a cropped image including the chassis area, Stitching the positions where the edge features of the cropped image and the surrounding images are the same to obtain the panoramic image.

9. An electronic device, characterized in that, Including a memory and a processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the method for generating the panoramic image of the vehicle as described in any one of claims 1 to 8 by running the programs in the memory.

10. A storage medium, characterized in that, A computer program is stored on the storage medium, and when the computer program is run by the processor, the method for generating the panoramic image of the vehicle as described in any one of claims 1 to 8 is implemented.

Images