Vehicle transparent chassis generation method and device

Abstract

The invention relates to the technical field of automobile auxiliary driving, in particular to a vehicle transparent chassis generation method and device, and the method comprises the steps: building a vehicle three-dimensional coordinate system, calibrating a vehicle sensor according to the vehicle three-dimensional coordinate system, and obtaining internal and external parameters; constructing a virtual camera above the vehicle according to the internal and external parameters of the vehicle sensor, and performing overlook projection on the vehicle according to the virtual camera to obtain a top view; obtaining motion data of the vehicle at the current moment, and obtaining an optimal vehicle speed value and an optimal front wheel steering angle value according to the motion data; establishing a motion state equation of the vehicle according to the vehicle speed optimal value and the front wheel steering angle optimal value, and obtaining a rotation translation matrix of the vehicle at the current moment relative to the previous moment according to the motion state equation; and splicing the top view of the previous moment and the top view of the current moment according to the rotation translation matrix to obtain a transparent chassis image. The method is used for overcoming the defects of insufficient precision or large computing power resource occupation in the prior art.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile assisted driving, and in particular to a method and device for generating a transparent chassis of a vehicle. Background Art

[0002] As cars continue to become more intelligent, electrified, lightweight, networked and shared, car consumers are paying more and more attention to safety during driving. Eliminating the driver's blind spot is an important safety requirement during driving. The driver can only obtain the driving picture during driving with the naked eye, and cannot judge whether there are obstacles under the chassis of the vehicle and their specific information. The transparent chassis function allows the driver to intuitively perceive the position of the tires and the specific information of obstacles such as potholes, stones, small animals or manhole covers on the road, so as to achieve safe driving.

[0003] The transparent chassis is an important function integrated in the 360-degree surround view system, which is mainly realized based on machine vision. At the current time t, the scene captured by each camera is converted into image information from the top view (bird's eye view) around the vehicle body through the on-board surrounding cameras. Since the image of the bottom of the vehicle body cannot be obtained directly, it is necessary to cache the top view (bird's eye view) information of the previous N moments. The top view (bird's eye view) information of the previous N moments will be used when the current frame is spliced. The top view (bird's eye view) image frames of the previous N moments are spliced ​​to the blind spot of the bottom of the vehicle body through the image splicing algorithm to form a transparent chassis effect.

[0004] The difficulty of transparent chassis technology lies in calculating the image changes that occur during the splicing process of the current frame and the previous N frames. The effect of splicing directly affects the driver's driving experience and safety judgment. The most classic transparent chassis splicing solution obtains vehicle driving signals, such as the vehicle's current speed, angular velocity, acceleration, and angular acceleration, calculates the image motion changes of the previous frame or the previous few frames, and fills in the chassis field of view blind area on the current top view. The advantages of this solution are simple implementation and low processing delay. However, since the vehicle itself is a nonlinear control motion system, the obtained vehicle driving signals such as vehicle speed, angular velocity, acceleration, and angular acceleration have transmission and measurement errors. The accuracy of the splicing information calculated from these signals is not enough, resulting in dislocation, ghosting, black edges, etc. in the splicing, which affects the driver's driving experience and safety judgment. In order to solve the problem of insufficient accuracy of splicing information caused by vehicle driving signal errors, a transparent chassis solution that introduces machine vision came into being. This solution performs feature matching on the images of the current frame and the previous few frames, thereby calculating the translation, rotation and other splicing information in the vehicle movement, and overcoming the problem of splicing information accuracy caused by vehicle driving signal errors. In practice, this solution has its limitations. Feature matching requires a lot of computing resources, and display delays are prone to occur on low-computing platforms, resulting in ghosting and freezing. The actual road conditions are relatively complex, and feature mismatches are prone to occur, resulting in poor splicing such as splicing misalignment. Therefore, a transparent chassis generation method that occupies little computing resources and has high splicing accuracy has high practical value. Summary of the invention

[0005] The present invention provides a method and device for generating a vehicle transparent chassis, which are used to solve the defects of insufficient accuracy or large computing power resource occupation in the prior art, and realize transparent chassis generation with small computing power resource occupation and high accuracy.

[0006] The present invention provides a method for generating a vehicle transparent chassis, comprising:

[0007] Establishing a three-dimensional vehicle coordinate system, calibrating the vehicle sensor according to the three-dimensional vehicle coordinate system, and obtaining internal and external parameters;

[0008] A virtual camera is constructed above the vehicle according to the internal and external parameters of the vehicle sensor, and a top-down projection of the vehicle is performed according to the virtual camera to obtain a top-down view;

[0009] Acquire the motion data of the vehicle at the current moment, and obtain the optimal value of the vehicle speed and the optimal value of the front wheel turning angle according to the motion data;

[0010] Establishing a motion state equation of the vehicle according to the optimal value of the vehicle speed and the optimal value of the front wheel steering angle, and obtaining a rotation and translation matrix of the vehicle at a current moment relative to a previous moment according to the motion state equation;

[0011] The top view at the previous moment and the top view at the current moment are spliced ​​according to the rotation and translation matrix to obtain a transparent chassis image.

[0012] According to a method for generating a transparent chassis of a vehicle provided by the present invention, the origin of the coordinate system of the vehicle's three-dimensional coordinate system is established directly below the vehicle's license plate, the Y axis of the vehicle's three-dimensional coordinate system is the vehicle's forward direction, the Z axis is the vertical upward direction, and the X axis is perpendicular to the Y axis and the Z axis respectively.

[0013] According to a method for generating a transparent chassis of a vehicle provided by the present invention, the vehicle sensor comprises: a distance sensor and a camera device, and the spatial three-dimensional coordinates (X) of each point P (u, v) in the top view under the perspective of the virtual camera are p , Y p , Z p ) is expressed as:

[0014]

[0015] Among them, s represents the scaling factor of the coordinate system, and M represents the intrinsic projection matrix of the virtual camera, which is specifically expressed as:

[0016]

[0017] Among them, the top view image is on the ground, assuming Z p =0, we get:

[0018]

[0019] According to the intrinsic projection matrix of the camera device, the actual pixel information of point P is expressed as:

[0020]

[0021] Among them, M f Represents the intrinsic projection matrix of the camera device.

[0022] According to a method for generating a transparent chassis of a vehicle provided by the present invention, the step of acquiring the motion data of the vehicle at the current moment and obtaining the optimal value of the vehicle speed and the optimal value of the front wheel turning angle according to the motion data comprises:

[0023] Get the vehicle's motion data at the current time t, including: vehicle speed v and front wheel steering angle δ. The vehicle speed at time t+1 is expressed as:

[0024] v(t+1)=v(t)+W v (t) ΔT

[0025] Among them, W v (t) represents the random acceleration noise caused by the driver according to the current road conditions, with a mean of 0 and a variance of Q v, ΔT represents the sampling interval of motion data;

[0026] The observed vehicle speed at time t is expressed as:

[0027] z v (t)=v(t)+V v (t)

[0028] Among them, V v (t) represents the transmission and detection noise, with a mean of 0 and a variance of R v ;

[0029] According to the state optimization algorithm, the state equation is established as follows:

[0030] v(t+1)=Av(t)+ΓW v (t)

[0031] According to the formula z v (t)=v(t)+V v (t) The observation equation is established as:

[0032] z v (t)=Hv(t)+V v (t)

[0033] According to the formula v(t+1)=v(t)+W v (t)·ΔT and v(t+1)=Av(t)+ΓW v (t) confirming the hyperparameters of the state equation, and obtaining: A=1, Γ=ΔT;

[0034] According to the formula z v (t)=v(t)+V v (t) and z v (t)=Hv(t)+V v (t) confirming the hyperparameters of the observation equation, and obtaining: H = 1;

[0035] According to the state optimization algorithm, the initial error covariance P is set v (0) = a, the prior error covariance is expressed as:

[0036]

[0037] According to the gain calculation formula, the gain is obtained:

[0038]

[0039] The optimal value of vehicle speed is expressed as:

[0040] v opt (t+1)=v opt(t)+K(t+1)[z v (t+1)-v opt (t)]

[0041] The corresponding error covariance matrix is ​​expressed as:

[0042] The front wheel turning angle at time t+1 is expressed as:

[0043] δ(t+1)=δ(t)+W δ (t)

[0044] Among them, W δ (t) represents the noise caused by the transmission, with a mean of 0 and a variance of Q δ , ΔT represents the sampling interval of the device;

[0045] The observed value of the front wheel turning angle at time t is expressed as:

[0046] z δ (t)=δ(t)+V δ (t)

[0047] Among them, V δ (t) represents the transmission and detection noise, with a mean of 0 and a variance of R δ .

[0048] According to the state optimization algorithm, the initial error covariance P is set δ (0) = b;

[0049] Confirm the hyperparameters of the state equation and observation equation and obtain:

[0050] A=1,Γ=1,H=1

[0051] The prior error covariance is then expressed as:

[0052]

[0053] According to the gain calculation formula, the gain is obtained:

[0054]

[0055] The optimal value of the front wheel steering angle is expressed as:

[0056] δ opt (t+1)=δ opt (t)+K(t+1)[z δ (t+1)-δ opt (t)]

[0057] The corresponding error covariance matrix is ​​expressed as:

[0058] According to a vehicle transparent chassis generation method provided by the present invention, the state optimization algorithm is a Kalman filter optimization algorithm.

[0059] According to a method for generating a transparent chassis of a vehicle provided by the present invention, the vehicle's motion state equation is established according to the optimal value of the vehicle speed and the optimal value of the front wheel turning angle, and the rotation and translation matrix of the vehicle at a current moment relative to a previous moment is obtained according to the motion state equation, including:

[0060] At the last moment t, a motion state equation of the vehicle is established according to the optimal value of the vehicle speed and the optimal value of the front wheel steering angle, and a rotation and translation matrix from the last moment t to the current moment t+1 is obtained according to the motion state equation;

[0061] At the current time t+1, the top view at the current time t+1 is spliced ​​with the top view at the previous time t, and the rotation and translation matrix of the current time t+1 relative to the previous time t is obtained by matrix inversion.

[0062] According to a method for generating a transparent chassis of a vehicle provided by the present invention, the top view at the previous moment and the top view at the current moment are spliced ​​according to the rotation and translation matrix to obtain a transparent chassis image, including:

[0063] According to the rotation and translation matrix, each pixel of the top view at the previous moment is filled with the top view at the current moment to obtain a transparent chassis image.

[0064] The present invention also provides a vehicle transparent chassis generating device, comprising:

[0065] A parameter acquisition module, used to establish a three-dimensional vehicle coordinate system, calibrate the vehicle sensor according to the three-dimensional vehicle coordinate system, and obtain internal and external parameters;

[0066] A top view acquisition module, used to construct a virtual camera above the vehicle according to the internal and external parameters of the vehicle sensor, and to perform a top view projection of the vehicle according to the virtual camera to obtain a top view;

[0067] An optimal value acquisition module, used to acquire the motion data of the vehicle at the current moment, and obtain the optimal value of the vehicle speed and the optimal value of the front wheel steering angle according to the motion data;

[0068] A matrix acquisition module, used to establish a vehicle motion state equation according to the optimal vehicle speed value and the optimal front wheel steering angle value, and obtain a rotation and translation matrix of the vehicle at a current moment relative to a previous moment according to the motion state equation;

[0069] The image acquisition module is used to splice the top view at the previous moment and the top view at the current moment according to the rotation and translation matrix to obtain a transparent chassis image.

[0070] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned methods for generating a transparent chassis of a vehicle when executing the program.

[0071] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for generating a transparent chassis of a vehicle described in any one of the above is implemented.

[0072] The present invention provides a method and device for generating a transparent chassis of a vehicle. By establishing a three-dimensional coordinate system of the vehicle, the vehicle sensor is calibrated according to the three-dimensional coordinate system of the vehicle to obtain internal and external parameters for subsequent calculation and analysis. A virtual camera is constructed above the vehicle according to the internal and external parameters of the vehicle sensor, and the vehicle is projected from above according to the virtual camera to obtain a top view. The top view is established from the perspective of the virtual camera according to the internal and external parameters as the framework of the transparent chassis image. The motion data of the vehicle at the current moment is obtained, and the optimal value of the vehicle speed and the optimal value of the front wheel turning angle are obtained according to the motion data. The vehicle speed and the front wheel turning angle are directly calculated according to the motion data of the vehicle and the optimal values ​​are taken to ensure the accuracy of subsequent calculations and reduce the burden on computing power. The motion state equation of the vehicle is established according to the optimal value of the vehicle speed and the optimal value of the front wheel turning angle, and the rotation and translation matrix of the vehicle at the current moment relative to the previous moment is obtained according to the motion state equation. The motion state equation of the vehicle is directly established through the optimal value of the vehicle speed and the optimal value of the front wheel turning angle, and the corresponding rotation and translation matrix is ​​obtained to establish a basis for the generation of the final transparent chassis image. Finally, the top view of the previous moment and the top view of the current moment are spliced ​​according to the rotation and translation matrix to obtain a transparent chassis image, which has the basic functions of a surround view transparent chassis, allowing the driver to intuitively feel the vehicle body conditions and road obstacles, eliminating the vehicle body blind spots, and improving driving safety. Compared with the existing transparent chassis implementation schemes using image feature matching, the present invention directly uses vehicle driving information, i.e., vehicle speed and front wheel turning angle, and does not rely on the collected surround view images. It can effectively overcome the display delay, ghosting and jamming problems caused by the image algorithm occupying the processor computing power resources, solve the feature mismatch problem caused by complex road conditions, and achieve transparent chassis generation with small computing power resources and high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0074] Figure 1It is a schematic diagram of the process of the method for generating a transparent chassis of a vehicle provided by the present invention;

[0075] Figure 2 It is a structural schematic diagram of a transparent chassis system of a vehicle transparent chassis generation method provided by the present invention;

[0076] Figure 3 It is a structural schematic diagram of a camera device for the method for generating a vehicle transparent chassis provided by the present invention;

[0077] Figure 4 It is a geometrical schematic diagram of the vehicle motion state of the vehicle transparent chassis generation method provided by the present invention;

[0078] Figure 5 It is a structural schematic diagram of a vehicle transparent chassis generating device provided by the present invention;

[0079] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0080] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0081] Combine the following Figure 1-Figure 4 A method for generating a transparent chassis of a vehicle according to a first embodiment of the present invention is described.

[0082] like Figure 1 As shown, a method for generating a vehicle transparent chassis in this embodiment specifically includes the following steps (the numbering of each step in the present invention is only used to distinguish the steps, and does not limit the specific execution order of each step):

[0083] Step S1: Establish a three-dimensional vehicle coordinate system, and calibrate the vehicle sensor according to the three-dimensional vehicle coordinate system to obtain internal and external parameters.

[0084] The vehicle sensors are calibrated according to the vehicle's three-dimensional coordinate system to obtain internal and external parameters for subsequent calculation and analysis.

[0085] Step S2: constructing a virtual camera above the vehicle according to the internal and external parameters of the vehicle sensor, and performing a top-down projection of the vehicle according to the virtual camera to obtain a top view.

[0086] A top view is established from the perspective of a virtual camera according to the internal and external parameters, which serves as the framework of the transparent chassis image.

[0087] Step S3: Obtain the motion data of the vehicle at the current moment, and obtain the optimal value of the vehicle speed and the optimal value of the front wheel turning angle according to the motion data.

[0088] The vehicle speed and front wheel turning angle are directly calculated based on the vehicle's motion data and the optimal value is taken to ensure the accuracy of subsequent calculations with less computing power burden.

[0089] Step S4: establishing a vehicle's motion state equation based on the optimal vehicle speed value and the optimal front wheel steering angle value, and obtaining a rotation and translation matrix of the vehicle at a current moment relative to a previous moment based on the motion state equation.

[0090] The vehicle's motion state equation is directly established through the optimal vehicle speed and the optimal front wheel angle, and the corresponding rotation and translation matrix is ​​obtained, which lays the foundation for the final generation of the transparent chassis image.

[0091] Step S5: splicing the top view at the previous moment and the top view at the current moment according to the rotation and translation matrix to obtain a transparent chassis image.

[0092] Finally, the top view at the previous moment and the top view at the current moment are spliced ​​according to the rotation and translation matrix to obtain the transparent chassis image, which has the basic functions of a surround transparent chassis, allowing the driver to intuitively feel the vehicle body condition and road obstacles, eliminating the vehicle body blind spots and improving driving safety.

[0093] The first embodiment of the present invention provides a method for generating a transparent chassis of a vehicle. By establishing a three-dimensional coordinate system of the vehicle, the vehicle sensor is calibrated according to the three-dimensional coordinate system of the vehicle to obtain internal and external parameters for subsequent calculation and analysis. A virtual camera is constructed above the vehicle according to the internal and external parameters of the vehicle sensor, and the vehicle is projected from above according to the virtual camera to obtain a top view. The top view is established from the perspective of the virtual camera according to the internal and external parameters as the framework of the transparent chassis image. The motion data of the vehicle at the current moment is obtained, and the optimal value of the vehicle speed and the optimal value of the front wheel angle are obtained according to the motion data. The vehicle speed and the front wheel angle are directly calculated according to the motion data of the vehicle and the optimal values ​​are taken to ensure the accuracy of subsequent calculations and reduce the burden on computing power. The motion state equation of the vehicle is established according to the optimal value of the vehicle speed and the optimal value of the front wheel angle, and the rotation and translation matrix of the vehicle at the current moment relative to the previous moment is obtained according to the motion state equation. The motion state equation of the vehicle is directly established through the optimal value of the vehicle speed and the optimal value of the front wheel angle, and the corresponding rotation and translation matrix is ​​obtained to establish a basis for the generation of the final transparent chassis image. Finally, the top view of the previous moment and the top view of the current moment are spliced ​​according to the rotation and translation matrix to obtain a transparent chassis image, which has the basic functions of a surround view transparent chassis, allowing the driver to intuitively feel the vehicle body conditions and road obstacles, eliminating the vehicle body blind spots, and improving driving safety. Compared with the existing transparent chassis implementation schemes using image feature matching, the present invention directly uses vehicle driving information, i.e., vehicle speed and front wheel turning angle, and does not rely on the collected surround view images. It can effectively overcome the display delay, ghosting and jamming problems caused by the image algorithm occupying the processor computing power resources, solve the feature mismatch problem caused by complex road conditions, and achieve transparent chassis generation with small computing power resources and high accuracy.

[0094] In this embodiment, the coordinate origin of the vehicle's three-dimensional coordinate system is established directly below the vehicle's license plate, the Y axis of the vehicle's three-dimensional coordinate system is the vehicle's forward direction, the Z axis is the vertical upward direction, and the X axis is perpendicular to the Y axis and the Z axis respectively.

[0095] The origin of the vehicle's three-dimensional coordinate system is established directly below the vehicle's license plate, with the Y axis being the vehicle's forward direction, the Z axis being the vertical upward direction, and the X axis being perpendicular to the Y axis and the Z axis, respectively, to conform to the vehicle's surrounding view and facilitate the positioning of subsequent coordinate points in the top view.

[0096] In this embodiment, the vehicle sensor includes: a distance sensor and a camera device, and the spatial three-dimensional coordinates (X x ) of each point P (u, v) in the top view under the perspective of the virtual camera are p , Y p , Z p ) is expressed as:

[0097]

[0098] Among them, s represents the scaling factor of the coordinate system, and M represents the intrinsic projection matrix of the virtual camera, which is specifically expressed as:

[0099]

[0100] Among them, the top view image is on the ground, assuming Z p =0, we get:

[0101]

[0102] According to the intrinsic projection matrix of the camera device, the actual pixel information of point P is expressed as:

[0103]

[0104] Among them, M f Represents the intrinsic projection matrix of the camera device.

[0105] like Figure 2 As shown, the vehicle sensor of this embodiment includes a distance sensor and a camera device. The hardware system of the transparent chassis includes a controller, a camera device, a display screen, and a driving parameter acquisition device, wherein the display screen, the camera device, and the driving parameter acquisition device are electrically connected to the controller respectively. Figure 3 As shown, cameras are installed in front, on the left, on the right and on the rear of the vehicle, respectively, to collect the exterior scene of the transparent chassis system, and collect the environment around the vehicle to obtain image data of the vehicle's surrounding environment. The pixel points in the top view are represented in the form of a matrix through three-dimensional coordinates and internal and external parameters, providing a basis for subsequent pixel filling to generate a transparent chassis image.

[0106] In this embodiment, the step of acquiring the motion data of the vehicle at the current moment and obtaining the optimal value of the vehicle speed and the optimal value of the front wheel steering angle according to the motion data includes:

[0107] Get the vehicle's motion data at the current time t, including: vehicle speed v and front wheel steering angle δ. The vehicle speed at time t+1 is expressed as:

[0108] v(t+1)=v(t)+W v (t) ΔT

[0109] Among them, W v (t) represents the random acceleration noise caused by the driver according to the current road conditions, with a mean of 0 and a variance of Q v , ΔT represents the sampling interval of motion data;

[0110] The observed vehicle speed at time t is expressed as:

[0111] z v (t)=v(t)+V v (t)

[0112] Among them, V v (t) represents the transmission and detection noise, with a mean of 0 and a variance of R v ;

[0113] According to the state optimization algorithm, the state equation is established as follows:

[0114] v(t+1)=Av(t)+ΓW v (t)

[0115] According to the formula z v (t)=v(t)+V v (t) The observation equation is established as:

[0116] z v (t)=Hv(t)+V v (t)

[0117] According to the formula v(t+1)=v(t)+W v (t)·ΔT and v(t+1)=Av(t)+ΓW v (t) confirming the hyperparameters of the state equation, and obtaining: A=1, Γ=ΔT;

[0118] According to the formula z v (t)=v(t)+V v (t) and z v (t)=Hv(t)+V v (t) confirming the hyperparameters of the observation equation, and obtaining: H = 1;

[0119] According to the state optimization algorithm, the initial error covariance P is set v (0) = a, the prior error covariance is expressed as:

[0120]

[0121] According to the gain calculation formula, the gain is obtained:

[0122]

[0123] The optimal value of vehicle speed is expressed as:

[0124] v opt (t+1)=v opt (t)+K(t+1)[z v (t+1)-v opt (t)]

[0125] The corresponding error covariance matrix is ​​expressed as:

[0126] The front wheel turning angle at time t+1 is expressed as:

[0127] δ(t+1)=δ(t)+W δ (t)

[0128] Among them, W δ (t) represents the noise caused by the transmission, with a mean of 0 and a variance of Q δ , ΔT represents the sampling interval of the device;

[0129] The observed value of the front wheel turning angle at time t is expressed as:

[0130] z δ (t)=δ(t)+V δ (t)

[0131] Among them, V δ (t) represents the transmission and detection noise, with a mean of 0 and a variance of R δ .

[0132] According to the state optimization algorithm, the initial error covariance P is set δ (0) = b;

[0133] Confirm the hyperparameters of the state equation and observation equation and obtain: A = 1, Γ = 1, H = 1;

[0134] The prior error covariance is then expressed as:

[0135]

[0136] According to the gain calculation formula, the gain is obtained:

[0137]

[0138] The optimal value of the front wheel steering angle is expressed as:

[0139] δ opt (t+1)=δ opt (t)+K(t+1)[z δ (t+1)-δ opt (t)]

[0140] The corresponding error covariance matrix is ​​expressed as:

[0141] At a certain time t, the front wheel steering angle δ and the speed v of the vehicle measured by the physical device for obtaining vehicle body information are sampled at regular intervals. The sampling time is [0, 1, 2...., t-1, t, t+1,...], and the sampling intervals are equal. The sampling sources of the front wheel steering angle and the speed of the vehicle are all from the same device.

[0142] The front wheel steering angle δ and vehicle speed v of the vehicle can be obtained through the vehicle body equipment, but the vehicle signal will inevitably produce errors in signal transmission and detection during the acquisition process, which will lead to errors in vehicle speed detection and steering angle. In the following steps, separate state optimization modeling will be performed for vehicle speed and front wheel steering angle.

[0143] Get the vehicle's motion data at the current time t, including: vehicle speed v and front wheel steering angle δ. The driver's unpredictable acceleration and deceleration behavior will cause the vehicle speed change to fail to keep up with the signal change. Define the sampling time t, and the vehicle speed at time t+1 is expressed as:

[0144] v(t+1)=v(t)+W v (t) ΔT

[0145] Among them, W v (t) represents the random acceleration noise caused by the driver according to the current road conditions, with a mean of 0 and a variance of Q v , ΔT represents the sampling interval of motion data;

[0146] When the vehicle-mounted equipment performs speed detection, there will be transmission and detection noise. The sampling time t is defined, and the vehicle speed observation value at time t is expressed as:

[0147] z v (t)=v(t)+V v (t)

[0148] Among them, V v (t) represents the transmission and detection noise, with a mean of 0 and a variance of R v ;

[0149] According to the state optimization algorithm, the state equation is established as follows:

[0150] v(t+1)=Av(t)+ΓW v (t)

[0151] According to the formula z v (t)=v(t)+V v (t) The observation equation is established as:

[0152] z v (t)=Hv(t)+V v (t)

[0153] According to the formula v(t+1)=v(t)+W v (t)·ΔT and v(t+1)=Av(t)+ΓW v (t) confirming the hyperparameters of the state equation, and obtaining: A=1, Γ=ΔT;

[0154] According to the formula zv (t)=v(t)+V v (t) and z v (t)=Hv(t)+V v (t) confirming the hyperparameters of the observation equation, and obtaining: H = 1;

[0155] According to the state optimization algorithm, the initial error covariance P is set v (0) = a (a is a preset value, which is set to 0 in this embodiment and can be adjusted according to actual debugging), and the prior error covariance is expressed as:

[0156]

[0157] According to the gain calculation formula, the gain is obtained:

[0158]

[0159] The optimal value of vehicle speed is expressed as:

[0160] v opt (t+1)=v opt (t)+K(t+1)[z v (t+1)-v opt (t)]

[0161] The corresponding error covariance matrix is ​​expressed as:

[0162] At each moment, after obtaining the vehicle speed v, a prediction is performed to obtain the optimal vehicle speed estimate v at the current moment t. opt

[0163] The front wheel angle is affected by rotation and ground friction, which affects the accurate measurement of the signal. Define the sampling time t, and the front wheel angle at time t+1 is expressed as:

[0164] δ(t+1)=δ(t)+W δ (t)

[0165] Among them, W δ (t) represents the noise caused by the transmission, with a mean of 0 and a variance of Q δ , ΔT represents the sampling interval of the device;

[0166] There will be detection noise when the vehicle-mounted equipment performs angle detection. The sampling time t is defined, and the front wheel angle observation value at time t is expressed as:

[0167] z δ (t)=δ(t)+V δ (t)

[0168] Among them, V δ(t) represents the transmission and detection noise, with a mean of 0 and a variance of R δ .

[0169] According to the state optimization algorithm, the initial error covariance P is set δ (0) = b (b is a preset value, which is set to 0 in this embodiment and can be adjusted according to actual debugging);

[0170] Confirm the hyperparameters of the state equation and observation equation and obtain: A = 1, Γ = 1, H = 1;

[0171] The prior error covariance is then expressed as:

[0172]

[0173] According to the gain calculation formula, the gain is obtained:

[0174]

[0175] The optimal value of the front wheel steering angle is expressed as:

[0176] δ opt (t+1)=δ opt (t)+K(t+1)[z δ (t+1)-δ opt (t)]

[0177] The corresponding error covariance matrix is ​​expressed as:

[0178] The vehicle speed and front wheel turning angle are directly calculated based on the vehicle's motion data and the optimal value is taken to ensure the accuracy of subsequent calculations with less computing power burden.

[0179] In this embodiment, the state optimization algorithm is a Kalman filter optimization algorithm.

[0180] By adopting the Kalman filter optimization algorithm as the state optimization algorithm, the collected vehicle driving information data is optimized, which improves the accuracy of the data and ensures the subsequent splicing accuracy of the transparent chassis.

[0181] In this embodiment, the vehicle's motion state equation is established according to the optimal value of the vehicle speed and the optimal value of the front wheel steering angle, and the rotation and translation matrix of the vehicle at the current moment relative to the previous moment is obtained according to the motion state equation, including:

[0182] At the last moment t, a motion state equation of the vehicle is established according to the optimal value of the vehicle speed and the optimal value of the front wheel steering angle, and a rotation and translation matrix from the last moment t to the current moment t+1 is obtained according to the motion state equation;

[0183] At the current time t+1, the top view at the current time t+1 is spliced ​​with the top view at the previous time t, and the rotation and translation matrix of the current time t+1 relative to the previous time t is obtained by matrix inversion.

[0184] At a certain time t, obtain the estimated front wheel steering angle δ of the vehicle opt and vehicle speed v opt (hereinafter, δ and v are used to represent this optimal value);

[0185] At time t, solve the rotation and translation matrix from time t to time t+1;

[0186] Assume that the coordinate position of the vehicle in the preset coordinate system at time t is (x_car t ,y_car t , z_car t ), the coordinate position of the vehicle in the preset coordinate system at time t+1 is (x_car t+1 ,y_car t+1 , z_car t+1 ), the rotation and translation matrix from time t to the next time t+1 is expressed as R t2t+1 and T t2t+1 , then the change in the vehicle coordinate position from the current time t to the next time t+1 can be expressed as follows:

[0187]

[0188] In the process of expressing the vehicle's motion, the vehicle's motion can be simplified to a bicycle model, such as Figure 4 As shown. Among them, ψ is the angle between the vehicle body and the x-axis, δ is the front wheel turning angle of the vehicle, Figure 3 Point C in the figure is the center of mass of the vehicle, l r is the rear overhang length, l f is the front overhang length. Figure 3 The geometric relationship shown in the figure, O is the center of instantaneous rotation of the vehicle, and the line segments OA and OB are perpendicular to the rolling direction of the two wheels. is the distance from the rear wheel to the center of rotation, R is the distance from the center of mass to the center of rotation, x t and t is the position coordinate of the center of mass C at time t in the preset coordinate system, which can be used to solve the vehicle motion state at time t:

[0189]

[0190] x t+1 =x t +v t cos(ψ t +β t )d t

[0191] y t+1 =y t +v t sin(ψ t +β t )d t

[0192] From this, the rotation matrix and translation matrix from time t to time t+1 can be constructed:

[0193]

[0194] At time t+1, the top view at time t+1 is spliced ​​with the top view at time t. At this time, the rotation and translation matrix of time t+1 relative to time t can be obtained by matrix inversion:

[0195] R t =R t+12t =R t2t+1 '

[0196] T t =T t+12t =T t2t+1 '

[0197] The vehicle's motion state equation is directly established through the optimal vehicle speed and the optimal front wheel angle, and the corresponding rotation and translation matrix is ​​obtained, which lays the foundation for the final generation of the transparent chassis image.

[0198] In this embodiment, the step of splicing the top view at the previous moment and the top view at the current moment according to the rotation and translation matrix to obtain the transparent chassis image includes:

[0199] According to the rotation and translation matrix, each pixel of the top view at the previous moment is filled with the top view at the current moment to obtain a transparent chassis image.

[0200] At a certain time t, according to the previous steps, we get R t and T t Matrix, for the current top view F t For example, the transparent chassis needs to fill any position P t , whose pixel coordinates are (u t , v t ), the texture information it needs comes from the top view F of the previous frame t-1 , in this frame of top view, P t The corresponding position of the point is P t-1 , whose pixel coordinates are (u t-1 , v t-1 ), there is the following corresponding relationship

[0201]

[0202] Top view F at time t t Each pixel in the image is transformed in the above formula, and finally a complete filled top view is obtained.

[0203] The vehicle transparent chassis generating device provided by the present invention is described below. The vehicle transparent chassis generating device described below and the vehicle transparent chassis generating method described above can be referred to each other.

[0204] like Figure 5 As shown, the second embodiment of the present invention further provides a vehicle transparent chassis generating device, comprising:

[0205] The parameter acquisition module 210 is used to establish a three-dimensional vehicle coordinate system, calibrate the vehicle sensor according to the three-dimensional vehicle coordinate system, and obtain internal and external parameters.

[0206] The top view acquisition module 220 is used to construct a virtual camera above the vehicle according to the internal and external parameters of the vehicle sensor, and perform a top view projection on the vehicle according to the virtual camera to obtain a top view.

[0207] The optimal value acquisition module 230 is used to acquire the motion data of the vehicle at the current moment, and obtain the optimal value of the vehicle speed and the optimal value of the front wheel steering angle according to the motion data.

[0208] The matrix acquisition module 240 is used to establish the vehicle's motion state equation according to the optimal vehicle speed value and the optimal front wheel steering angle value, and obtain the vehicle's rotation and translation matrix at the current moment relative to the previous moment according to the motion state equation.

[0209] The image acquisition module 250 is used to splice the top view at the previous moment and the top view at the current moment according to the rotation and translation matrix to obtain a transparent chassis image.

[0210] The second embodiment of the present invention provides a vehicle transparent chassis generation device, which establishes a vehicle three-dimensional coordinate system, calibrates the vehicle sensor according to the vehicle three-dimensional coordinate system, and obtains internal and external parameters for subsequent calculation and analysis. A virtual camera is constructed above the vehicle according to the internal and external parameters of the vehicle sensor, and the vehicle is projected from above according to the virtual camera to obtain a top view. The top view is established from the perspective of the virtual camera according to the internal and external parameters as the framework of the transparent chassis image. The motion data of the vehicle at the current moment is obtained, and the optimal value of the vehicle speed and the optimal value of the front wheel angle are obtained according to the motion data. The vehicle speed and the front wheel angle are directly calculated according to the vehicle motion data and the optimal values ​​are taken to ensure the accuracy of subsequent calculations and reduce the burden on computing power. The vehicle's motion state equation is established according to the optimal value of the vehicle speed and the optimal value of the front wheel angle, and the rotation and translation matrix of the vehicle at the current moment relative to the previous moment is obtained according to the motion state equation. The vehicle's motion state equation is directly established through the optimal value of the vehicle speed and the optimal value of the front wheel angle, and the corresponding rotation and translation matrix is ​​obtained to establish a basis for the generation of the final transparent chassis image. Finally, the top view of the previous moment and the top view of the current moment are spliced ​​according to the rotation and translation matrix to obtain a transparent chassis image, which has the basic functions of a surround view transparent chassis, allowing the driver to intuitively feel the vehicle body conditions and road obstacles, eliminating the vehicle body blind spots, and improving driving safety. Compared with the existing transparent chassis implementation schemes using image feature matching, the present invention directly uses vehicle driving information, i.e., vehicle speed and front wheel turning angle, and does not rely on the collected surround view images. It can effectively overcome the display delay, ghosting and jamming problems caused by the image algorithm occupying the processor computing power resources, solve the feature mismatch problem caused by complex road conditions, and achieve transparent chassis generation with small computing power resources and high accuracy.

[0211] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the vehicle transparent chassis generation method, which includes:

[0212] Establishing a three-dimensional vehicle coordinate system, calibrating the vehicle sensor according to the three-dimensional vehicle coordinate system, and obtaining internal and external parameters;

[0213] A virtual camera is constructed above the vehicle according to the internal and external parameters of the vehicle sensor, and a top-down projection of the vehicle is performed according to the virtual camera to obtain a top-down view;

[0214] Acquire the motion data of the vehicle at the current moment, and obtain the optimal value of the vehicle speed and the optimal value of the front wheel turning angle according to the motion data;

[0215] Establishing a motion state equation of the vehicle according to the optimal value of the vehicle speed and the optimal value of the front wheel steering angle, and obtaining a rotation and translation matrix of the vehicle at a current moment relative to a previous moment according to the motion state equation;

[0216] The top view at the previous moment and the top view at the current moment are spliced ​​according to the rotation and translation matrix to obtain a transparent chassis image.

[0217] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0218] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the vehicle transparent chassis generation method provided by the above methods, the method includes:

[0219] Establishing a three-dimensional vehicle coordinate system, calibrating the vehicle sensor according to the three-dimensional vehicle coordinate system, and obtaining internal and external parameters;

[0220] A virtual camera is constructed above the vehicle according to the internal and external parameters of the vehicle sensor, and a top-down projection of the vehicle is performed according to the virtual camera to obtain a top-down view;

[0221] Acquire the motion data of the vehicle at the current moment, and obtain the optimal value of the vehicle speed and the optimal value of the front wheel turning angle according to the motion data;

[0222] Establishing a motion state equation of the vehicle according to the optimal value of the vehicle speed and the optimal value of the front wheel steering angle, and obtaining a rotation and translation matrix of the vehicle at a current moment relative to a previous moment according to the motion state equation;

[0223] The top view at the previous moment and the top view at the current moment are spliced ​​according to the rotation and translation matrix to obtain a transparent chassis image.

[0224] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for generating a vehicle transparent chassis provided by the above methods is implemented, and the method comprises:

[0225] Establishing a three-dimensional vehicle coordinate system, calibrating the vehicle sensor according to the three-dimensional vehicle coordinate system, and obtaining internal and external parameters;

[0226] A virtual camera is constructed above the vehicle according to the internal and external parameters of the vehicle sensor, and a top-down projection of the vehicle is performed according to the virtual camera to obtain a top-down view;

[0227] Acquire the motion data of the vehicle at the current moment, and obtain the optimal value of the vehicle speed and the optimal value of the front wheel turning angle according to the motion data;

[0228] Establishing a motion state equation of the vehicle according to the optimal value of the vehicle speed and the optimal value of the front wheel steering angle, and obtaining a rotation and translation matrix of the vehicle at a current moment relative to a previous moment according to the motion state equation;

[0229] The top view at the previous moment and the top view at the current moment are spliced ​​according to the rotation and translation matrix to obtain a transparent chassis image.

[0230] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0231] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0232] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating a vehicle transparent chassis, characterized in that: include: Establishing a three-dimensional vehicle coordinate system, calibrating the vehicle sensor according to the three-dimensional vehicle coordinate system, and obtaining internal and external parameters; A virtual camera is constructed above the vehicle according to the internal and external parameters of the vehicle sensor, and a top-down projection of the vehicle is performed according to the virtual camera to obtain a top-down view; Acquire the motion data of the vehicle at the current moment, and obtain the optimal value of the vehicle speed and the optimal value of the front wheel turning angle according to the motion data; Establishing a motion state equation of the vehicle according to the optimal value of the vehicle speed and the optimal value of the front wheel steering angle, and obtaining a rotation and translation matrix of the vehicle at a current moment relative to a previous moment according to the motion state equation; The top view at the previous moment and the top view at the current moment are spliced ​​according to the rotation and translation matrix to obtain a transparent chassis image.

2. The method for generating a vehicle transparent chassis according to claim 1, characterized in that: The coordinate origin of the vehicle's three-dimensional coordinate system is established directly below the vehicle's license plate. The Y axis of the vehicle's three-dimensional coordinate system is the vehicle's forward direction, the Z axis is the vertical upward direction, and the X axis is perpendicular to the Y axis and the Z axis respectively.

3. The method for generating a vehicle transparent chassis according to claim 2, characterized in that: The vehicle sensor includes: a distance sensor and a camera device, and the spatial three-dimensional coordinates (X) of each point P (u, v) in the top view under the perspective of the virtual camera p ,Y p ,Z p ) is expressed as: Among them, s represents the scaling factor of the coordinate system, and M represents the intrinsic projection matrix of the virtual camera, which is specifically expressed as: Among them, the top view image is on the ground, assuming Z p =0, we get: According to the intrinsic projection matrix of the camera device, the actual pixel information of point P is expressed as: Among them, M f Represents the intrinsic projection matrix of the camera device.

4. The method for generating a vehicle transparent chassis according to claim 1, characterized in that: The step of obtaining the motion data of the vehicle at the current moment and obtaining the optimal value of the vehicle speed and the optimal value of the front wheel turning angle according to the motion data includes: Get the vehicle's motion data at the current time t, including: vehicle speed v and front wheel steering angle δ. The vehicle speed at time t+1 is expressed as: v(t+1)=v(t)+W v (t)·ΔT Among them, W v (t) represents the random acceleration noise caused by the driver according to the current road conditions, with a mean of 0 and a variance of Q v , ΔT represents the sampling interval of motion data; The observed vehicle speed at time t is expressed as: z v (t)=v(t)+V v (t) Among them, V v (t) represents the transmission and detection noise, with a mean of 0 and a variance of R v ; According to the state optimization algorithm, the state equation is established as follows: v(t+1)=Av(t)+ΓW v (t) According to the formula z v (t)=v(t)+V v (t) The observation equation is established as: z v (t)=Hv(t)+V v (t) According to the formula v(t+1)=v(t)+W v (t)·ΔT and v(t)+1)=Av(t)+ΓW v (t) confirming the hyperparameters of the state equation, and obtaining: A=1, Γ=ΔT; According to the formula z v (t)=v(t)+V v (t) and z v (t) = H v (t)+V v (t) confirming the hyperparameters of the observation equation, and obtaining: H = 1; According to the state optimization algorithm, the initial error covariance P is set v (0) = a, the prior error covariance is expressed as: According to the gain calculation formula, the gain is obtained: The optimal value of vehicle speed is expressed as: v opt (t+1)=v opt (t)+K(t+1)[z v (t+1)-v opt (t)] The corresponding error covariance matrix is ​​expressed as: The front wheel turning angle at time t+1 is expressed as: δ(t+1)=δ(t)+W δ (t) Among them, W δ (t) represents the noise caused by the transmission, with a mean of 0 and a variance of Q δ , ΔT represents the sampling interval of the device; The observed value of the front wheel turning angle at time t is expressed as: z δ (t)=δ(t)+V δ (t) Among them, V δ (t) represents the transmission and detection noise, with a mean of 0 and a variance of R δ . According to the state optimization algorithm, the initial error covariance P is set δ (0) = b; Confirm the hyperparameters of the state equation and observation equation and obtain: A = 1, Γ = 1, H = 1; The prior error covariance is then expressed as: According to the gain calculation formula, the gain is obtained: The optimal value of the front wheel steering angle is expressed as: δ opt (t+1)=δ opt (t)+K(t+1)[z δ (t+1)-δ opt (t)] The corresponding error covariance matrix is ​​expressed as:

5. The method for generating a vehicle transparent chassis according to claim 4, characterized in that: The state optimization algorithm is a Kalman filter optimization algorithm.

6. The method for generating a vehicle transparent chassis according to claim 1, characterized in that: The step of establishing a vehicle motion state equation according to the optimal vehicle speed value and the optimal front wheel steering angle value, and obtaining a rotation and translation matrix of the vehicle at a current moment relative to a previous moment according to the motion state equation, includes: At the last moment t, a motion state equation of the vehicle is established according to the optimal value of the vehicle speed and the optimal value of the front wheel steering angle, and a rotation and translation matrix from the last moment t to the current moment t+1 is obtained according to the motion state equation; At the current time t+1, the top view at the current time t+1 is spliced ​​with the top view at the previous time t, and the rotation and translation matrix of the current time t+1 relative to the previous time t is obtained by matrix inversion.

7. The method for generating a vehicle transparent chassis according to claim 6, characterized in that: The step of stitching the top view at the previous moment and the top view at the current moment according to the rotation and translation matrix to obtain a transparent chassis image includes: According to the rotation and translation matrix, each pixel of the top view at the previous moment is filled with the top view at the current moment to obtain a transparent chassis image.

8. A vehicle transparent chassis generating device, characterized in that: include: A parameter acquisition module, used to establish a three-dimensional vehicle coordinate system, calibrate the vehicle sensor according to the three-dimensional vehicle coordinate system, and obtain internal and external parameters; A top view acquisition module, used to construct a virtual camera above the vehicle according to the internal and external parameters of the vehicle sensor, and to perform a top view projection of the vehicle according to the virtual camera to obtain a top view; An optimal value acquisition module, used to acquire the motion data of the vehicle at the current moment, and obtain the optimal value of the vehicle speed and the optimal value of the front wheel steering angle according to the motion data; A matrix acquisition module, used to establish a vehicle motion state equation according to the optimal vehicle speed value and the optimal front wheel steering angle value, and obtain a rotation and translation matrix of the vehicle at a current moment relative to a previous moment according to the motion state equation; The image acquisition module is used to splice the top view at the previous moment and the top view at the current moment according to the rotation and translation matrix to obtain a transparent chassis image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the vehicle transparent chassis generation method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for generating a vehicle transparent chassis according to any one of claims 1 to 7 is implemented.

Images