Vehicle bottom perspective method and device
Through image stitching and rendering technology, a perspective image of the vehicle bottom is generated, which solves the problem that traditional panoramic systems cannot display the vehicle bottom image, and improves driving experience and safety.
Patent Information
- Application Number
- CN202110740603.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-06-30
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-06-30
AI Technical Summary
Traditional vehicle panoramic systems cannot display images of the bottom of the vehicle, resulting in users being unable to obtain environmental information on the bottom of the vehicle body, affecting driving experience and safety.
By obtaining the current panoramic video image, determining the vehicle speed and wheel angle signals, combining the camera internal and external parameters, the corresponding relationship between the occlusion area at the bottom of the vehicle and the historical image is calculated, the image is stitched, and the preset vehicle model is rendered onto the virtual image. Generate a perspective image of the bottom of the vehicle.
It realizes perspective on the bottom of the vehicle, improves driving experience and safety index, and helps drivers have a more comprehensive understanding of the driving environment.
Smart Images

Figure CN115209037B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of assisted driving, and in particular to a vehicle bottom perspective method and device. Background Art
[0002] With the rapid development of automotive electronic technology, users have higher and higher requirements for vehicle driving experience. For example, the traditional vehicle panoramic system can only display the visible range captured by the camera around the vehicle body, but cannot display the image of the bottom of the vehicle, so it cannot obtain the environmental information of the bottom of the vehicle body, resulting in poor user experience.
[0003] Therefore, there is an urgent need to improve the existing panoramic system to achieve vehicle bottom perspective, thereby helping the driver to understand the driving environment more comprehensively and improving driving safety. Summary of the invention
[0004] The present invention provides a vehicle bottom perspective method and device, aiming to solve the defects in the prior art, realize the perspective of the vehicle bottom, and improve the driving experience and safety index.
[0005] In order to achieve the above object, the technical solution adopted by the present invention is:
[0006] One aspect of the present invention provides a vehicle bottom perspective method, comprising:
[0007] Step 1: Get the current panoramic video image;
[0008] Step 2: Determine the current vehicle speed;
[0009] Step 3: determining a wheel angle signal according to the current steering wheel angle signal;
[0010] Step 4: Obtain the internal and external parameters of the camera module;
[0011] Step 5: Calculate a first corresponding relationship based on the current vehicle speed, the wheel angle signal, the camera module internal parameters and external parameters, where the first corresponding relationship is a corresponding relationship between the blocked area at the bottom of the vehicle at the current moment and the image of the same area in the historical video image;
[0012] Step 6: read the corresponding historical video image and the current video image according to the first corresponding relationship, and splice them to obtain a virtual vehicle bottom image;
[0013] Step 7: Rendering the preset vehicle model onto the virtual vehicle bottom image to obtain a vehicle bottom perspective image.
[0014] Furthermore, after step 7, the method further includes the following steps:
[0015] Step 8: Encode the vehicle bottom perspective image into a video format that matches a display terminal and then output it.
[0016] Specifically, the step 6 includes:
[0017] Step 601, determining a vehicle running trajectory equation between two frames of images according to the current vehicle speed, the current wheel turning angle and the historical vehicle speed and historical wheel turning angle corresponding to the previous frame of image;
[0018] Step 602: Determine the image trajectory equation of the same feature point on the image according to the vehicle trajectory equation, the camera module internal parameters and external parameters;
[0019] Step 603, calculating the second position of the first position of the feature point at the current moment in the previous frame image according to the image running trajectory equation;
[0020] Step 604: According to the second position, read the corresponding historical video image and stitch it with the current panoramic video image to obtain a virtual vehicle bottom image.
[0021] Specifically, step 604 includes:
[0022] Step 604-a, taking the interval formed by the first position, the second position and the corresponding image motion trajectory as a stitching area;
[0023] Step 602-b: fuse the pixels in the stitching area using a weighted method.
[0024] Specifically, the step 7 includes:
[0025] Step 71-a, drawing the virtual vehicle bottom image as a background image;
[0026] Step 71-b, loading the preset vehicle model image as the first foreground image into the temporary cache;
[0027] Step 71-c, reading the first pixel value of each pixel point of the first foreground image, and setting the transparency of each pixel value to a preset transparency, to obtain a second pixel value;
[0028] Step 71-d, writing the second pixel value into the corresponding pixel point of the foreground image to generate a second foreground image;
[0029] Step 71 - e: read the second foreground image from the temporary cache and draw it.
[0030] Specifically, the step 7 includes:
[0031] Step 72-a, using the virtual vehicle bottom image as a background texture;
[0032] Step 72-b, reading the pre-stored vehicle model image into the local cache;
[0033] Step 72-c, calling the OPENGL interface to load the vehicle model image as a foreground rendering texture;
[0034] Step 72-e: call the OPENGL rendering interface to render the foreground rendering texture onto the background texture.
[0035] Specifically, the step 2 includes:
[0036] Step 201, determining whether the current vehicle speed is less than a first speed threshold, if yes, proceeding to the next step, otherwise calculating the current vehicle speed according to the data sent by the wheel speed sensor;
[0037] Step 202: Use the optical flow method to detect the instantaneous speed of the camera, and calculate the current vehicle speed according to the instantaneous speed of the camera.
[0038] Specifically, in step 202, using the optical flow method to detect the instantaneous speed of the camera includes:
[0039] Step 202-a, calibrating a first relationship between the image coordinate system of the camera and the world coordinate system;
[0040] Step 202-b, extracting two consecutive frames of images from the historical video image as a first image frame and a second image frame;
[0041] Step 202-c, constructing a first image pyramid and a second image pyramid of preset layers for the first image frame and the second image frame respectively;
[0042] Step 202-d, calculating a first optical flow value, where the first optical flow value is the optical flow of the top layer of the image pyramid;
[0043] Step 202-e, using the first optical flow value as an iterative initial value for calculating the optical flow of the previous layer of the image pyramid, and performing an iterative operation;
[0044] Step 202-f, determine whether the current layer number is 0, if yes, output the current optical flow value, otherwise return to execute step 202-e;
[0045] Step 202-g, calculating the horizontal optical flow velocity and the vertical optical flow velocity of any point in the first image frame according to the current optical flow value;
[0046] Step 202-h, calculating the world coordinates of any point in the first image frame according to the first relationship, the horizontal optical flow velocity and the vertical optical flow velocity;
[0047] Step 202-i, calculate the current vehicle speed V according to the world coordinates.
[0048] Specifically, the method for determining the number of layers of the image pyramid in step 202-c includes:
[0049] Step c1, shifting each pixel of the first image frame a preset number of times according to a preset pixel step length, subtracting the image after each pixel shift from the second image frame, and recording the difference result;
[0050] Step c2, determining a first value assignment set for each pixel point in the first image frame, wherein the first value assignment set is a set of current translation pixel values corresponding to the minimum value in the difference results of each pixel point;
[0051] Step c3, counting the number of identical values in the first value set, and calculating a first ratio set of the identical values, where the first ratio set is a set of ratios of the number of identical values to the total number of pixels in the first image frame;
[0052] Step c4: determining a second value assignment set according to the first ratio set, wherein the second value assignment set is a set of identical values corresponding to each ratio in the first ratio set that is greater than a preset ratio threshold;
[0053] Step c5, selecting the largest one from the second value assignment set as the maximum movement value;
[0054] Step c6: determining the number of layers of the image pyramid according to the maximum movement value.
[0055] Specifically, m={B / 20}+2, where m represents the number of layers of the image pyramid, B represents the maximum movement value, and {} represents rounding up.
[0056] Another aspect of the present invention provides a vehicle bottom perspective device, comprising:
[0057] A splicing module, and a real-time video data acquisition module, a historical video data storage module, a vehicle speed determination module, a turning angle determination module, and a rendering module connected to the splicing module; the real-time video data acquisition module is also connected to the camera module and the historical video data storage module, and the vehicle speed determination module is also connected to the wheel speed sensor;
[0058] The camera module includes a plurality of cameras, and the images captured by the cameras constitute a panoramic image of the vehicle from above;
[0059] The real-time video data acquisition module is used to acquire the video image at the current moment from the camera module and store it in the historical video data storage module;
[0060] The historical video data storage module is used to store the video images taken by the camera module;
[0061] The wheel speed sensor is used to output wheel rotation data;
[0062] The vehicle speed determination module is used to determine the current vehicle speed according to the relevant signal;
[0063] The turning angle determination module is used to determine the direction and degree of the wheel turning angle according to the turning direction and degree of the steering wheel;
[0064] The splicing module is used to calculate a first corresponding relationship through the current vehicle speed, the wheel angle signal, the internal parameters and external parameters of the camera module, and read the corresponding historical video image and the current video image according to the first corresponding relationship to splice them to obtain a virtual vehicle bottom image, wherein the first corresponding relationship is the corresponding relationship between the blocked area of the vehicle bottom at the current moment and the image of the same area in the historical video image;
[0065] The rendering module is used to render a preset vehicle model onto the virtual vehicle bottom image to obtain a vehicle bottom perspective image.
[0066] Specifically, the vehicle speed determination module calculates the current vehicle speed according to the data sent by the wheel speed sensor.
[0067] Furthermore, the vehicle bottom perspective device also includes: a video encoding module connected to the splicing module, which is used to encode the vehicle bottom perspective image into a video format matching a display terminal and then output it.
[0068] Furthermore, the vehicle bottom perspective device also includes: an optical flow detection module connected to the vehicle speed determination module, used to calculate the instantaneous speed of the camera through the LK optical flow method, and the optical flow detection module is started when the vehicle speed is lower than the first speed threshold and is turned off when the vehicle speed is higher than the first speed threshold.
[0069] Specifically, the vehicle speed determination module calculates the wheel speed of the vehicle according to the instantaneous speed of the camera calculated by the optical flow detection module, combined with the Ackerman steering principle, the internal parameters and external parameters of the camera module.
[0070] The beneficial effects of the present invention are as follows: the present invention determines a wheel angle signal according to a current steering wheel angle signal, and calculates a first corresponding relationship according to the current vehicle speed, the wheel angle signal, the internal parameters and the external parameters of the camera module, and then reads the corresponding historical video image according to the first corresponding relationship and splices it with the current video image to obtain a virtual vehicle bottom image, and finally renders a preset vehicle model onto the virtual vehicle bottom image to obtain a vehicle bottom perspective image, thereby realizing the perspective of the vehicle bottom and improving the driving experience and safety index. BRIEF DESCRIPTION OF THE DRAWINGS
[0071] Figure 1 It is a schematic flow chart of the vehicle bottom perspective method of the present invention;
[0072] Figure 2 It is a structural schematic diagram of the vehicle bottom perspective device of the present invention;
[0073] Figure 3 is another structural schematic diagram of the vehicle bottom perspective device of the present invention;
[0074] Figure 4 It is another structural schematic diagram of the vehicle bottom perspective device of the present invention. DETAILED DESCRIPTION
[0075] The following specifically describes the embodiments of the present invention in conjunction with the accompanying drawings, which are only for reference and illustration and do not constitute a limitation on the scope of patent protection of the present invention.
[0076] Example 1
[0077] like Figure 1 As shown, this embodiment provides a vehicle bottom perspective method, including:
[0078] Step 1: Get the current panoramic video image.
[0079] The current panoramic video image can be obtained by acquiring the road images taken by the panoramic system camera and stitching them together.
[0080] Step 2: Determine the current vehicle speed.
[0081] In a specific implementation, the current vehicle speed may be calculated based on data sent by a wheel speed sensor.
[0082] Step 3: Determine the wheel angle signal according to the current steering wheel angle signal.
[0083] Step 4: Get the internal and external parameters of the camera module.
[0084] Step 5: Calculate a first correspondence based on the current vehicle speed, the wheel angle signal, the internal and external parameters of the camera module. The first correspondence is a correspondence between the blocked area at the bottom of the vehicle at the current moment and the image of the same area in the historical video image.
[0085] Step 6: read the corresponding historical video image according to the first corresponding relationship and splice it with the current video image to obtain a virtual vehicle bottom image.
[0086] It is easy to understand that the virtual vehicle bottom image is a part of the panoramic image.
[0087] Step 7: Rendering the preset vehicle model onto the virtual vehicle bottom image to obtain a vehicle bottom perspective image.
[0088] In another embodiment of the present invention, the step 7 is followed by the following steps:
[0089] Step 8: Encode the vehicle bottom perspective image into a video format that matches a display terminal and then output it.
[0090] Example 2
[0091] Different from Example 1, this example provides a specific implementation method of step 6, including:
[0092] Step 601: Determine the vehicle trajectory equation V(x, y) between two frames of images according to the current vehicle speed, the current wheel turning angle, and the historical vehicle speed and historical wheel turning angle corresponding to the previous frame of image.
[0093] The angle change of the steering wheel curve and the moving distance of the vehicle form the vehicle's running trajectory. Therefore, by calculating the vehicle speed signal and the steering wheel angle signal in real time, the vehicle running trajectory equation V(x,y) can be obtained.
[0094] For example, the steering wheel is turned 15 degrees to the left, the vehicle is driven for 1 meter, and then turned 15 degrees to the right, and the vehicle is driven for 1 meter. The entire process is the vehicle's trajectory formed by the curved angle change of the steering wheel and the moving distance.
[0095] Step 602: Determine the image trajectory equation M(u,v) of the same feature point on the image according to the vehicle trajectory equation V(x,y) and the camera module internal and external parameters.
[0096] There is a predetermined relationship between a point on the road (i.e., a point in the world coordinate system) and its corresponding point in the image. This predetermined relationship can be obtained through the homography matrix H, and then combined with the Ackermann steering principle to obtain the image trajectory equation M(u,v) of the point.
[0097] Step 603: Calculate the second position p(u0', v0') of the first position p(u0, v0) of the feature point at the current moment in the previous frame image according to the image trajectory equation M(u, v).
[0098] Step 604: According to the second position p(u0′, v0′), read the corresponding historical video image and stitch it with the current panoramic video image to obtain a virtual vehicle bottom image.
[0099] In this embodiment, step 604 includes:
[0100] Step 604 - a, taking the interval formed by the first position p (u0, v0), the second position p (u0′, v0′) and the corresponding image motion trajectory as the stitching area.
[0101] Step 602-b: fuse the pixels in the stitching area using a weighted method.
[0102] Example 3
[0103] Different from Example 1, this example provides a specific implementation method of step 7, including:
[0104] Step 71 - a, drawing the virtual vehicle bottom image as a background image.
[0105] Step 71 - b: load the preset vehicle model image into the temporary cache as the first foreground image.
[0106] Step 71 - c, reading the first pixel value of each pixel point of the first foreground image, and setting the transparency of each pixel value to a preset transparency, to obtain a second pixel value.
[0107] In this embodiment, the preset transparency is 125.
[0108] Step 71 - d: write the second pixel value into the corresponding pixel point of the foreground image to generate a second foreground image.
[0109] Step 71 - e: read the second foreground image from the temporary cache and draw it.
[0110] Example 4
[0111] Different from Example 1, this example provides another specific implementation method of step 7, including:
[0112] Step 72-a: Use the virtual vehicle bottom image as a background texture.
[0113] Step 72-b, read the pre-stored vehicle model image into the local cache, wherein the vehicle model image is a 4-channel RGBAPNG image, wherein the A channel contains transparency information.
[0114] Step 72-c, calling the OPENGL interface to load the vehicle model image as the foreground rendering texture.
[0115] Step 72-e: call the OPENGL rendering interface to render the foreground rendering texture onto the background texture.
[0116] Example 5
[0117] Different from Example 1, step 2 includes:
[0118] Step 201, determine whether the current vehicle speed is less than a first speed threshold, if yes, proceed to the next step, otherwise calculate the current vehicle speed according to the data sent by the wheel speed sensor.
[0119] In this embodiment, the first speed threshold is 2-4 km / h.
[0120] Step 202: Use the optical flow method to detect the instantaneous speed of the camera, and calculate the current vehicle speed according to the instantaneous speed of the camera.
[0121] Example 6
[0122] Different from Embodiment 5, this embodiment provides a specific implementation method of using the optical flow method to detect the instantaneous speed of the camera in step 202, including:
[0123] Step 202 - a, calibrate a first relationship between the image coordinate system of the camera and the world coordinate system.
[0124] The first relationship is:
[0125]
[0126] Among them, (Xw, Yw, Zw) represents the coordinates of the calibration point in the world coordinate system, (u, v) represents the coordinates of the calibration point in the image coordinate system, h represents the height of the camera from the ground, d represents the horizontal distance between the camera and the calibration object, Ki represents the intrinsic parameter of the camera, Ko represents the extrinsic parameter of the camera, and the intrinsic and extrinsic parameters of the camera can be obtained through calibration.
[0127] Step 202 - b, extracting two consecutive frames of images from the historical video image as a first image frame P1 and a second image frame P2.
[0128] The first image frame P1 is an image frame at time t, and the second image frame P2 is an image frame after an interval of 1 / f time. It is easy to understand that the first image frame P1 and the second image frame P2 are both extracted from the video image captured by the camera after the vehicle speed drops to the first speed threshold.
[0129] Step 202 - c : constructing a first image pyramid IP1 and a second image pyramid IP2 for the first image frame P1 and the second image frame P2 , respectively. The number of layers of the first image pyramid IP1 and the second image pyramid IP2 is m.
[0130] An image pyramid is a collection of images of different resolutions originating from the same original image. These images are arranged in a pyramid shape with gradually decreasing resolutions. The bottom of the pyramid is a high-resolution representation of the image to be processed (such as the first image frame P1 and the second image frame P2), and the top is a low-resolution approximation of these images to be processed, that is, the higher the level, the smaller the image and the lower the resolution.
[0131] Step 202 - d: Calculate a first optical flow value, where the first optical flow value is the optical flow (um, vm) of the top layer of the image pyramid.
[0132] The top layer of the image pyramid is the layer with the lowest resolution, that is, the mth layer.
[0133] Step 202-e: Use the first optical flow value (um, vm) as the iterative initial value of the optical flow calculation of the previous layer of the image pyramid to perform an iterative operation.
[0134] The previous layer of the image pyramid is a layer with a resolution increased by one level. For example, if the current layer is the kth layer, the previous layer is the k-1th layer.
[0135] Step 202-f, determine whether the current layer number k is 0, if so, output the current optical flow value, otherwise return to execute step 202-e.
[0136] Step 202-g, calculating the horizontal optical flow velocity u and the vertical optical flow velocity v of any point in the first image frame according to the current optical flow value.
[0137] Step 202-h: Calculate the world coordinates (xw, yw) of any point in the first image frame according to the first relationship.
[0138] Step 202-i, calculate the current vehicle speed V according to the world coordinates (xw, yw).
[0139] In this embodiment, the current vehicle speed Where f represents the frame rate of the camera.
[0140] Example 7
[0141] Different from Embodiment 6, this embodiment provides a specific method for determining the number of layers m of the image pyramid in step 202-c, including:
[0142] Step c1, shift each pixel of the first image frame P1 a preset number of times according to a preset pixel step, subtract the image after each pixel shift from the second image frame P2, and record the difference result T(q) (q=1,2,3...Q).
[0143] For example, the preset pixel step is p pixels, and the preset number of times is Q times. The first translation is p pixels, the second translation is 2p pixels, the third translation is 3p pixels... and so on. The Qth translation is Q*p pixels. The values of p and Q can be set according to the experimental results.
[0144] Obviously, T(q) is a matrix of size m*n, where m is the horizontal resolution of the image and n is the vertical resolution; each element in the matrix is a 1*Q subarray.
[0145] Step c2: determine a first value assignment set for each pixel point in the first image frame P1, where the first value assignment set is a set of current translation pixel values corresponding to the minimum value in the difference results of each pixel point.
[0146] For example, for the pixel point with a coordinate value of (100, 200) in the first image frame P1, after the image is translated Q times and subtracted from the second image frame P2, it has Q difference values, such as -1, 2, -3...8, a total of Q. The minimum value of these Q differences is -10, and the corresponding number of movements is the 10th time, that is, the current translation pixel value is 10*q. Then the pixel point with a coordinate value of (100, 200) is assigned the current translation pixel value 10*q.
[0147] The pixel points with other coordinate values are determined in the same way. It is easy to understand that the values corresponding to each pixel point may be the same or different. The values assigned to all pixel points constitute the first value set.
[0148] Step c3, counting the number of identical values in the first value set, and calculating a first ratio set of the identical values, wherein the first ratio set is a set of ratios of the number of identical values to the total number of pixels in the first image frame P1.
[0149] For example, the number of pixels assigned with a value of 10*q is 200,000, and the total number of pixels in the first image frame P1 is 480,000. Then, the proportion of the number of pixels assigned with a value of 10*q is 200,000 / 480,000=41.7%.
[0150] Step c4: determine a second value assignment set according to the first ratio set, where the second value assignment set is a set of identical values corresponding to each ratio in the first ratio set that is greater than a preset ratio threshold.
[0151] The preset ratio threshold can be obtained through experimental effect calibration.
[0152] Step c5: Select the largest one from the second value assignment set as the maximum movement value.
[0153] Step c6: Determine the number of layers of the image pyramid according to the maximum movement value B.
[0154] In this embodiment, m={B / 20}+2, where m represents the number of layers of the image pyramid, B represents the maximum movement value, and {} represents rounding up.
[0155] Example 8
[0156] like Figure 2 As shown, this embodiment provides a vehicle bottom perspective device, including:
[0157] A splicing module, and a real-time video data acquisition module, a historical video data storage module, a vehicle speed determination module, a turning angle determination module, and a rendering module connected to the splicing module; the real-time video data acquisition module is also connected to the camera module and the historical video data storage module, and the vehicle speed determination module is also connected to the wheel speed sensor;
[0158] The camera module includes a plurality of cameras, and the images captured by the cameras constitute a panoramic image of the vehicle from above;
[0159] The real-time video data acquisition module is used to acquire the video image at the current moment from the camera module and store it in the historical video data storage module;
[0160] The historical video data storage module is used to store the video images taken by the camera module;
[0161] The wheel speed sensor is used to output wheel rotation data;
[0162] The vehicle speed determination module is used to determine the current vehicle speed according to the relevant signal;
[0163] The turning angle determination module is used to determine the direction and degree of the wheel turning angle according to the turning direction and degree of the steering wheel;
[0164] The splicing module is used to calculate a first corresponding relationship through the current vehicle speed, the wheel angle signal, the internal parameters and external parameters of the camera module, and read the corresponding historical video image and the current video image according to the first corresponding relationship to splice them to obtain a virtual vehicle bottom image, wherein the first corresponding relationship is the corresponding relationship between the blocked area of the vehicle bottom at the current moment and the image of the same area in the historical video image;
[0165] The rendering module is used to render a preset vehicle model onto the virtual vehicle bottom image to obtain a vehicle bottom perspective image.
[0166] As an implementable example, the camera module includes four cameras, which are respectively installed on the front bumper, rear bumper, left exterior mirror and right exterior mirror of the vehicle. Of course, according to actual needs or with the advancement of technology, the number of cameras in the camera module is not necessarily four, and the present invention does not limit this, as long as the images captured by the cameras can form a panoramic image of the vehicle from above.
[0167] In this embodiment, the vehicle speed determination module calculates the current vehicle speed based on data sent by the wheel speed sensor.
[0168] Example 9
[0169] The vehicle speed determination module in Example 1 calculates the current vehicle speed based on the data sent by the wheel speed sensor. However, when the vehicle is traveling at a low speed (the speed is less than 2 km / h), the data sent by the wheel speed sensor has a large error, resulting in a large cumulative error in the first corresponding relationship calculated by the splicing module, which in turn causes a certain misalignment between the virtual vehicle bottom image and the actual situation.
[0170] Therefore, if Figure 3 As shown, different from Example 1, the vehicle bottom perspective device of this embodiment adds an optical flow detection module connected to the vehicle speed determination module, which is used to calculate the instantaneous speed of the camera through the LK optical flow method. The optical flow detection module is started when the vehicle speed is lower than the first speed threshold (for example, 2km / h) and is turned off when the vehicle speed is higher than the first speed threshold (for example, 3km / h).
[0171] The vehicle speed determination module calculates the wheel speed of the vehicle according to the instantaneous speed of the camera calculated by the optical flow detection module, combined with the Ackerman steering principle, the internal and external parameters of the camera module.
[0172] The stitching module calculates the first corresponding relationship according to the wheel speed calculated by the vehicle speed determination module, thereby achieving accurate image stitching when the vehicle is traveling at a low speed.
[0173] Example 10
[0174] like Figure 4 As shown, different from Embodiments 8 and 9, the vehicle bottom perspective device of this embodiment further includes: a video encoding module connected to the splicing module, for encoding the vehicle bottom perspective image into a video format matching the display terminal and then outputting it.
[0175] The above disclosure is only a preferred embodiment of the present invention, which cannot be used to limit the scope of protection of the present invention. Therefore, equivalent changes made according to the scope of the patent application of the present invention are still within the scope covered by the present invention.
Claims
1. A vehicle bottom perspective method, characterized in that: include: Step 1: Get the current panoramic video image; Step 2: Determine the current vehicle speed; Step 3: determining a wheel angle signal according to the current steering wheel angle signal; Step 4: Obtain the internal and external parameters of the camera module; Step 5: Calculate a first corresponding relationship based on the current vehicle speed, the wheel angle signal, the camera module internal parameters and external parameters, where the first corresponding relationship is a corresponding relationship between the blocked area at the bottom of the vehicle at the current moment and the image of the same area in the historical video image; Step 6: read the corresponding historical video image and the current video image according to the first corresponding relationship, and splice them to obtain a virtual vehicle bottom image; Step 7: Rendering the preset vehicle model onto the virtual vehicle bottom image to obtain a vehicle bottom perspective image; The step 2 comprises: Step 201, determining whether the current vehicle speed is less than a first speed threshold, if yes, proceeding to the next step, otherwise calculating the current vehicle speed according to the data sent by the wheel speed sensor; Step 202: Detect the instantaneous speed of the camera using the optical flow method, and calculate the current vehicle speed according to the instantaneous speed of the camera; The step 202 of using the optical flow method to detect the instantaneous speed of the camera includes: Step 202-a, calibrating a first relationship between the image coordinate system of the camera and the world coordinate system; Step 202-b, extracting two consecutive frames of images from the historical video image as a first image frame and a second image frame; Step 202-c, constructing a first image pyramid and a second image pyramid of preset layers for the first image frame and the second image frame respectively; Step 202-d, calculating a first optical flow value, where the first optical flow value is the optical flow of the top layer of the image pyramid; Step 202-e, using the first optical flow value as an iterative initial value for calculating the optical flow of the previous layer of the image pyramid, and performing an iterative operation; Step 202-f, determine whether the current layer number is 0, if yes, output the current optical flow value, otherwise return to execute step 202-e; Step 202-g, calculating the horizontal optical flow velocity and the vertical optical flow velocity of any point in the first image frame according to the current optical flow value; Step 202-h, calculating the world coordinates of any point in the first image frame according to the first relationship, the horizontal optical flow velocity and the vertical optical flow velocity; Step 202-i, calculating the current vehicle speed V according to the world coordinates; The method for determining the number of layers of the image pyramid in step 202-c includes: Step c1, shifting each pixel of the first image frame a preset number of times according to a preset pixel step length, subtracting the image after each pixel shift from the second image frame, and recording the difference result; Step c2, determining a first value assignment set for each pixel point in the first image frame, wherein the first value assignment set is a set of current translation pixel values corresponding to the minimum value in the difference results of each pixel point; Step c3, counting the number of identical values in the first value set, and calculating a first ratio set of the identical values, where the first ratio set is a set of ratios of the number of identical values to the total number of pixels in the first image frame; Step c4: determining a second value assignment set according to the first ratio set, wherein the second value assignment set is a set of identical values corresponding to each ratio in the first ratio set that is greater than a preset ratio threshold; Step c5, selecting the largest one from the second value assignment set as the maximum movement value; Step c6, determining the number of layers of the image pyramid according to the maximum movement value; The number of layers of the image pyramid is m={B / 20}+2, where m represents the number of layers of the image pyramid, B represents the maximum movement value, and {} represents rounding up.
2. The vehicle bottom perspective method according to claim 1, characterized in that: After step 7, the method further includes the following steps: Step 8: Encode the vehicle bottom perspective image into a video format that matches a display terminal and then output it.
3. The vehicle bottom perspective method according to claim 1, characterized in that: The step 6 comprises: Step 601, determining a vehicle running trajectory equation between two frames of images according to the current vehicle speed, the current wheel turning angle and the historical vehicle speed and historical wheel turning angle corresponding to the previous frame of image; Step 602: Determine the image trajectory equation of the same feature point on the image according to the vehicle trajectory equation, the camera module internal parameters and external parameters; Step 603, calculating the second position of the first position of the feature point at the current moment in the previous frame image according to the image running trajectory equation; Step 604: According to the second position, read the corresponding historical video image and stitch it with the current panoramic video image to obtain a virtual vehicle bottom image.
4. The vehicle bottom perspective method according to claim 3, characterized in that: The step 604 includes: Step 604-a, taking the interval formed by the first position, the second position and the corresponding image motion trajectory as a stitching area; Step 602-b: fuse the pixels in the stitching area using a weighted method.
5. The vehicle bottom perspective method according to claim 1, characterized in that: The step 7 comprises: Step 71-a, drawing the virtual vehicle bottom image as a background image; Step 71-b, loading the preset vehicle model image as the first foreground image into the temporary cache; Step 71-c, reading the first pixel value of each pixel point of the first foreground image, and setting the transparency of each pixel value to a preset transparency, to obtain a second pixel value; Step 71-d, writing the second pixel value into the corresponding pixel point of the foreground image to generate a second foreground image; Step 71 - e: read the second foreground image from the temporary cache and draw it.
6. The vehicle bottom perspective method according to claim 1, characterized in that: The step 7 comprises: Step 72-a, using the virtual vehicle bottom image as a background texture; Step 72-b, reading the pre-stored vehicle model image into the local cache; Step 72-c, calling the OPENGL interface to load the vehicle model image as a foreground rendering texture; Step 72-e: call the OPENGL rendering interface to render the foreground rendering texture onto the background texture.
