Method, device and system for correlating underbody area buffer images and storage medium

Abstract

The application discloses a related method, device and system for a vehicle bottom area buffer image and a storage medium, wherein the updating method of the vehicle bottom area buffer image comprises the following steps: acquiring a current frame vehicle bottom area image of a vehicle by using a current frame vehicle bottom area buffer image; judging whether the vehicle bottom area buffer image needs to be updated based on a current state parameter of the vehicle and a buffer interval; if yes, obtaining a next frame vehicle bottom area buffer image by using the current frame vehicle bottom area image; and if not, taking the current frame vehicle bottom area buffer image as the next frame vehicle bottom area buffer image. The above scheme can display the road conditions under the vehicle in real time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle electronics, in particular to a related method, device, system and storage medium for buffering images of a vehicle bottom area. BACKGROUND

[0002] The current mature panoramic image technology 360-degree panoramic image is a kind of more intuitive, safer and more reliable auxiliary driving technology, which can view the panorama around the vehicle through the vehicle display screen, has a super wide viewing angle, and can eliminate the visual blind area. The common practice is to install four wide-angle fisheye cameras around the vehicle to cover all the field of view of the vehicle, to splice the multi-path video images collected at the same time, to synthesize a panoramic surround view containing 360-degree view of the vehicle surroundings, and finally to display it on the screen of the center console. However, the traditional 360-degree panoramic image can only view the scenery around the vehicle in real time, and cannot display the road conditions under the vehicle in real time. The driver cannot see the road conditions under the vehicle, which poses a certain safety hazard.

[0003] The existing scheme can fill the vehicle bottom part of the current frame or the current moment by buffering the fisheye image and the vehicle bottom image data of the previous frame or the previous moment, or by buffering the spliced surround view data (containing vehicle bottom image data) of the previous frame or the previous moment to fill the vehicle bottom part of the current frame or the current moment. Through frequent buffering and filling operations, the complete perspective effect of the vehicle bottom can be achieved.

[0004] However, the existing scheme has the following problems: (1) using buffer data to fill the bottom part of the vehicle, which is theoretically a one-time image sampling process. Since there is a large overlap between the bottom parts of two consecutive frames, if the bottom data of the previous frame is used to fill the corresponding bottom area of the current frame, it will inevitably cause frequent sampling of a certain area of the bottom, ultimately leading to an abnormally blurred bottom scene, which is very different from the clarity of the surrounding scene. If the data at the previous time is used to fill the bottom area at the current time, the time interval in between is difficult to determine. If it is set too short, the bottom image will become blurred, and if it is set too long, it is not suitable for scenes with high speed, and it is also difficult to determine the number of buffer pictures. (2) Frequent buffering operation or large amount of buffer data. If the data of the previous frame is buffered for the current frame, data needs to be buffered and released every frame. If the data at a certain historical time is buffered for a certain time later, multiple frames of data may need to be buffered. Both of these two schemes have high requirements for the performance of cpu or gpu, which is not conducive to the actual application implementation. (3) If the fisheye data is buffered to fill the bottom area, since the original image data has not been subjected to brightness equalization, when the brightness of the current, previous, left and right four images is inconsistent, there will be a significant brightness imbalance problem in the display of the bottom part and the surrounding area. If the panoramic surround view is directly buffered, since the surround view is an image displayed on the screen after four fisheye original images are spliced, although it may solve the brightness equalization problem, frequent and large amount of data buffering on the gpu is very performance-consuming and is not conducive to the actual application implementation. SUMMARY

[0005] The technical problem solved by the present application is to provide a related method, device, system and storage medium for buffering images of the bottom area of a vehicle, which can display the road conditions of the bottom area of the vehicle in real time.

[0006] To solve the above problems, the first aspect of the present application provides a method for updating the buffer image of the bottom area of a vehicle, which comprises: obtaining the current frame bottom area image of the vehicle by using the current frame bottom area buffer image; determining whether the bottom area buffer image needs to be updated based on the current state parameters of the vehicle and the buffer interval; if so, obtaining the next frame bottom area buffer image by using the current frame bottom area image; if not, using the current frame bottom area buffer image as the next frame bottom area buffer image.

[0007] To solve the above problems, the second aspect of the present application provides a display method of vehicle panoramic image, the method comprising: acquiring current frame images of four sides of a vehicle and current state parameters of the vehicle; obtaining a current frame surrounding area image of the vehicle by using the current frame images of four sides of the vehicle and texture coordinates of a surrounding area of the vehicle in the current frame images of four sides of the vehicle; determining texture coordinates of a bottom area in a current frame bottom area buffer image by using the current state parameters of the vehicle, and obtaining a current frame bottom area image of the vehicle; obtaining a current frame panoramic overhead splicing image of the vehicle according to the current frame surrounding area image and the current frame bottom area image, and displaying the current frame panoramic overhead splicing image; wherein the current frame bottom area buffer image is obtained by the updating method of the bottom area buffer image of the first aspect.

[0008] To solve the above problems, the third aspect of the present application provides an updating device of bottom area buffer image, the updating device of bottom area buffer image comprising: an acquisition module, the acquisition module being configured to obtain a current frame bottom area image of a vehicle by using a current frame bottom area buffer image; a judgment module, the judgment module being configured to judge whether the bottom area buffer image needs to be updated based on current state parameters of the vehicle and a buffer interval; and a processing module, the processing module being configured to obtain a next frame bottom area buffer image by using the current frame bottom area image when the bottom area buffer image needs to be updated, and to take the current frame bottom area buffer image as the next frame bottom area buffer image when the bottom area buffer image does not need to be updated.

[0009] To solve the above problems, the fourth aspect of the present application provides a display system of vehicle panoramic image, comprising a camera, a display, a memory and a processor which are coupled with each other; the camera is configured to collect images of four sides of a vehicle; the display is configured to display a panoramic overhead splicing image of the vehicle; and the processor is configured to execute program instructions stored in the memory to implement the display method of vehicle panoramic image of the second aspect.

[0010] To solve the above problems, the fifth aspect of the present application provides a computer readable storage medium, which stores program data capable of being executed by a processor, and the program data is used to implement the updating method of bottom area buffer image of the first aspect or the display method of vehicle panoramic image of the second aspect.

[0011] The beneficial effects of the present application are: different from the prior art, the present application can obtain the current frame vehicle bottom area image of the vehicle by using the current frame vehicle bottom area buffer image, and after obtaining the vehicle bottom area image each time, whether the vehicle bottom area buffer image needs to be updated can be judged based on the current state parameter of the vehicle and the buffer interval, when the vehicle bottom area buffer image needs to be updated, the next frame vehicle bottom area buffer image is obtained by using the current frame vehicle bottom area image, and when the vehicle bottom area buffer image does not need to be updated, the current frame vehicle bottom area buffer image is used as the next frame vehicle bottom area buffer image, so that the next frame vehicle bottom area image of the vehicle can be obtained according to the next frame vehicle bottom area buffer image, that is, the vehicle bottom area image of the vehicle can be continuously obtained, and the vehicle bottom road condition can be displayed in real time. In addition, whether the vehicle bottom area buffer image needs to be updated can be judged according to the current state parameter of the vehicle and the buffer interval, if not, the current frame vehicle bottom area buffer image is used as the next frame vehicle bottom area buffer image, so that the image content of one buffer image can be used for the following continuous multiple frames, so that repeated buffering and filling of the vehicle bottom overlapping area of the continuous frames can be avoided, and the final clarity of the vehicle bottom image display can be effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 is a flowchart of an embodiment of the vehicle bottom area buffer image updating method of the present application;

[0013] Figure 2 is Figure 1 is a flowchart of an embodiment of step S12 in the present application;

[0014] Figure 3a and Figure 3b is a display diagram of the relationship between the buffer image of the vehicle and the motion direction of the vehicle in an embodiment;

[0015] Figure 3c and Figure 3d is a display diagram of the relationship between the buffer interval and the buffer image in an embodiment;

[0016] Figure 4 is Figure 1 is a flowchart of an embodiment of step S13 in the present application;

[0017] Figure 5a is a display diagram of the current frame surrounding area image of the vehicle in an embodiment;

[0018] Figure 5b is a display diagram of the current frame vehicle bottom area image of the vehicle in an embodiment;

[0019] Figure 5c is a display diagram of the current frame panoramic overhead splicing image of the vehicle in an embodiment;

[0020] Figure 6 This is a schematic diagram showing the relationship between the vehicle's buffer area and the current frame's panoramic top-down stitched image in one embodiment;

[0021] Figure 7 This is a schematic flowchart of another embodiment of the method for updating the buffer image of the vehicle underside region in this application;

[0022] Figure 8 This is a schematic diagram showing the maximum distance the vehicle body moves within the vehicle's buffer zone in one embodiment;

[0023] Figure 9 This is a schematic flowchart of an embodiment of the method for displaying a panoramic image of a vehicle according to this application;

[0024] Figure 10 yes Figure 9 A flowchart illustrating an embodiment of step S93;

[0025] Figure 11 This is a schematic diagram of the framework of an embodiment of the device for updating the buffer image of the vehicle underside region in this application;

[0026] Figure 12 This is a schematic diagram of the framework of an embodiment of the vehicle panoramic image display system of this application;

[0027] Figure 13 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0028] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0029] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0030] In this paper, the terms "system" and "network" are often used interchangeably. The term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this paper means two or more.

[0031] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the method for updating a buffered image of the vehicle underside region according to this application. Specifically, it may include the following steps:

[0032] Step S11: obtaining the current frame vehicle underbody region image of the vehicle by using the current frame vehicle underbody region buffer image.

[0033] In the present application, the next frame vehicle underbody region buffer image is part of the current frame panoramic overhead mosaic image, i.e. the current frame vehicle underbody region buffer image is part of the previous frame panoramic overhead mosaic image, which is known information. The texture coordinates of the vehicle underbody region in the current frame vehicle underbody region buffer image can be determined according to the image frame rate, the vehicle speed and the wheel rotation angle and other current state parameters, and then the current frame vehicle underbody region image of the vehicle can be obtained through the current frame vehicle underbody region buffer image.

[0034] Step S12: determining whether the vehicle underbody region buffer image needs to be updated based on the current state parameters of the vehicle and the buffer interval. If the vehicle underbody region buffer image needs to be updated, step S13 is performed; if the vehicle underbody region buffer image does not need to be updated, step S14 is performed.

[0035] In the present application, the current frame vehicle underbody region buffer image is part of the previous frame panoramic overhead mosaic image, and the next frame vehicle underbody region buffer image is part of the current frame panoramic overhead mosaic image. Since the previous frame panoramic overhead mosaic image and the current frame panoramic overhead mosaic image can be different, according to the current state parameters of the vehicle, the current frame panoramic overhead mosaic image can completely include the current frame vehicle underbody region buffer image, or the current frame panoramic overhead mosaic image can not completely include the current frame vehicle underbody region buffer image. When the current frame panoramic overhead mosaic image completely includes the current frame vehicle underbody region buffer image, the next frame vehicle underbody region buffer image is determined to be the current frame vehicle underbody region buffer image, and the buffer image does not need to be updated. When the current frame panoramic overhead mosaic image does not completely include the current frame vehicle underbody region buffer image, the next frame vehicle underbody region buffer image needs to be determined according to the current frame panoramic overhead mosaic image, and the buffer image needs to be updated. Alternatively, if the vehicle underbody region is buffered every frame, i.e. the buffer image is updated every frame, since the time interval between frames is very small, there is a large overlap region between the vehicle underbody regions of adjacent two frames, and the repeated regions of consecutive frames are frequently buffered and filled, i.e. the performance is wasted, and the clarity of the overlap region is reduced. Therefore, by setting a buffer interval, when the buffer interval condition is not met, the next frame vehicle underbody region buffer image is determined to be the current frame vehicle underbody region buffer image, and the buffer image does not need to be updated. When the buffer interval condition is met, the next frame vehicle underbody region buffer image needs to be determined according to the current frame panoramic overhead mosaic image, and the buffer image needs to be updated.

[0036] Step S13: obtaining the next frame vehicle underbody region buffer image by using the current frame vehicle underbody region image.

[0037] When the buffer image needs to be updated, since the next frame vehicle bottom region buffer image is part of the current frame panoramic overhead mosaic image, the current frame surrounding region image of the vehicle can be obtained, then spliced with the current frame vehicle bottom region image to obtain the current frame panoramic overhead mosaic image, and then part of the region in the current frame panoramic overhead mosaic image is read and saved to obtain the next frame vehicle bottom region buffer image.

[0038] Step S14: taking the current frame vehicle bottom region buffer image as the next frame vehicle bottom region buffer image.

[0039] If the buffer image does not need to be updated, the original buffer image is reused, that is, the current frame vehicle bottom region buffer image can be taken as the next frame vehicle bottom region buffer image.

[0040] The above scheme can obtain the current frame vehicle bottom region image of the vehicle by using the current frame vehicle bottom region buffer image, and after obtaining the vehicle bottom region image each time, whether the vehicle bottom region buffer image needs to be updated can be determined based on the current state parameter of the vehicle and the buffer interval. When the vehicle bottom region buffer image needs to be updated, the next frame vehicle bottom region buffer image is obtained by using the current frame vehicle bottom region image, and when the vehicle bottom region buffer image does not need to be updated, the current frame vehicle bottom region buffer image is taken as the next frame vehicle bottom region buffer image, so that the next frame vehicle bottom region image of the vehicle can be obtained according to the next frame vehicle bottom region buffer image, that is, the vehicle bottom region image of the vehicle can be continuously obtained, and the vehicle bottom road condition can be displayed in real time. In addition, whether the vehicle bottom region buffer image needs to be updated can be determined according to the current state parameter of the vehicle and the buffer interval, and if not, the current frame vehicle bottom region buffer image is taken as the next frame vehicle bottom region buffer image, so that the image content of one buffer image can be used for continuous multiple frames, which can avoid repeated buffering and filling of the vehicle bottom overlapping region of continuous frames, and can effectively improve the final clarity of the vehicle bottom image display.

[0041] Please combine Figure 2 , Figure 2 is Figure 1 a flowchart of an embodiment of step S12 in the method. In an embodiment, step S12 can specifically include:

[0042] Step S121: determining whether the motion direction of the vehicle changes according to the current state parameter of the vehicle. The motion direction of the vehicle includes the forward direction of the vehicle and the backward direction of the vehicle. If the motion direction of the vehicle changes, step S122 is performed; if the motion direction of the vehicle does not change, step S123 is performed.

[0043] Step S122: determining that the vehicle bottom region buffer image needs to be updated. At this time, step S13 can be further performed.

[0044] Step S123: judging whether the frame number between the current frame vehicle bottom region image and the buffering time of the current frame vehicle bottom region buffered image is less than the buffering interval number. If the frame number is less than the buffering interval number, step S124 is performed; if the frame number is greater than or equal to the buffering interval number, step S125 is performed.

[0045] It can be understood that after the current frame vehicle bottom region image is acquired, it can be firstly judged according to the motion direction of the vehicle whether the buffered image needs to be updated. If the motion direction of the vehicle is changed, the buffered image needs to be updated; otherwise, it needs to be further judged according to the buffering interval whether the buffered image needs to be updated. Please combine with Figure 3a and Figure 3b , Figure 3a and Figure 3b are display schematic diagrams of the relationship between the buffered image of the vehicle and the motion direction of the vehicle in an embodiment. Since the region displayed by the current frame vehicle bottom region buffered image of the vehicle includes the vehicle body region and the extended region of the vehicle body motion, for example, when the vehicle is in the forward direction (including straight forward, left forward and right forward), the current frame vehicle bottom region buffered image of the vehicle is as shown in Figure 3a , when the vehicle is in the backward direction (including straight backward, left backward and right backward), the current frame vehicle bottom region buffered image of the vehicle is as shown in Figure 3b When the motion direction of the vehicle is changed, the position of the vehicle at the next time is probably not in the region displayed by the current frame vehicle bottom region buffered image of the vehicle. Therefore, if the motion direction of the vehicle is changed, the buffered image needs to be updated.

[0046] If the motion direction is not changed, it needs to be further judged according to the buffering interval whether the buffered image needs to be updated. Please combine with Figure 3c and Figure 3d , Figure 3c and Figure 3d are display schematic diagrams of the relationship between the buffering interval and the buffered image in an embodiment. If the vehicle bottom part is buffered every frame, that is, the buffered image is updated every frame, as shown in Figure 3cThe figure assumes that the vehicle travels from left to right and downwards, and the rectangular regions S1, S2 and S3 are the positions of the vehicle in the nth frame, the nth+1 frame and the nth+2 frame respectively. Since the time interval between frames is very small, there is a large overlapping region between the vehicle bottom regions of two adjacent frames. If the vehicle bottom region of the nth frame and the surrounding region are used to fill the vehicle bottom region of the nth+1 frame, and then the vehicle bottom region of the nth+1 frame and the surrounding region are used to fill the vehicle bottom region of the nth+2 frame, the repeated region of continuous frames will be frequently buffered and filled, which results in waste of performance and reduction of the clarity of the overlapping region. Since filling is actually an image texture sampling process, repeated buffering and filling of the overlapping region is essentially a sampling and resampling process of the same region, which results in abnormal blur of the image of the vehicle bottom region in the final display. If the image is buffered at an interval of N frames, and the buffered image is used for the following continuous N frames, as shown in Figure 3d The figure assumes that the rectangular regions S1, S2 and S3 are the positions of the vehicle in the nth frame, the nth+N frame and the nth+N+N frame respectively. The larger N is, the smaller the overlapping region between adjacent buffered frames is. In this way, the repeated sampling process of the same region can be effectively reduced, and the clarity of the final display of the vehicle bottom image can be effectively improved. In addition, buffering one frame of image is used for the following continuous N frames, which can effectively reduce the buffering times and improve the performance utilization. The size of N is adaptively calculated according to the buffering region and the vehicle speed.

[0047] Step S124: determining that the vehicle bottom region buffered image does not need to be updated. At this time, step S14 can be further performed.

[0048] Step S125: determining that the vehicle bottom region buffered image needs to be updated. At this time, step S13 can be further performed.

[0049] Therefore, according to the buffering interval, it can be determined whether the buffered image needs to be updated. If the frame number between the time corresponding to the image of the vehicle bottom region of the current frame and the buffering time of the vehicle bottom region buffered image of the current frame is greater than or equal to the buffering interval N, it is determined that the buffered image needs to be updated, otherwise it is determined that the buffered image does not need to be updated. According to this operation, the image content of one buffered image can be used for the following continuous N frames, which can avoid repeated buffering and filling of the overlapping region of continuous frames, and can effectively improve the final clarity of the display of the vehicle bottom image.

[0050] Specifically, please refer to Figure 4 , Figure 4 is Figure 1 the flowchart of an embodiment of step S13. In an embodiment, step S13 can specifically include the following steps.

[0051] Step S131: acquiring the current frame image of the four sides of the vehicle and the current state parameter of the vehicle.

[0052] Specifically, the images around the vehicle can be fisheye images, by setting a wide-angle fisheye camera on each side of the vehicle, the wide-angle fisheye cameras in front, back, left and right directions can cover all field of view ranges around the vehicle, and one frame of fisheye image can be acquired by each wide-angle fisheye camera at the same time; thus, for the current frame, the current frame fisheye images of the four sides of the vehicle can be acquired, and the current state parameters of the vehicle at the corresponding time of the current frame can be acquired, the current state parameters of the vehicle can include real-time information such as image frame rate, vehicle speed and wheel rotation angle.

[0053] Step S132: obtaining the current frame surrounding area image of the vehicle by using the current frame images of the four sides of the vehicle and the texture coordinates of the surrounding area of the vehicle in the current frame fisheye images of the four sides of the vehicle.

[0054] As shown in Figure 5a , Figure 5a is a display schematic diagram of the current frame surrounding area image of the vehicle in an embodiment, according to the 360-degree panoramic calibration algorithm, the texture coordinates of the surrounding area of the vehicle in the current frame fisheye images of the four sides of the vehicle can be determined, and then the current frame surrounding area image of the vehicle can be spliced by using the current frame fisheye images of the four sides of the vehicle, the current frame surrounding area image of the vehicle can display the surrounding area of the vehicle, but cannot display the underbody area.

[0055] Step S133: obtaining the current frame panoramic overhead splicing image of the vehicle according to the current frame surrounding area image and the current frame underbody area image.

[0056] The current frame underbody area image of the vehicle can be acquired from the current frame underbody area buffer image, as shown in Figure 5b , in combination with the results of step S132 and step S11, a complete panoramic overhead splicing image can be drawn, for example, the current frame underbody area image is pasted into the current frame surrounding area image, and the current frame panoramic overhead splicing image of the vehicle can be obtained, as shown in Figure 5c .

[0057] Step S134: determining the buffer area of the vehicle according to the moving direction of the vehicle.

[0058] Step S135: determining the corresponding position of the buffer area of the vehicle in the current frame panoramic overhead splicing image.

[0059] Step S136: intercepting the image of the corresponding position of the buffer area of the vehicle in the current frame panoramic overhead splicing image as the next frame underbody area buffer image.

[0060] As shown in Figure 6 , Figure 6is a schematic diagram of the relationship between the buffer area of the vehicle and the current frame panoramic overhead mosaic in an embodiment. First, the buffer area of the vehicle needs to be determined according to the moving direction of the vehicle, and the current area of the vehicle plus the extended area in the moving direction is the buffer area of the vehicle. Then, the corresponding position of the buffer area of the vehicle in the current frame panoramic overhead mosaic is determined. By intercepting the image of the corresponding position of the buffer area of the vehicle in the current frame panoramic overhead mosaic, the next frame vehicle bottom area buffer image can be obtained. Since the panoramic overhead mosaic Figure 1 is generally directly drawn on the screen display end by the GPU, so this step needs to obtain part of the memory content, which consumes a lot of performance. Therefore, the operation times of this step should be as few as possible, and the buffer area should be as small as possible.

[0061] Further, please refer to Figure 3a and Figure 3b , the buffer area of the vehicle includes the vehicle bottom area and an extended area, the extended area includes two first sub-areas Sa on both sides of the vehicle and one second sub-area Sb on one side of the moving direction of the vehicle; wherein the length of the first sub-area Sa is the length of the vehicle body, the width W of the first sub-area Sa is related to the width of the vehicle body, the width of the second sub-area Sb is the width of the vehicle body plus the width W of the two first sub-areas Sa, and the length H of the second sub-area Sb is related to the length of the vehicle body.

[0062] Specifically, in order to control the size of the buffer image, the embodiments of the present application adjust the buffer area according to the moving direction of the vehicle. If the vehicle moves forward (including left front and right front), the buffer area is as shown in Figure 3a , if the vehicle moves backward (including left rear and right rear), the buffer area is as shown in Figure 3b . Wherein H and W are the extended buffer areas in the width and length directions of the vehicle, and the sizes can be self-defined. The size of the H and W values will directly affect the final display clarity of the vehicle bottom image.

[0063] Further, the width W of the first sub-area Sa is 0.25 times the width of the vehicle body, and the length H of the second sub-area Sb is 0.5-1 times the length of the vehicle body. Specifically, considering that the left and right moving distances of the vehicle during operation are generally less than the front and rear moving distances, W is generally set to be small, and H is generally set to be large. In addition, in order to balance the image clarity and the display range of the panoramic overhead view, H can be set to 0.5-1 times the length of the vehicle, and W can be set to about 0.25 times the width of the vehicle.

[0064] Please refer to Figure 7 , Figure 7is a flowchart of another embodiment of the method for updating the underbody area buffer image of the present application. In an embodiment, the method for updating the underbody area buffer image comprises the following steps:

[0065] Step S71: obtaining the current frame underbody area image of the vehicle using the current frame underbody area buffer image.

[0066] Step S72: determining whether the underbody area buffer image needs to be updated based on the current state parameters of the vehicle and the buffer interval. If the underbody area buffer image needs to be updated, step S73 is performed; if the underbody area buffer image does not need to be updated, step S74 is performed.

[0067] Step S73: obtaining the next frame underbody area buffer image using the current frame underbody area image.

[0068] Step S74: taking the current frame underbody area buffer image as the next frame underbody area buffer image.

[0069] Steps S71 to S74 in the present embodiment are basically the same as steps S11 to S14 in the foregoing embodiment, and will not be described again here.

[0070] Further, the current state parameters of the vehicle include the image frame rate, the vehicle speed and the wheel trajectory. It can be understood that, since the state parameters of the vehicle are always changing during the movement of the vehicle, after the next frame underbody area buffer image is obtained in steps S73 and S74, the buffer interval number N needs to be calculated and updated in real time according to the buffer area size, the vehicle speed, the frame rate and the like. The difference between the present embodiment and the foregoing embodiment is that the present embodiment can further comprise:

[0071] Step S75: determining the buffer area of the vehicle according to the movement direction of the vehicle.

[0072] Step S76: calculating the maximum moving distance of the vehicle body in the buffer area of the vehicle using the wheel trajectory line model.

[0073] Specifically, please refer to Figure 8 , Figure 8 is a display diagram of the maximum moving distance of the vehicle body in the buffer area of the vehicle in an embodiment. According to the size of the buffer area and the wheel trajectory line model, the maximum moving distance d of the vehicle body in the buffer area can be calculated. For example, if the vehicle drives forward, d is maximum as the buffer expansion width H; if the vehicle drives to the left front, d is determined by the length H, the width W of the buffer expansion and the trajectory line application model.

[0074] Step S77: calculating the buffer interval number according to the maximum moving distance, the vehicle speed and the image frame rate.

[0075]

[0076] distance, f is the image frame rate, and v is the vehicle speed.

[0077] It can be understood that, based on the above formula, the buffer interval number N can be calculated according to the maximum moving distance d (km) of the vehicle body, the vehicle speed v (km / h) and the image frame rate f (fps). For example, when the maximum moving distance of the vehicle body in the buffer area is 3 m, the image frame rate is 30 fps, and the vehicle speed is 30 km / h, the calculated buffer interval number is 10 frames, that is, the buffer image is buffered every 10 frames. When d is constant, the smaller the vehicle speed v is, the larger the buffer interval number N is. Since the trajectory path and the vehicle speed can change in real time, the buffer interval number N also needs to be updated in real time every frame.

[0078] Referring to Figure 9 , Figure 9 is a flowchart of an embodiment of a display method of a vehicle panoramic image. In an embodiment, the display method of the vehicle panoramic image comprises the following steps:

[0079] Step S91: acquiring current frame images of four sides of a vehicle and current state parameters of the vehicle.

[0080] Step S92: obtaining a current frame surrounding area image of the vehicle by using the current frame images of the four sides of the vehicle and texture coordinates of a surrounding area of the vehicle in the current frame images of the four sides of the vehicle.

[0081] Step S93: obtaining a current frame underbody area image of the vehicle by using the current state parameters of the vehicle to determine texture coordinates of the underbody area in the current frame underbody area buffer image.

[0082] Step S94: obtaining and displaying a current frame panoramic overhead mosaic image of the vehicle according to the current frame surrounding area image and the current frame underbody area image.

[0083] The results of steps S92 and S93 can be combined to draw a complete panoramic overhead mosaic image. For example, the current frame underbody area image is pasted into the current frame surrounding area image to obtain the current frame panoramic overhead mosaic image of the vehicle, which is displayed.

[0084] The current frame underbody area buffer image is obtained by using any one of the above-mentioned update methods of the underbody area buffer image.

[0085] The above scheme can obtain the current frame surrounding area image of the vehicle by using the current frame images of the four sides of the vehicle and the texture coordinates of the surrounding area of the vehicle in the current frame images of the four sides of the vehicle, and can obtain the current frame underbody area image of the vehicle by using the current state parameters of the vehicle to determine the texture coordinates of the underbody area in the current frame underbody area buffer image, so that the current frame panoramic overhead splicing image of the vehicle can be obtained according to the current frame surrounding area image and the current frame underbody area image, and then the next frame underbody area buffer image can be determined after the current frame panoramic overhead splicing image is drawn, so as to facilitate the drawing of subsequent panoramic overhead splicing images, and the complete perspective effect of the underbody can be realized. In addition, whether the buffer image needs to be updated can be determined according to the current state parameters of the vehicle and the buffer interval, if yes, the next frame underbody area buffer image is obtained by using the current frame panoramic overhead splicing image, and if no, the current frame underbody area buffer image is used as the next frame underbody area buffer image, so that the image content of one buffer image can be used for continuous multiple frames, which can avoid repeated buffering and filling of the overlapping area of the underbody of continuous frames, and can effectively improve the final clarity of the underbody image display. Moreover, since the next frame underbody area buffer image is part of the current frame panoramic overhead splicing image, that is, only the limited area of the underbody and the surrounding area is buffered, instead of the entire panoramic overhead splicing image, the image data of each buffering can be greatly reduced. In addition, since the brightness of the panoramic splicing image is balanced, the buffer image is part of the panoramic overhead splicing image, so there is no brightness mutation problem between the underbody area image and the surrounding area image determined by the buffer image, and the image brightness balance can be ensured.

[0086] Please combine Figure 10 , Figure 10 is Figure 9 a flowchart of an embodiment of step S93. In an embodiment, the above step S93 can specifically include:

[0087] Step S931: determining a displacement transformation matrix between the current time of the vehicle and the buffering time of the current frame underbody area buffer image according to the current state parameters of the vehicle.

[0088] Step S932: determining the texture coordinates of the underbody area in the current frame underbody area buffer image by using the displacement transformation matrix.

[0089] Step S933: determining the corresponding position of the underbody area in the current frame underbody area buffer image by using the texture coordinates of the underbody area in the current frame underbody area buffer image.

[0090] Step S934: intercepting the image of the corresponding position of the underbody area in the current frame underbody area buffer image as the current frame underbody area image of the vehicle.

[0091] Specifically, the current state parameters of the vehicle include an image frame rate, a vehicle speed and a wheel rotation angle. Firstly, a displacement transformation matrix Mat of the vehicle body can be calculated according to the image frame rate, the vehicle speed, the wheel rotation angle and the like, that is, the displacement transformation matrix Mat of the vehicle body at a current time relative to a buffering time of a current frame vehicle bottom region buffer image; in this application, the buffering time of the current frame vehicle bottom region buffer image refers to a time corresponding to image content of the current frame vehicle bottom region buffer image generated for the first time, for example, the A frame buffer image is generated by updating the buffer image at the A time, and the B frame buffer image corresponding to the next time is completely the same as the A frame buffer image, that is, the buffer image is not updated at the next time, therefore, the buffering time of the B frame buffer image is the A time. Then, according to the displacement transformation matrix Mat, the texture coordinates of the vehicle bottom region in the current frame vehicle bottom region buffer image can be determined, so that it can be calculated which block region of the current frame vehicle bottom region buffer image corresponds to the vehicle bottom region, the corresponding position of the vehicle bottom region in the current frame vehicle bottom region buffer image is determined, and then the image of the corresponding position of the vehicle bottom region in the current frame vehicle bottom region buffer image is intercepted as the current frame vehicle bottom region image of the vehicle.

[0092] Please refer to Figure 11 , Figure 11 is a schematic diagram of an embodiment of the vehicle bottom region buffer image updating device. The vehicle bottom region buffer image updating device 110 includes an acquisition module 1100, a judgment module 1102 and a processing module 1104. The acquisition module 1100 is configured to acquire a current frame vehicle bottom region image of a vehicle by using a current frame vehicle bottom region buffer image; the judgment module 1102 is configured to judge whether the vehicle bottom region buffer image needs to be updated based on current state parameters of the vehicle and a buffering interval; and the processing module 1104 is configured to obtain a next frame vehicle bottom region buffer image by using the current frame vehicle bottom region image when the vehicle bottom region buffer image needs to be updated, and configured to take the current frame vehicle bottom region buffer image as the next frame vehicle bottom region buffer image when the vehicle bottom region buffer image does not need to be updated.

[0093] In the above scheme, the acquisition module 1100 can acquire the current frame undercarriage region image of the vehicle using the current frame undercarriage region buffer image. After each acquisition of the undercarriage region image, the judgment module 1102 can determine whether the undercarriage region buffer image needs to be updated based on the current state parameters of the vehicle and the buffer interval. When the undercarriage region buffer image needs to be updated, the processing module 1104 uses the current frame undercarriage region image to obtain the next frame undercarriage region buffer image. When the undercarriage region buffer image does not need to be updated, the current frame undercarriage region buffer image is used as the next frame undercarriage region buffer image. Thus, the next frame undercarriage region image of the vehicle can be acquired based on the next frame undercarriage region buffer image, which means that the undercarriage region image of the vehicle can be continuously acquired to achieve real-time display of the road conditions under the vehicle. In addition, based on the vehicle's current state parameters and buffer interval, it can be determined whether the undercarriage buffer image needs to be updated. If not, the current frame's undercarriage buffer image is used as the next frame's undercarriage buffer image. Thus, the image content of one buffer image can be used for subsequent consecutive frames. This avoids repeatedly buffering and filling the overlapping areas of the undercarriage in consecutive frames, effectively improving the final clarity of the undercarriage image display.

[0094] In one embodiment, the determination module 1102 performs a step of determining whether the undercarriage region buffer image needs to be updated based on the vehicle's current state parameters and buffer interval, including: determining whether the vehicle's movement direction has changed according to the vehicle's current state parameters; wherein the vehicle's movement direction includes the vehicle's forward direction and the vehicle's backward direction; if it has changed, then it is determined that the undercarriage region buffer image needs to be updated; if it has not changed, then it is further determined whether the number of frames between the current frame undercarriage region image and the buffer time of the current frame undercarriage region buffer image is less than the buffer interval number; if yes, then it is determined that the undercarriage region buffer image does not need to be updated; if no, then it is determined that the undercarriage region buffer image needs to be updated.

[0095] In one embodiment, the processing module 1104 performs the step of obtaining a next frame vehicle underside region buffer image using the current frame vehicle underside region image, including: acquiring current frame images of the four sides of the vehicle and the current state parameters of the vehicle; obtaining a current frame surrounding region image of the vehicle using the current frame images of the four sides of the vehicle and the texture coordinates of the surrounding region in the current frame images of the four sides of the vehicle; obtaining a current frame panoramic top-down stitched image of the vehicle based on the current frame surrounding region image and the current frame vehicle underside region image; determining the buffer region of the vehicle based on the vehicle's movement direction; determining the corresponding position of the vehicle's buffer region in the current frame panoramic top-down stitched image; and cropping the image of the vehicle's buffer region at the corresponding position in the current frame panoramic top-down stitched image as the next frame vehicle underside region buffer image.

[0096] In an embodiment, the current state parameters of the vehicle include an image frame rate, a vehicle speed, and a wheel trajectory; the processing module 1104 is further configured to determine a buffer area of the vehicle according to a moving direction of the vehicle; calculate a maximum moving distance of the vehicle body in the buffer area of the vehicle by using a wheel trajectory line model; and calculate the buffer interval number according to the maximum moving distance, the vehicle speed, and the image frame rate.

[0097] Referring to Figure 12 , Figure 12 is a schematic diagram of an embodiment of a vehicle panoramic image display system. The vehicle panoramic image display system 120 includes a camera 123, a display 124, a memory 121, and a processor 122 coupled with each other. The camera 123 is configured to capture images of four sides of a vehicle. The display 124 is configured to display a panoramic overhead mosaic image of the vehicle. The processor 122 is configured to execute program instructions stored in the memory 121 to implement the steps of any of the vehicle panoramic image display method embodiments described above. In a specific implementation scenario, the vehicle panoramic image display system 120 can include, but is not limited to, a microcomputer, a server.

[0098] Specifically, the processor 122 is configured to control itself and the memory 121 to implement the steps of any of the vehicle panoramic image display method embodiments described above. The processor 122 can also be referred to as a CPU (Central Processing Unit). The processor 122 can be an integrated circuit chip with a processing capability of signals. The processor 122 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. In addition, the processor 122 can be implemented by an integrated circuit chip together.

[0099] Referring to Figure 13 , Figure 13 is a schematic diagram of an embodiment of a computer readable storage medium. The computer readable storage medium 130 stores program instructions 1300 executable by a processor, which are used to implement the steps of any of the vehicle bottom area buffer image updating method or any of the vehicle panoramic image display method embodiments described above.

[0100] In several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other manners. For example, the described device embodiments are merely illustrative, and the division of modules or units can be different, for example, some features can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0101] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed on network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0102] In addition, the functional units in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present separately, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0103] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

Claims

1. A method for updating a buffered image of the underside of a vehicle, characterized in that, The method includes: Obtain the current frame undercarriage region image of the vehicle using the buffered image of the current frame undercarriage region; Based on the vehicle's current state parameters and buffer interval, determine whether it is necessary to update the buffer image of the vehicle's underside area. If necessary, the next frame's vehicle underside buffer image can be obtained using the current frame's vehicle underside region image; If not needed, the current frame undercarriage buffer image is used as the next frame undercarriage buffer image. The step of determining whether the buffered image of the undercarriage region needs to be updated based on the vehicle's current state parameters and buffer interval includes: The vehicle's direction of motion is determined based on its current state parameters; wherein the vehicle's direction of motion includes the vehicle's forward direction and the vehicle's backward direction; the vehicle's current state parameters include the image frame rate, vehicle speed, and wheel trajectory. If the image changes, then it is determined that the buffered image of the vehicle underside area needs to be updated; If it does not change, then it is further determined whether the number of frames between the current frame undercarriage region image and the buffer time of the current frame undercarriage region buffer image is less than the number of buffer intervals. If so, then it is determined that the buffered image of the vehicle underside area does not need to be updated; If not, then it is determined that the buffer image of the vehicle underside area needs to be updated; The method further includes: The vehicle's buffer zone is determined based on its direction of movement. The maximum distance the vehicle body moves within the vehicle's buffer zone is calculated using a wheel trajectory model. The number of buffer intervals is calculated based on the maximum travel distance, the vehicle speed, and the image frame rate.

2. The method for updating the buffered image of the vehicle underside region according to claim 1, characterized in that, The step of obtaining the next frame's vehicle underside buffer image using the current frame's vehicle underside region image includes: Acquire the current frame images of the four sides of the vehicle and the current state parameters of the vehicle; Using the current frame images of the four sides of the vehicle and the texture coordinates of the area surrounding the vehicle in the current frame images of the four sides of the vehicle, the current frame image of the area surrounding the vehicle is obtained. Based on the surrounding area image of the current frame and the undercarriage area image of the current frame, a panoramic top-down stitched image of the vehicle in the current frame is obtained; The vehicle's buffer zone is determined based on its direction of movement. Determine the corresponding position of the vehicle's buffer area in the current frame panoramic top-down stitched image; The image of the vehicle's buffer area at the corresponding position in the current frame's panoramic top-down stitched image is extracted and used as the vehicle's underside buffer image for the next frame.

3. The method for updating the buffered image of the vehicle underside region according to claim 2, characterized in that, The vehicle underside buffer image includes the area directly facing the vehicle underside and an extended area. The extended area includes two first sub-regions on both sides of the vehicle and a second sub-region on one side of the vehicle's direction of movement. Wherein, the length of the first sub-region is the vehicle body length, the width of the first sub-region is related to the vehicle body width, the width of the second sub-region is the vehicle body width plus the width of two first sub-regions, and the length of the second sub-region is related to the vehicle body length.

4. The method for updating the buffered image of the vehicle underside region according to claim 3, characterized in that, The width of the first sub-region is 0.25 times the width of the vehicle body, and the length of the second sub-region is 0.5-1 times the length of the vehicle body.

5. A method for displaying a panoramic image of a vehicle, characterized in that, The method includes: Acquire the current frame images of the vehicle from all four sides and the vehicle's current state parameters; Using the current frame images of the four sides of the vehicle and the texture coordinates of the area surrounding the vehicle in the current frame images of the four sides of the vehicle, the current frame image of the area surrounding the vehicle is obtained. The texture coordinates of the vehicle's undercarriage region in the current frame undercarriage region buffer image are determined using the vehicle's current state parameters, thus obtaining the vehicle's current frame undercarriage region image. Based on the surrounding area image of the current frame and the undercarriage area image of the current frame, a panoramic top-down view of the vehicle in the current frame is obtained and displayed; The current frame undercarriage buffer image is obtained by the undercarriage buffer image update method according to any one of claims 1 to 4.

6. An apparatus for updating a buffered image of a vehicle underside region, characterized in that, The device for updating the buffered image of the vehicle underside region includes: The acquisition module is used to acquire the current frame undercarriage region image of the vehicle using the current frame undercarriage region buffer image; The judgment module is used to determine whether the buffered image of the vehicle's undercarriage region needs to be updated based on the vehicle's current state parameters and buffer interval; it is also used to determine whether the vehicle's movement direction has changed based on the vehicle's current state parameters; wherein the vehicle's movement direction includes the vehicle's forward direction and the vehicle's backward direction; the vehicle's current state parameters include the image frame rate, vehicle speed, and wheel trajectory; if it has changed, it is determined that the buffered image of the vehicle's undercarriage region needs to be updated; if it has not changed, it further determines whether the number of frames between the current frame of the vehicle's undercarriage region image and the buffer time of the current frame of the vehicle's undercarriage region image is less than the buffer interval number; if yes, it is determined that the buffered image of the vehicle's undercarriage region does not need to be updated; if not, it is determined that the buffered image of the vehicle's undercarriage region needs to be updated. The processing module is configured to obtain the next frame of the vehicle underside buffer image using the current frame of the vehicle underside buffer image when it is necessary to update the vehicle underside buffer image, and to use the current frame of the vehicle underside buffer image as the next frame of the vehicle underside buffer image when it is not necessary to update the vehicle underside buffer image; it is also configured to determine the vehicle's buffer area based on the vehicle's direction of motion; calculate the maximum movement distance of the vehicle body within the vehicle's buffer area using a wheel trajectory model; and calculate the number of buffer intervals based on the maximum movement distance, the vehicle speed, and the image frame rate.

7. A display system for a panoramic vehicle image, characterized in that, This includes interconnected cameras, displays, memory, and processors; The camera is used to capture images from all four sides of the vehicle; the display is used to display a panoramic top-down stitched image of the vehicle; the processor is used to execute program instructions stored in the memory to implement the method for displaying panoramic vehicle images as described in claim 5.

8. A computer-readable storage medium, characterized in that, The system stores program data that can be executed by a processor, the program data being used to implement the method for updating the buffer image of the vehicle underside region as described in any one of claims 1-4, or the method for displaying the panoramic image of the vehicle as described in claim 5.

Images