Vehicle-mounted panoramic image picture adjusting method and device and electronic equipment

Abstract

The invention relates to a method and device for adjusting a vehicle-mounted panoramic image and electronic equipment, and belongs to the technical field of visual effects of the vehicle-mounted panoramic image, and the method comprises the steps: obtaining an adjustment coefficient of a real-time frame aerial view according to the real-time frame aerial view and a dynamic adjustment coefficient algorithm, adjusting the real-time frame aerial view through the adjustment coefficient to obtain real-time target aerial views corresponding to four directions, and performing alpha fusion on the real-time target aerial views in the four directions to obtain a real-time frame of the vehicle-mounted panoramic image; the dynamic adjustment coefficient algorithm can be combined with the adjustment coefficient of the previous moment to obtain the real-time adjustment coefficient of the real-time frame aerial view, and the image formed by the real-time image frame obtained based on the real-time adjustment coefficient and the image frame of the previous moment is the image of one process, so that the frame flicker between frames is effectively avoided, and the fluency of the vehicle-mounted panoramic image is improved.

Description

Technical Field

[0001] The present application relates to the technical field of visual effects of vehicle-mounted panoramic image pictures, and in particular to a method, device and electronic equipment for adjusting vehicle-mounted panoramic image pictures. Background Art

[0002] The current on-board panoramic images are generally based on a series of images obtained by four cameras around the vehicle body, front, back, left and right, which are transformed into a bird's-eye view, and the bird's-eye view is spliced ​​to obtain the frame of the panoramic film and television, which is output to the large screen of the vehicle. In different scenes, the image color and brightness captured by different cameras and finally displayed on the vehicle screen have certain differences, which are particularly obvious in certain specific scenes (such as obstacles blocking the field of view of a camera). In this case, if the existing technology is used to adjust the panoramic image, the frame of the panoramic film and television will cause the picture to flicker due to factors such as sudden changes in the environment. Summary of the invention

[0003] To this end, the present invention provides a method, device and electronic device for adjusting a vehicle-mounted panoramic image to solve the problem of image flickering between frames of panoramic video caused by environmental changes when adjusting the panoramic image in the prior art.

[0004] In a first aspect, a method for adjusting a vehicle-mounted panoramic image is provided, the method comprising:

[0005] Get a real-time frame bird's-eye view of the vehicle body in four directions;

[0006] Obtaining an adjustment coefficient of the real-time frame bird's-eye view according to the real-time frame bird's-eye view and a dynamic adjustment coefficient algorithm;

[0007] The real-time frame bird's-eye view is adjusted by the adjustment coefficient to obtain real-time target bird's-eye views corresponding to four directions;

[0008] The real-time target bird's-eye views in the four directions are fused through alpha fusion to obtain a real-time frame of the vehicle-mounted panoramic image.

[0009] Furthermore, the dynamic adjustment coefficient algorithm includes:

[0010] Acquire the four-directional bird's-eye view of the first frame and the four-directional bird's-eye view of the (n+1)th frame collected in the first adjustment period; one adjustment period is n frames;

[0011] Obtaining a first adjustment coefficient according to the four-directional bird's-eye view of the first frame, and obtaining a second adjustment coefficient according to the four-directional bird's-eye view of the (n+1)th frame;

[0012] According to the first adjustment coefficient and the second adjustment coefficient, an adjustment coefficient corresponding to each frame of the bird's-eye view in a second adjustment period is obtained by a dynamic adjustment formula; the second adjustment period is an adjustment period next to the first adjustment period;

[0013] The adjustment coefficient corresponding to each frame of the bird's-eye view in the third adjustment period is obtained by a dynamic adjustment formula according to the start and end frame adjustment coefficients in the second to third adjustment periods; the start and end frame adjustment coefficients include the adjustment coefficient of the first frame of the bird's-eye view in the second adjustment period and the adjustment coefficient of the start frame of the bird's-eye view in the next adjustment period;

[0014] The adjustment coefficient corresponding to each frame of the bird's-eye view in the subsequent adjustment cycles starting from the third adjustment cycle is obtained by a dynamic adjustment formula depending on the adjustment coefficient of the starting frame in the previous adjustment cycle.

[0015] Furthermore, the dynamic adjustment formula is:

[0016] K ia channel =F i1 channel +a×(F ip1 channel –F i1 channel ) / n;

[0017] Among them, K ia channel F is the dynamic adjustment coefficient corresponding to the a-th frame of the bird's-eye view in the current adjustment period, 1≤a≤n; i1 channel is the preliminary adjustment coefficient corresponding to the first frame of the bird's-eye view in the previous adjustment period of the current adjustment period; F ip1 channel It is the adjustment coefficient corresponding to the first frame of the bird's-eye view in the current adjustment period; where i∈(1,2,3,4); channel∈(R,G,B).

[0018] Further, obtaining the first adjustment coefficient according to the four-directional bird's-eye view of the first frame includes:

[0019] Four overlapping areas are obtained according to the initial bird's-eye view in four directions of the first frame, front, back, left, and right; the initial bird's-eye view is an RGB color mode picture;

[0020] The adjustment coefficients of the four initial bird's-eye views are obtained through an error algorithm according to the number of pixels and the pixel mean of the image in the overlapped area.

[0021] Furthermore, the real-time frame bird's-eye view includes four bird's-eye views of the front, back, left, and right sides of the vehicle body; the real-time frame bird's-eye view is adjusted by the adjustment coefficient to obtain real-time target bird's-eye views corresponding to four directions, including:

[0022] A real-time frame target bird's-eye view is obtained through an image adjustment formula according to the adjustment coefficient; the image adjustment formula is:

[0023]

[0024] Among them, dstImg i channel is the real-time target bird's-eye view of the i-th bird's-eye view in the real-time frame bird's-eye view, where i∈(1,2,3,4); srcImg i channel is the i-th bird's-eye view in the real-time frame bird's-eye view; k i channel is the adjustment coefficient of the i-th bird's-eye view in the real-time frame bird's-eye view; channel∈(R, G, B).

[0025] Furthermore, the real-time target bird's-eye view images in the four directions are fused by alpha to obtain a real-time frame of the vehicle-mounted panoramic image, including:

[0026] The real-time target bird's-eye view images in the four directions are fused based on the alpha fusion formula to obtain a real-time frame of the vehicle-mounted panoramic image; the alpha fusion formula is:

[0027] C = αCb + (1-α)Ca;

[0028] Among them, C a A bird's-eye view of any real-time target in the left or right direction; C b For C a A bird's-eye view of any real-time target with overlapping areas; α is the ratio of the Euclidean distance from the current point to the left or right boundary to the sum of the Euclidean distances from the current point to the left and right boundaries, that is, the normalized weight.

[0029] In a second aspect, a device for adjusting a vehicle-mounted panoramic image is provided, the device comprising:

[0030] An acquisition module is used to acquire a real-time frame bird's-eye view of the vehicle body in four directions;

[0031] An adjustment coefficient module, used for obtaining the adjustment coefficient of the real-time frame bird's-eye view according to the real-time frame bird's-eye view and a dynamic adjustment coefficient algorithm;

[0032] An adjustment module, used for adjusting the real-time frame bird's-eye view by the adjustment coefficient to obtain real-time target bird's-eye views corresponding to four directions;

[0033] The fusion module is used to obtain a real-time frame of the vehicle-mounted panoramic image by alpha fusion of the real-time target bird's-eye views in the four directions.

[0034] In a third aspect, an electronic device is provided, including:

[0035] at least one processor; and

[0036] a memory communicatively connected to the at least one processor; wherein,

[0037] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the above-mentioned methods for adjusting the vehicle-mounted panoramic image screen.

[0038] The present invention adopts the above technical solution and has at least the following beneficial effects:

[0039] Provided are a method, device and electronic device for adjusting a vehicle-mounted panoramic image. The adjustment coefficient of the real-time frame bird's-eye view is obtained according to a real-time frame bird's-eye view and a dynamic adjustment coefficient algorithm. The real-time frame bird's-eye view is adjusted by the adjustment coefficient to obtain real-time target bird's-eye views corresponding to four directions. The real-time target bird's-eye views in the four directions are alpha-fused to obtain a real-time frame of the vehicle-mounted panoramic image. The dynamic adjustment coefficient algorithm can obtain the real-time adjustment coefficient of the real-time frame bird's-eye view in combination with the adjustment coefficient at the previous moment. The image composed of the real-time image frame obtained based on the real-time adjustment coefficient and the image frame at the previous moment is an image of a process, which effectively avoids image flickering between frames and improves the fluency of the vehicle-mounted panoramic image.

[0040] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 is a flow chart of a method for adjusting a vehicle-mounted panoramic image screen shown in an exemplary embodiment of the present invention;

[0043] Figure 2 is a schematic diagram A showing an overlap area according to an exemplary embodiment of the present invention;

[0044] Figure 3 is a schematic diagram B showing an overlapped area according to an exemplary embodiment of the present invention;

[0045] Figure 4is a schematic block diagram of an adjustment device for a vehicle-mounted panoramic image screen shown in an exemplary embodiment of the present invention;

[0046] Figure 5 It is a schematic block diagram of an electronic device shown in an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] At present, the in-vehicle panoramic images are generally based on a series of image transformations based on the images around the vehicle body obtained by four cameras in front, behind, left and right of the vehicle body, to obtain the final virtual bird's-eye view, which is then output to the large screen of the vehicle. Due to the different installation positions of the cameras, the images captured by different cameras in different scenes and finally displayed on the vehicle screen have certain differences in color and brightness, which is particularly obvious in certain specific scenes (such as obstacles blocking the field of view of a camera). If no corresponding processing is performed, the directly spliced ​​bird's-eye view will show significant color and brightness unevenness and obvious splicing marks, which will affect the overall user experience.

[0049] The embodiments of the present application provide a method, device and electronic device for adjusting a vehicle-mounted panoramic image. The adjustment coefficient of the real-time frame bird's-eye view is obtained based on the real-time frame bird's-eye view and the dynamic adjustment coefficient algorithm. The real-time frame bird's-eye view is adjusted by the adjustment coefficient to obtain real-time target bird's-eye views corresponding to four directions. The real-time target bird's-eye views in the four directions are alpha-fused to obtain a real-time frame of the vehicle-mounted panoramic image. The dynamic adjustment coefficient algorithm can obtain the real-time adjustment coefficient of the real-time frame bird's-eye view in combination with the adjustment coefficient at the previous moment. The image composed of the real-time image frame obtained based on the real-time adjustment coefficient and the image frame at the previous moment is an image of a process, which effectively avoids screen flickering between frames and improves the smoothness of the vehicle-mounted panoramic image.

[0050] The method and device in this application are described below through specific embodiments.

[0051] See also Figure 1 , Figure 1 is a flow chart of a method for adjusting a vehicle-mounted panoramic image screen according to an exemplary embodiment of the present invention, see Figure 1 , the method comprising:

[0052] Step S11, obtaining a real-time frame bird's-eye view of four directions around the vehicle body;

[0053] Step S12, obtaining an adjustment coefficient of the real-time frame bird's-eye view according to the real-time frame bird's-eye view and a dynamic adjustment coefficient algorithm;

[0054] Step S13, adjusting the real-time frame bird's-eye view by adjusting the coefficient to obtain real-time target bird's-eye views corresponding to four directions;

[0055] Step S14: The real-time bird's-eye views of the targets in four directions are alpha-fused to obtain a real-time frame of the vehicle-mounted panoramic image.

[0056] It should be noted that the technical solution provided in this embodiment can be added to the existing vehicle-mounted panoramic imaging system in the form of a small program in specific practice, or it can also be in the form of an independent application to provide an interface to the outside world to complete the panoramic image screen adjustment function; applicable scenarios include but are not limited to: vehicle-mounted panoramic image screen adjustment.

[0057] It can be understood that the method provided in this embodiment obtains the adjustment coefficient of the real-time frame bird's-eye view based on the real-time frame bird's-eye view and the dynamic adjustment coefficient algorithm, adjusts the real-time frame bird's-eye view through the adjustment coefficient to obtain the real-time target bird's-eye view corresponding to the four directions, and obtains the real-time frame of the vehicle-mounted panoramic image by alpha fusion of the real-time target bird's-eye view in the four directions; the dynamic adjustment coefficient algorithm can obtain the real-time adjustment coefficient of the real-time frame bird's-eye view in combination with the adjustment coefficient at the previous moment, and the image composed of the real-time image frame obtained based on the real-time adjustment coefficient and the image frame at the previous moment is an image of a process, which effectively avoids the flickering of the picture between frames and improves the smoothness of the vehicle-mounted panoramic image.

[0058] In specific practice, step S11 "obtaining a real-time frame bird's-eye view of the four directions around the vehicle body" includes: obtaining the current frames of the four fisheye cameras in front, behind, left and right of the vehicle body, and converting the current frames into real-time frame bird's-eye views of the corresponding four directions.

[0059] It should be noted that four fisheye images of the current frame are collected by the on-board surround-view cameras in the front, back, left and right directions of the car body, and the fisheye images are converted into four bird's-eye images in RGB color mode through the existing algorithm. In the same pixel coordinate system, the parts with the same coordinates are defined as the overlapping areas of the initial bird's-eye images.

[0060] In specific practice, step S12 "dynamic adjustment coefficient algorithm" includes: obtaining the four-directional bird's-eye view of the first frame and the four-directional bird's-eye view of the n+1th frame collected in the first adjustment period; one adjustment period is n frames; obtaining the first adjustment coefficient based on the four-directional bird's-eye view of the first frame, and obtaining the second adjustment coefficient based on the four-directional bird's-eye view of the n+1th frame; obtaining the dynamic adjustment coefficient corresponding to each frame of the bird's-eye view in the second adjustment period through the dynamic adjustment formula based on the first adjustment coefficient and the second adjustment coefficient; the second adjustment period is the next adjustment period of the first adjustment period; obtaining the dynamic adjustment coefficient corresponding to each frame of the bird's-eye view in the third adjustment period through the dynamic adjustment formula based on the second adjustment coefficient and the starting frame adjustment coefficient in the third adjustment period; the start and end frame adjustment coefficients include the adjustment coefficient of the first frame of the bird's-eye view in the current adjustment period and the adjustment coefficient of the first frame of the bird's-eye view in the next adjustment period; starting from the third adjustment period, the adjustment coefficient corresponding to each frame of the bird's-eye view in the subsequent adjustment periods depends on the starting frame adjustment coefficient in the previous adjustment period and is obtained through the dynamic adjustment formula.

[0061] Specifically, the dynamic adjustment formula is: K ia channel =F i1 channel +a×(F ip1 channel –F i1 channel ) / n; where K ia channel F is the dynamic adjustment coefficient corresponding to the a-th frame of the bird's-eye view in the current adjustment period, 1≤a≤n; i1 channel is the preliminary adjustment coefficient corresponding to the first frame of the bird's-eye view in the previous adjustment period of the current adjustment period; F ip1 channel is the adjustment coefficient corresponding to the first frame of the bird's-eye view in the current adjustment cycle; where i∈(1,2,3,4); channel∈(R,G,B). Where 1, 2, 3, and 4 in i∈(1,2,3,4) represent the front, left, rear, and right of the vehicle body respectively; for example, K 1a R It is the adjustment coefficient of the R color corresponding to the bird's-eye view of the front position of the vehicle body in the a-th frame bird's-eye view.

[0062] It should be noted that the value of n depends on the specific image quality requirements. As an adjustment cycle, n can be set to 10-30. When n is larger, the adjustment coefficient will change less, which corresponds to high image quality. That is, the larger n is, the less obvious the picture flickering is. Among them, the first frame to the nth frame in the first adjustment cycle do not adjust the bird's-eye view picture, which provides basic data support for subsequent adjustments. The specific adjustment process is: ①, record the adjustment coefficient calculated for the first frame as c1, record the adjustment coefficient calculated for the current frame after n frames as c2, and record the difference D1 between c2 and c1. During this period, the bird's-eye view image is not adjusted from the first frame to the nth frame; ②, starting from the n+1th frame, the final adjustment coefficient is the adjustment coefficient c1 of the first frame plus the difference D1 / n, and the final adjustment coefficient of the n+2th frame is the adjustment coefficient c1 of the first frame plus the difference 2*D1 / n; ③, and so on, and finally approaching the adjustment coefficient of the nth frame, that is, the final adjustment coefficient of the 2nth frame; ④, starting from the 2n+1th frame, record the calculated adjustment coefficient as c3, and record the difference D2 between c3 and c2, and the final adjustment coefficient is The adjustment coefficient c2 of the n+1th frame is added with the difference D2 / n, and the final adjustment coefficient of the 2n+2th frame is the adjustment coefficient c2 of the n+1th frame plus the difference 2*D2 / n; ⑤, and so on, and finally approach the adjustment coefficient of the 2nth frame, which is the final adjustment coefficient of the 3nth frame; ⑥, starting from the 3n+1th frame, record the calculated adjustment coefficient as c4, and record the difference D3 between c4 and c3, and repeat the above steps to calculate the final adjustment coefficient of each frame in every n frames, so as to achieve the bird's-eye view picture adjustment and effectively avoid the flicker of the picture. Among them, the value of n can be determined according to the actual test situation, and the final frame rate can be stabilized between 25-30FPS.

[0063] In specific practice, the first adjustment coefficient is obtained according to the bird's-eye view of the first frame in four directions, including: obtaining four overlapping areas according to the initial bird's-eye view in four directions of front, back, left, and right of the first frame; the initial bird's-eye view is an RGB color mode image; and obtaining the adjustment coefficients of the four initial bird's-eye views through an error algorithm according to the number of pixels and the pixel mean of the image in the overlapping area.

[0064] Specifically, see Figure 2 , Figure 2Schematic diagram A of an overlap area shown in an exemplary embodiment of the present invention, four overlap areas are obtained according to the initial bird's-eye view in four directions of the first frame, including: the vehicle-mounted surround view camera collects four fisheye images of the front, back, left, and right of the vehicle body, and generates four bird's-eye views in RGB color format through any bird's-eye view algorithm; the four bird's-eye views are all rectangular, and the four bird's-eye views are placed in front of, to the left, to the right, and to the rear of the corresponding vehicle with the centroid of the top view of the vehicle as the center, and a plane rectangular coordinate system is established with the center as the origin, and the area formed by the pixel points with the same pixel coordinates is taken as the overlap area, and the overlap area is a rectangle; this method defines the bird's-eye views scattered in the front, back, left, and right of the vehicle body in the same pixel coordinate system, and the parts with the same coordinates are the overlap areas of the bird's-eye views.

[0065] In specific practice, the adjustment coefficients of the four initial bird's-eye views are obtained through an error algorithm based on the number of pixels and the pixel mean of the image in the overlapped area, including: taking different overlapped areas as the calculation range, respectively establishing corresponding minimization error functions, which can be established by the following standard formula: In the standard formula, N is the area of ​​the overlap between the current i and j images, n is the number of images, and F i c and F j c is the adjustment coefficient of the corresponding color channel of the corresponding image, and the image color format is not limited to RGB, YUV, etc. Take RGB as an example, where c∈{R,G,B}; i, j∈{0,1,2,3}, respectively correspond to the overlapping areas of the front left and rear right images. It should be noted that when establishing the formula, it must be ensured that image i and image j are in the same formula, and the overlapping areas of the two are the corresponding overlapping areas. That is, there is no overlapping area between the bird's-eye view corresponding to the front camera and the bird's-eye view corresponding to the rear camera, and the minimum error function cannot be constructed between the two, that is, e=0. M i c and M j c is the arithmetic mean of the color channels of the overlapping areas of the corresponding images, σ N and σ g They are the channel error standard deviation and the gain coefficient standard deviation of the corresponding channel, generally σ N Take the empirical value between 5.0 and 10.0, σ g Take the empirical value of 0.1 and adjust it according to the actual situation. In this example, the specific overlap area is as follows Figure 3 As shown, I, II, III, and IV are overlapping ranges, F0 and F1 are overlapping parts of the front camera image with the left camera and the right camera, B0 and B1 are overlapping parts of the rear camera image with the left camera and the right camera, and so on... More division methods can be determined manually, not limited to Figure 2The division method shown in the figure; the minimized error function established by combining all the images, using methods such as but not limited to Gaussian elimination method to solve the equations, and obtain the corresponding adjustment coefficients F of the channels corresponding to the four images n c , n∈{0,1,2,3} (where n represents four screen images), as the preliminary adjustment coefficient of the corresponding image. Specifically, taking the front camera and the left camera as an example, determine the overlapping area between the two and establish the minimization error function of each independent channel, and solve the adjustment coefficient of each channel as the preliminary adjustment coefficient of the current corresponding camera.

[0066] It can be understood that the method provided in this embodiment obtains the adjustment coefficient through an error algorithm, does not rely on a specific encapsulation function library and complex iterative solution, has a low computational cost, and can improve the color consistency of the four bird's-eye views.

[0067] In specific practice, step S13 "adjusting the real-time frame bird's-eye view by adjusting the coefficient to obtain the real-time target bird's-eye view corresponding to four directions" includes: obtaining the real-time frame target bird's-eye view by an image adjustment formula according to the adjustment coefficient; the image adjustment formula is:

[0068]

[0069] Among them, dstImg i channel is the real-time target bird's-eye view of the i-th bird's-eye view in the real-time frame bird's-eye view, where i∈(1,2,3,4); srcImg i channel is the i-th bird's-eye view in the real-time frame bird's-eye view; k i channel is the adjustment coefficient of the i-th bird's-eye view in the real-time frame bird's-eye view; channel∈(R, G, B).

[0070] It can be understood that the method provided in this embodiment makes the color consistency of the four bird's-eye views better by adjusting the coefficients.

[0071] In specific practice, step S14 "obtaining a real-time frame of the vehicle-mounted panoramic image by alpha fusion of the real-time target bird's-eye view images in four directions" includes: obtaining a real-time frame of the vehicle-mounted panoramic image by fusing the real-time target bird's-eye view images in four directions based on the alpha fusion formula; the alpha fusion formula is: C = αCb + (1-α)Ca; wherein, C a A bird's-eye view of any real-time target in the left or right direction; C b For C a A bird's-eye view of any real-time target with overlapping areas; α is the ratio of the Euclidean distance from the current point to the left or right boundary to the sum of the Euclidean distances from the current point to the left and right boundaries, that is, the normalized weight.

[0072] It should be noted that, with C a Take the left view as an example, where the left view C a As the reference screen, C b As the picture bordering the reference picture (here, the front camera picture), with the seam as the left boundary and the right boundary of the overlapped area as the end, the points in this area are calculated by applying the above formula to the final alpha value and updated to the image. Among them, the left boundary can be abstracted by an approximate straight line equation, and α is the ratio of the Euclidean distance from the current point to the left or right boundary to the sum of the Euclidean distances from the current point to the left and right boundaries, that is, the normalized weight.

[0073] It can be understood that the technical solution provided in this embodiment effectively solves the problem of seams generated in overlapping areas and achieves a good uniform effect.

[0074] In a specific embodiment, the above operations can be implemented through GPU chip programming. The use of APIs such as opengl and vulkan for GPU image rendering has significant advantages, mainly reflected in its powerful parallel computing capabilities. Solving the system of equations consisting of the minimum error function, performing lag approximation and feathering transition processing all require expensive calculations. Compared with the CPU, the GPU has a large number of stream processor cores, which can significantly speed up the processing speed of large-scale data, thereby shortening the rendering time, thereby ensuring the stability and real-time performance of the entire AVM system.

[0075] Specifically, it is possible to consider using the opengl compute shader to process the calculation process involved in this method. The compute shader can be specifically expressed as setting workgroups and local work items according to the actual size of the image by utilizing the highly parallel computing capability of the GPU. Each workgroup consists of multiple threads, and the GPU executes these threads in parallel, which greatly improves the efficiency of the computing task. In this method, the relevant calculations involving overlapping areas can be carried out by the workgroup and local work items in the compute shader to reduce computational redundancy and improve efficiency. At the same time, the workgroup and local work items can cover the entire image as much as possible to avoid the result of missing calculations. It should be noted that due to the high concurrency of the compute shader, certain memory barrier mechanisms (Memory Barriers) and synchronization functions within the workgroup must be adopted to ensure data access consistency between different workgroups or threads and avoid data competition and read errors. In addition, multiple compute shaders can be used to work together within the rendering of a frame. Data can be transferred between compute shaders through Shader Storage Buffer Object (SSBO). SSBO is a type of buffer object introduced in the advanced version of OpenGL, which allows shaders to read and write large amounts of data, and can also share data between multiple compute shaders to achieve cross-compute shader usage. Using this mechanism, complex computing tasks are split into different stages, each stage is executed by a corresponding compute shader, and data is transferred and shared between them through SSBO.

[0076] It can be understood that the technical solution provided in this embodiment is developed based on the on-board GPU chip, and uses the GPU image rendering API interface to maximize the parallel processing of high-resolution images and large-scale data, improve rendering efficiency, shorten rendering time, and achieve high frame rate and low latency.

[0077] See also Figure 4 , Figure 4 is a schematic block diagram of an adjustment device for a vehicle-mounted panoramic image according to an exemplary embodiment of the present invention, see Figure 4 The vehicle-mounted panoramic image adjustment device 100 includes:

[0078] An acquisition module 101 is used to acquire a real-time frame bird's-eye view of the vehicle body in four directions;

[0079] An adjustment coefficient module 102, used to obtain an adjustment coefficient of the real-time frame bird's-eye view according to the real-time frame bird's-eye view and a dynamic adjustment coefficient algorithm;

[0080] An adjustment module 103 is used to adjust the real-time frame bird's-eye view by adjusting the coefficient to obtain real-time target bird's-eye views corresponding to four directions;

[0081] The fusion module 104 is used to obtain a real-time frame of the vehicle-mounted panoramic image by alpha fusion of the real-time bird's-eye views of the target in four directions.

[0082] It should be noted that the technical solution provided by this embodiment can be applied to scenarios including, but not limited to: adjusting the in-vehicle panoramic image.

[0083] It can be understood that the device provided in this embodiment obtains the adjustment coefficient of the real-time frame bird's-eye view based on the real-time frame bird's-eye view and the dynamic adjustment coefficient algorithm, adjusts the real-time frame bird's-eye view through the adjustment coefficient to obtain the real-time target bird's-eye view corresponding to the four directions, and obtains the real-time frame of the vehicle-mounted panoramic image by alpha fusion of the real-time target bird's-eye view in the four directions; the dynamic adjustment coefficient algorithm can obtain the real-time adjustment coefficient of the real-time frame bird's-eye view in combination with the adjustment coefficient at the previous moment, and the image composed of the real-time image frame obtained based on the real-time adjustment coefficient and the image frame at the previous moment is an image of a process, which effectively avoids the flickering of the picture between frames and improves the smoothness of the vehicle-mounted panoramic image.

[0084] See also Figure 5 , Figure 5 is a schematic block diagram of an electronic device according to an exemplary embodiment of the present invention, see Figure 5 , an electronic device 200, comprising:

[0085] at least one processor 202; and

[0086] A memory 201 is communicatively connected to at least one processor 202; wherein,

[0087] The memory 201 stores instructions that can be executed by at least one processor 202. The instructions are executed by at least one processor 202 so that the at least one processor 202 can execute any of the above-mentioned methods for adjusting the vehicle-mounted panoramic image screen.

[0088] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

[0089] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0090] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for adjusting a vehicle-mounted panoramic image, characterized in that: The method comprises: Get a real-time frame bird's-eye view of the vehicle body in four directions; Obtaining an adjustment coefficient of the real-time frame bird's-eye view according to the real-time frame bird's-eye view and a dynamic adjustment coefficient algorithm; The real-time frame bird's-eye view is adjusted by the adjustment coefficient to obtain real-time target bird's-eye views corresponding to four directions; The real-time target bird's-eye views in the four directions are fused through alpha fusion to obtain a real-time frame of the vehicle-mounted panoramic image.

2. The adjustment method according to claim 1, characterized in that: The dynamic adjustment coefficient algorithm includes: Acquire the four-directional bird's-eye view of the first frame and the four-directional bird's-eye view of the (n+1)th frame collected in the first adjustment period; one adjustment period is n frames; Obtaining a first adjustment coefficient according to the four-directional bird's-eye view of the first frame, and obtaining a second adjustment coefficient according to the four-directional bird's-eye view of the (n+1)th frame; According to the first adjustment coefficient and the second adjustment coefficient, an adjustment coefficient corresponding to each frame of the bird's-eye view in a second adjustment period is obtained by a dynamic adjustment formula; the second adjustment period is an adjustment period next to the first adjustment period; The adjustment coefficient corresponding to each frame of the bird's-eye view in the third adjustment period is obtained by a dynamic adjustment formula according to the start and end frame adjustment coefficients in the second to third adjustment periods; the start and end frame adjustment coefficients include the adjustment coefficient of the first frame of the bird's-eye view in the second adjustment period and the adjustment coefficient of the start frame of the bird's-eye view in the next adjustment period; The adjustment coefficient corresponding to each frame of the bird's-eye view in the subsequent adjustment cycles starting from the third adjustment cycle is obtained by a dynamic adjustment formula depending on the adjustment coefficient of the starting frame in the previous adjustment cycle.

3. The adjustment method according to claim 2, characterized in that: The dynamic adjustment formula is: K ia channel =F i1 channel +a×(F ip1 channel –F i1 channel ) / n; Among them, K ia channel F is the dynamic adjustment coefficient corresponding to the a-th frame of the bird's-eye view in the current adjustment period, 1≤a≤n; i1 channel is the preliminary adjustment coefficient corresponding to the first frame of the bird's-eye view in the previous adjustment period of the current adjustment period; F ip1 channel It is the adjustment coefficient corresponding to the first frame of the bird's-eye view in the current adjustment period; where i∈(1,2,3,4); channel∈(R,G,B).

4. The adjustment method according to claim 2, characterized in that: The obtaining of the first adjustment coefficient according to the four-directional bird's-eye view of the first frame includes: Four overlapping areas are obtained according to the initial bird's-eye view in four directions of the first frame, front, back, left, and right; the initial bird's-eye view is an RGB color mode picture; The adjustment coefficients of the four initial bird's-eye views are obtained through an error algorithm according to the number of pixels and the pixel mean of the image in the overlapped area.

5. The adjustment method according to claim 1, characterized in that: The real-time frame bird's-eye view includes four bird's-eye views of the front, back, left, and right sides of the vehicle body; the real-time frame bird's-eye view is adjusted by the adjustment coefficient to obtain real-time target bird's-eye views corresponding to four directions, including: A real-time frame target bird's-eye view is obtained through an image adjustment formula according to the adjustment coefficient; the image adjustment formula is: Among them, dstImg i channel is the real-time target bird's-eye view of the i-th bird's-eye view in the real-time frame bird's-eye view, where i∈(1,2,3,4); srcImg i channel is the i-th bird's-eye view in the real-time frame bird's-eye view; k i channel is the adjustment coefficient of the i-th bird's-eye view in the real-time frame bird's-eye view; channel∈(R, G, B).

6. The adjustment method according to claim 1, characterized in that: The method of obtaining a real-time frame of a vehicle-mounted panoramic image by alpha fusion of the real-time target bird's-eye view images in the four directions includes: The real-time target bird's-eye view images in the four directions are fused based on the alpha fusion formula to obtain a real-time frame of the vehicle-mounted panoramic image; the alpha fusion formula is: C = αCb + (1-α)Ca; Among them, C a A bird's-eye view of any real-time target in the left or right direction; C b For C a A bird's-eye view of any real-time target with overlapping areas; α is the ratio of the Euclidean distance from the current point to the left or right boundary to the sum of the Euclidean distances from the current point to the left and right boundaries, that is, the normalized weight.

7. A vehicle-mounted panoramic image adjustment device, characterized in that: The device comprises: An acquisition module is used to acquire a real-time frame bird's-eye view of the vehicle body in four directions; An adjustment coefficient module, used for obtaining the adjustment coefficient of the real-time frame bird's-eye view according to the real-time frame bird's-eye view and a dynamic adjustment coefficient algorithm; An adjustment module, used for adjusting the real-time frame bird's-eye view by the adjustment coefficient to obtain real-time target bird's-eye views corresponding to four directions; The fusion module is used to obtain a real-time frame of the vehicle-mounted panoramic image by alpha fusion of the real-time target bird's-eye views in the four directions.

8. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for adjusting the vehicle-mounted panoramic image screen as described in any one of claims 1-6.

Images