Method, device and equipment for adjusting brightness of panoramic image

Abstract

The invention discloses a method, a device and equipment for adjusting the brightness of a panoramic image. The method comprises the following steps: acquiring front and rear views and left and right views for splicing the panoramic image; based on a first brightness difference between the front and rear views and a second brightness difference between the left and right views, determining a first view of which the brightness is to be adjusted and a reference view for adjusting the brightness of the first view in the front and rear views and the left and right views; adjusting the brightness of the overlapping area in the first view based on the brightness of the overlapping area in the reference view; and splicing the reference view and the first view after brightness adjustment to obtain a panoramic image. According to the scheme of the invention, the brightness balance of the panoramic image can be realized, the brightness transition at the splicing junction of the views is smoother and more natural, a large amount of calculation caused by a global brightness adjustment algorithm can be effectively avoided, and the resource consumption is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to, but not limited to, a method, a device and an apparatus for adjusting the brightness of a panoramic image. Background Art

[0002] As an advanced automobile assistance system, the panoramic monitoring imaging system can provide the driver with all-round and full-angle panoramic images by stitching together multiple angle views around the vehicle, thereby effectively assisting the vehicle in reversing, parking and driving in complex road conditions, helping to improve driving safety and convenience.

[0003] In actual applications, the top view of the on-board panoramic image is usually stitched together by the front view, rear view, left view, and right view captured by the four cameras in the front, rear, left, and right directions of the vehicle. However, due to the differences in the ambient light conditions of each camera, there are differences in brightness between the views captured by each camera, resulting in uneven brightness in the stitched panoramic image top view, which seriously affects the user's visual experience. Summary of the invention

[0004] One of the purposes of the present invention is to provide a method, device and equipment for adjusting the brightness of a panoramic image, which can not only achieve brightness balance of the panoramic image, making the brightness transition at the junction of view stitching smoother and more natural, but also effectively avoid the large amount of calculation brought by the global brightness adjustment algorithm and reduce resource consumption.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a method for adjusting the brightness of a panoramic image, comprising: obtaining front and rear views and left and right views for stitching a panoramic image; determining a first view to be adjusted in brightness and a reference view for adjusting the brightness of the first view in the front and rear views and the left and right views based on a first brightness difference between the front and rear views and a second brightness difference between the left and right views; adjusting the brightness of an overlapping area in the first view based on the brightness of an overlapping area in the reference view; and stitching the reference view with the first view after brightness adjustment to obtain a panoramic image.

[0007] According to the above technical means, by calculating the first brightness difference between the front and rear views and the second brightness difference between the left and right views, the brightness of the overlapping areas in the other two views can be adjusted according to the brightness of the overlapping areas in the two views with the smallest brightness difference, thereby realizing the stitching of the four views to obtain a panoramic image. The scheme of the present invention fully considers the influence of the brightness balance of the overlapping areas of the views on the brightness balance of the panoramic image, and by using the brightness of the overlapping areas in the front and rear views to adjust the brightness of the overlapping areas in the left and right views, or by using the brightness of the overlapping areas in the left and right views to adjust the brightness of the overlapping areas in the front and rear views, not only can the brightness balance of the panoramic image be achieved, but also the brightness transition at the junction of the views is smoother and more natural, and the large amount of calculation caused by the global brightness adjustment algorithm can be effectively avoided, thereby reducing resource consumption.

[0008] In a possible implementation, based on the brightness of the overlapping area in the baseline view, the brightness of the overlapping area in the first view is adjusted, including: for each first overlapping area in at least one first overlapping area included in the first view, performing the following processing: determining a target overlapping area that overlaps with the first overlapping area in at least one baseline overlapping area included in the baseline view; and adjusting the brightness of the first overlapping area based on the brightness of the target overlapping area.

[0009] According to the above technical means, by determining the target overlapping area that overlaps with the first overlapping area in at least one reference overlapping area, the brightness of the first overlapping area can be adjusted based on the brightness of the target overlapping area. The scheme of the present invention fully considers the overlapping correspondence between different overlapping areas, and by adjusting the brightness of one of the two overlapping overlapping areas based on the brightness of the other overlapping area, the brightness balance and smooth transition between the two overlapping overlapping areas can be achieved, and the implementation scheme is simple and efficient.

[0010] In a possible implementation, the brightness of the first overlapping area is adjusted based on the brightness of the target overlapping area, including: determining the current brightness of the first overlapping area as the initial brightness of the first overlapping area, and determining the current brightness of the target overlapping area as the reference brightness of the first overlapping area; raising the initial brightness to the power of a brightness adjustment factor to determine a brightness adjustment value of the first overlapping area; the brightness adjustment factor is used to characterize the ratio of the initial brightness to the reference brightness; and adjusting the brightness of the first overlapping area to the brightness adjustment value.

[0011] According to the above technical means, by determining the ratio of the initial brightness to the reference brightness as the brightness adjustment factor, the brightness adjustment value of the first overlapping area can be determined based on the initial brightness and the brightness adjustment factor. The embodiment of the present invention takes into account that the human eye's perception of brightness has nonlinear characteristics, and describes this nonlinear characteristic by using a power function with the initial brightness as the base and the brightness adjustment factor as the exponent, so as to more accurately simulate the difference in the human eye's perception of different brightness areas, so that when the brightness of the first overlapping area is adjusted, it can better adapt to the visual habits of the human eye, so that the adjusted image is more natural, comfortable and visually coherent.

[0012] In a possible implementation, the method for adjusting the brightness of a panoramic image further includes: determining a non-overlapping area and a reference overlapping area adjacent to the non-overlapping area in the first view; and adjusting the brightness of the non-overlapping area based on a brightness adjustment value of the reference overlapping area.

[0013] According to the above technical means, considering the influence of the brightness difference between the non-overlapping area and the overlapping area on the overall brightness balance of the panoramic image, by determining the reference overlapping area adjacent to the non-overlapping area, the brightness of the non-overlapping area can be adjusted based on the brightness adjustment value of the reference overlapping area. In this way, the visual disharmony caused by the excessive brightness difference between the non-overlapping area and the overlapping area can be effectively avoided, and the brightness of the non-overlapping area is more coordinated and consistent with the brightness of the adjacent overlapping area, so as to achieve the overall brightness balance of the panoramic image and natural transition.

[0014] In one possible implementation, the brightness of the non-overlapping area is adjusted based on the brightness adjustment value of the reference overlapping area, including: determining the starting point coordinates and the ending point coordinates of the non-overlapping area; performing the following processing for each pixel point of at least one pixel point included in the non-overlapping area: determining the pixel brightness adjustment value of the pixel point based on the current coordinates of the pixel point, the starting point coordinates, the ending point coordinates, and the brightness adjustment value of the reference overlapping area; and adjusting the brightness of the pixel point to the pixel brightness adjustment value.

[0015] According to the above technical means, by determining the starting point coordinates and the end point coordinates of the non-overlapping area, it is possible to achieve the brightness adjustment value based on the starting point coordinates, the end point coordinates and the reference overlapping area, determine the pixel brightness adjustment value pixel by pixel in the non-overlapping area, and then achieve the brightness adjustment of all pixels in the non-overlapping area. The scheme of the present invention takes into account the position characteristics of different pixels in the non-overlapping area. By adjusting the brightness pixel by pixel in the non-overlapping area, it can avoid the unnatural feeling and local over-brightness or over-darkness problems caused by the uniform brightness adjustment of all pixels, and more accurately control the brightness adjustment of the non-overlapping area to ensure that the brightness transition between the non-overlapping area and the adjacent reference overlapping area is natural and coherent.

[0016] In a possible implementation, based on a first brightness difference between front and rear views and a second brightness difference between left and right views, determining a first view whose brightness is to be adjusted and a reference view for adjusting the brightness of the first view in the front and rear views and the left and right views includes: determining a first brightness difference based on the brightness of an overlapping area in the front view and the brightness of an overlapping area in the rear view, and determining a second brightness difference based on the brightness of an overlapping area in the left view and the brightness of an overlapping area in the right view; if the first brightness difference is less than or equal to the second brightness difference, determining the left view and the right view as the first view, and determining the front view and the rear view as the reference view; if the first brightness difference is greater than the second brightness difference, determining the front view and the rear view as the first view, and determining the left view and the right view as the reference view.

[0017] According to the above technical means, by determining the size relationship between the first brightness difference between the front and rear views and the second brightness difference between the left and right views, it is possible to determine the two views with the smallest brightness difference as the reference view, and the two views with the largest brightness difference as the first view. The scheme of the present invention takes into account the different orientation relationships between different views, and by comparing the brightness difference between the front and rear views and the brightness difference between the left and right views, it is possible to achieve more targeted brightness adjustment of the first view with reference to the reference view, which not only helps to quickly and efficiently achieve brightness balance between different views during the stitching process of panoramic images, but also can reduce additional calculation and adjustment processes, and improve the efficiency and quality of the panoramic image stitching process.

[0018] In a possible implementation, when the front view includes a front left overlapping area and a front right overlapping area, and the rear view includes a rear left overlapping area and a rear right overlapping area, a first brightness difference is determined based on the brightness of the overlapping areas in the front view and the brightness of the overlapping areas in the rear view, including: respectively determining the average brightness of the front left overlapping area, the front right overlapping area, the rear left overlapping area, and the rear right overlapping area to obtain the front left average brightness, the front right average brightness, the rear left average brightness, and the rear right average brightness; determining a first sum of the front left average brightness and the front right average brightness, and a second sum of the rear left average brightness and the rear right average brightness; and determining the difference between the first sum and the second sum as the first brightness difference.

[0019] According to the above technical means, by determining the sum of the brightness of the front left overlapping area and the front right overlapping area, and the sum of the brightness of the rear left overlapping area and the rear right overlapping area, the difference between the two sums can be determined as the brightness difference between the front view and the rear view. In this way, since the overlapping relationship between the front view and the rear view and the left and right views is taken into account, the brightness difference between the front view and the rear view in the overlapping area can be more accurately measured, and the brightness difference obtained by comprehensive consideration of the brightness of the corresponding overlapping area is more representative and reliable.

[0020] In a possible implementation, the method for adjusting the brightness of a panoramic image also includes: using a graphics processor to bind the overlapping area as a texture image to an initial frame buffer object based on a target graphics drawing library to obtain a target frame buffer object; the number of pixels P of the target frame buffer object is less than the number of pixels Q of the front left overlapping area; P and Q are both integers greater than 0; extracting the brightness value of each pixel of the P pixels included in the target frame buffer object through the graphics processor to obtain P brightness values; and determining the brightness of the overlapping area based on the P brightness values ​​through the central processing unit.

[0021] According to the above technical means, the graphics processor binds the overlapping area with a large number of pixels as a texture image to the initial frame buffer object with a small number of pixels and then extracts the brightness value, and the central processing unit determines the brightness of the overlapping area. This can fully utilize the multi-core concurrent computing power of the graphics processor to effectively reduce the performance consumption of the central processing unit. In addition, since the number of pixels in the calculation process of the scheme of the present invention is less than the actual number of pixels included in the overlapping area, the time for determining each brightness adjustment value is faster and the performance consumption is less, which can better achieve the effect of real-time brightness adjustment and no jamming of the panoramic image.

[0022] In a second aspect, the present invention provides a device for adjusting the brightness of a panoramic image, comprising:

[0023] An acquisition unit, used for acquiring front and rear views and left and right views for stitching a panoramic image;

[0024] a determining unit, configured to determine, based on a first brightness difference between the front and rear views and a second brightness difference between the left and right views, a first view whose brightness is to be adjusted and a reference view for adjusting the brightness of the first view from among the front and rear views and the left and right views;

[0025] an adjusting unit, configured to adjust the brightness of the overlapping area in the first view based on the brightness of the overlapping area in the reference view;

[0026] The stitching unit is used to stitch the reference view and the first view after brightness adjustment to obtain a panoramic image.

[0027] In a third aspect, the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the above-mentioned method for adjusting the brightness of the panoramic image is implemented.

[0028] In a fourth aspect, the present invention further provides a storage medium having a computer program stored thereon, and when the computer program on the storage medium is executed, the above-mentioned method for adjusting the brightness of the panoramic image is implemented.

[0029] In a fifth aspect, the present invention further provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the above-mentioned method for adjusting the brightness of a panoramic image.

[0030] It should be noted that the technical effects of the second to fifth aspects can refer to the detailed description of the first aspect above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of a first optional flow chart of a method for adjusting the brightness of a panoramic image provided by an embodiment of the present invention;

[0032] Figure 2 A schematic diagram of a second optional flow chart of a method for adjusting the brightness of a panoramic image provided by an embodiment of the present invention;

[0033] Figure 3 A schematic diagram of a third optional flow chart of the method for adjusting the brightness of a panoramic image provided by an embodiment of the present invention;

[0034] Figure 4 A fourth optional flow chart of the method for adjusting the brightness of a panoramic image provided by an embodiment of the present invention;

[0035] Figure 5 A fifth optional flow chart of the method for adjusting the brightness of a panoramic image provided by an embodiment of the present invention;

[0036] Figure 6 A sixth optional flow chart of the method for adjusting the brightness of a panoramic image provided by an embodiment of the present invention;

[0037] Figure 7 A seventh optional flow chart of the method for adjusting the brightness of a panoramic image provided by an embodiment of the present invention;

[0038] Figure 8 A schematic diagram of an eighth optional flow chart of the method for adjusting the brightness of a panoramic image provided by an embodiment of the present invention;

[0039] Fig. 9 A ninth optional flow chart of the method for adjusting the brightness of a panoramic image provided by an embodiment of the present invention;

[0040] Fig.10 An optional schematic diagram of an overlapping area of ​​original size in the method for adjusting the brightness of a panoramic image provided by an embodiment of the present invention;

[0041] Fig.11 An optional schematic diagram of an overlapping area of ​​a specified FBO size in a method for adjusting the brightness of a panoramic image provided by an embodiment of the present invention;

[0042] Fig.12 A schematic diagram of an optional structure of a panoramic image brightness adjustment device provided by an embodiment of the present invention;

[0043] Fig.13 An optional structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] To make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the specific technical solution of the application will be further described in detail below in conjunction with the drawings in the embodiments of the present invention. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0045] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0046] In the following description, the terms "first\second\third" are used only as examples to distinguish different objects, and do not represent a specific order for the objects, nor do they have a limitation on the order of precedence. It is understood that "first\second\third" can be interchanged with a specific order or order of precedence where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0048] The embodiments of the present invention provide a method, device, equipment, storage medium and computer program product for adjusting the brightness of a panoramic image. In practical applications, the method for adjusting the brightness of a panoramic image can be implemented by a device for adjusting the brightness of a panoramic image, and each functional entity in the device for adjusting the brightness of a panoramic image can be collaboratively implemented by hardware resources of an electronic device, such as computing resources such as a processor, and communication resources (such as for supporting various communication methods such as optical cables and cellular).

[0049] Hereinafter, various embodiments of the method, apparatus, device, storage medium and computer program product for adjusting the brightness of a panoramic image provided by the embodiments of the present invention will be described.

[0050] In a first aspect, an embodiment of the present invention provides a method for adjusting the brightness of a panoramic image. The function implemented by the method can be implemented by calling a program code by a processor in an electronic device (such as a terminal device or a controller), and of course the program code can be stored in a computer storage medium. It can be seen that the electronic device at least includes a processor and a storage medium.

[0051] In a possible implementation, the electronic device may be a terminal device or a controller in a terminal device, and the terminal device at least includes a vehicle device.

[0052] Next, the method for adjusting the brightness of a panoramic image provided by an embodiment of the present invention is described by taking an electronic device as an example of an execution subject.

[0053] refer to Figure 1 As shown in the content, the process may include but is not limited to the following S101 to S104.

[0054] S101: The electronic device obtains front and rear views and left and right views for stitching a panoramic image.

[0055] The front and rear views refer to the front and rear views used to stitch the panoramic image.

[0056] The left and right views refer to the left and right views used to stitch the panoramic image.

[0057] The embodiment of the present invention does not limit the attribute parameters such as size, shape, brightness, contrast, color saturation, etc. of the front and rear views and the left and right views, and they can be set according to actual conditions.

[0058] In an embodiment of the present invention, the panoramic image of the electronic device is stitched together by the front view, rear view, left view, and right view of the electronic device. In practical applications, the front view, rear view, left view, and right view of the electronic device can be captured by cameras installed at the front, rear, left, and right of the electronic device, respectively.

[0059] Exemplarily, S101 may be implemented as follows: using cameras installed at the front, rear, left, and right positions of the electronic device to capture the front view, rear view, left view, and right view of the electronic device in real time.

[0060] S102: The electronic device determines a first view whose brightness is to be adjusted and a reference view for adjusting the brightness of the first view from among the front and rear views and the left and right views based on a first brightness difference between the front and rear views and a second brightness difference between the left and right views.

[0061] The first brightness difference is used to represent the brightness difference between the front view and the rear view.

[0062] The second brightness difference is used to represent the brightness difference between the left view and the right view.

[0063] The embodiment of the present invention does not limit the specific calculation method of the first brightness difference and the second brightness difference, and can be set according to actual conditions.

[0064] In a possible implementation, the electronic device may determine the difference between the average brightness of the front view and the average brightness of the rear view as the first brightness difference, and determine the difference between the average brightness of the left view and the average brightness of the right view as the second brightness difference.

[0065] In another possible implementation, the electronic device may determine the difference between the average brightness of the overlapping area in the front view and the average brightness of the overlapping area in the rear view as the first brightness difference, and determine the difference between the average brightness of the overlapping area in the left view and the average brightness of the overlapping area in the right view as the second brightness difference.

[0066] The first view refers to the view whose brightness needs to be adjusted among the front and rear views and the left and right views.

[0067] The reference view refers to a view that does not require brightness adjustment among the front and rear views and the left and right views and is used to adjust the brightness of the first view.

[0068] It should be noted that, if the first view is a front view and a rear view, the reference view is a left view and a right view; if the first view is a left view and a right view, the reference view is a front view and a rear view.

[0069] Exemplarily, S102 can be implemented as follows: the electronic device compares the size relationship between the first brightness difference and the second brightness difference; if the first brightness difference is less than or equal to the second brightness difference, the left view and the right view are determined as the first view, and the front view and the rear view are determined as the reference view; if the first brightness difference is greater than the second brightness difference, the front view and the rear view are determined as the first view, and the left view and the right view are determined as the reference view.

[0070] S103: The electronic device adjusts the brightness of the overlapping area in the first view based on the brightness of the overlapping area in the reference view.

[0071] The overlapping area refers to the partial area with common image content between different views when the views are stitched.

[0072] In actual applications, due to the installation position, installation angle and moving object detection requirements of the camera, there are two overlapping areas in the views captured by each of the four cameras installed on the front, back, left and right of the electronic device. The front view has two overlapping areas, front left and front right, the rear view has two overlapping areas, rear left and rear right, the left view has two overlapping areas, front left and rear left, and the right view has two overlapping areas, front right and rear right.

[0073] The embodiment of the present invention does not limit the specific size and shape of the overlapping area, and can be set according to actual conditions. For example, when the installation positions of the four cameras at the front, back, left, and right of the electronic device remain unchanged, the overlapping area captured by two adjacent cameras at the same time is always fixed, so the overlapping area can be determined by calibration.

[0074] It can be understood that when the front view, rear view, left view and right view are spliced, the front left overlapping area overlaps with the left front overlapping area, the front right overlapping area overlaps with the right front overlapping area, the rear left overlapping area overlaps with the left rear overlapping area, and the rear right overlapping area overlaps with the right rear overlapping area.

[0075] The embodiment of the present invention does not limit the specific manner in which the electronic device adjusts the brightness of the overlapping area in the first view based on the brightness of the overlapping area in the reference view, and the manner can be set according to actual conditions.

[0076] Exemplarily, S103 can be implemented as follows: the electronic device determines the overlapping area included in the first view to obtain at least one first overlapping area; determines the overlapping area included in the baseline view to obtain at least one baseline overlapping area; based on a preset overlapping correspondence and the brightness of at least one baseline overlapping area, adjusts at least one first overlapping area; the preset overlapping correspondence is used to characterize the correspondence between the first overlapping area and the baseline overlapping area.

[0077] For example, since the front left overlapping area and the left front overlapping area overlap with each other when the front view and the left view are spliced, it can be considered that the front left overlapping area and the left front overlapping area correspond to each other, that is, there is an overlapping correspondence between the front left overlapping area and the left front overlapping area. At this time, the brightness of the left front overlapping area can be adjusted based on the brightness of the front left overlapping area, or the brightness of the front left overlapping area can be adjusted based on the brightness of the left front overlapping area, so that there is no brightness difference between the front view and the left view at the junction of the views.

[0078] In practical applications, if the brightness difference between the front view and the rear view is the smallest, the brightness of the left front overlapping area of ​​the left view can be aligned with the front left overlapping area of ​​the front view, the brightness of the left rear overlapping area of ​​the left view can be aligned with the rear left overlapping area of ​​the rear view, the brightness of the right front overlapping area of ​​the right view can be aligned with the front right overlapping area of ​​the front view, and the brightness of the right rear overlapping area of ​​the right view can be aligned with the rear right overlapping area of ​​the rear view, so that the brightness of the non-overlapping areas in the left view and the non-overlapping areas in the right view are evenly transitioned; if the brightness difference between the left view and the right view is the smallest, the brightness of the front left overlapping area of ​​the front view can be aligned with the left front overlapping area of ​​the left view, the front right overlapping area of ​​the front view can be aligned with the right front overlapping area of ​​the right view, the brightness of the rear left overlapping area of ​​the rear view can be aligned with the left rear overlapping area of ​​the left view, and the brightness of the rear right overlapping area of ​​the rear view can be aligned with the right rear overlapping area of ​​the right view, so that the brightness of the non-overlapping areas in the front view and the non-overlapping areas in the rear view can be evenly transitioned.

[0079] S104: The electronic device stitches the reference view and the first view after brightness adjustment to obtain a panoramic image.

[0080] It should be noted that, in the embodiment of the present invention, it is the first view that needs to adjust the brightness, and the brightness of the reference view does not need to be adjusted. After the brightness of the first view is adjusted, there is no brightness difference between the reference view and the first view after the brightness adjustment at the view stitching junction. At this time, the reference view and the first view after the brightness adjustment can be stitched to obtain a panoramic image.

[0081] Exemplarily, S104 may be implemented as follows: the electronic device uses an image stitching technology to stitch the reference view and the first view after brightness adjustment; after the stitching is completed, a panoramic image is obtained.

[0082] An embodiment of the present invention provides a method for adjusting the brightness of a panoramic image, the method comprising: obtaining front and rear views and left and right views for stitching a panoramic image; determining a first view whose brightness is to be adjusted and a reference view for adjusting the brightness of the first view in the front and rear views and the left and right views based on a first brightness difference between the front and rear views and a second brightness difference between the left and right views; adjusting the brightness of an overlapping area in the first view based on the brightness of an overlapping area in the reference view; and stitching the reference view with the first view after the brightness adjustment to obtain a panoramic image.

[0083] According to the above technical means, by calculating the first brightness difference between the front and rear views and the second brightness difference between the left and right views, the brightness of the overlapping areas in the other two views can be adjusted according to the brightness of the overlapping areas in the two views with the smallest brightness difference, thereby realizing the stitching of the four views to obtain a panoramic image. The scheme of the present invention fully considers the influence of the brightness balance of the overlapping areas of the views on the brightness balance of the panoramic image, and by using the brightness of the overlapping areas in the front and rear views to adjust the brightness of the overlapping areas in the left and right views, or by using the brightness of the overlapping areas in the left and right views to adjust the brightness of the overlapping areas in the front and rear views, not only can the brightness balance of the panoramic image be achieved, but also the brightness transition at the junction of the views is smoother and more natural, and the large amount of calculation caused by the global brightness adjustment algorithm can be effectively avoided, thereby reducing resource consumption.

[0084] Next, the process in S103 in which the electronic device adjusts the brightness of the overlapping area in the first view based on the brightness of the overlapping area in the reference view is described.

[0085] refer to Figure 2 The process may include but is not limited to the following S201 and S202.

[0086] For each first overlapping region in the at least one first overlapping region included in the first view, the following process is performed:

[0087] S201: The electronic device determines, in at least one reference overlapping region included in a reference view, a target overlapping region that overlaps with a first overlapping region.

[0088] The first overlapping area refers to the overlapping area included in the first view.

[0089] The reference overlap area refers to the overlap area included in the reference view.

[0090] The embodiment of the present invention does not limit the size, shape, brightness, contrast, color saturation, quantity and other attribute parameters of the first overlapping area and the reference overlapping area, and they can be set according to actual conditions.

[0091] The target overlapping area refers to an overlapping area in at least one reference overlapping area that overlaps with the first overlapping area.

[0092] Exemplarily, an overlapping correspondence relationship between the first overlapping area and the reference overlapping area is pre-determined and stored in the electronic device. Correspondingly, S201 can be implemented as follows: the electronic device traverses at least one first overlapping area; for each first overlapping area, the first overlapping area is searched in the overlapping correspondence relationship; and the reference overlapping area corresponding to the first overlapping area in the overlapping correspondence relationship is determined as the target overlapping area.

[0093] S202: The electronic device adjusts the brightness of the first overlapping area based on the brightness of the target overlapping area.

[0094] In a possible implementation, when the first views are the left view and the right view and the reference views are the front view and the rear view, S202 may be implemented as follows: the electronic device adjusts the brightness of the left front overlapping area in the left view based on the brightness of the front left overlapping area in the front view; adjusts the brightness of the right front overlapping area in the right view based on the brightness of the front right overlapping area in the front view; adjusts the brightness of the left rear overlapping area in the left view based on the brightness of the rear left overlapping area in the rear view; and adjusts the brightness of the right rear overlapping area in the right view based on the brightness of the rear right overlapping area in the rear view.

[0095] In another possible implementation, when the first view is a front view and a rear view and the reference view is a left view and a right view, S202 may be implemented as follows: the electronic device adjusts the brightness of the front left overlapping area in the front view based on the brightness of the left front overlapping area in the left view; adjusts the brightness of the rear left overlapping area in the rear view based on the brightness of the left rear overlapping area in the left view; adjusts the brightness of the front right overlapping area in the front view based on the brightness of the right front overlapping area in the right view; and adjusts the brightness of the rear right overlapping area in the rear view based on the brightness of the right rear overlapping area in the right view.

[0096] According to the above technical means, by determining the target overlapping area that overlaps with the first overlapping area in at least one reference overlapping area, the brightness of the first overlapping area can be adjusted based on the brightness of the target overlapping area. The scheme of the present invention fully considers the overlapping correspondence between different overlapping areas, and by adjusting the brightness of one of the two overlapping overlapping areas based on the brightness of the other overlapping area, the brightness balance and smooth transition between the two overlapping overlapping areas can be achieved, and the implementation scheme is simple and efficient.

[0097] Next, the process of adjusting the brightness of the first overlapping area based on the brightness of the target overlapping area by the electronic device in S202 is described.

[0098] refer to Figure 3 As shown in the content, the process may include but is not limited to the following S301 to S303.

[0099] S301: The electronic device determines the current brightness of the first overlapping area as the initial brightness of the first overlapping area, and determines the current brightness of the target overlapping area as the reference brightness of the first overlapping area.

[0100] Initial brightness refers to the brightness of the overlapping area before the brightness is adjusted.

[0101] Reference brightness refers to the reference standard brightness value used to adjust the brightness of the overlapping area.

[0102] The embodiment of the present invention does not limit the specific values ​​of the initial brightness and the reference brightness of the first overlapping area, and they can be set according to actual conditions.

[0103] It can be understood that since the brightness of the first overlapping area needs to be adjusted based on the brightness of the target overlapping area, the current brightness of the first overlapping area can be determined as the initial brightness of the first overlapping area, and the current brightness of the target overlapping area can be determined as the reference brightness of the first overlapping area.

[0104] Exemplarily, S301 can be implemented as: the electronic device calculates the first average brightness of all pixels in the first overlapping area, and determines the first average brightness as the initial brightness of the first overlapping area; calculates the second average brightness of all pixels in the target overlapping area, and determines the second average brightness as the reference brightness of the first overlapping area.

[0105] S302: The electronic device determines the brightness adjustment factor raised to the power of the initial brightness as the brightness adjustment value of the first overlapping area; the brightness adjustment factor is used to represent the ratio of the initial brightness to the reference brightness.

[0106] The brightness adjustment factor refers to an adjustment parameter used to adjust the brightness of the overlapping area.

[0107] In the embodiment of the present invention, the brightness adjustment factor may be expressed as a ratio of the initial brightness to the reference brightness.

[0108] It should be noted that since the human eye's perception of brightness has nonlinear characteristics, in low-brightness areas, the human eye is relatively insensitive to brightness changes, while in high-brightness areas, the human eye's sensitivity to brightness changes increases. Therefore, this nonlinear characteristic can be described by an approximate power function.

[0109] Exemplarily, S302 can be implemented as follows: the electronic device calculates the ratio of the initial brightness to the reference brightness to obtain a brightness adjustment factor; calculates the value of a power function with the initial brightness as the base and the brightness adjustment factor as the exponent to obtain a brightness adjustment value of the first overlapping area.

[0110] S303: The electronic device adjusts the brightness of the first overlapping area to a brightness adjustment value.

[0111] Exemplarily, S303 may be implemented as follows: the electronic device traverses all pixel points in the first overlapping area; and for each pixel point, adjusts the brightness of the pixel point to a brightness adjustment value of the first overlapping area.

[0112] According to the above technical means, by determining the ratio of the initial brightness to the reference brightness as the brightness adjustment factor, the brightness adjustment value of the first overlapping area can be determined based on the initial brightness and the brightness adjustment factor. The embodiment of the present invention takes into account that the human eye's perception of brightness has nonlinear characteristics, and describes this nonlinear characteristic by using a power function with the initial brightness as the base and the brightness adjustment factor as the exponent, so as to more accurately simulate the difference in the human eye's perception of different brightness areas, so that when the brightness of the first overlapping area is adjusted, it can better adapt to the visual habits of the human eye, so that the adjusted image is more natural, comfortable and visually coherent.

[0113] The embodiment of the present invention may also include a brightness adjustment process for a non-overlapping area of ​​views.

[0114] refer to Figure 4 The process may include but is not limited to the following S401 and S402.

[0115] For each first overlapping region in the at least one first overlapping region included in the first view, the following process is performed:

[0116] S401: The electronic device determines a non-overlapping area and a reference overlapping area adjacent to the non-overlapping area in a first view.

[0117] The non-overlapping area refers to the area in each view that does not overlap with other views when the views are stitched.

[0118] The reference overlapping area refers to the overlapping area adjacent to the non-overlapping area in the view.

[0119] The embodiment of the present invention does not limit the size, shape, brightness, contrast, color saturation, quantity and other attribute parameters of the non-overlapping area and the reference overlapping area, and can be set according to actual conditions. For example, when the installation positions of the front, rear, left and right cameras of the electronic device remain unchanged, similar to the overlapping area, the non-overlapping area can be determined by calibration.

[0120] In practical applications, since there are two overlapping areas, front left and front right, in the front view, the non-overlapping area in the front view can be the middle area between the front left overlapping area and the front right overlapping area, and since the front left overlapping area and the front right overlapping area are adjacent to the non-overlapping area in the front view, the reference overlapping area can include the front left overlapping area and the front right overlapping area. Similarly, the left view, right view, and rear view are the same.

[0121] Exemplarily, S401 can be implemented as follows: the electronic device determines a non-overlapping area in the first view using a calibration method; for each non-overlapping area, based on a positional relationship between the non-overlapping area and each first overlapping area in at least one first overlapping area, a reference overlapping area adjacent to the non-overlapping area is determined in at least one first overlapping area.

[0122] S402: The electronic device adjusts the brightness of the non-overlapping area based on the brightness adjustment value of the reference overlapping area.

[0123] It can be understood that a non-overlapping area has at least two adjacent reference overlapping areas.

[0124] Exemplarily, S402 can be implemented as follows: the electronic device traverses all pixel points in the non-overlapping area; for each pixel point, based on the brightness adjustment value of the reference overlapping area, a brightness adjustment value of the pixel point is calculated by linear interpolation; and the brightness of the pixel point is adjusted to the brightness adjustment value.

[0125] According to the above technical means, considering the influence of the brightness difference between the non-overlapping area and the overlapping area on the overall brightness balance of the panoramic image, by determining the reference overlapping area adjacent to the non-overlapping area, the brightness of the non-overlapping area can be adjusted based on the brightness adjustment value of the reference overlapping area. In this way, the visual disharmony caused by the excessive brightness difference between the non-overlapping area and the overlapping area can be effectively avoided, and the brightness of the non-overlapping area is more coordinated and consistent with the brightness of the adjacent overlapping area, so as to achieve the overall brightness balance of the panoramic image and natural transition.

[0126] Next, the process of adjusting the brightness of the non-overlapping area by the electronic device based on the brightness adjustment value of the reference overlapping area in S402 is described.

[0127] refer to Figure 5 As shown in the content, the process may include but is not limited to the following S501 to S503.

[0128] S501: The electronic device determines the starting point coordinates and the ending point coordinates of a non-overlapping area.

[0129] The embodiments of the present invention do not limit the specific values ​​of the starting point coordinates and the ending point coordinates of the non-overlapping area, and they can be set according to actual conditions. For example, when the installation positions of the front, rear, left, and right cameras of the electronic device remain unchanged, similar to the overlapping area, the starting point coordinates and the ending point coordinates of the non-overlapping area can be determined by calibration.

[0130] In actual applications, when there are two reference overlapping areas in a non-overlapping area, since the non-overlapping area is adjacent to the reference overlapping area, the starting coordinates of the non-overlapping area can be the end coordinates of one of the reference overlapping areas, and the end coordinates of the non-overlapping area can be the starting coordinates of the other reference overlapping area.

[0131] For example, since the non-overlapping area in the front view may be the middle area between the front left overlapping area and the front right overlapping area, and the reference overlapping area in the front view may include the front left overlapping area and the front right overlapping area, then the starting coordinates of the non-overlapping area in the front view may be the ending coordinates of the front left overlapping area, and the ending coordinates of the non-overlapping area in the front view may be the starting coordinates of the front right overlapping area. Similarly, the left view, the right view, and the rear view are the same.

[0132] Exemplarily, S501 may be implemented as follows: the electronic device determines the starting point coordinates and the ending point coordinates of the non-overlapping area by a calibration method according to the installation positions of the front, rear, left, and right cameras.

[0133] S502: The electronic device performs the following processing for each pixel point in at least one pixel point included in the non-overlapping area: determining a pixel brightness adjustment value of the pixel point based on the current coordinates, starting point coordinates, end point coordinates, and the brightness adjustment value of the reference overlapping area of ​​the pixel point.

[0134] Exemplarily, S502 can be implemented as follows: the electronic device traverses at least one pixel point included in the non-overlapping area; for each pixel point, based on the current coordinates, starting point coordinates, and ending point coordinates of the pixel point, determines the first distance between the pixel point and the starting point coordinates, and the second distance between the pixel point and the ending point coordinates; based on the first distance, the second distance, and the brightness adjustment value of the reference overlapping area, determines the pixel brightness adjustment value of the pixel point.

[0135] For example, when there are two reference overlapping areas in the non-overlapping area, the pixel brightness adjustment value of the pixel point in the non-overlapping area can be determined by the following formula (1):

[0136] light=((light2-light1) / (border2-border1+1.0))×(coord-border1)+light1 Formula (1);

[0137] Among them, light is the pixel brightness adjustment value of the current pixel in the non-overlapping area, light1 and light2 are the brightness adjustment values ​​of the two reference overlapping areas of the non-overlapping area, border1 and border2 are the starting point coordinates and end point coordinates of the non-overlapping area, and coord is the current coordinates of the current pixel in the non-overlapping area.

[0138] S503: The electronic device adjusts the brightness of the pixel point to a pixel brightness adjustment value.

[0139] According to the above technical means, by determining the starting point coordinates and the end point coordinates of the non-overlapping area, it is possible to achieve the brightness adjustment value based on the starting point coordinates, the end point coordinates and the reference overlapping area, determine the pixel brightness adjustment value pixel by pixel in the non-overlapping area, and then achieve the brightness adjustment of all pixels in the non-overlapping area. The scheme of the present invention takes into account the position characteristics of different pixels in the non-overlapping area. By adjusting the brightness pixel by pixel in the non-overlapping area, it can avoid the unnatural feeling and local over-brightness or over-darkness problems caused by the uniform brightness adjustment of all pixels, and more accurately control the brightness adjustment of the non-overlapping area to ensure that the brightness transition between the non-overlapping area and the adjacent reference overlapping area is natural and coherent.

[0140] Next, the process of determining the first view to be adjusted in brightness and the reference view for adjusting the brightness of the first view in S103 by the electronic device based on the first brightness difference between the front and rear views and the second brightness difference between the left and right views is described.

[0141] refer to Figure 6 As shown in the content, the process may include but is not limited to the following S601 to S603.

[0142] S601. The electronic device determines a first brightness difference based on the brightness of the overlapping area in the front view and the brightness of the overlapping area in the rear view, and determines a second brightness difference based on the brightness of the overlapping area in the left view and the brightness of the overlapping area in the right view.

[0143] It can be understood that the brightness of the overlapping area in the front view includes the brightness of the front left overlapping area and the brightness of the front right overlapping area; the brightness of the overlapping area in the rear view includes the brightness of the rear left overlapping area and the brightness of the rear right overlapping area; the brightness of the overlapping area in the left view includes the brightness of the left front overlapping area and the brightness of the left rear overlapping area; the brightness of the overlapping area in the right view includes the brightness of the right front overlapping area and the brightness of the right rear overlapping area.

[0144] Exemplarily, S601 can be implemented as follows: the electronic device determines the brightness of the front view based on the brightness of the front left overlapping area and the front right overlapping area in the front view; determines the brightness of the rear view based on the brightness of the rear left overlapping area and the rear right overlapping area in the rear view; determines the brightness of the left view based on the brightness of the left front overlapping area and the left rear overlapping area in the left view; determines the brightness of the right view based on the brightness of the right front overlapping area and the right rear overlapping area in the right view; determines the difference between the brightness of the front view and the brightness of the rear view as a first brightness difference; and determines the difference between the brightness of the left view and the brightness of the right view as a second brightness value.

[0145] S602: If the first brightness difference is less than or equal to the second brightness difference, the electronic device determines the left view and the right view as first views, and determines the front view and the rear view as reference views.

[0146] It can be understood that the brightness difference of the reference view is smaller, and its brightness is relatively balanced, and can be used as a standard for adjusting brightness, while the brightness difference of the first view is larger, and its brightness is relatively more unbalanced, and can be used as a target for adjusting brightness.

[0147] Exemplarily, S602 may be implemented as follows: the electronic device compares the size relationship between the first brightness difference and the second brightness difference; if the first brightness difference is less than or equal to the second brightness difference, the left view and the right view are determined as the first view, and the front view and the rear view are determined as the reference view.

[0148] S603: If the first brightness difference is greater than the second brightness difference, the electronic device determines the front view and the rear view as first views, and determines the left view and the right view as reference views.

[0149] Exemplarily, S603 may be implemented as follows: the electronic device compares the first brightness difference with the second brightness difference; if the first brightness difference is greater than the second brightness difference, the front view and the rear view are determined as the first view, and the left view and the right view are determined as the reference view.

[0150] According to the above technical means, by determining the size relationship between the first brightness difference between the front and rear views and the second brightness difference between the left and right views, it is possible to determine the two views with the smallest brightness difference as the reference view, and the two views with the largest brightness difference as the first view. The scheme of the present invention takes into account the different orientation relationships between different views, and by comparing the brightness difference between the front and rear views and the brightness difference between the left and right views, it is possible to achieve more targeted brightness adjustment of the first view with reference to the reference view, which not only helps to quickly and efficiently achieve brightness balance between different views during the stitching process of panoramic images, but also can reduce additional calculation and adjustment processes, and improve the efficiency and quality of the panoramic image stitching process.

[0151] Next, the process of determining the first brightness difference in S601 by the electronic device based on the brightness of the overlapping area in the front view and the brightness of the overlapping area in the rear view is described.

[0152] In the case where the front view includes a front left overlap region and a front right overlap region, and the rear view includes a rear left overlap region and a rear right overlap region, reference Figure 7 As shown in the content, the process may include but is not limited to the following S701 to S703.

[0153] S701. The electronic device determines the average brightness of the front left overlapping area, the front right overlapping area, the rear left overlapping area, and the rear right overlapping area respectively, and obtains the front left average brightness, the front right average brightness, the rear left average brightness, and the rear right average brightness.

[0154] The embodiment of the present invention does not limit the specific manner in which the electronic device determines the average brightness of the front left overlapping area, the front right overlapping area, the rear left overlapping area, and the rear right overlapping area, which can be set according to actual conditions.

[0155] Exemplarily, S701 can be implemented as: the electronic device sums and averages the brightness of all pixels in the front left overlapping area, the front right overlapping area, the rear left overlapping area, and the rear right overlapping area, respectively, to obtain the average brightness of the front left overlapping area, the front right overlapping area, the rear left overlapping area, and the rear right overlapping area.

[0156] S702: The electronic device determines a first sum of the front left average brightness and the front right average brightness, and a second sum of the rear left average brightness and the rear right average brightness.

[0157] The first sum value refers to the sum of the front left average brightness and the front right average brightness.

[0158] The second sum value refers to the sum of the average brightness of the rear left and the average brightness of the rear right.

[0159] Exemplarily, S702 may be implemented as follows: the electronic device calculates the sum of the front left average brightness and the front right average brightness to obtain a first sum value; and calculates the sum of the rear left average brightness and the rear right average brightness to obtain a second sum value.

[0160] S703: The electronic device determines the difference between the first sum value and the second sum value as a first brightness difference.

[0161] Exemplarily, S703 may be implemented as follows: the electronic device calculates the difference between the first sum value and the second sum value to obtain a first brightness difference.

[0162] According to the above technical means, by determining the sum of the brightness of the front left overlapping area and the front right overlapping area, and the sum of the brightness of the rear left overlapping area and the rear right overlapping area, the difference between the two sums can be determined as the brightness difference between the front view and the rear view. In this way, since the overlapping relationship between the front view and the rear view and the left and right views is taken into account, the brightness difference between the front view and the rear view in the overlapping area can be more accurately measured, and the brightness difference obtained by comprehensive consideration of the brightness of the corresponding overlapping area is more representative and reliable.

[0163] Regarding the process in S601 where the electronic device determines the second brightness difference based on the brightness of the overlapping area in the left view and the brightness of the overlapping area in the right view, reference may be made to the detailed description of S701 to S703 above, which will not be repeated here.

[0164] The embodiment of the present invention may further include a process for determining the brightness of the overlapping area.

[0165] refer to Figure 8 As shown in the content, the process may include but is not limited to the following S801 to S803.

[0166] S801: The electronic device uses a graphics processor to bind the overlapping area as a texture image to an initial frame buffer object based on a target graphics rendering library to obtain a target frame buffer object.

[0167] The number of pixels P of the target frame buffer object is less than the number of pixels Q of the overlapping area; P and Q are both integers greater than 0.

[0168] The overlapping area may be any overlapping area.

[0169] The embodiment of the present invention does not limit the specific type of the target graphics rendering library, and it can be set according to actual conditions. Exemplarily, the target graphics rendering library can be an open graphics library (Open Graphics Library, OpenGL), and OpenGL runs on a GPU.

[0170] The initial frame buffer object refers to a frame buffer object that has been created but has not yet been bound to a texture image.

[0171] As you can understand, the initial framebuffer object has a size specified when it is created.

[0172] The target framebuffer object refers to the framebuffer object to which the texture image has been bound.

[0173] It should be noted that the size of the initial frame buffer object is consistent with that of the target frame buffer object, which is the size specified when they are created.

[0174] The embodiment of the present invention does not limit the specific values ​​of the number of pixels P of the target frame buffer object and the number of pixels Q of the overlapping area, which can be set according to actual conditions. For example, the number of pixels P of the target frame buffer object can be 50*50, and the number of pixels Q of the overlapping area can range from 5,000 to 30,000.

[0175] Exemplarily, S801 may be implemented as follows: the electronic device runs a target graphics rendering library through a graphics processor; the target graphics rendering library creates an initial frame buffer object and specifies a size; and the four vertex coordinates of the overlapping area are bound to the initial frame buffer object as texture coordinates to obtain a target frame buffer object.

[0176] S802: The electronic device extracts the brightness value of each pixel point among the P pixels included in the target frame buffer object through a graphics processor to obtain P brightness values.

[0177] Exemplarily, S802 can be implemented as follows: the electronic device traverses the P pixel points included in the target frame buffer object pixel by pixel through a graphics processor; for each pixel point, bilinear interpolation sampling is used to obtain the color of the pixel point according to the coordinates of the pixel point; the color of the pixel point is converted into a color space to obtain a brightness value of the pixel point; after the traversal is completed, P brightness values ​​are obtained.

[0178] For example, the color space conversion of the pixel point can be to convert the sampled RGB color value into the HSL color space so as to extract the brightness separately. In the HSL color space, the value of the L channel is the brightness. The brightness value of the pixel point can be determined by the following formula (2):

[0179] L = (max (R, G, B) + min (R, G, B)) / 2 formula (2);

[0180] Among them, L is the brightness value of the pixel, R is the red component of the pixel, G is the green component of the pixel, B is the blue component of the pixel, max is the maximum value, and min is the minimum value.

[0181] S803: The electronic device determines the brightness of the overlapping area based on the P brightness values ​​through a central processor.

[0182] Exemplarily, S803 may be implemented as follows: the electronic device obtains P brightness values ​​through a central processor; and determines the quotient of the sum of the P brightness values ​​and the number of valid pixels in the overlapping area as the brightness of the overlapping area.

[0183] According to the above technical means, the graphics processor binds the overlapping area with a large number of pixels as a texture image to the initial frame buffer object with a small number of pixels and then extracts the brightness value, and the central processing unit determines the brightness of the overlapping area. This can fully utilize the multi-core concurrent computing power of the graphics processor to effectively reduce the performance consumption of the central processing unit. In addition, since the number of pixels in the calculation process of the scheme of the present invention is less than the actual number of pixels included in the overlapping area, the time for determining each brightness adjustment value is faster and the performance consumption is less, which can better achieve the effect of real-time brightness adjustment and no jamming of the panoramic image.

[0184] The following describes a method for adjusting the brightness of a panoramic image provided by the present invention through a detailed embodiment.

[0185] The on-board panoramic image top view is composed of the front view, rear view, left view and right view captured by the four cameras in the front, rear, left and right directions of the vehicle. During the driving process of the vehicle, due to the differences in the ambient light conditions of each camera, there will inevitably be brightness differences between the images captured by the cameras. If not processed, the directly stitched panoramic top view will appear abrupt in brightness due to overexposure or dark light in some areas, and the picture will appear not harmonious and natural, which greatly affects the user's visual experience and overall perception.

[0186] In order to solve the problem of inconsistent brightness of the top view of the panoramic image, most of the existing technologies calculate the brightness adjustment value of the corresponding view based on the average brightness of the pixels in the overlapping area of ​​the images taken by two adjacent cameras. However, the actual number of pixels in the overlapping area usually ranges from 5,000 to 30,000. In the process of real-time brightness adjustment, the time required to calculate the brightness adjustment value for each frame of the image will exceed 50 milliseconds, and the current camera may have 30 or 50 frames of images per second, with one frame of image every 30 milliseconds or 20 milliseconds. Therefore, the technical solution in the prior art that calculates the brightness adjustment value based on the pixels in the overlapping area has a too long processing time for brightness adjustment. If the effect of real-time brightness correction for each frame of the image is to be achieved, the only option is to reduce the frame rate and slightly freeze the panoramic image, or use a single thread to calculate the brightness adjustment value, which in turn increases processor overhead and reduces performance.

[0187] Based on this, the present invention utilizes the bilinear interpolation capability of the Open Graphics Library (OpenGL) to upload the overlapping area as a texture image to a frame buffer object (FBO) of a specified size, and changes the number of pixels in the overlapping area from 5,000 to 30,000 to the specified FBO size. If we set the canvas size to 50x50, the number of pixels in the overlapping area becomes 2500. If there are 5,000 pixels in the overlapping area, the solution of the present invention reduces the number of pixels by 50%, and if there are 30,000 pixels in the overlapping area, the solution of the present invention reduces the number of pixels by 90%. Because OpenGL runs on a graphics processing unit (GPU), the present invention also uses the multi-core concurrent computing capability of the GPU to reduce the performance consumption of the central processing unit (CPU).

[0188] In the on-board panoramic image, each view captured by the camera has two overlapping areas. For example, the front view has two overlapping areas, the front left and the front right. Therefore, the four views of front, back, left and right have two overlapping areas. Figure 1There are eight overlapping areas, namely, front left, front right, rear left, rear right, left front, left rear, right front, and right rear. The overlapping areas in the present invention are obtained during vehicle calibration. As long as the installation positions of the front, rear, left, and right cameras remain unchanged, the overlapping areas taken by two adjacent cameras at the same time are always fixed, so the overlapping areas obtained during calibration can be directly reused.

[0189] See also Fig. 9 , the specific steps of the embodiment of the present invention include:

[0190] S901: The electronic device converts vertex coordinates of the overlapping area into texture coordinates.

[0191] Get the coordinates of the corresponding vertices of the eight overlapping areas obtained during vehicle calibration and convert them into corresponding texture coordinates. Each overlapping area has four vertex coordinates, and each view has two overlapping areas, that is, eight vertex coordinates. Pass the corresponding eight texture coordinates of the front, rear, left, and right views to OpenGL in turn, which is equivalent to passing them to the GPU.

[0192] S902: The electronic device processes the pixels of the overlapping area pixel by pixel through the GPU and calculates the brightness value.

[0193] Construct an FBO of a specified size. OpenGL binds this FBO, that is, the GPU binds this FBO, that is, this FBO binds the overlapping area represented by the texture coordinates passed in the previous step. The GPU then traverses this FBO pixel by pixel, and according to the pixel coordinates, bilinear interpolation samples are taken from the images captured by the front, back, left, and right cameras to obtain the specific color. The GPU then performs color space conversion and converts the sampled RGB color value to the HSL color space. This step is to extract the brightness separately. In the HSL color space, the value of the L channel is the brightness. Finally, the GPU converts the pixels in the overlapping area of ​​the FBO of the specified size into the brightness values ​​corresponding to the pixels in the overlapping area. To calculate the brightness L in HSL based on RGB, refer to the above formula (2).

[0194] As an example, the original overlap area of ​​a view can be seen in Fig.10 As shown, the overlapping area after specifying the FBO size can be seen in Fig.11 shown.

[0195] S903: The electronic device calculates the average brightness of the overlapping area according to the brightness values ​​of the pixels through the CPU.

[0196] After the GPU multi-core concurrent calculation is completed, the CPU reads the brightness values ​​of all pixels output on the FBO, and uses the brightness and / or number of effective pixels to calculate the average brightness of the eight overlapping areas: front left, front right, back left, back right, left front, left back, right front, and right back. The effective pixels in this step refer to pixels whose colors are not black. Taking RGB color as an example, the values ​​of the three channels R, G, and B are not equal to 0.

[0197] S904: The electronic device calculates a brightness adjustment value of the overlapping area through a CPU.

[0198] The CPU calculates the absolute value of the difference between (front left brightness + front right brightness) - (rear left brightness + rear right brightness), and regards it as the brightness difference between the front view and the rear view. The CPU calculates the absolute value of the difference between (left front brightness + left rear brightness) - (right front brightness + right rear brightness), and regards it as the brightness difference between the left view and the right view. Based on these two brightness differences, find the two views with the smallest brightness difference.

[0199] The CPU uses the brightness of the overlapping area of ​​the two views with the smallest brightness difference as the target brightness. If the brightness difference between the front view and the rear view is the smallest, align the left front brightness with the front left brightness, align the left rear brightness with the rear left brightness, align the right front brightness with the front right brightness, and align the right rear brightness with the rear right brightness, and calculate the brightness adjustment values ​​corresponding to the four overlapping areas of the left front, left rear, right front, and right rear. Taking the left front as an example, the brightness adjustment value is a power function with the left front brightness as the base and (left front brightness / front left brightness) as the exponent.

[0200] S905: The electronic device adjusts the brightness of the overlapping area according to the brightness adjustment value through the GPU.

[0201] The CPU passes the brightness adjustment value and the end coordinates of the first overlapping area and the start coordinates of the second overlapping area to the GPU. Taking the left view as an example, this step requires passing the end coordinates border1 of the left front area and the start coordinates border2 of the left back area, as well as the brightness adjustment values ​​light1 and light2 of the two areas.

[0202] The GPU accurately samples the color values ​​of all pixels on the left and right views pixel by pixel. The GPU first performs color space conversion and converts the sampled RGB color values ​​into the HSL color space. Then, based on the transmitted brightness adjustment value and the starting coordinates of the overlapping area, the brightness adjustment value of the pixel is calculated. Taking the left view as an example, for the pixels in the left front and left rear overlapping areas, the calculated brightness adjustment value corresponding to the overlapping area is used, while for the pixels outside the left front and left rear overlapping areas, the brightness adjustment value of the pixel is calculated linearly. Finally, the value of the L channel is adjusted separately according to the brightness adjustment value, and the adjusted HSL color space value is converted into the value of the RGB color space. Finally, a linear gradient brightness adjustment effect is achieved, making the overall brightness transition of the image smooth and natural.

[0203] The linear calculation of the brightness adjustment value can refer to the above formula (1).

[0204] Compared with the existing solutions, the present invention calculates fewer pixels, calculates the brightness adjustment value faster, consumes less performance, and can better achieve the effect of frame-by-frame real-time brightness correction without freezing the panoramic image.

[0205] In a second aspect, an embodiment of the present invention provides a device for adjusting the brightness of a panoramic image, such as Fig.12 As shown, the panoramic image brightness adjustment device 120 includes: an acquisition unit 1201, a determination unit 1202, an adjustment unit 1203, and a splicing unit 1204. Among them:

[0206] An acquisition unit 1201 is used to acquire front and rear views and left and right views for stitching a panoramic image;

[0207] A determining unit 1202, configured to determine a first view whose brightness is to be adjusted and a reference view for adjusting the brightness of the first view from among the front and rear views and the left and right views based on a first brightness difference between the front and rear views and a second brightness difference between the left and right views;

[0208] An adjusting unit 1203, configured to adjust the brightness of the overlapping area in the first view based on the brightness of the overlapping area in the reference view;

[0209] The stitching unit 1204 is configured to stitch the reference view and the first view after brightness adjustment to obtain a panoramic image.

[0210] In some embodiments, the adjustment unit 1203 is specifically used to: perform the following processing for each first overlapping area in at least one first overlapping area included in the first view: determine a target overlapping area that overlaps with the first overlapping area in at least one reference overlapping area included in the reference view; and adjust the brightness of the first overlapping area based on the brightness of the target overlapping area.

[0211] In some embodiments, the adjustment unit 1203 is also used to: determine the current brightness of the first overlapping area as the initial brightness of the first overlapping area, and determine the current brightness of the target overlapping area as the reference brightness of the first overlapping area; raise the initial brightness to the power of the brightness adjustment factor to determine the brightness adjustment value of the first overlapping area; the brightness adjustment factor is used to characterize the ratio of the initial brightness to the reference brightness; and adjust the brightness of the first overlapping area to the brightness adjustment value.

[0212] In some embodiments, the adjusting unit 1203 is further configured to: determine a non-overlapping area and a reference overlapping area adjacent to the non-overlapping area in the first view; and adjust the brightness of the non-overlapping area based on a brightness adjustment value of the reference overlapping area.

[0213] In some embodiments, the adjustment unit 1203 is also used to: determine the starting point coordinates and the ending point coordinates of the non-overlapping area; perform the following processing for each pixel point of at least one pixel point included in the non-overlapping area: determine the pixel brightness adjustment value of the pixel point based on the current coordinates of the pixel point, the starting point coordinates, the ending point coordinates, and the brightness adjustment value of the reference overlapping area; and adjust the brightness of the pixel point to the pixel brightness adjustment value.

[0214] In some embodiments, the determination unit 1202 is specifically used to: determine a first brightness difference based on the brightness of the overlapping area in the front view and the brightness of the overlapping area in the rear view, and determine a second brightness difference based on the brightness of the overlapping area in the left view and the brightness of the overlapping area in the right view; if the first brightness difference is less than or equal to the second brightness difference, determine the left view and the right view as the first view, and determine the front view and the rear view as the reference view; if the first brightness difference is greater than the second brightness difference, determine the front view and the rear view as the first view, and determine the left view and the right view as the reference view.

[0215] In some embodiments, when the front view includes a front left overlapping area and a front right overlapping area, and the rear view includes a rear left overlapping area and a rear right overlapping area, the determination unit 1202 is further used to: respectively determine the average brightness of the front left overlapping area, the front right overlapping area, the rear left overlapping area, and the rear right overlapping area to obtain the front left average brightness, the front right average brightness, the rear left average brightness, and the rear right average brightness; determine a first sum of the front left average brightness and the front right average brightness, and a second sum of the rear left average brightness and the rear right average brightness; and determine the difference between the first sum and the second sum as a first brightness difference.

[0216] In some embodiments, the brightness adjustment device 120 for a panoramic image also includes a processing unit, which is used to: bind the overlapping area as a texture image to an initial frame buffer object based on a target graphics drawing library through a graphics processor to obtain a target frame buffer object; the number of pixels P of the target frame buffer object is less than the number of pixels Q of the overlapping area; P and Q are both integers greater than 0; extract the brightness value of each pixel of the P pixels included in the target frame buffer object through a graphics processor to obtain P brightness values; determine the brightness of the overlapping area based on the P brightness values ​​through a central processing unit.

[0217] It should be noted that the various units included in the panoramic image brightness adjustment device provided in the embodiment of the present invention can be implemented by a processor in an electronic device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU, Central Processing Unit), a microprocessor (MPU, MicroProcessor Unit), a digital signal processor (DSP, Digital Signal Processor) or a field programmable gate array (FPGA, Field-Programmable Gate Array), etc.

[0218] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0219] It should be noted that, in the embodiment of the present invention, if the above-mentioned method for adjusting the brightness of a panoramic image is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the method for adjusting the brightness of a panoramic image described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present invention is not limited to any specific combination of hardware and software.

[0220] In a third aspect, an embodiment of the present invention provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the steps in the method for adjusting the brightness of a panoramic image provided in the above embodiment are implemented.

[0221] Combine the following Fig.13 The electronic device 130 shown is used to illustrate a structural diagram of the electronic device.

[0222] In one example, if Fig.13 As shown, the electronic device 130 includes: a processor 1301, at least one communication bus 1302, at least one external communication interface 1303 and a memory 1304. The communication bus 1302 is configured to realize the connection and communication between these components. The external communication interface 1303 may include a standard wired interface and a wireless interface.

[0223] The memory 1304 is configured to store instructions and applications executable by the processor 1301, and can also cache data to be processed or processed by the processor 1301 and various modules in the electronic device (for example, view data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0224] In another example, the electronic device may be a terminal device or a controller. The terminal device or the controller is used to execute the steps in the method for adjusting the brightness of a panoramic image provided in the above embodiment.

[0225] In a fourth aspect, the present invention further provides a storage medium having a computer program stored thereon. When the computer program on the storage medium is executed, the steps in the method for adjusting the brightness of a panoramic image provided in an embodiment of the present invention are implemented.

[0226] In a fifth aspect, the present invention further provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps in the above-mentioned method for adjusting the brightness of a panoramic image.

[0227] It should be noted that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of the present invention, please refer to the description of the method embodiments of the present invention for understanding.

[0228] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in some embodiments" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0229] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0230] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0231] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0232] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0233] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0234] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the brightness adjustment method of the panoramic image described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0235] The above is only an embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for adjusting the brightness of a panoramic image, characterized in that: The method comprises: Obtaining front and rear views and left and right views for stitching panoramic images; Determining, based on a first brightness difference between the front and rear views and a second brightness difference between the left and right views, a first view whose brightness is to be adjusted and a reference view for adjusting the brightness of the first view from among the front and rear views and the left and right views; adjusting the brightness of the overlapping area in the first view based on the brightness of the overlapping area in the reference view; The reference view is spliced ​​with the first view after brightness adjustment to obtain a panoramic image.

2. The brightness adjustment method according to claim 1, characterized in that: The adjusting the brightness of the overlapping area in the first view based on the brightness of the overlapping area in the reference view includes: For each of the at least one first overlapping area included in the first view, the following process is performed: Determine, in at least one reference overlapping region included in the reference view, a target overlapping region that overlaps with the first overlapping region; Based on the brightness of the target overlapping area, the brightness of the first overlapping area is adjusted.

3. The brightness adjustment method according to claim 2, characterized in that: The adjusting the brightness of the first overlapping area based on the brightness of the target overlapping area includes: Determining the current brightness of the first overlapping area as the initial brightness of the first overlapping area, and determining the current brightness of the target overlapping area as the reference brightness of the first overlapping area; Raising the initial brightness to the power of a brightness adjustment factor is determined as a brightness adjustment value of the first overlapping area; the brightness adjustment factor is used to represent a ratio of the initial brightness to the reference brightness; The brightness of the first overlapping area is adjusted to the brightness adjustment value.

4. The method according to claim 3, characterized in that The method further comprises: determining a non-overlapping area and a reference overlapping area adjacent to the non-overlapping area in the first view; The brightness of the non-overlapping area is adjusted based on the brightness adjustment value of the reference overlapping area.

5. The method according to claim 4, characterized in that The step of adjusting the brightness of the non-overlapping area based on the brightness adjustment value of the reference overlapping area includes: Determine the starting point coordinates and the ending point coordinates of the non-overlapping area; For each of the at least one pixel point included in the non-overlapping area, perform the following processing: Determine a pixel brightness adjustment value of the pixel point based on the current coordinates of the pixel point, the starting point coordinates, the end point coordinates, and the brightness adjustment value of the reference overlapping area; The brightness of the pixel point is adjusted to the pixel brightness adjustment value.

6. The method according to any one of claims 1 to 5, characterized in that: The determining, based on the first brightness difference between the front and rear views and the second brightness difference between the left and right views, a first view to be adjusted in brightness and a reference view for adjusting the brightness of the first view from among the front and rear views and the left and right views, comprises: Determine the first brightness difference based on the brightness of the overlapping area in the front view and the brightness of the overlapping area in the rear view, and determine the second brightness difference based on the brightness of the overlapping area in the left view and the brightness of the overlapping area in the right view; If the first brightness difference is less than or equal to the second brightness difference, the left view and the right view are determined as the first views, and the front view and the rear view are determined as the reference views; If the first brightness difference is greater than the second brightness difference, the front view and the rear view are determined as the first views, and the left view and the right view are determined as the reference views.

7. The method according to claim 6, characterized in that In a case where the front view includes a front left overlapping area and a front right overlapping area, and the rear view includes a rear left overlapping area and a rear right overlapping area, determining the first brightness difference based on the brightness of the overlapping area in the front view and the brightness of the overlapping area in the rear view includes: Respectively determine the average brightness of the front left overlapping area, the front right overlapping area, the rear left overlapping area, and the rear right overlapping area to obtain the front left average brightness, the front right average brightness, the rear left average brightness, and the rear right average brightness; Determine a first sum of the front left average brightness and the front right average brightness, and a second sum of the rear left average brightness and the rear right average brightness; The difference between the first sum value and the second sum value is determined as the first brightness difference.

8. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: The overlapping area is bound to the initial frame buffer object as a texture image by a graphics processor based on a target graphics rendering library to obtain a target frame buffer object; the number of pixels P of the target frame buffer object is less than the number of pixels Q of the overlapping area; and both P and Q are integers greater than 0; Extracting, by the graphics processor, the brightness value of each of the P pixels included in the target frame buffer object to obtain P brightness values; The brightness of the overlapping area is determined by a central processing unit based on the P brightness values.

9. A device for adjusting the brightness of a panoramic image, characterized in that: The device comprises: An acquisition unit, used for acquiring front and rear views and left and right views for stitching a panoramic image; a determining unit, configured to determine, based on a first brightness difference between the front and rear views and a second brightness difference between the left and right views, a first view whose brightness is to be adjusted and a reference view for adjusting the brightness of the first view from among the front and rear views and the left and right views; an adjusting unit, configured to adjust the brightness of the overlapping area in the first view based on the brightness of the overlapping area in the reference view; The stitching unit is used to stitch the reference view with the first view after brightness adjustment to obtain a panoramic image.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

Images