White balance processing method, device and equipment

Abstract

The invention relates to the field of vehicle-mounted image processing, in particular to a white balance processing method, device and equipment. The white balance processing method comprises the following steps: acquiring a target fisheye image stream; determining a target pixel point in the target fisheye image stream based on a preset fisheye image coordinate library according to the target coordinate of a first sampling point in the fisheye image coordinate library; determining a white balance adjustment coefficient through the target pixel point; and performing white balance processing on a target aerial view generated by splicing the target fisheye image flow according to the white balance adjustment coefficient. White balance calculation is divided into a preprocessing stage and a real-time adjustment stage, coordinate calculation of sampling points is completed in advance through the preprocessing stage, direct calling is carried out in the adjustment stage, the sequence of white balance coefficient calculation and aerial view generation is adjusted, the algorithm architecture is optimized, the overall time consumption of the image processing flow is reduced, and the image processing efficiency is improved. And the real-time performance of the vehicle-mounted panoramic image system is improved.

Description

Technical Field

[0001] This application relates to the field of vehicle image processing, and in particular to a white balance processing method, apparatus and device. Background Technology

[0002] In vehicular panoramic imaging systems, white balance algorithms are a key technology for ensuring color consistency after stitching together images captured by multiple vehicle cameras. Currently, common white balance algorithms mainly calculate the white balance of overlapping areas in the stitched bird's-eye view, using the color information of these overlapping areas to determine the adjustment coefficients of the panoramic view. This allows the system to adapt to changes in camera shooting angles and content to a certain extent, thereby generating the final panoramic view.

[0003] However, common algorithms have significant drawbacks: the calculation of the white balance adjustment coefficient is usually performed after the bird's-eye view is generated. On general-purpose hardware platforms, especially in resource-constrained scenarios such as in-vehicle environments, white balance calculation and single-view adjustment consume a large amount of system resources, resulting in a significant decrease in the real-time performance of the panoramic imaging system, making it difficult to meet the demand for real-time image updates during vehicle operation. Summary of the Invention

[0004] This invention provides a white balance processing method, apparatus, and device to solve the problem that white balance calculation in the prior art consumes a large amount of system resources, resulting in a significant decrease in the real-time performance of panoramic imaging systems.

[0005] In a first aspect, embodiments of the present invention provide a white balance processing method, the method comprising: Obtain the target fisheye image stream; Based on a preset fisheye image coordinate library, the target pixel is determined in the target fisheye image stream according to the target coordinates of the first sampling point in the fisheye image coordinate library. The white balance adjustment coefficient is determined using the target pixels; The target bird's-eye view generated by stitching together the target fisheye image stream is subjected to white balance processing according to the white balance adjustment coefficient.

[0006] Optionally, before acquiring the target fisheye image stream, the method further includes: A first bird's-eye view is generated by capturing a first fisheye image stream from an onboard camera, and a first sampling point is determined from the first bird's-eye view. The coordinates of the first sampling point are converted into target coordinates in the fisheye diagram coordinate system, and each of the first sampling points and the target coordinates corresponding to each of the first sampling points are stored in the fisheye diagram coordinate library.

[0007] Optionally, determining the first sampling point from the first bird's-eye view includes: When generating the first bird's-eye view by stitching together the first fisheye image streams, a sampling area is determined in the overlapping area of ​​two adjacent first fisheye images, and the sampling area is a closed regular shape; The first sampling point is selected in the sampling area based on a preset sampling point selection rule.

[0008] Optionally, selecting the first sampling point in the sampling area based on a preset sampling point selection rule includes: In the vertical direction of the sampling area, the first sampling point is preferentially selected from the direction closer to the vehicle body; In the horizontal direction of the sampling area, the first sampling point is preferentially selected from the direction closer to the vehicle body; Using the fusion line of the overlapping region as a reference, the first sampling point is selected preferentially from the direction closer to the fusion line.

[0009] Optionally, converting the coordinates of the first sampling point into target coordinates in the fisheye diagram coordinate system includes: By using the pseudo-inverse matrix of the perspective transformation matrix of the bird's-eye view, the coordinates of the first sampling point are transformed from the coordinates in the bird's-eye view coordinate system to the coordinates in the vehicle coordinate system; By using the camera's extrinsic parameter matrix, the coordinates of the first sampling point are converted from those in the vehicle coordinate system to those in the camera coordinate system. The first sampling point is converted from the coordinates in the camera coordinate system to the target coordinates in the fisheye image coordinate system using the fisheye projection model.

[0010] Optionally, the method further includes: Determine if the vehicle calibration parameters need to be updated; When it is determined that the vehicle calibration parameters need to be updated, the target coordinates of the first sampling point are updated to update the fisheye image coordinate library. When it is determined that there is no need to update the vehicle calibration parameters, the pre-generated fisheye coordinate library is reused.

[0011] Secondly, embodiments of the present invention provide a white balance processing device, the device comprising: The acquisition module acquires the target fisheye image stream; The first determining module, based on a preset fisheye coordinate library, determines the target pixel in the target fisheye image stream according to the target coordinates of the first sampling point in the fisheye coordinate library; The second determining module determines the white balance adjustment coefficient based on the target pixel. The processing module performs white balance processing on the target bird's-eye view generated by stitching together the target fisheye image stream according to the white balance adjustment coefficient.

[0012] Optionally, before acquiring the target fisheye image stream, the method further includes: A first bird's-eye view is generated by capturing a first fisheye image stream from an onboard camera, and a first sampling point is determined from the first bird's-eye view. The coordinates of the first sampling point are converted into target coordinates in the fisheye diagram coordinate system, and each of the first sampling points and the target coordinates corresponding to each of the first sampling points are stored in the fisheye diagram coordinate library.

[0013] Thirdly, embodiments of the present invention provide an electronic device, including: At least one processor; and At least one memory communicatively connected to the processor, wherein: The memory stores program instructions that can be executed by the processor, which can invoke the program instructions to perform the method as described in any of the first aspects.

[0014] Fourthly, embodiments of the present invention provide a storage medium including a stored program, wherein, when the program is executed, it controls the device where the storage medium is located to perform the method described in any of the first aspects.

[0015] In this embodiment of the invention, white balance adjustment coefficients are calculated based on a pre-set fisheye dynamic coordinate library, indexing target pixels using the library. After determining the white balance adjustment coefficients, white balance processing for the current frame image stream and the calculation of the white balance adjustment coefficients for the next frame image stream can be performed simultaneously. This ensures that when the white balance processing for the current frame image stream is complete, the white balance adjustment coefficients for the next frame image stream have already been calculated, allowing the calculated white balance adjustment coefficients to be directly used for subsequent white balance processing. This avoids consuming significant system resources for white balance processing, reduces the overall white balance processing time, and meets the need for real-time updates of in-vehicle images during vehicle operation. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 The diagram shown is a flowchart of a white balance processing method provided in an embodiment of this application; Figure 2 The image shown is a schematic diagram of a bird's-eye view provided in an embodiment of this application; Figure 3 The diagram shown is a structural schematic of a white balance processing device provided in an embodiment of this application; Figure 4 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0019] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0020] In vehicular panoramic imaging systems, white balance algorithms are a key technology for ensuring color consistency after stitching together images captured by multiple vehicle cameras. Currently, common white balance algorithms mainly calculate the white balance of overlapping areas in the stitched bird's-eye view, using the color information of these overlapping areas to determine the adjustment coefficients of the panoramic view. This allows the system to adapt to changes in camera shooting angles and content to a certain extent, thereby generating the final panoramic view.

[0021] However, common algorithms have significant drawbacks: the calculation of the white balance adjustment coefficient is usually performed after the bird's-eye view is generated. On general-purpose hardware platforms, especially in resource-constrained scenarios such as in-vehicle environments, white balance calculation and single-view adjustment consume a large amount of system resources, resulting in a significant decrease in the real-time performance of the panoramic imaging system, making it difficult to meet the demand for real-time image updates during vehicle operation.

[0022] Specifically, after the vehicle-mounted camera captures a fisheye image stream, it needs to perform image processing such as projection transformation and stitching to generate a bird's-eye view. Then, white balance adjustment coefficients are calculated based on the overlapping areas in the bird's-eye view. The calculated white balance adjustment coefficients are used to adjust the image, followed by stitching and fusion, and finally, a panoramic view is output, ending the processing of the current frame. After processing the current frame, the system continues to acquire the next fisheye image stream, repeating the steps of generating the bird's-eye view, calculating the white balance adjustment coefficients, and image adjustment.

[0023] The overall white balance processing needs to be performed linearly, which consumes a lot of system resources and causes a significant decrease in the real-time performance of the panoramic imaging system.

[0024] This invention provides a white balance processing method that places the calculation of the white balance adjustment coefficient after acquiring the image stream and before generating the bird's-eye view. Based on a pre-set fisheye dynamic coordinate library, the white balance adjustment coefficient is calculated by indexing the target pixel points through the coordinate library. This ensures that the coordinate points used to calculate the white balance adjustment coefficient each time match the current camera status, adapting to changes such as camera replacement, position shift, and recalibration, and does not affect the real-time performance of the panoramic imaging system in resource-constrained in-vehicle environments.

[0025] like Figure 1 The diagram shown is a flowchart of a white balance processing method provided in an embodiment of the present invention. See also... Figure 1 The steps of this method include: S101, Obtain the target fisheye image stream.

[0026] Specifically, a single frame of the target fisheye image stream is acquired. This single frame comprises fisheye images captured by multiple vehicle-mounted cameras. The acquired fisheye image stream is then stored in a buffer.

[0027] In one specific embodiment, the vehicle is equipped with four onboard cameras: front, rear, left, and right. A single frame of target fisheye image stream is acquired, including the front view, rear view, left view, and right view captured by the four onboard cameras, respectively.

[0028] S102, based on the preset fisheye image coordinate library, determine the target pixel in the target fisheye image stream according to the target coordinates of the first sampling point in the fisheye image coordinate library.

[0029] Specifically, the target coordinates are determined directly from the preset fisheye image coordinate library. Since the target coordinates are in the fisheye image coordinate system, the target pixel can be directly determined in the target fisheye image using the target coordinates.

[0030] S103 determines the white balance adjustment coefficient based on the target pixel.

[0031] Specifically, after calculating the white balance adjustment coefficients of the target fisheye image stream using the target pixels, the calculation of the white balance adjustment coefficients of the next frame fisheye image stream will be performed immediately, without waiting for the target fisheye images to be stitched together to generate the target bird's-eye view before the calculation is performed.

[0032] After completing S103, S104 is executed to perform white balance processing using the calculated white balance adjustment coefficients, and S101 of the next loop is executed simultaneously to calculate the white balance adjustment coefficients for the next frame of the fisheye image stream.

[0033] S104, perform white balance processing on the target bird's-eye view generated by stitching together the target fisheye image stream according to the white balance adjustment coefficient.

[0034] Specifically, the target fisheye image stream is processed by projection transformation, stitching and fusion to generate a target bird's-eye view, and the bird's-eye view is white-balanced using the white balance adjustment coefficient calculated by S103 to generate the final surround view displayed on the vehicle's infotainment system.

[0035] In this embodiment of the invention, white balance adjustment coefficients are calculated based on a pre-set fisheye dynamic coordinate library, indexing target pixels using the library. After determining the white balance adjustment coefficients, white balance processing for the current frame image stream and the calculation of the white balance adjustment coefficients for the next frame image stream can be performed simultaneously. This ensures that when the white balance processing for the current frame image stream is complete, the white balance adjustment coefficients for the next frame image stream have already been calculated, allowing the calculated white balance adjustment coefficients to be directly used for subsequent white balance processing. This avoids consuming significant system resources for white balance processing, reduces the overall white balance processing time, and meets the need for real-time updates of in-vehicle images during vehicle operation.

[0036] In this embodiment of the invention, before executing S101 to obtain the target fisheye image stream, it is necessary to establish a fisheye coordinate library in advance and store the target coordinates in the fisheye coordinate library.

[0037] Specifically, a first fisheye image stream is captured using an onboard camera, and this first fisheye image stream is stitched together to generate a first bird's-eye view. First sampling points are determined from the first bird's-eye view. The coordinates of the first sampling points are converted from the bird's-eye view coordinate system to target coordinates in the fisheye view coordinate system. Each determined first sampling point, along with its corresponding target coordinates, is stored in a fisheye view coordinate library.

[0038] In determining the first sampling point from the first bird's-eye view, it is necessary to first determine the sampling area, and then determine the first sampling point from the sampling area.

[0039] When generating the first bird's-eye view by stitching together the first fisheye images from the first fisheye image stream, a sampling area is determined in the overlapping region of two adjacent first fisheye images. When determining the sampling area, a closed regular graphic frame is used to select the sampling area, such as a rectangle or sector, to ensure the continuity of the sampling area. Furthermore, when determining the sampling area, the fusion line of the overlapping area must be used as a reference for selection, ensuring that the fusion line passes through the center point of the sampling area.

[0040] Optionally, the sampling area needs to occupy more than 40% of the overlapping area.

[0041] like Figure 2 The image shown is a schematic diagram of a first bird's-eye view provided in an embodiment of the present invention. See also... Figure 2The vehicle is equipped with four onboard cameras. The first fisheye image stream captured includes front view, rear view, left view, and right view. Figure 4 The first fisheye image is stitched together to generate the first bird's-eye view. During the stitching process, adjacent first fisheye images have overlapping areas. For example, in the left front view of the vehicle shown in the figure, there is an overlapping area between the left view and the front view. The images are then fused based on a fusion line as the central axis. A bounding box is then selected within the overlapping area.

[0042] After determining the sampling area, a first sampling point is selected within the sampling area based on preset sampling point selection rules. The sampling point selection rules are as follows: in the vertical direction of the sampling area, the first sampling point is selected preferentially from the direction closer to the vehicle body; in the horizontal direction of the sampling area, the first sampling point is selected preferentially from the direction closer to the vehicle body; and with the fusion line of the overlapping area as a reference, the first sampling point is selected preferentially from the direction closer to the fusion line.

[0043] In the vertical direction, the ground area closer to the vehicle body captures relatively stable content with less interference from the dynamic environment, thus more accurately reflecting the true color information of the overlapping area. Therefore, the first sampling point is selected from the direction closer to the vehicle body. In the horizontal direction, the side closer to the vehicle body is less affected by external factors, ensuring the stability and representativeness of the selected points. Therefore, the first sampling point is selected from the direction closer to the vehicle body. Using the blending line of the overlapping area as a reference, selecting points in all directions increases the weight of color information in the blended area, resulting in a better blending effect. Therefore, the first sampling point is selected from the direction closer to the blending line.

[0044] Optionally, downsampling can be performed in the sampling area to select points, which can reduce the amount of computation while preserving the actual color information of the vehicle camera.

[0045] This method of setting the sampling area and selecting the first sampling point ensures a balance between the effective use of color information and computational efficiency, and is applicable to most vehicle models with multi-camera configurations.

[0046] After determining the first sampling point, since the defined sampling area will not change, a fisheye diagram coordinate library is generated based on the determined first sampling point. The first sampling point is stored in the fisheye diagram coordinate library, and the coordinates of each first sampling point are transformed before being stored.

[0047] Specifically, the first sampling point is converted from the coordinates in the bird's-eye view coordinate system to the coordinates in the vehicle coordinate system by using the pseudo-inverse matrix of the perspective transformation matrix of the bird's-eye view.

[0048] The transformation from the bird's-eye view coordinate system to the vehicle coordinate system is achieved using the following formula: ; In the above formula, These are the coordinates of the first sampling point in the bird's-eye view coordinate system. It is the pseudo-inverse matrix of the perspective transformation matrix of the bird's-eye view. These are the coordinates of the first sampling point in the vehicle coordinate system. The general assumption is that the height relative to the ground is 0, i.e., taking... .

[0049] Then, by using the extrinsic parameter matrix of the camera, the coordinates of the first sampling point are converted from the vehicle coordinate system to the camera coordinate system.

[0050] The transformation from the vehicle coordinate system to the camera coordinate system is achieved using the following formula: ; In the above formula, It is the extrinsic parameter matrix from the vehicle to the camera. It is the coordinate of the first sampling point in the camera coordinate system.

[0051] Finally, using a fisheye projection model, the coordinates of the first sampling point are converted from those in the camera coordinate system to the target coordinates in the fisheye image coordinate system.

[0052] The fisheye projection model is as follows: ; In the above formula, It is the normalized distance from a pixel to the center of the image. It is the angle between the light source and the camera's main axis. It is the angle after distortion correction; It is the distortion coefficient, determined through calibration experiments, with a typical order of magnitude of [missing value]. , , , These are the camera's intrinsic parameters (i.e., focal length and optical center). It is the target coordinate of the first sampling point in the fisheye diagram coordinate system.

[0053] Based on this method, white balance calculation is divided into two parts: a preprocessing stage and a real-time adjustment stage. The coordinate calculation of the sampling points is completed in advance in the preprocessing stage and directly called in the adjustment stage. This optimizes the algorithm architecture, reduces the overall time consumption of the image processing process, and improves the real-time performance of the panoramic imaging system in the resource-constrained vehicle environment.

[0054] Optionally, in some embodiments, it is determined whether the vehicle calibration parameters need to be updated. Generally, the vehicle calibration parameters need to be updated after the vehicle's onboard camera is replaced or the onboard camera's position shifts.

[0055] When it is determined that the vehicle calibration parameters need to be updated, the target coordinates of the first sampling point are updated to update the fisheye coordinate library; when it is determined that the vehicle calibration parameters do not need to be updated, the pre-generated fisheye coordinate library is reused.

[0056] This method ensures that even if a camera is replaced, shifted, or its installation location changes, accurate target coordinates can be obtained again through the coordinate transformation process once new calibration parameters are acquired. This effectively avoids the problem of inaccurate coordinates in the overlapping area due to camera changes, which in turn leads to poor white balance. Inverse projection involves extensive computation; this mechanism only updates the coordinate library when calibration parameters change, without affecting overall system resource utilization.

[0057] Corresponding to the white balance processing method described above, this application also provides a white balance processing apparatus. See [link to previous document]. Figure 3 This is a schematic diagram of a white balance processing device provided in an embodiment of this application. The white balance processing device may include: an acquisition module 301, a first determination module 302, a second determination module 303, and a processing module 304.

[0058] Module 301 acquires the target fisheye image stream.

[0059] The first determining module 302 determines the target pixel in the target fisheye image stream based on a preset fisheye coordinate library and the target coordinates of the first sampling point in the fisheye coordinate library.

[0060] The second determining module 303 determines the white balance adjustment coefficient based on the target pixel.

[0061] The processing module 304 performs white balance processing on the target bird's-eye view generated by stitching together the target fisheye image stream according to the white balance adjustment coefficient.

[0062] Before acquiring the target fisheye image stream, the device needs to generate a first bird's-eye view from the first fisheye image stream captured by the vehicle-mounted camera, and determine the first sampling points from the first bird's-eye view. The coordinates of the first sampling points are converted into target coordinates in the fisheye coordinate system, and each of the first sampling points, along with the target coordinates corresponding to each of the first sampling points, is stored in the fisheye coordinate library.

[0063] Figure 4This is a schematic diagram illustrating the structure of one embodiment of the electronic device described in this specification. Specifically, the electronic device can be implemented as an image forming apparatus for performing the white balance processing method provided in this embodiment. Figure 4 As shown, the above-mentioned electronic device may include at least one processor; and at least one memory communicatively connected to the processing unit, wherein: the memory stores program instructions executable by the processing unit, and the processor can execute the white balance processing method provided in this embodiment by calling the program instructions.

[0064] The aforementioned electronic device can be a device capable of intelligent dialogue with the user, such as a cloud server. This specification does not limit the specific form of the electronic device in the embodiments. It is understood that the electronic device here refers to the machine mentioned in the method embodiments.

[0065] Figure 4 A block diagram of an exemplary electronic device suitable for implementing embodiments of this specification is shown. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments described in this specification.

[0066] like Figure 4 As shown, the electronic device is represented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: one or more processors 410, communication interface 420, memory 430, and communication bus 440 connecting different system components (including memory 430, communication interface 420 and processor 410).

[0067] Communication bus 440 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MAC) buses, Enhanced ISA buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses.

[0068] Electronic devices typically include a variety of computer-readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, and removable and non-removable media.

[0069] Memory 430 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. Memory 430 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments described herein.

[0070] A program / utility having a set (at least one) of program modules may be stored in memory 430. Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. The program modules typically perform the functions and / or methods described in the embodiments of this specification.

[0071] The processor 410 executes various functional applications and data processing by running programs stored in the memory 430, such as implementing the white balance processing method provided in the embodiments shown in this specification.

[0072] This specification provides a non-transitory computer-readable storage medium that stores computer instructions that cause the computer to execute the white balance processing method provided in the embodiments shown in this specification.

[0073] The aforementioned non-transitory computer-readable storage medium may be any combination of one or more computer-readable media. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in connection with an instruction execution system, apparatus, or device.

[0074] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device.

[0075] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0076] Computer program code for performing the operations described herein can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0077] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0078] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this specification, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0079] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this specification includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which the embodiments of this specification pertain.

[0080] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0081] It should be noted that the terminals involved in the embodiments of this specification may include, but are not limited to, personal computers (PCs), personal digital assistants (PDAs), wireless handheld devices, tablet computers, mobile phones, MP3 players, MP4 players, etc.

[0082] In the embodiments provided in this specification, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0083] Furthermore, the functional units in the various embodiments of this specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0084] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this specification.

[0085] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

1. A white balance processing method, characterized in that, The method includes: Obtain the target fisheye image stream; Based on a preset fisheye image coordinate library, the target pixel is determined in the target fisheye image stream according to the target coordinates of the first sampling point in the fisheye image coordinate library. The white balance adjustment coefficient is determined using the target pixels; The target bird's-eye view generated by stitching together the target fisheye image stream is subjected to white balance processing according to the white balance adjustment coefficient.

2. The method according to claim 1, characterized in that, Before acquiring the target fisheye image stream, the method further includes: A first bird's-eye view is generated by capturing a first fisheye image stream from an onboard camera, and a first sampling point is determined from the first bird's-eye view. The coordinates of the first sampling point are converted into target coordinates in the fisheye diagram coordinate system, and each of the first sampling points and the target coordinates corresponding to each of the first sampling points are stored in the fisheye diagram coordinate library.

3. The method according to claim 2, characterized in that, Determining the first sampling point from the first bird's-eye view includes: When generating the first bird's-eye view by stitching together the first fisheye image streams, a sampling area is determined in the overlapping area of ​​two adjacent first fisheye images, and the sampling area is a closed regular shape; The first sampling point is selected in the sampling area based on a preset sampling point selection rule.

4. The method according to claim 3, characterized in that, The step of selecting the first sampling point in the sampling area based on a preset sampling point selection rule includes: In the vertical direction of the sampling area, the first sampling point is preferentially selected from the direction closer to the vehicle body; In the horizontal direction of the sampling area, the first sampling point is preferentially selected from the direction closer to the vehicle body; Using the fusion line of the overlapping region as a reference, the first sampling point is selected preferentially from the direction closer to the fusion line.

5. The method according to claim 2, characterized in that, The step of converting the coordinates of the first sampling point into target coordinates in the fisheye coordinate system includes: By using the pseudo-inverse matrix of the perspective transformation matrix of the bird's-eye view, the coordinates of the first sampling point are transformed from the coordinates in the bird's-eye view coordinate system to the coordinates in the vehicle coordinate system; By using the camera's extrinsic parameter matrix, the coordinates of the first sampling point are converted from those in the vehicle coordinate system to those in the camera coordinate system. The first sampling point is converted from the coordinates in the camera coordinate system to the target coordinates in the fisheye image coordinate system using the fisheye projection model.

6. The method according to claim 2, characterized in that, The method further includes: Determine if the vehicle calibration parameters need to be updated; When it is determined that the vehicle calibration parameters need to be updated, the target coordinates of the first sampling point are updated to update the fisheye image coordinate library. When it is determined that there is no need to update the vehicle calibration parameters, the pre-generated fisheye coordinate library is reused.

7. A white balance processing device, characterized in that, The device includes: The acquisition module acquires the target fisheye image stream; The first determining module, based on a preset fisheye coordinate library, determines the target pixel in the target fisheye image stream according to the target coordinates of the first sampling point in the fisheye coordinate library; The second determining module determines the white balance adjustment coefficient based on the target pixel. The processing module performs white balance processing on the target bird's-eye view generated by stitching together the target fisheye image stream according to the white balance adjustment coefficient.

8. The apparatus according to claim 7, characterized in that, Before acquiring the target fisheye image stream, the process also includes: A first bird's-eye view is generated by capturing a first fisheye image stream from an onboard camera, and a first sampling point is determined from the first bird's-eye view. The coordinates of the first sampling point are converted into target coordinates in the fisheye diagram coordinate system, and each of the first sampling points and the target coordinates corresponding to each of the first sampling points are stored in the fisheye diagram coordinate library.

9. An electronic device, characterized in that, include: At least one processor; as well as At least one memory communicatively connected to the processor, wherein: The memory stores program instructions that can be executed by the processor, and the processor can execute the method as described in any one of claims 1 to 6 by calling the program instructions.

10. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the storage medium is located to perform the method according to any one of claims 1 to 6.

Images