A method, apparatus, and device for processing brightness and color after image stitching.

Abstract

This invention discloses a method, apparatus, and device for processing the brightness and color of images after image stitching. The method includes: acquiring a stitched image; extracting the stitched region from the stitched image, denoted as a first image; predicting the brightness and color of the first image; correcting the first image based on the prediction result to obtain a corrected image, denoted as a second image; and generating a processed image based on the second image. The processed image refers to the image after correcting the brightness and color of the first image in the stitched image. Through this method, the present invention can improve the realism of brightness and color correction for stitched images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method, apparatus, and device for processing the brightness and color of images after image stitching. Background Technology

[0002] Currently, the main process of panoramic camera imaging involves acquiring images from multiple perspectives using multiple image sensors, and then stitching these images together using image registration technology. However, because the stitched image was obtained from multiple image sensors, the brightness and color of the images obtained from each sensor differ.

[0003] In existing technologies, brightness and color correction of stitched images mainly relies on detecting the brightness of overlapping areas and then correcting based on the detection results. However, this method has significant limitations because existing image signal processing techniques are becoming increasingly complex. After adjustment, the parameters of the image output differ from those of the original data output by each image sensor, making it difficult to inversely recover the original brightness and color information of the image. This also means that the brightness and color adjustment parameters calculated from the overlapping areas using traditional methods are difficult to apply to the correction of brightness and color across the entire image. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a method, apparatus and device for processing the brightness and color of images after image stitching to overcome or at least partially solve the above problems.

[0005] According to one aspect of the present invention, a method for processing the brightness and color of images after image stitching is provided, comprising:

[0006] Obtain the stitched image;

[0007] Extract the image of the stitched region from the stitched image, and denote it as the first image;

[0008] Predict the brightness and color of the first image, and correct the first image based on the prediction results to obtain the corrected image, which is denoted as the second image;

[0009] Based on the second image, a processed image is generated, wherein the processed image refers to the image after correcting the brightness and color of the first image in the stitched image.

[0010] Optionally, obtain the stitched image, including:

[0011] At least two images to be synthesized are obtained, and the at least two images to be synthesized are correlated with each other;

[0012] The at least two images to be synthesized are stitched together to obtain the stitched image.

[0013] Optionally, after extracting the image of the stitched region from the stitched image, the method further includes:

[0014] Obtain the image of the unstitched area, denoted as the third image.

[0015] Optionally, based on the second image, generating the processed image includes:

[0016] The third image is combined with the second image to generate the processed image.

[0017] Optionally, after obtaining at least two images to be synthesized, the process may also include:

[0018] The at least two images to be synthesized are input into the trained neural network.

[0019] Optionally, the neural network is trained using the following method:

[0020] Acquire at least two training images that are correlated with each other;

[0021] The at least two training images are input into the neural network to be optimized, and the neural network to be optimized stitches the at least two training images together to obtain the stitched training image.

[0022] Based on the stitched training image, obtain the standard image of the stitched region;

[0023] The standard image is mapped onto the stitched training image to obtain the processed training image. The processed training image refers to the image after correcting the brightness and color of the stitched region image in the stitched training image. The processed training image is used to characterize the effectiveness of the neural network in correcting the brightness and color of the stitched region image.

[0024] The parameters of the neural network to be optimized are adjusted based on the processed training images to obtain the neural network.

[0025] According to another aspect of the present invention, an image stitching brightness and color processing apparatus is provided, comprising:

[0026] The acquisition module is used to acquire the stitched image;

[0027] The processing module is used to extract the image of the stitched region from the stitched image, denoted as the first image; predict the brightness and color of the first image; and correct the first image according to the prediction result to obtain the corrected image, denoted as the second image.

[0028] The output module is used to generate a processed image based on the second image, wherein the processed image refers to the image after correcting the brightness and color of the first image in the stitched image.

[0029] According to another aspect of the present invention, a computing device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus;

[0030] The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the above-described image stitching brightness and color processing method.

[0031] According to another aspect of the present invention, a computer storage medium is provided, the storage medium storing at least one executable instruction, the executable instruction causing a processor to perform operations corresponding to the above-described image stitching brightness and color processing method.

[0032] According to the solution provided in the above embodiments of the present invention, the following steps are taken: acquiring a stitched image; extracting the image of the stitched region from the stitched image, denoted as a first image; predicting the brightness and color of the first image; correcting the first image based on the prediction result to obtain a corrected image, denoted as a second image; and generating a processed image based on the second image. The processed image refers to the image after correcting the brightness and color of the first image in the stitched image, which can improve the realism of brightness and color correction of the stitched image.

[0033] The above description is merely an overview of the technical solutions of the embodiments of the present invention. In order to better understand the technical means of the embodiments of the present invention and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of the present invention more obvious and understandable, specific implementation methods of the embodiments of the present invention are described below. Attached Figure Description

[0034] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0035] Figure 1 A flowchart illustrating the image stitching brightness and color processing method provided in an embodiment of the present invention is shown.

[0036] Figure 2 This diagram illustrates a specific panoramic camera provided by an embodiment of the present invention.

[0037] Figure 3 This illustration shows a schematic diagram of a panoramic camera used in a specific outdoor tourism scenario, according to an embodiment of the present invention.

[0038] Figure 4 This illustration shows a schematic diagram of a panoramic camera used in another specific outdoor tourism scenario, as provided in an embodiment of the present invention.

[0039] Figure 5 A flowchart of the neural network training method provided in an embodiment of the present invention is shown;

[0040] Figure 6 This illustration shows two images acquired by different sensors in a specific panoramic camera according to an embodiment of the present invention.

[0041] Figure 7 This diagram illustrates a specific method for obtaining a standard image of the stitched area in a stitched graphic, according to an embodiment of the present invention.

[0042] Figure 8 A schematic diagram of a neural network training method provided by an embodiment of the present invention is shown;

[0043] Figure 9 This invention provides a specific schematic diagram of a gimbal angle for acquiring training input images, according to an embodiment of the present invention.

[0044] Figure 10 This illustration shows a specific gimbal angle diagram for acquiring training output images according to an embodiment of the present invention;

[0045] Figure 11 A schematic diagram of a camera with a specific hardware configuration provided in an embodiment of the present invention is shown;

[0046] Figure 12 A schematic diagram of the structure of the image stitching brightness and color processing device provided in an embodiment of the present invention is shown;

[0047] Figure 13 A schematic diagram of the structure of a computing device provided in an embodiment of the present invention is shown. Detailed Implementation

[0048] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0049] Figure 1 A flowchart illustrating the method for processing brightness and color after image stitching according to an embodiment of the present invention is shown. Figure 1 As shown, the method includes the following steps:

[0050] Step 11: Obtain the stitched image;

[0051] Step 12: Extract the image of the stitched region from the stitched image and denote it as the first image;

[0052] Step 13: Predict the brightness and color of the first image, and correct the first image according to the prediction results to obtain the corrected image, which is recorded as the second image;

[0053] Step 14: Generate a processed image based on the second image. The processed image refers to the image after correcting the brightness and color of the first image in the stitched image.

[0054] In this embodiment, the process involves acquiring a stitched image; extracting the stitched region from the stitched image and designating it as a first image; predicting the brightness and color of the first image; correcting the first image based on the prediction results to obtain a corrected image, designated as a second image; and generating a processed image based on the second image. The processed image refers to the image after correcting the brightness and color of the first image in the stitched image, which can improve the realism of brightness and color correction in the stitched image.

[0055] In an optional embodiment of the present invention, step 11 may include:

[0056] Step 111: Obtain at least two images to be synthesized, wherein the at least two images to be synthesized are correlated;

[0057] Step 112: The at least two images to be synthesized are stitched together to obtain the stitched image.

[0058] In this embodiment, the at least two images to be synthesized are correlated; for example, they can be images obtained from different sensors under the same panoramic camera, but are not limited to those described above. The at least two images to be synthesized can be stitched together using image registration or image stitching techniques, but are not limited to those described above.

[0059] Figure 2 The illustration shows a specific panoramic camera according to an embodiment of the present invention. This camera has multiple image sensors, and the images acquired by the different sensors can be correlated. For example... Figure 2As shown, the panoramic camera includes a bottom pillar and a top panoramic camera. The bottom pillar houses a battery module, a control module, and a communication module; the top includes multiple lens groups and corresponding image sensors and image signal processors (ISPs). The control module controls the camera's startup, shooting, and image processing. Specifically, the control module can individually or simultaneously control each lens group and its corresponding image sensor and image signal processor.

[0060] Figure 3 This illustration shows a schematic diagram of a panoramic camera used in a specific outdoor tourism scenario, as provided by an embodiment of the present invention. Figure 3 As shown, sensor 1, sensor 2, sensor 3 and sensor 4 are all sensors under the same panoramic camera. Since the field of view of sensor 2 is facing the sun, the image obtained will inevitably have serious brightness and color deviations compared with the images obtained by other sensors under the same panoramic camera. Therefore, the stitched image will appear unrealistic and the brightness and color of the stitched image need to be corrected.

[0061] Figure 4 This illustration shows a practical schematic diagram of a panoramic camera used in another specific outdoor tourism scenario provided by an embodiment of the present invention, such as... Figure 4 As shown, the panoramic camera contains four lenses. Lens 1 faces the direction in which the tourist is standing and captures an image in the direction of the tourist. Lenses 2, 3, and 4 capture the scenery. If the images captured by lenses 1, 2, 3, and 4 are to be stitched together into a relatively realistic panoramic image, the brightness and color of the stitched image need to be corrected.

[0062] In yet another optional embodiment of the invention, step 12 may be followed by:

[0063] Step 121: Obtain the image of the unstitched area, denoted as the third image.

[0064] In yet another optional embodiment of the invention, step 14 may include:

[0065] Step 141: Combine the third image with the second image to generate the processed image.

[0066] In this embodiment, the second image differs from the first image in that its brightness and color may be different.

[0067] In yet another optional embodiment of the invention, step 11 may be followed by:

[0068] Step 113: Input the at least two images to be synthesized into the trained neural network.

[0069] In this embodiment, the at least two images to be synthesized are input into the trained neural network, which can directly output a stitched image after brightness and color correction, thus achieving the beneficial effects of automated stitching and brightness and color correction of the stitched image.

[0070] Figure 5 A flowchart of a neural network training method provided in an embodiment of the present invention is shown. Figure 5 As shown, the method includes the following steps:

[0071] Step 51: Obtain at least two training images, wherein the at least two training images are correlated;

[0072] Step 52: Input the at least two training images into the neural network to be optimized, and the neural network to be optimized stitches the at least two training images together to obtain the stitched training image.

[0073] Step 53: Obtain the standard image of the stitched region based on the stitched training image;

[0074] Step 54: Map the standard image onto the stitched training image to obtain the processed training image. The processed training image refers to the image after correcting the brightness and color of the stitched region image in the stitched training image. The processed training image is used to characterize the effectiveness of the neural network in correcting the brightness and color of the stitched region image.

[0075] Step 55: Adjust the parameters of the neural network to be optimized based on the processed training image to obtain the neural network.

[0076] In this embodiment, in step 51, the at least two training images are correlated. This correlation includes the presence of the same scene, images obtained from two different sensors at the same time and location, or images of the same scene obtained from the same sensor at different times, but is not limited to those described above. For example… Figure 6 The two images, Image 1 and Image 2, obtained from different sensors in a specific panoramic camera, are correlated. Because Image 1 and Image 2 were obtained from different sensors, even if the overlapping area is the same region, there may be unevenness in brightness and color. This is because each image sensor only optimizes its own overall imaging effect and cannot be aware of the imaging parameters of neighboring sensors.

[0077] In step 52, for example in Figure 6In this process, image registration or image stitching techniques can be used to stitch together images to obtain a stitched image. However, it is not limited to the above. When stitching, even if image 1 and image 2 do not have overlapping areas, they can still be stitched together. The stitched image is not limited to the two images having overlapping parts.

[0078] In step 53, a separate image sensor is used to image the area containing the overlapping region. Since a separate image sensor can optimize the overall brightness and color within its field of view, the image produced by this sensor in the overlapping region can be considered natural, resulting in a standard image that includes the stitched region of the two training images from step 52. For example... Figure 7 As shown, image X was acquired separately by a single image sensor. Figure 6 The image is the stitched region between image 1 and image 2. Therefore, the brightness and color distribution of image X can be considered the optimal brightness and color distribution for that region in the synthesized panoramic image. Its specific brightness and color values ​​are not equal to the linear average of the edge regions, but rather correspond to the semantic targets within the image.

[0079] In step 54, for example, when it is confirmed that image X is the standard image of the stitching area between image 1 and image 2, image X is mapped to the stitching area of ​​the stitched image through image registration. Parameters such as resolution and image size are automatically corrected, but not limited to resolution and image size. This results in a stitched image after brightness and color correction, and an image of image X within the mapped area of ​​the stitched image. When correcting parameters, the position of image X in the stitching area can be corrected, or only some pixels in the image can be corrected. Then, the image is stitched together with the stitched image to finally obtain the synthesized panoramic image, i.e., the processed training image. This processed image has a natural brightness and color distribution, especially within the stitching area, where there are no longer abrupt changes in brightness and color, and the brightness and color distribution matches the semantic features within the panoramic image.

[0080] In step 55, the network parameters can be automatically optimized using labeled data, and the network learns from the distribution of special images using a large amount of training data, thus gaining optimization capabilities to adapt to various image features. In an embodiment of the present invention, a deep neural network, such as a deep convolutional neural network (DCNN), is used to process the corresponding region of the stitched image to obtain a target image with overall optimized brightness and color in the corresponding region. During image registration and stitching, if there is an overlapping region between two images, a region image is cut out from the stitched image. This image includes the overlapping region, and the cut-out image needs to be much larger than the overlapping region. This cut-out image serves as the image to be adjusted for brightness and color, is input into the deep neural network, and an output image is obtained. The output image serves as the image after brightness and color adjustment. At this point, the brightness and color transitions within the overlapping region are processed by the deep neural network, completing the overall adjustment of brightness and color distribution, resulting in an automatic brightness and color correction stitching network.

[0081] Corresponding to Figure 5 The process of neural network training methods, Figure 8 This diagram illustrates a neural network training method according to an embodiment of the present invention. This neural network is primarily used for global adjustment of images exhibiting abrupt changes in brightness or color. Its adjustment performance depends on the parameters of different modules within the deep neural network. Figure 8 The neural network shown can be a U-Net or a fully convolutional network (FCN), in which case it directly outputs a high-resolution image with corrected brightness and color. Figure 8 The neural network in this example only outputs correction parameters; in this case, the neural network is a regression network. It then performs brightness and color correction on the stitched image based on these correction parameters.

[0082] The performance of deep neural networks in brightness and color adjustment depends on the high-quality data used to generate the network. Theoretically, the target image output by the neural network is a high-quality target image with optimized brightness and color. Therefore, the training process of this neural network requires paired two-dimensional image training data (x... i ,y i ), where x i For training, the input image data, y i The output image data is used for training. In existing panoramic camera systems, x i It is easy to obtain because x i It is an intermediate product in the panoramic imaging process. It only requires outputting a predefined-sized image containing overlapping areas from an existing panoramic imaging process in the cloud or on the camera. The cloud-based background program has control over the camera, thus easily obtaining massive amounts of image data. Conversely, y i It is relatively difficult to obtain because y iIt is a target image with high-quality brightness and color distribution from a virtual perspective. Therefore, in the embodiments of the present invention, different implementation methods are proposed to obtain y i data.

[0083] like Figure 9-10 As shown, obtain y i The first way to obtain data is by using a camera with feature settings to obtain, for example... Figure 7 The image shown is of the region corresponding to image X. Specifically, using a camera with a gimbal, the angle of the gimbal is adjusted to obtain two sets of corresponding images. One set is used as the traditional data, and the other set of images is obtained by rotating the gimbal angle 45 degrees horizontally, which is used as the y-axis image. i . Figure 9 Combination Figure 10 This is a schematic diagram of the method shown, where the training input image data x is obtained at a gimbal angle of 1. i Adjust the gimbal angle to, for example Figure 10 The gimbal angle shown is 2, and the output image data y for training is obtained. i .

[0084] like Figure 11 As shown, obtain y i The second method of data collection involves using a camera with specialized hardware, deployed in tourist areas with direct power supply. Specifically, this camera is equipped with a second set of panoramic lenses arranged in a cross configuration. Figure 11 In the diagram, lenses 11, 12, 13, and 14 are panoramic camera lenses located at the lower level; lenses 21, 22, 23, and 24 are panoramic camera lenses located at the higher level, with a 45-degree difference in lens normal angle. Specifically, both layers of cameras acquire images simultaneously, with the lower-level camera used to acquire the x-axis. i Data, while high-level cameras are used to obtain y i Data. Compared to the first method, this method can acquire two sets of data simultaneously, but the image must remain unchanged during the movement of the gimbal, so it can only be used in unmanned scenarios.

[0085] Get y i The third way to obtain the data is by manually selecting y. i Data. Specifically, images with good stitching effects in different scenarios are manually selected, and the overlapping areas are extracted. Brightness and color are adjusted manually as needed to obtain y. i Data. Compared to camera-based methods, manually generating training data is less efficient. However, manually refined data has a brightness and color distribution that better matches human vision. Furthermore, manual correction can also correct targets in non-overlapping areas, such as locally correcting overexposure caused by the sun.

[0086] The above three methods of obtaining y i Data can also be used in combination to obtain a large amount of high-quality training data. Because... Figure 8 The neural network in this system is only used for brightness and color adjustment, so there is no problem with generalization. As long as a sufficient amount of training data is obtained once, the neural network can be repeatedly used in the cloud to correct each image.

[0087] In the above embodiments of the present invention, the following steps are taken: acquiring a stitched image; extracting the image of the stitched region from the stitched image, denoted as a first image; predicting the brightness and color of the first image; correcting the first image based on the prediction result to obtain a corrected image, denoted as a second image; and generating a processed image based on the second image. The processed image refers to the image after correcting the brightness and color of the first image in the stitched image. This can improve the realism of brightness and color correction of the stitched image, and can also apply the image stitching brightness and color processing method to an image sensor to achieve automatic adjustment of the brightness and color of the stitched image.

[0088] Figure 12 A schematic diagram of the image stitching brightness and color processing device 120 provided in an embodiment of the present invention is shown. Figure 12 As shown, the device includes:

[0089] Module 121 is used to acquire the stitched image;

[0090] Processing module 122 is used to extract the image of the stitched region in the stitched image, denoted as the first image; predict the brightness and color of the first image; and correct the first image according to the prediction result to obtain the corrected image, denoted as the second image.

[0091] The output module 123 is used to generate a processed image based on the second image, wherein the processed image refers to the image after correcting the brightness and color of the first image in the stitched image.

[0092] Optionally, the acquisition module 121 is further configured to acquire at least two images to be synthesized, wherein the at least two images to be synthesized are correlated; and to stitch the at least two images to be synthesized together to obtain the stitched image.

[0093] Optionally, the processing module 122 is further configured to obtain an image of the unstitched area, referred to as the third image.

[0094] Optionally, the output module 123 is further configured to combine the third image with the second image to generate a processed image.

[0095] Optionally, the processing module 122 is further configured to input the at least two images to be synthesized into the trained neural network.

[0096] Optionally, the processing module 122 is further configured to acquire at least two training images, wherein the at least two training images are correlated;

[0097] The at least two training images are input into the neural network to be optimized, and the neural network to be optimized stitches the at least two training images together to obtain the stitched training image.

[0098] Based on the stitched training image, obtain the standard image of the stitched region;

[0099] The standard image is mapped onto the stitched training image to obtain the processed training image. The processed training image refers to the image after correcting the brightness and color of the stitched region image in the stitched training image. The processed training image is used to characterize the effectiveness of the neural network in correcting the brightness and color of the stitched region image.

[0100] The parameters of the neural network to be optimized are adjusted based on the processed training images to obtain the neural network.

[0101] It should be noted that this embodiment is a device embodiment corresponding to the above method embodiment. All implementation methods in the above method embodiment are applicable to this device embodiment and can achieve the same technical effect.

[0102] This invention provides a non-volatile computer storage medium storing at least one executable instruction that can execute the image stitching brightness and color processing method described in any of the above method embodiments.

[0103] Figure 13 The diagram shows a structural schematic of a computing device provided in an embodiment of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the computing device.

[0104] like Figure 13 As shown, the computing device may include a processor, a communications interface, memory, and a communications bus.

[0105] The processor, communication interface, and memory communicate with each other via a communication bus. The communication interface is used to communicate with other network elements, such as clients or other servers. The processor executes programs, specifically the steps described in the embodiment of the method for processing the brightness and color of images after image stitching.

[0106] Specifically, the program may include program code, which includes computer operation instructions.

[0107] The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The computing device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0108] Memory is used to store programs. Memory may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device.

[0109] Specifically, the program can be used to cause the processor to execute the image stitching brightness and color processing method in any of the above method embodiments. The specific implementation of each step in the program can be found in the corresponding steps and units described in the above-described image stitching brightness and color processing method embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0110] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of the present invention are not directed to any particular programming language. It should be understood that the embodiments of the present invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the embodiments of the present invention.

[0111] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0112] Similarly, it should be understood that, in order to streamline the embodiments of the invention and aid in understanding one or more of the various inventive aspects, features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed embodiments of the invention require more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0113] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0114] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0115] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The embodiments of the present invention can also be implemented as device or apparatus programs (e.g., computer programs and computer program products) for performing part or all of the methods described herein. Such programs implementing the embodiments of the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0116] It should be noted that the above embodiments are illustrative of the present invention and not restrictive of the invention, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. Embodiments of the present invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A method for processing brightness and color after image stitching, characterized in that, The method is applied to a panoramic camera, which includes four sensors, one of which has its field of view facing the sun, and four imaging lenses. The method includes: The stitched image includes: Using a panoramic camera with different sensors, at least two images to be synthesized are obtained, and the at least two images to be synthesized are correlated. The at least two images to be synthesized are stitched together to obtain the stitched image; Extract the image of the stitched region from the stitched image, and denote it as the first image; Predict the brightness and color of the first image, and correct the first image based on the prediction results to obtain the corrected image, which is denoted as the second image; Based on the second image, a processed image is generated. The processed image refers to the image after correcting the brightness and color of the first image in the stitched image. After obtaining at least two images to be synthesized, the process also includes: The at least two images to be synthesized are input into the trained neural network. The neural network was trained using the following method: Acquire at least two training images that are correlated with each other; The at least two training images are input into the neural network to be optimized, and the neural network to be optimized stitches the at least two training images together to obtain the stitched training image. Based on the stitched training image, obtain the standard image of the stitched region; The standard image is mapped onto the stitched training image to obtain the processed training image. The processed training image refers to the image after correcting the brightness and color of the stitched region image in the stitched training image. The processed training image is used to characterize the effectiveness of the neural network in correcting the brightness and color of the stitched region image. The parameters of the neural network to be optimized are adjusted based on the processed training images to obtain the neural network.

2. The method for processing brightness and color after image stitching according to claim 1, characterized in that, After extracting the image of the stitched region from the stitched image, the process further includes: Obtain the image of the unstitched area, denoted as the third image.

3. The method for processing brightness and color after image stitching according to claim 2, characterized in that, Based on the second image, a processed image is generated, including: The third image is combined with the second image to generate the processed image.

4. An apparatus for processing the brightness and color of images after stitching, characterized in that, The device includes: The acquisition module, used to acquire the stitched image, includes: Using a panoramic camera with different sensors, at least two images to be synthesized are obtained, and the at least two images to be synthesized are correlated. The at least two images to be synthesized are stitched together to obtain the stitched image; The processing module is used to extract the image of the stitched region from the stitched image, denoted as the first image; predict the brightness and color of the first image; and correct the first image according to the prediction result to obtain the corrected image, denoted as the second image. The output module is used to generate a processed image based on the second image. The processed image refers to the image after correcting the brightness and color of the first image in the stitched image. After obtaining at least two images to be synthesized, the process also includes: The at least two images to be synthesized are input into the trained neural network. The neural network was trained using the following method: Acquire at least two training images that are correlated with each other; The at least two training images are input into the neural network to be optimized, and the neural network to be optimized stitches the at least two training images together to obtain the stitched training image. Based on the stitched training image, obtain the standard image of the stitched region; The standard image is mapped onto the stitched training image to obtain the processed training image. The processed training image refers to the image after correcting the brightness and color of the stitched region image in the stitched training image. The processed training image is used to characterize the effectiveness of the neural network in correcting the brightness and color of the stitched region image. The parameters of the neural network to be optimized are adjusted based on the processed training images to obtain the neural network.

5. A computing device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, which, when executed, causes the processor to perform the image stitching brightness and color processing method as described in any one of claims 1-3.

6. A computer storage medium storing at least one executable instruction, wherein the executable instruction, when executed, causes a computing device to perform a method for processing the brightness and color of an image after stitching as described in any one of claims 1-3.

Images