Image generation method and device, electronic equipment and storage medium

Abstract

The invention discloses an image generation method and device, electronic equipment and a storage medium, and belongs to the technical field of image processing. The method comprises the steps that a panoramic image is acquired, and the panoramic image is obtained by synthesizing original images which are acquired by a plurality of acquisition devices in a surrounding mode at the same moment; the panoramic image comprises at least one target object; separating the panoramic image into a background image and an object area image corresponding to each target object; decomposing the background image into a plurality of intermediate background images respectively corresponding to the plurality of acquisition devices; performing equalization processing on the plurality of intermediate background images to obtain an equalized background image; and fusing the balanced background image and the object area image to generate a result image.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to an image generation method and device, electronic equipment and a storage medium. BACKGROUND

[0002] With the continuous improvement of the intelligent level of automobiles, a panoramic surround view system has become a standard configuration of many vehicle models. The system collects images through fisheye cameras installed around the vehicle, and synthesizes a panoramic picture with a bird's eye view in real time, which greatly eliminates the visual blind area and provides key visual assistance for drivers in complex scenes such as low-speed parking and narrow road passing, and has important significance for improving driving safety and convenience.

[0003] At present, the surround view splicing technology of the panoramic surround view system (AVM) has significant limitations in actual application. On the one hand, motion blur caused by vehicle and pedestrian movement may be misjudged as low-quality images and filtered, resulting in the absence of key dynamic targets in the surround view picture and introducing safety hazards. On the other hand, it is difficult to cope with scenes where the light difference between cameras is huge, and it is easy to produce uneven brightness of the spliced picture, affecting the visual experience. Moreover, a simple screening mechanism will destroy the continuity of the video stream, resulting in frame loss or jumping of the spliced output, affecting the real-time judgment of the driver.

[0004] Therefore, the current surround view splicing technology cannot balance real-time and safety. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide an image generation method and device, electronic equipment and a storage medium, which can solve the problem that the AVM surround view splicing technology cannot balance real-time and safety.

[0006] In a first aspect, the embodiments of the present application provide an image generation method, which comprises: obtaining a panoramic image, wherein the panoramic image is synthesized from original images of the same moment collected by multiple collection devices; and the panoramic image includes at least one target object; separating the panoramic image into a background image and an object region image corresponding to each target object; decomposing the background image into multiple intermediate background images corresponding to the multiple collection devices, respectively; performing equalization processing on the multiple intermediate background images to obtain an equalized background image; fusing the equalized background image and the object region image to generate a result image.

[0007] In a second aspect, the embodiments of the present application provide an image generation device, which comprises: An acquisition module is configured to acquire a panoramic image, the panoramic image being synthesized by original images of the same moment captured by a plurality of acquisition devices; A separation module is configured to separate the panoramic image into a background image and an object region image corresponding to each target object; A decomposition module is configured to decompose the background image into a plurality of intermediate background images corresponding to the plurality of acquisition devices respectively; An equalization module is configured to perform equalization processing on the plurality of intermediate background images to obtain an equalized background image; A generation module is configured to fuse the equalized background image and the object region image to generate a result image.

[0008] In a third aspect, an electronic device is provided, which includes a processor and a memory, the memory stores programs or instructions executable on the processor, and the programs or instructions are executed by the processor to implement the steps of the method according to the first aspect.

[0009] In a fourth aspect, a readable storage medium is provided, which stores programs or instructions, and the programs or instructions are executed by a processor to implement the steps of the method according to the first aspect.

[0010] In a fifth aspect, a chip is provided, which includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run programs or instructions to implement the method according to the first aspect.

[0011] In a sixth aspect, a computer program product is provided, which is stored in a storage medium, and the program product is executed by at least one processor to implement the method according to the first aspect.

[0012] In the embodiment of the present application, by acquiring a panoramic image synthesized by the original images of the same moment captured by the plurality of acquisition devices, the panoramic image includes at least one target object, ensuring the integrity and consistency of environmental information in the space-time dimension, and establishing an accurate data basis for subsequent processing. The panoramic image is separated into a background image and an object region image corresponding to each target object, realizing accurate decoupling of scene structure and dynamic elements, so that key targets can be independently extracted and retained, ensuring the perception ability of dynamic risk factors. The background image is decomposed into a plurality of intermediate background images corresponding to the plurality of acquisition devices, creating conditions for subsequent local targeted processing, facilitating subsequent identification and positioning of color and brightness differences caused by different light environments of each acquisition device. The plurality of intermediate background images are balanced to obtain a balanced background image, which can eliminate the visual differences between multi-source images and improve the visual consistency and comfort of the background picture. The balanced background image and the object region image are fused to generate a result image, which improves the overall picture quality while ensuring the prominent display and natural integration of key targets, and can efficiently generate a result image containing complete environmental perception information, balancing real-time and safety. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a flowchart of an image generation method provided by an embodiment of the present application; Figure 2 is a schematic diagram of a morphological mutation region provided by an embodiment of the present application; Figure 3 is a structural diagram of an image generation device provided by an embodiment of the present application; Figure 4 is a hardware structure schematic diagram of an electronic device of an embodiment of the present application. DETAILED DESCRIPTION

[0014] The technical solutions of the embodiments of the present application will be described in detail below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0015] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally a class and do not limit the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / " generally indicates that the front and rear associated objects are in an "or" relationship.

[0016] The image generation method provided by the embodiments of the present application can be applied at least to the following application scenarios, which are described below.

[0017] Tunnel, tree-lined road and other alternating light environment scenarios: This scenario mainly tests the visual consistency maintenance ability of the surround view system under severe light changes. When the vehicle enters or exits the tunnel, the cameras in different directions will capture pictures with different brightness and color temperature due to the instantaneous difference in the light conditions. The forward camera may have entered the dark area of the tunnel, while the lateral and rear cameras are still in the high-brightness environment outside, resulting in a clear bright-dark division in the spliced picture. When passing through a tree-lined road section, the mottled light spots projected by the gaps between the leaves will form a rapidly moving light and shadow pattern on the surface of the vehicle body, causing the images captured by adjacent cameras to have a complex situation of local overexposure and shadow interweaving.

[0018] Continuous environment perception scenario under urban congestion road conditions: This scenario mainly tests the timing stability and data integrity of the surround view system under high dynamic and multi-target environment. When the vehicle is in a low-speed or stationary state, pedestrians, non-motor vehicles and other road users in the surrounding environment still maintain continuous motion. The system needs to complete the acquisition, splicing and output of each frame of image in a very short processing period. Any picture jump, target loss or motion trajectory break caused by image screening or processing delay may interfere with the driver's accurate judgment of the surrounding situation.

[0019] In view of the problems in the related art, the embodiments of the present application provide an image generation method, device, electronic equipment and storage medium, which can solve the problem that the AVM surround view splicing technology in the related art cannot balance real-time performance and safety.

[0020] The image generation method provided by the embodiments of the present application will be described in detail below in combination with the drawings, specific embodiments and their application scenarios.

[0021] Figure 1 A flowchart of an image generation method provided by the embodiments of the present application.

[0022] As Figure 1As shown, the image generation method can include steps 110-150, which is applied to an image generation device, as shown below: Step 110, obtaining a panoramic image, the panoramic image being composed of original images captured by multiple acquisition devices at the same time; the panoramic image including at least one target object; The panoramic image refers to a bird's-eye view synthetic image formed by stitching after geometric correction and perspective transformation of original images synchronously captured by multiple fisheye cameras deployed around the vehicle.

[0023] The time consistency of the captured image frames is ensured by the space-time synchronization of the multiple cameras, and seamless mapping of multi-view images to a unified bird's-eye view coordinate system is achieved by using pre-calibrated stitching parameters. For example, when the vehicle enters a tunnel, the cameras facing the inside of the tunnel and those facing the outside of the tunnel will capture images with a large difference in brightness. This step combines these images into a panoramic view containing complete environmental information.

[0024] Step 120, separating the panoramic image into a background image and an object region image corresponding to each target object; The background image is the part remaining after removing all target objects from the panoramic image, and its content mainly includes road surface, static buildings, and other scene elements. The object region image refers to an independent target region extracted from the panoramic image by image segmentation technology, and each region corresponds to a recognized dynamic or specific category target, such as a pedestrian or a vehicle.

[0025] Based on the target detection and semantic segmentation model of computer vision, the target objects meeting the preset conditions in the panoramic view are first identified, such as targets belonging to the pedestrian category or targets determined to be in motion, and then the contour regions of each target object are accurately extracted by pixel-level segmentation. For example, a pedestrian crossing can be accurately identified in the panoramic image and separated from the background to form an independent pedestrian object region image, and the area in the original panoramic image where the pedestrian is located forms a corresponding vacancy in the background image.

[0026] Step 130, decomposing the background image into multiple intermediate background images corresponding to the multiple acquisition devices respectively; The intermediate background image is a sub-image obtained by inversely decomposing the complete background image according to the original perspective relationship of the acquisition devices, and these sub-images correspond to the viewing ranges of the cameras.

[0027] Based on the inverse mapping relationship of the panoramic stitching process, the fused background image is restored to the sub-background images under the perspective of each camera by querying the pre-established image synthesis path. For example, for a surround view composed of four fisheye cameras, this step will decompose the complete background image into intermediate background images corresponding to the front, rear, left, and right perspectives respectively.

[0028] Step 140, the plurality of intermediate background images are balanced to obtain balanced background images; Balancing refers to the consistency adjustment of color and brightness of the plurality of intermediate background images, so that the visual style tends to be unified.

[0029] The visual differences between the intermediate background images are eliminated by the color balancing algorithm. In specific implementation, one of the intermediate background images can be selected as a reference, and then the color distribution of the other images is adjusted to be consistent with the reference image through histogram matching and other techniques. For example, when one side of the vehicle is in direct sunlight and the other side is in the shadow, this step can adjust the brightness and contrast of the intermediate background image in the shadow to a level consistent with the sunlight area.

[0030] Step 150, the balanced background image is fused with the object region image to generate a result image.

[0031] The fusion operation refers to the process of re-integrating the processed background and the enhanced target image into a complete output image.

[0032] In the unified image coordinate system, the object region image after enhancement is accurately placed in the corresponding position of the balanced background image through a pixel-level fusion algorithm. For example, the pedestrian image after clear enhancement is re-embedded into the background image after color balancing, ensuring that the transition between the pedestrian outline and the background is smooth.

[0033] In one possible embodiment, step 140 can specifically include the following steps: From the plurality of intermediate background images, determine the image to be processed; According to the color distribution characteristics of the background image, adjust the color distribution characteristics of the image to be processed to obtain an adjusted intermediate background image; According to the adjusted intermediate background image, synthesize a balanced background image.

[0034] The image to be processed refers to a sub-image identified from the plurality of intermediate background images, which has a significant difference in color performance from other images or the overall background. The color distribution characteristics are a quantitative description of color information in the image, which is usually obtained by statistical distribution of color channels in the image, such as through a color histogram. Adjusting the color distribution characteristics refers to the process of changing the color properties of the image through image processing algorithms so that it tends to be consistent with the reference standard.

[0035] In the step of determining the image to be processed, the consistency degree of each intermediate background image and the overall background image in color features is quantitatively compared to identify abnormal images with large deviations. In specific implementation, the similarity measure value between the color histogram of each intermediate background image and the color histogram of the background image is calculated, for example, using Bhattacharyya coefficient or correlation coefficient for quantitative comparison. For example, when one side of the vehicle is directly exposed to sunlight and the other side is in the shadow, the color feature similarity of the intermediate background image facing the shadow and the overall background image will be significantly low, so as to be identified as the image to be processed.

[0036] In the step of adjusting the color distribution features of the image to be processed, the mapping relationship between the current image and the target color features is established, and the consistency adjustment is realized through color transformation. This is usually realized through histogram matching technology, that is, taking the color distribution of the image to be processed as input and taking the color distribution of the background image as target, the corresponding transformation function is calculated. For example, for the image to be processed which is overall blue due to white balance deviation, a color correction curve which can eliminate the color deviation is calculated, which enhances the red channel and weakens the blue channel, so that the color tone is coordinated with the overall background.

[0037] In the step of synthesizing the balanced background image, the intermediate background images after color adjustment are re-integrated into a complete background picture. This process needs to follow the same geometric mapping relationship as the panoramic image synthesis to ensure seamless connection of each image block. For example, four intermediate background images which are respectively color-optimized are re-pasted into a complete bird's eye view background picture according to the perspective relationship of front, back, left and right.

[0038] By establishing a color correction mechanism based on the overall background, color deviations caused by different light conditions of individual cameras can be effectively identified and corrected. By processing the image to be processed instead of global adjustment, the original quality of normal image areas is maintained, and the visual consistency of the spliced picture is significantly improved. While ensuring the processing efficiency, the output stability of surround view under various complex light conditions is effectively improved.

[0039] In one possible embodiment, the step of determining the image to be processed from the plurality of intermediate background images involves the following steps: The similarity of the color distribution features of each intermediate background image and the color distribution features of the background image is obtained; wherein the image to be processed is the intermediate background image with a similarity less than a preset similarity threshold.

[0040] Color distribution feature is a statistical representation of color information of an image, which is usually obtained by analyzing the numerical distribution of each color channel in the image, such as constructing a color histogram to describe the frequency of different color values. Similarity is a quantitative index for measuring the consistency between two color distribution features, and the matching degree of two feature vectors is calculated by mathematical method. The preset similarity threshold is a numerical boundary line set in advance, which is used to determine whether two color distribution features are similar enough.

[0041] In the step of calculating similarity, the difference between two color distribution features can be quantified by mathematical method. The color distribution features of the intermediate background image and the background image are extracted respectively, and then a specific similarity measurement algorithm is used for calculation. For example, Bhattacharyya distance or histogram intersection algorithm can be used to compare the similarity of two color histograms, which can effectively capture the overall shape and statistical characteristic difference of color distribution. For example, when an intermediate background image is overall blue due to white balance deviation, the distribution of its blue channel will deviate significantly from the overall distribution of the background image, resulting in a significant decrease in the similarity calculation value.

[0042] The preset threshold mechanism can be used to automatically identify the intermediate background image that needs to be adjusted. The calculated similarity value is compared with the preset threshold, and a classification decision is made according to the comparison result. For example, if the similarity threshold is set to 0.8, and the color distribution similarity between an intermediate background image and the background image is only 0.6, the intermediate background image will be marked as a to-be-processed image.

[0043] By determining the intermediate background image with a similarity less than the preset similarity threshold as a to-be-processed image, the color abnormality problem caused by the difference in acquisition conditions of individual acquisition devices can be effectively identified. By quantifying the similarity and setting a reasonable threshold, automatic screening of intermediate background images is realized, providing accurate targets for subsequent targeted processing, ensuring that the color correction of individual images can be consistent with the overall picture, thereby improving the visual uniformity of the surround view output picture.

[0044] In one possible implementation, the step 120 can specifically include the following steps: Perform target detection on the panoramic image to determine a target object satisfying a preset condition, and the preset condition includes at least one of the following: belonging to a preset category, being in a motion state; Separate the panoramic image into a background image and an object region image corresponding to each target object based on the target object.

[0045] Target detection refers to a method of automatically identifying the location and category of specific objects in an image through computer vision algorithms. Preset categories refer to pre-defined object types that require special attention, such as pedestrians, vehicles, and other objects closely related to driving safety. Motion state refers to the dynamic characteristics of target objects that change in position between consecutive frames. Separation operation refers to the accurate extraction of target objects from the original image, forming independent object region images and background images after removing these objects.

[0046] In the step of target detection on the panoramic image, multi-scale feature extraction and classification regression on the input image are based on a deep learning model. A pre-trained neural network model will be used, which can output the detected target bounding box coordinates and corresponding category confidence at the same time. For example, when the panoramic image captured by the surround view contains a pedestrian crossing, the target detection algorithm can accurately frame the location of the pedestrian and assign it a high pedestrian category confidence. When determining the motion state, temporal information will be combined to judge the motion characteristics of the same target by comparing its position change between the current frame and the previous frame. For example, for a vehicle parked on the roadside, although it belongs to the preset category, it may not be determined to be in motion state because its position does not change between consecutive frames.

[0047] In the step of image separation based on target objects, image segmentation technology is used to accurately extract the pixels of the target region. After obtaining the bounding box of target detection, an instance segmentation algorithm is further used to classify the region within each bounding box at the pixel level, generating an accurate target mask. For example, for a detected vehicle target, the segmentation algorithm can accurately outline the vehicle's contour, distinguishing all pixels belonging to the vehicle from background pixels. Subsequently, according to the generated target mask, the image region corresponding to each target object is extracted from the original panoramic image, forming independent object region images, while the background image is formed by removing these target regions from the original image.

[0048] By combining target detection and image segmentation technology, accurate identification and separation of key targets in panoramic images are achieved. Through the dual screening mechanism of preset categories and motion states, dynamic targets and specific category targets that are important for driving safety are completely retained. This targeted separation process lays the foundation for subsequent differential optimization of background and targets, avoiding the safety hazards of important target information loss that may be caused by traditional image screening methods, and creating favorable conditions for improving the overall image quality.

[0049] Among the above steps related to separating the panoramic image into a background image and an object region image corresponding to each target object based on target objects, the steps can include the following: The semantic segmentation of the area where each target object is located in the panoramic image is performed to generate a target mask for distinguishing target pixels and background pixels; all target pixels are extracted from the panoramic image according to all target masks to form an object region image; and the area in the panoramic image except the target pixels is taken as a background image.

[0050] Semantic segmentation refers to a technology of performing pixel-level classification on an image through a deep learning algorithm, which can accurately identify each pixel point belonging to a specific target. The target mask is a binary image data in which the target pixels are marked as a specific value and the background pixels are marked as another specific value, so as to form a clear pixel-level segmentation boundary.

[0051] The full convolutional neural network architecture is used to perform end-to-end pixel-level classification on the input panoramic image. The network gradually extracts deep features of the image and restores the spatial resolution through multiple convolution and deconvolution operations, and outputs the probability distribution of the category to which each pixel belongs. The probability map output by the semantic segmentation model is thresholded, and the pixels belonging to the target category are set to 1 and the background pixels are set to 0, so as to generate a binary mask image. For example, for a detected vehicle target, a mask with the same size as the panoramic image is generated, in which the pixels in the area where the vehicle is located are 1 and the pixels in other areas are 0, accurately outlining the complete contour of the vehicle.

[0052] In the process of extracting the object region image, the generated target mask is used as a selection filter to extract the target region from the original panoramic image through pixel-by-pixel multiplication operation. All target masks are traversed, and the pixel positions marked as 1 in each mask are copied from the panoramic image to form independent object region images. For example, according to the pedestrian segmentation mask, an independent image block containing the complete image of the pedestrian can be extracted while retaining its original color and texture information.

[0053] In the process of forming the background image, a blank image with the same size as the panoramic image is created, and then the panoramic image pixels corresponding to the pixel positions marked as 0 in all target masks are copied into the blank image. For example, after all targets such as pedestrians and vehicles are extracted, the remaining pixels of the road surface, buildings, sky and the like are retained to form a pure background image containing no dynamic targets.

[0054] The target and the background are accurately separated by pixel-level semantic segmentation, which can effectively handle complex situations such as similar target and background colors and blurred boundaries, and ensure the preservation of the complete contour of the key target. Precise pixel-level operation avoids the background residue or target cropping problem caused by traditional bounding box extraction, lays a high-quality data foundation for subsequent optimization processing of the background and target, and ensures that the dynamic target is completely preserved while the background area can be processed for image enhancement without interference in the surround view.

[0055] In the step of detecting the target object from the panoramic image and determining the target object satisfying the preset condition, the following steps can be included: associating the position of the candidate object at the current time with the position of the candidate object at the previous historical time to obtain position change information of each candidate object; determining the candidate object in a motion state as the target object according to the position change information of each candidate object.

[0056] Position association refers to establishing the correspondence between the candidate objects detected at different times through a target tracking algorithm to determine the motion trajectory of the same target in a time sequence. Position change information is a motion parameter obtained by calculating the coordinate change of the same target in consecutive frames of images, including displacement size and direction and other quantitative indicators. Motion state refers to a significant position change of the target exceeding a preset threshold within a continuous time interval.

[0057] In the position association step, the appearance features and spatial position information of the target are used to match the candidate objects detected in the current frame with the targets tracked in the previous frame through a data association algorithm. A motion model will be established for each target detected at the previous time to predict its possible position in the current frame, and then the correspondence will be established by calculating the similarity between the predicted position and the actual detection position. For example, the Hungarian algorithm or joint probability data association method can be used to solve the multi-target matching problem. For example, when a moving bicycle is tracked at the previous time, its position at the current time will be predicted according to its motion speed and direction, and matched with all bicycle candidate objects detected in the current frame.

[0058] In the step of determining the motion state, the motion characteristics of the target are judged by analyzing the displacement amount between consecutive frames. The displacement vector of each successfully associated candidate object between two frames will be calculated, including the moving distance and direction, and the displacement amount will be compared with the preset motion threshold. For example, a threshold based on image pixel distance can be set, and when the displacement of the target between two frames exceeds 5 pixels, it is determined that the target is in a motion state. For example, a car coming from a distance will show a significant position change in consecutive surround view image frames, and its displacement amount will exceed the preset threshold, so it is determined as a moving target.

[0059] By correlating the position in time with the motion state analysis, the precise identification of moving targets in the panoramic image is realized. This dynamic target detection method based on continuous frame analysis can effectively distinguish between stationary objects and moving objects, avoiding misjudgment of temporarily parked vehicles or fixed facilities on the roadside as targets that require special treatment. By accurately identifying moving targets, computing resources can be concentrated on objects that truly require attention, ensuring timely response to dynamic risk factors in the surround view and improving the efficiency of the entire image processing process.

[0060] In one possible embodiment, step 150 can specifically include the following steps: Performing enhancement processing on the at least one object region image to obtain an enhanced object region image; Fusing the equalized background image and the enhanced object region image to generate a result image.

[0061] The enhancement processing refers to improving the visual quality of the object region image through image processing algorithms, including but not limited to improving image clarity, optimizing contrast, reducing noise, etc. Fusing refers to re-embedding the processed object region image into the corresponding position in the equalized background image, and implementing a natural transition image synthesis process through edge fusion technology.

[0062] In the enhancement processing step, special image optimization algorithms are applied to the independent object regions separated from the panoramic image. Since these object regions often have degraded quality due to factors such as motion blur, insufficient lighting, or shooting distance, algorithms such as deep learning-based super-resolution reconstruction, adaptive contrast enhancement, or intelligent deblurring are used for targeted processing. For example, for a pedestrian image with unclear details due to motion blur, a trained deconvolutional neural network model can be used to restore its outline and texture details, making the character features more clear and identifiable.

[0063] In the image fusion step, the enhanced object region is seamlessly integrated into the background environment. First, according to the coordinate information of the object region in the original panoramic image, its accurate position in the equalized background image is determined. Then, advanced algorithms such as multi-resolution fusion or gradient domain fusion are used to make the edge of the inserted object naturally transition with the background, avoiding obvious seams or halo effects. For example, when a vehicle image that has been processed to be clear is reinserted into the surround view scene, alpha blending technology is used to smooth the transition area between the vehicle outline and the road background, ensuring that the synthesized visual effect is realistic and natural.

[0064] The overall optimization of the surround view image quality is achieved by the strategies of processing and fusion respectively. The specific enhancement processing of the object area effectively solves the image quality problems caused by target motion or environmental factors, ensuring the visual recognizability of the key target. The intelligent fusion with the balanced background ensures that the processed target can harmoniously integrate into the environmental scene, maintaining the spatial consistency and visual authenticity of the surround view. The image quality of the key target can be improved without affecting the quality of the background picture, thereby improving the overall visual effect of the surround view.

[0065] In a possible embodiment, the step 110 can specifically include the following steps: obtaining original images collected by a plurality of collection devices; performing color space conversion and normalization processing on each original image to obtain a plurality of preprocessed images; synthesizing a panoramic image according to the plurality of preprocessed images.

[0066] Color space conversion is the process of converting an image from one color representation system to another, such as from RGB color space to Lab color space. Normalization processing is an operation that maps image pixel values to a specific range through mathematical transformation, aiming to eliminate the effects of different lighting conditions. A preprocessed image is an image that has undergone a series of standardized processing, providing a unified input standard for subsequent image synthesis. Panoramic image synthesis is a technical process of forming a complete surround view scene by geometric transformation and pixel fusion of multiple preprocessed images.

[0067] In the step of obtaining original images and performing color space conversion, the characteristics of different color spaces are utilized to optimize the image processing effect. First, the original images collected by each fisheye camera are synchronized, and then they are converted from the standard RGB color space to the Lab color space which is more suitable for visual perception. The Lab color space separates the brightness information from the color information, allowing subsequent processing to independently adjust the brightness and chroma. For example, after converting a dark image taken at dusk to the Lab space, the brightness channel can be enhanced independently without affecting the original color balance.

[0068] In the normalization processing step, the brightness differences caused by different exposure parameters or lighting conditions of each camera are eliminated through numerical transformation. The pixel values of each color channel are scaled linearly or nonlinearly to be distributed within a unified numerical range. For example, for a dark image taken in backlight, the pixel values are redistributed to the complete range of 0 to 255, thereby restoring the proper contrast performance.

[0069] In the panoramic image synthesis step, according to the pre-calibrated camera parameters and the stitching mapping table, the multiple pre-processed images are projected into a unified bird's eye view coordinate system. Through feature point matching and overlapping area fusion algorithm, the geometric distortion and stitching gap between images are eliminated. For example, four pre-processed images respectively from front, rear, left and right cameras are mapped to a virtual ground plane coordinate system through perspective transformation, and seamless splicing is realized through multi-band fusion algorithm, forming a complete 360-degree surround view picture.

[0070] The standardized preprocessing process lays an important foundation for subsequent image processing. Color space conversion enables image information to be processed in a manner more consistent with human visual perception, creating favorable conditions for subsequent color balancing. Normalization effectively suppresses exposure differences between cameras, avoiding image information loss due to local overexposure or underexposure. The standardized synthesis process ensures the geometric accuracy and visual coherence of the surround view image. The overall cooperation of these preprocessing steps significantly improves the stability and reliability of subsequent target detection, image separation, and color balancing processes.

[0071] In the embodiments of the present application, by acquiring a panoramic image synthesized from raw images of the same moment captured by multiple acquisition devices surrounding the acquisition, the panoramic image includes at least one target object, ensuring the completeness and consistency of environmental information in the time and space dimensions, and establishing an accurate data foundation for subsequent processing. Separating the panoramic image into a background image and an object region image corresponding to each target object realizes precise decoupling of scene structure and dynamic elements, enabling key targets to be independently extracted and retained, and ensuring the perception ability of dynamic risk factors. Decomposing the background image into multiple intermediate background images corresponding to the multiple acquisition devices creates conditions for subsequent local targeted processing, facilitating subsequent identification and positioning of color and brightness differences caused by different light environments of each acquisition device. Equalizing the multiple intermediate background images obtains an equalized background image, which can eliminate visual differences between multiple source images and improve the visual consistency and comfort of the background picture. Fusing the equalized background image and the object region image generates a result image, which improves the overall picture quality while ensuring the prominent display and natural integration of key targets, and can efficiently generate a result image containing complete environmental perception information, balancing real-time performance and safety.

[0072] The specific process of image generation of the embodiments of the present application will be explained as follows: Figure 2 The specific process of image generation of the embodiments of the present application will be explained as follows: Raw images of the same moment captured by multiple acquisition devices surrounding the acquisition are acquired, which can be 4-way surround view data. Then, key target detection based on a segmentation model, i.e., identifying at least one target object in the panoramic image, can be specifically realized by analyzing the synthesized panoramic view through a deep learning segmentation model to detect key targets meeting the preset conditions.

[0073] Subsequently, a logical judgment is made to detect key targets; if not, a surround rendering process is performed; if so, key target extraction and migration are executed, which means separating the panoramic image into a background image and an object region image corresponding to each target object. Specifically, the detected key targets can be accurately separated from the original panoramic image to form independent target images and background images in which these targets have been removed.

[0074] Simultaneously, image frames are filtered based on panoramic style differences, which involves equalizing multiple intermediate background images. By analyzing the color distribution characteristics among the intermediate background images corresponding to each camera's viewpoint, it is possible to identify image frames whose style differences exceed a preset threshold.

[0075] Next, it is determined whether there are frames with significant differences. If not, the background image is used directly for compositing; if so, image style transfer is performed on the image frames with significant style differences. This involves color equalization processing, for example, using histogram matching techniques to adjust the color distribution of the differing frames to match that of the reference frame.

[0076] Finally, the key target image is compared with the style-transfer processed background. Figure 1 The process begins with a surround-view rendering workflow, which involves merging the balanced background image with the object area image to generate a final image. This outputs a surround-view view that is both complete with key information and has a unified visual style. While improving the overall image quality, it ensures the prominent display and natural integration of key targets, efficiently generating a final image containing complete environmental awareness information, and balancing real-time performance with security.

[0077] The image generation method provided in this application can be executed by an image generation device. This application uses an image generation device executing the image generation method as an example to illustrate the image generation device provided in this application.

[0078] Figure 3 This is a block diagram of an image generation apparatus provided in an embodiment of this application. The apparatus 300 includes: The acquisition module 310 is used to acquire a panoramic image, which is synthesized from original images acquired at the same moment by multiple acquisition devices; the panoramic image includes at least one target object. The separation module 320 is used to separate the panoramic image into a background image and an object region image corresponding to each target object; The decomposition module 330 is used to decompose the background image into multiple intermediate background images corresponding to multiple acquisition devices respectively; The equalization module 340 is used to equalize multiple intermediate background images to obtain an equalized background image; The generating module 350 is configured to fuse the balanced background image and the object region image to generate a result image.

[0079] In a possible embodiment, the balancing module 340 is specifically configured to: determine the to-be-processed image from the plurality of intermediate background images; adjust color distribution features of the to-be-processed image according to color distribution features of the background image to obtain an adjusted intermediate background image; synthesize the balanced background image according to the adjusted intermediate background image.

[0080] In a possible embodiment, the balancing module 340 is specifically configured to: obtain a similarity between color distribution features of each intermediate background image and color distribution features of the background image; wherein the to-be-processed image is an intermediate background image with a similarity less than a preset similarity threshold.

[0081] In a possible embodiment, the separating module 320 is specifically configured to: perform target detection on the panoramic image to determine a target object satisfying a preset condition, the preset condition including at least one of the following: belonging to a preset category, being in a motion state; separate the panoramic image into the background image and an object region image corresponding to each target object based on the target object.

[0082] In a possible embodiment, the separating module 320 is specifically configured to: associate a candidate object at a current moment with a candidate object at a previous historical moment to obtain position change information of each candidate object; determine a candidate object in a motion state as the target object according to the position change information of each candidate object.

[0083] In a possible embodiment, the generating module 350 is specifically configured to: perform enhancement processing on the at least one object region image to obtain an enhanced object region image; fuse the balanced background image and the enhanced object region image to generate a result image.

[0084] In a possible embodiment, the acquiring module 310 is specifically configured to: acquire original images collected by a plurality of collection devices; perform color space conversion and normalization processing on each original image to obtain a plurality of preprocessed images; synthesize the panoramic image according to the plurality of preprocessed images.

[0085] In the embodiment of the present application, by acquiring a panoramic image synthesized by the original images of the same moment captured by the plurality of acquisition devices, the panoramic image includes at least one target object, ensuring the completeness and consistency of the environmental information in the space-time dimension, and establishing an accurate data basis for subsequent processing. The panoramic image is separated into a background image and an object region image corresponding to each target object, realizing accurate decoupling of scene structure and dynamic elements, so that key targets can be independently extracted and retained, ensuring the perception ability of dynamic risk factors. The background image is decomposed into a plurality of intermediate background images corresponding to the plurality of acquisition devices, creating conditions for subsequent local targeted processing, facilitating subsequent identification and positioning of color and brightness difference problems caused by different light environments of each acquisition device. The plurality of intermediate background images are balanced to obtain a balanced background image, which can eliminate the visual difference between multi-source images and improve the visual consistency and comfort of the background picture. The balanced background image and the object region image are fused to generate a result image, which improves the overall picture quality while ensuring the prominent display and natural integration of key targets, and can efficiently generate a result image containing complete environmental perception information, taking into account real-time and safety.

[0086] The image generation apparatus provided by the embodiments of the present application can realize each process realized by the method embodiments, and to avoid repetition, details are not repeated here.

[0087] Optionally, Figure 4 A hardware structure schematic diagram of an electronic device provided by an embodiment of the present application is shown.

[0088] The electronic device can include a processor 401 and a memory 402 having computer program instructions stored therein.

[0089] Specifically, the processor 401 can include a central processing unit (CPU), or a specific integrated circuit (Application Specific Integrated Circuit, ASIC), or can be configured as one or more integrated circuits that implement one or more embodiments of the present application.

[0090] The memory 402 can include mass storage for data or instructions. By way of example, and not limitation, the memory 402 can include a hard disk drive (HDD), floppy disk drive, flash memory, compact disk, digital versatile disk, optical disk, tape, or universal serial bus (USB) drive or a combination of two or more of these. The memory 402, in some embodiments, can include removable or non-removable (or fixed) media. The memory 402, in some embodiments, can be internal or external to the integrated gateway disaster recovery appliance. In certain embodiments, the memory 402 is non-volatile solid-state memory. In certain embodiments, the memory 402 includes read-only memory (ROM). The ROM, in some embodiments, can be mask- programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0091] The processor 401 implements any of the image generation methods in the embodiments shown in the figures by reading and executing computer program instructions stored in the memory 402.

[0092] In one example, the electronic device can further include a communication interface 404 and a bus 410. As shown, the processor 401, the memory 402, the communication interface 404 are connected through the bus 410 and complete the communication between each other. Figure 4

[0093] The communication interface 404 is mainly used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application.

[0094] The bus 410 includes hardware, software, or both, that couples components of the electronic device to each other. By way of example, and not limitation, the bus can include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand (IB) interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or another suitable bus or interconnect, or a combination of two or more of these. Where appropriate, the bus 410 can include one or more buses. Although the present application describes and illustrates a particular bus, this application contemplates any suitable bus or interconnect.

[0095] ​The electronic device can execute the image generation method in the embodiments of the present application, thereby realizing the image generation method combined with Figure 2 described above.

[0096] In addition, in combination with the image generation method in the embodiments described above, the embodiments of the present application can provide a computer readable storage medium to realize. The computer readable storage medium has computer program instructions stored thereon; the computer program instructions are executed by a processor to realize Figure 1 the image generation method.

[0097] It needs to be clear that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after understanding the spirit of the present application.

[0098] The functional blocks shown in the structure block diagram described above can be implemented as hardware, software, firmware or their combination. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the present application are program or code segment used to perform the required tasks. The program or code segment can be stored in a machine readable medium or transmitted on a transmission medium or communication link through a data signal carried in a carrier wave. The "machine readable medium" can include any medium capable of storing or transmitting information. Examples of machine readable medium include electronic circuit, semiconductor memory device, ROM, flash memory, erasable ROM (EROM), floppy disk, CD-ROM, optical disk, hard disk, optical fiber medium, radio frequency (RF) link, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0099] It also needs to be noted that the exemplary embodiments mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be executed in the order mentioned in the embodiments, or in an order different from the embodiments, or several steps can be executed simultaneously.

[0100] The above merely describes a specific implementation of the present application. Those skilled in the art can clearly understand the specific working processes of the system, modules and units described above for the convenience and brevity of description, and can refer to the corresponding processes in the foregoing method embodiments, which will not be described herein again. It should be understood that the protection scope of the present application is not limited to this, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application.

Claims

1. An image generation method, characterized in that, The method includes: A panoramic image is acquired, which is synthesized from original images captured at the same moment by multiple acquisition devices; the panoramic image includes at least one target object. The panoramic image is separated into a background image and an object region image corresponding to each target object; The background image is decomposed into multiple intermediate background images corresponding to the multiple acquisition devices respectively; The multiple intermediate background images are subjected to equalization processing to obtain an equalized background image; The balanced background image and the object region image are fused to generate the result image.

2. The method according to claim 1, characterized in that, The process of equalizing the plurality of intermediate background images to obtain an equalized background image includes: The image to be processed is determined from the plurality of intermediate background images; Based on the color distribution characteristics of the background image, the color distribution characteristics of the image to be processed are adjusted to obtain the adjusted intermediate background image; The balanced background image is synthesized based on the adjusted intermediate background image.

3. The method according to claim 2, characterized in that, The step of determining the image to be processed from the plurality of intermediate background images includes: Obtain the similarity between the color distribution features of each intermediate background image and the color distribution features of the background image; wherein, the image to be processed is an intermediate background image with a similarity less than a preset similarity threshold.

4. The method according to any one of claims 1-3, characterized in that, The step of separating the panoramic image into a background image and an object region image corresponding to each target object includes: Target detection is performed on the panoramic image to determine target objects that meet preset conditions, the preset conditions including at least one of the following: belonging to a preset category and being in motion; Based on the target object, the panoramic image is separated into a background image and an object region image corresponding to each target object.

5. The method according to claim 4, characterized in that, The step of performing target detection on the panoramic image to determine target objects that meet preset conditions includes: The position of the candidate object at the current moment is associated with that of the candidate object at the previous historical moment to obtain the position change information of each candidate object; Based on the position change information of each candidate object, the candidate object in motion is determined as the target object.

6. The method according to claims 1-3 or 5, characterized in that, The step of fusing the equalized background image with the object region image to generate a result image includes... At least one of the object region images is enhanced to obtain an enhanced object region image; The equalized background image is fused with the enhanced object region image to generate the result image.

7. The method according to claims 1-3 or 5, characterized in that, The acquisition of panoramic images includes: Acquire the original images captured by the multiple acquisition devices; Each of the original images is subjected to color space conversion and normalization to obtain multiple preprocessed images; The panoramic image is synthesized based on the multi-frame preprocessed images.

8. An image generation apparatus, characterized in that, The device includes: An acquisition module is used to acquire a panoramic image, which is synthesized from original images acquired at the same moment by multiple acquisition devices; the panoramic image includes at least one target object. A separation module is used to separate the panoramic image into a background image and an object region image corresponding to each target object; The decomposition module is used to decompose the background image into multiple intermediate background images corresponding to the multiple acquisition devices respectively; An equalization module is used to perform equalization processing on the multiple intermediate background images to obtain an equalized background image; The generation module is used to fuse the equalized background image with the object region image to generate a result image.

9. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the image generation method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the image generation method as described in any one of claims 1-7.

Images