Image synthesis method, device, electronic device and storage medium

Abstract

The present disclosure provides an image synthesis method, apparatus, electronic device, and storage medium, which relate to the field of artificial intelligence technology, specifically to technical fields such as deep learning, image processing, and computer vision, and can be applied to scenarios such as image synthesis, including: obtaining a plurality of initial images, determining a plurality of target light source information respectively corresponding to the plurality of initial images, processing the corresponding plurality of initial images respectively according to the plurality of target light source information to obtain a plurality of images to be synthesized, and synthesizing the plurality of images to be synthesized to obtain a target image. Thereby, corresponding optimization processing is realized according to the target light source information of the initial images. When the target image is synthesized based on the processed images to be synthesized, the quality of image synthesis can be effectively improved, and the representation accuracy of the obtained target image for the relevant information of the plurality of initial images can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technologies, specifically to technologies such as deep learning, image processing, and computer vision, and can be applied to scenarios such as image synthesis. In particular, it relates to an image synthesis method, apparatus, electronic device, and storage medium. Background Art

[0002] Artificial intelligence is a discipline that studies how to make a computer simulate certain human thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and it has both hardware-level technologies and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include several major directions such as computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, and knowledge graph technology.

[0003] In related technologies, multiple images to be synthesized are usually synthesized to obtain a synthesized image. Summary of the Invention

[0004] The present disclosure provides an image synthesis method, apparatus, electronic device, storage medium, and computer program product.

[0005] According to a first aspect of the present disclosure, there is provided an image synthesis method, including: obtaining a plurality of initial images; determining a plurality of target light source information respectively corresponding to the plurality of initial images; processing the corresponding plurality of initial images respectively according to the plurality of target light source information to obtain a plurality of images to be synthesized; and synthesizing the plurality of images to be synthesized to obtain a target image.

[0006] According to a second aspect of the present disclosure, there is provided an image synthesis apparatus, including: an obtaining module for obtaining a plurality of initial images; a determining module for determining a plurality of target light source information respectively corresponding to the plurality of initial images; a first processing module for processing the corresponding plurality of initial images respectively according to the plurality of target light source information to obtain a plurality of images to be synthesized; and a second processing module for synthesizing the plurality of images to be synthesized to obtain a target image.

[0007] According to a third aspect of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image synthesis method according to the first aspect of the present disclosure.

[0008] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the image synthesis method according to the first aspect of the present disclosure.

[0009] According to a fifth aspect of the present disclosure, there is provided a computer program product including a computer program which, when executed by a processor, implements the steps of the image synthesis method according to the first aspect of the present disclosure.

[0010] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0012] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure;

[0013] Figure 2 is a schematic diagram according to the second embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram of an image synthesis proposed by an embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram according to the third embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram of an image synthesis process proposed by an embodiment of the present disclosure;

[0017] Figure 6 is a schematic diagram according to the fourth embodiment of the present disclosure;

[0018] Figure 7 is a schematic diagram according to the fifth embodiment of the present disclosure;

[0019] Figure 8 shows a schematic block diagram of an exemplary electronic device that can be used to implement the image synthesis method of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to help understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0021] Figure 1 It is a schematic diagram according to the first embodiment of the present disclosure.

[0022] It should be noted that, in this embodiment, the execution subject of the image synthesis method is an image synthesis device, which can be implemented in software and / or hardware, and can be configured in an electronic device, which may include but is not limited to a terminal, a server, etc.

[0023] The embodiments of the present disclosure relate to the field of artificial intelligence technology, specifically to the fields of deep learning, image processing, computer vision, etc., and can be applied to scenarios such as image synthesis.

[0024] Among them, Artificial Intelligence (AI) is a new technical science that studies, develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence.

[0025] Deep learning is to learn the internal laws and representation levels of sample data, and the information obtained during these learning processes is very helpful for the interpretation of data such as text, images, and sounds. The ultimate goal of deep learning is to enable machines to have the ability of analysis and learning like humans, and be able to recognize data such as text, images, and sounds.

[0026] Image processing is to use computer devices to analyze and process images to meet expected requirements. Common methods of image processing include: image transformation, image compression coding, image enhancement, image restoration, image segmentation, image description, and image classification, etc.

[0027] Computer vision refers to using cameras and computers to replace human eyes to perform machine vision such as target recognition, tracking, and measurement on targets, and further perform graphic processing to make the images processed by the computer more suitable for human eye observation or transmission to instrument detection.

[0028] Image synthesis refers to the technology of stitching several overlapping images (which may be obtained at different times, different perspectives, or by different sensors) into a seamless panoramic image or high-resolution image.

[0029] In the technical solution of the present disclosure, the processing of collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0030] As Figure 1 shown, the image synthesis method includes:

[0031] S101: Obtain multiple initial images.

[0032] Among them, the initial image refers to the image to be subjected to image synthesis. This initial image can be captured by a visible light camera, or alternatively, it can also be captured by an infrared camera. There is no limitation in this regard.

[0033] In the embodiments of the present disclosure, when obtaining a plurality of initial images, an image acquisition device can be pre-configured in the execution entity of the embodiments of the present disclosure to obtain the initial images from other storage media or networks. Or alternatively, a data interface can be pre-configured for the image synthesis device, and an image synthesis request is received via this data interface, and then the initial images are parsed from the image synthesis request. There is no limitation in this regard.

[0034] It can be understood that due to the limitation of the size of the photographed object or the shooting accuracy, a plurality of initial images in the embodiments of the present disclosure can be continuously used by the imaging device to collect images of different regions of the shooting scene or object in various directions, so as to obtain a plurality of initial images describing different regions of the same shooting scene or object.

[0035] In the embodiments of the present disclosure, when obtaining a plurality of initial images, subsequent steps can be triggered in a timely manner to implement the splicing process of the plurality of initial images, so that the obtained target image can effectively represent the relevant information of the plurality of initial images.

[0036] S102: Determine a plurality of target light source information respectively corresponding to the plurality of initial images.

[0037] Among them, the light source information refers to the relevant information of the light source used by the imaging device when acquiring the image. The target light source information refers to the light source information corresponding to the initial image.

[0038] In the embodiments of the present disclosure, the plurality of target light source information may be the same light source information, or alternatively, it may also correspond to different light source information. There is no limitation in this regard.

[0039] For example, the target light source information can be visible light, infrared light, light processed by a filter, etc. There is no limitation in this regard.

[0040] In the embodiments of the present disclosure, when determining a plurality of target light source information respectively corresponding to the plurality of initial images, the plurality of initial images can be respectively input into a pre-trained light source recognition model to obtain a plurality of target light source information respectively corresponding to the plurality of initial images. Or alternatively, the source information of the plurality of initial images can be determined, and then a plurality of target light source information respectively corresponding to the plurality of initial images is determined according to the source information. There is no limitation in this regard.

[0041] It can be understood that during the image synthesis process, the light source information may affect the matching effect among multiple initial images. When determining multiple target light source information corresponding to multiple initial images respectively, it can provide a reliable reference basis for the subsequent processing of multiple initial images.

[0042] S103: Process the corresponding multiple initial images respectively according to the multiple target light source information to obtain multiple images to be synthesized.

[0043] Among them, the image to be synthesized refers to the image obtained by processing the initial image based on the target light source information.

[0044] In the embodiments of the present disclosure, when processing the corresponding multiple initial images respectively according to the multiple target light source information to obtain multiple images to be synthesized, it may be to pre-train corresponding image processing models for the multiple target light source information respectively, and then input the multiple initial images into the corresponding image processing models respectively based on the target light source information to obtain multiple images to be synthesized. Or, a third-party image processing device may also be used to process the corresponding multiple initial images respectively according to the multiple target light source information to obtain multiple images to be synthesized, and there is no limitation on this.

[0045] In the embodiments of the present disclosure, when processing the corresponding multiple initial images respectively according to the multiple target light source information to obtain multiple images to be synthesized, flexible processing of the initial images can be realized based on the target light source information, thereby effectively improving the matching degree among the obtained multiple images to be synthesized and enhancing the success rate of this image synthesis method.

[0046] S104: Synthesize the multiple images to be synthesized to obtain a target image.

[0047] Among them, the target image refers to the image obtained by fusing and processing the multiple images to be synthesized.

[0048] In some embodiments, when synthesizing the multiple images to be synthesized to obtain a target image, it may be to obtain the gray-scale information of the multiple images to be synthesized, and then use the least squares method to obtain the gray-scale value difference among the multiple images to be synthesized, and determine the similarity degree among the multiple images to be synthesized based on this gray-scale value difference, thereby obtaining the range and position of the overlapping area among the multiple images to be synthesized, so as to realize image synthesis and obtain the target image.

[0049] In other embodiments, when synthesizing the multiple images to be synthesized to obtain a target image, it may also be to obtain the coordinate information corresponding to the multiple images to be synthesized, and then complete image synthesis based on this coordinate information to obtain the target image.

[0050] Certainly, in some embodiments, other arbitrary possible methods may also be adopted to synthesize multiple images to be synthesized to obtain a target image, such as methods like sequential puzzle assembly, axial fusion, etc., and there is no limitation thereto.

[0051] In this embodiment, by obtaining multiple initial images, determining multiple target light source information respectively corresponding to the multiple initial images, processing the corresponding multiple initial images respectively according to the multiple target light source information to obtain multiple images to be synthesized, and synthesizing the multiple images to be synthesized to obtain a target image, thereby, realizing corresponding optimization processing of the initial images according to the target light source information thereof. When the target image is synthesized based on the processed images to be synthesized, the image synthesis quality can be effectively improved, and the representation accuracy of the relevant information of the multiple initial images in the obtained target image can be improved.

[0052] Figure 2 It is a schematic diagram according to the second embodiment of the present disclosure.

[0053] Such as Figure 2 shown, the image synthesis method includes:

[0054] S201: Obtain multiple initial images.

[0055] For the description of S201, reference can be specifically made to the above embodiments, and details will not be repeated here.

[0056] S202: Determine multiple target image features respectively corresponding to the multiple initial images.

[0057] Among them, the image feature may refer to the relevant information describing features such as image gray scale / color, average brightness, chromaticity, saturation, etc. The target image feature refers to the image feature corresponding to the initial image.

[0058] In the embodiments of the present disclosure, since the image features of the initial images corresponding to different light source information may be different, when determining multiple target image features respectively corresponding to the multiple initial images, it can provide a reliable analysis basis for subsequent determination of multiple target light source information.

[0059] S203: Determine multiple target light source information respectively according to the multiple target image features.

[0060] In some embodiments, when determining multiple target light source information respectively according to the multiple target image features, it may be to perform comparative analysis on the multiple target image features, divide the multiple target image features into different groups according to the analysis and comparison results, and then determine multiple target light source information respectively according to the grouping information.

[0061] In some other embodiments, when respectively determining multiple target light source information according to multiple target image features, a relationship table may also be used, and the relationship table may record multiple target light source information adapted to multiple target image features.

[0062] Alternatively, any other possible method may also be used to respectively determine multiple target light source information according to multiple target image features, such as engineering or mathematical methods, which are not limited herein.

[0063] Optionally, in some embodiments, when respectively determining multiple target light source information according to multiple target image features, multiple reference light source information may be obtained, multiple reference image features respectively corresponding to the multiple reference light source information may be determined, reference image features matching the target image features may be determined from the multiple reference image features, and the reference light source information corresponding to the matching reference image features may be used as the target light source information to obtain multiple target light source information. Thus, the multiple reference image features can provide a reliable reference basis for the determination process of the target light source information. When performing a matching process based on the multiple reference image features and the target image features, the working efficiency and accuracy of determining multiple target light source information can be effectively improved.

[0064] Among them, the reference light source information refers to multiple light source information pre-configured for the initial image.

[0065] Among them, the reference image features refer to the image features corresponding to the reference light source information.

[0066] It can be understood that under the same light source information, the image features corresponding to multiple images have similarities. By determining the reference image features, a reliable reference object can be provided for the determination process of the target light source information.

[0067] For example, visible light images and filter images contain image chromaticity information. The saturation of visible light images is higher than that of filter images. Infrared light images are grayscale images and do not have chromaticity information. Based on the above characteristic information, by analyzing and comparing with the reference image features corresponding to the initial image, multiple target light source information can be obtained.

[0068] That is to say, after obtaining multiple initial images in the embodiments of the present disclosure, multiple target image features respectively corresponding to the multiple initial images may be determined, and multiple target light source information may be respectively determined according to the multiple target image features. Since the target image features corresponding to different target light source information may be different, when respectively determining multiple target light source information according to the multiple target image features, an accurate judgment of the target light source information can be achieved, and the reliability of the obtained target light source information can be effectively improved.

[0069] S204: Determine multiple image description information of multiple initial images respectively based on corresponding multiple target light source information.

[0070] Among them, the image description information may refer to the relevant information describing the quality of the corresponding initial image.

[0071] It can be understood that the image description information corresponding to multiple initial images may be different. When determining the image description information of multiple initial images respectively based on the corresponding multiple target light source information, it can provide a reliable reference basis for subsequent determination of multiple image enhancement methods, effectively improving the image enhancement processing effect.

[0072] S205: Determine multiple image enhancement methods respectively corresponding to the corresponding multiple target light source information according to the multiple image description information.

[0073] Among them, the image enhancement method refers to an image processing method for the corresponding initial image generated based on the image description information, and can be used to indicate the enhancement processing process corresponding to multiple initial images.

[0074] In some embodiments, when determining multiple image enhancement methods respectively corresponding to the corresponding multiple target light source information according to the multiple image description information, it may be to pre-determine multiple preset image description information respectively based on the multiple target light source information, and the preset image description information meets the image synthesis requirements, and then analyze and compare the multiple image description information with the corresponding preset image description information, and determine the corresponding image enhancement method according to the analysis and comparison results.

[0075] In other embodiments, when determining multiple image enhancement methods respectively corresponding to the corresponding multiple target light source information according to the multiple image description information, it may also be to input the multiple image description information and the corresponding target light source information into a pre-trained machine learning model respectively to obtain the corresponding image enhancement method.

[0076] Or, any other possible method may also be used to determine multiple image enhancement methods respectively corresponding to the corresponding multiple target light source information according to the multiple image description information, and this is not limited.

[0077] In the embodiments of the present disclosure, by determining multiple image enhancement methods respectively corresponding to the corresponding multiple target light source information according to the multiple image description information, the applicability of the obtained multiple image enhancement methods can be ensured, thereby providing a reliable execution basis for the enhancement processing process of the initial image.

[0078] S206: Perform enhancement processing on the corresponding multiple initial images respectively according to the multiple image enhancement methods to obtain multiple images to be synthesized.

[0079] In the embodiments of the present disclosure, when enhancing corresponding multiple initial images according to multiple image enhancement methods to obtain multiple images to be synthesized, it may be to configure corresponding image enhancement devices according to multiple image enhancement methods, and then use the multiple image enhancement devices to process the corresponding initial images respectively to obtain multiple images to be synthesized. Alternatively, it may also be to input multiple image enhancement methods and corresponding multiple initial images into a pre-trained image enhancement model respectively to obtain multiple images to be synthesized, and there is no limitation thereto.

[0080] For example, for initial images corresponding to different target light source information, appropriate image enhancement methods can be selected correspondingly, such as edge extraction, Gaussian filtering, etc. For infrared light images, gray information enhancement can be adopted; for visible light images, edge feature enhancement can be adopted; for filter images, chromaticity information enhancement can be adopted.

[0081] That is to say, in the embodiments of the present disclosure, after determining multiple target light source information according to multiple target image features, multiple image description information of multiple initial images respectively based on the corresponding multiple target light source information can be determined. According to the multiple image description information, multiple image enhancement methods respectively corresponding to the corresponding multiple target light source information are determined. The corresponding multiple initial images are enhanced respectively according to the multiple image enhancement methods to obtain multiple images to be synthesized. Thus, the corresponding image enhancement methods can be determined quickly and accurately based on the multiple image description information, and the flexible determination of the image enhancement methods corresponding to multiple initial images can be realized, thereby effectively improving the applicability of the obtained image enhancement methods to the initial images.

[0082] S207: Determine reference synthesis parameters according to multiple target light source information.

[0083] Among them, the reference synthesis parameters refer to the parameter information of multiple images to be synthesized during the image synthesis process. For example, it can be the brightness contrast of two adjacent images to be synthesized in the splicing area.

[0084] In the embodiments of the present disclosure, by determining reference synthesis parameters according to multiple target light source information, the obtained reference synthesis parameters can provide a reliable reference basis for the subsequent image synthesis process, so as to determine corresponding image synthesis strategies according to different reference synthesis parameters.

[0085] S208: Synthesize multiple images to be synthesized according to the reference synthesis parameters to obtain a target image.

[0086] For example, as Figure 3 shown, Figure 3 is an image synthesis schematic diagram proposed by the embodiments of the present disclosure. Among them, 9 images to be synthesized can adopt corresponding image splicing strategies based on the reference synthesis parameters to obtain the target image on the right.

[0087] That is to say, after the embodiments of the present disclosure perform enhancement processing on a corresponding plurality of initial images according to a plurality of image enhancement methods respectively to obtain a plurality of images to be synthesized, the reference synthesis parameters can be determined according to a plurality of target light source information, and then the plurality of images to be synthesized can be synthesized according to the reference synthesis parameters to obtain a target image. Thus, when synthesizing the plurality of images to be synthesized based on the reference synthesis parameters, the applicability of the image synthesis process to the plurality of images to be synthesized can be effectively improved, and the flexibility in the image synthesis process can be effectively improved to adapt to personalized application scenarios.

[0088] In this embodiment, by determining a plurality of target image features respectively corresponding to the plurality of initial images, and determining a plurality of target light source information according to the plurality of target image features, since the target image features corresponding to different target light source information may be different, when determining the plurality of target light source information according to the plurality of target image features respectively, an accurate judgment of the target light source information can be achieved, and the reliability of the obtained target light source information can be effectively improved. By obtaining a plurality of reference light source information, determining a plurality of reference image features respectively corresponding to the plurality of reference light source information, determining the reference image features matching the target image features from the plurality of reference image features, and using the reference light source information corresponding to the matching reference image features as the target light source information to obtain a plurality of target light source information. Thus, the plurality of reference image features can provide a reliable reference basis for the determination process of the target light source information. When performing matching processing based on the plurality of reference image features and the target image features, the working efficiency and accuracy of determining the plurality of target light source information can be effectively improved. By determining a plurality of image description information of the plurality of initial images respectively based on the corresponding plurality of target light source information, determining a plurality of image enhancement methods respectively corresponding to the corresponding plurality of target light source information according to the plurality of image description information, and performing enhancement processing on the corresponding plurality of initial images according to the plurality of image enhancement methods respectively to obtain a plurality of images to be synthesized. Thus, the corresponding image enhancement methods can be determined quickly and accurately based on the plurality of image description information, and the flexible determination of the image enhancement methods corresponding to the plurality of initial images can be realized, thereby effectively improving the applicability of the obtained image enhancement methods to the initial images. By determining the reference synthesis parameters according to the plurality of target light source information, and then synthesizing the plurality of images to be synthesized according to the reference synthesis parameters to obtain a target image. Thus, when synthesizing the plurality of images to be synthesized based on the reference synthesis parameters, the applicability of the image synthesis process to the plurality of images to be synthesized can be effectively improved, and the flexibility in the image synthesis process can be effectively improved to adapt to personalized application scenarios.

[0089] Figure 4 It is a schematic diagram according to the third embodiment of the present disclosure.

[0090] Such as Figure 4As shown, the image synthesis method includes:

[0091] S401: Obtain multiple initial images.

[0092] S402: Determine multiple target light source information respectively corresponding to the multiple initial images.

[0093] S403: Determine multiple image description information of the multiple initial images respectively based on the corresponding multiple target light source information.

[0094] For the descriptions of S401 - S403, specific reference can be made to the above - mentioned embodiments, which will not be elaborated here.

[0095] S404: According to the multiple image description information, determine multiple to - be - enhanced feature information respectively corresponding to the corresponding multiple target light source information.

[0096] Among them, the to - be - enhanced feature information refers to the feature information to be enhanced among the multiple feature information of the initial image.

[0097] It can be understood that the image features corresponding to different light source information may be different. When determining multiple to - be - enhanced feature information respectively corresponding to the corresponding multiple target light source information according to the multiple image description information, the accurate positioning of the to - be - enhanced features in the initial image can be realized, providing a reliable analysis object for determining the image enhancement method subsequently.

[0098] S405: According to the multiple to - be - enhanced feature information, determine corresponding multiple image enhancement methods respectively.

[0099] That is to say, after the embodiments of the present disclosure determine multiple image description information of the multiple initial images respectively based on the corresponding multiple target light source information, they can determine multiple to - be - enhanced feature information respectively corresponding to the corresponding multiple target light source information according to the multiple image description information, and then determine corresponding multiple image enhancement methods according to the multiple to - be - enhanced feature information. Since the to - be - enhanced feature information of different initial images may be different, when determining corresponding multiple image enhancement methods based on the multiple to - be - enhanced feature information respectively, the pertinence in the execution process of the multiple image enhancement methods can be effectively improved, so as to accurately and quickly realize the enhancement process for the multiple to - be - enhanced feature information.

[0100] S406: Obtain multiple calibration images respectively corresponding to the multiple target light source information.

[0101] Among them, the calibration image refers to an image that can be used as an image to be synthesized for image synthesis operations.

[0102] In the embodiments of the present disclosure, multiple reference images to be synthesized corresponding to multiple target light source information can be obtained in advance, and the reference images to be synthesized can be used as calibration images, so as to perform feature analysis between the initial image and the calibration image, and perform an image enhancement process according to the analysis result.

[0103] S407: Extract multiple calibration image features from the corresponding multiple calibration images respectively based on multiple image enhancement methods.

[0104] The calibration image feature refers to the image feature corresponding to the calibration image. The calibration image feature can meet the feature requirements corresponding to the image to be synthesized.

[0105] In the embodiments of the present disclosure, when multiple calibration image features are extracted from the corresponding multiple calibration images respectively based on multiple image enhancement methods, the calibration image features can be used as reference standards to effectively indicate the subsequent enhancement process.

[0106] S408: Perform enhancement processing on the corresponding multiple initial images respectively according to the multiple calibration image features to obtain multiple images to be synthesized.

[0107] In some embodiments, when performing enhancement processing on the corresponding multiple initial images respectively according to the multiple calibration image features to obtain multiple images to be synthesized, corresponding feature thresholds can be configured based on the multiple calibration image features, and then the multiple initial images can be enhanced based on the feature thresholds to obtain multiple images to be synthesized.

[0108] In other embodiments, when performing enhancement processing on the corresponding multiple initial images respectively according to the multiple calibration image features to obtain multiple images to be synthesized, an enhancement multiple can also be configured in advance, the initial image can be enhanced based on the enhancement multiple, and the image features after processing can be obtained. Then, the image features are analyzed and compared with the calibration image features to determine whether to enhance the initial image again based on the enhancement multiple. When the image features after processing meet the relevant information of the calibration image features, the image obtained at this time is determined as the image to be synthesized.

[0109] Alternatively, any other possible method can also be used to perform enhancement processing on the corresponding multiple initial images respectively according to the multiple calibration image features to obtain multiple images to be synthesized, such as engineering or mathematical methods, which are not limited herein.

[0110] Optionally, in some embodiments, when enhancing corresponding multiple initial images respectively according to multiple calibrated image features to obtain multiple images to be synthesized, it may be based on multiple image enhancement methods to extract multiple image features to be enhanced from the corresponding multiple initial images respectively, enhance the corresponding multiple image features to be enhanced according to the multiple calibrated image features respectively to obtain multiple target image features, and describe the corresponding multiple initial images according to the multiple target image features to obtain multiple images to be synthesized. Thus, accurate positioning of multiple image features to be enhanced in the initial images can be achieved, and the image features to be enhanced can be accurately enhanced based on the calibrated image features, thereby avoiding wasting resources by enhancing other features in the initial images and effectively improving the resource utilization rate in the image synthesis process.

[0111] Among them, the image feature to be enhanced refers to the image feature to be enhanced among the multiple image features of the initial image.

[0112] That is to say, after respectively determining corresponding multiple image enhancement methods according to multiple pieces of information on features to be enhanced in the embodiments of the present disclosure, multiple calibrated images respectively corresponding to multiple target light source information can be obtained, multiple calibrated image features are extracted from the corresponding multiple calibrated images respectively based on the multiple image enhancement methods, and the corresponding multiple initial images are enhanced according to the multiple calibrated image features respectively to obtain multiple images to be synthesized. Thus, the multiple calibrated image features can provide an accurate reference standard for the enhancement process of the initial images, effectively improving the reliability of the enhancement process corresponding to the initial images and enhancing the representational clarity of the obtained images to be synthesized for their own image features.

[0113] S409: Synthesize multiple images to be synthesized to obtain a target image.

[0114] For the description of S409, specific reference may be made to the above embodiments, which will not be elaborated here.

[0115] For example, as Figure 5 shown, Figure 5It is a schematic diagram of an image synthesis process proposed in an embodiment of the present disclosure. Among them, while obtaining the initial image, the image stitching coordinates corresponding to the initial image can be obtained to assist the subsequent image stitching process; judging the target light source information of the initial image can be divided into three types: visible light, infrared light, and filter; judging the image quality of the initial image to determine the image enhancement method for the initial image; the data enhancement module performs enhancement processing on the initial image based on the image enhancement method and the target light source information to obtain the image to be synthesized; the correction module selects different mapping matrices according to the target light source information to perform correction processing on the image to be synthesized; the puzzle module performs stitching processing on multiple images to be synthesized after the correction processing to obtain the target image, and the stitching process can flexibly select the stitching strategy based on the reference synthesis parameters.

[0116] In this embodiment, by determining multiple to-be-enhanced feature information respectively corresponding to multiple target light source information according to multiple image description information, and then determining multiple corresponding image enhancement methods according to the multiple to-be-enhanced feature information. Since the to-be-enhanced feature information of different initial images may be different, when determining multiple corresponding image enhancement methods based on the multiple to-be-enhanced feature information, the pertinence of the multiple image enhancement methods during execution can be effectively improved, so as to accurately and quickly implement the enhancement processing process for the multiple to-be-enhanced feature information. By obtaining multiple calibration images respectively corresponding to the multiple target light source information, extracting multiple calibration image features from the corresponding multiple calibration images based on the multiple image enhancement methods, and performing enhancement processing on the corresponding multiple initial images according to the multiple calibration image features to obtain multiple images to be synthesized. Thus, the multiple calibration image features can provide an accurate reference standard for the enhancement processing process of the initial image, effectively improving the reliability of the enhancement processing process corresponding to the initial image and enhancing the clarity of the obtained image to be synthesized for its own image features. By extracting multiple to-be-enhanced image features from the corresponding multiple initial images based on the multiple image enhancement methods, performing enhancement processing on the corresponding multiple to-be-enhanced image features according to the multiple calibration image features to obtain multiple target image features, and performing description processing on the corresponding multiple initial images according to the multiple target image features to obtain multiple images to be synthesized. Thus, the accurate positioning of multiple to-be-enhanced image features in the initial image can be realized, and the to-be-enhanced image features can be accurately enhanced based on the calibration image features, thereby avoiding wasting resources by enhancing other features in the initial image and effectively improving the resource utilization rate in the image synthesis process.

[0117] Figure 6 It is a schematic diagram according to the fourth embodiment of the present disclosure.

[0118] As Figure 6 shown, the image synthesis device 60 includes:

[0119] An acquisition module 601, configured to acquire a plurality of initial images;

[0120] A determination module 602, configured to determine a plurality of target light source information respectively corresponding to the plurality of initial images;

[0121] A first processing module 603, configured to process the corresponding plurality of initial images respectively according to the plurality of target light source information to obtain a plurality of images to be synthesized; and

[0122] A second processing module 604, configured to synthesize the plurality of images to be synthesized to obtain a target image.

[0123] In some embodiments of the present disclosure, as Figure 7 shown, Figure 7 is a schematic diagram according to the fifth embodiment of the present disclosure. The image synthesis device 70 includes: an acquisition module 701, a determination module 702, a first processing module 703, and a second processing module 704. Among them, the determination module 702 includes:

[0124] A first determination sub-module 7021, configured to determine a plurality of target image features respectively corresponding to the plurality of initial images;

[0125] A second determination sub-module 7022, configured to respectively determine a plurality of target light source information according to the plurality of target image features.

[0126] In some embodiments of the present disclosure, wherein the second determination sub-module 7022 is specifically configured to:

[0127] Acquire a plurality of reference light source information;

[0128] Determine a plurality of reference image features respectively corresponding to the plurality of reference light source information;

[0129] Determine the reference image features matching the target image features from the plurality of reference image features; and

[0130] Use the reference light source information corresponding to the matching reference image features as the target light source information to obtain a plurality of target light source information.

[0131] In some embodiments of the present disclosure, wherein the first processing module 703 includes:

[0132] A third determination sub-module 7031, configured to determine a plurality of image description information of the plurality of initial images respectively based on the corresponding plurality of target light source information;

[0133] A fourth determination sub-module 7032, configured to determine a plurality of image enhancement methods respectively corresponding to the corresponding plurality of target light source information according to the plurality of image description information; and

[0134] The processing sub-module 7033 is configured to perform enhancement processing on corresponding multiple initial images respectively according to multiple image enhancement methods to obtain multiple images to be synthesized.

[0135] In some embodiments of the present disclosure, the fourth determination sub-module 7032 is specifically configured to:

[0136] Determine multiple to-be-enhanced feature information respectively corresponding to the corresponding multiple target light source information according to the multiple image description information;

[0137] Determine corresponding multiple image enhancement methods respectively according to the multiple to-be-enhanced feature information.

[0138] In some embodiments of the present disclosure, the processing sub-module 7033 is specifically configured to:

[0139] Obtain multiple calibration images respectively corresponding to the multiple target light source information;

[0140] Extract multiple calibration image features respectively from the corresponding multiple calibration images based on the multiple image enhancement methods; and

[0141] Perform enhancement processing on the corresponding multiple initial images respectively according to the multiple calibration image features to obtain multiple images to be synthesized.

[0142] In some embodiments of the present disclosure, the processing sub-module 7033 is further configured to:

[0143] Extract multiple to-be-enhanced image features respectively from the corresponding multiple initial images based on the multiple image enhancement methods;

[0144] Perform enhancement processing on the corresponding multiple to-be-enhanced image features respectively according to the multiple calibration image features to obtain multiple target image features; and

[0145] Perform description processing on the corresponding multiple initial images according to the multiple target image features to obtain multiple images to be synthesized.

[0146] In some embodiments of the present disclosure, the second processing module 704 is specifically configured to:

[0147] Determine reference synthesis parameters according to the multiple target light source information;

[0148] Synthesize the multiple images to be synthesized according to the reference synthesis parameters to obtain a target image.

[0149] It can be understood that this embodiment attaches Figure 7The image synthesis device 70 in [the present embodiment] and the image synthesis device 60 in the above-mentioned embodiment, the acquisition module 701 and the acquisition module 601 in the above-mentioned embodiment, the determination module 702 and the determination module 602 in the above-mentioned embodiment, the first processing module 703 and the first processing module 603 in the above-mentioned embodiment, and the second processing module 704 and the second processing module 604 in the above-mentioned embodiment may have the same functions and structures.

[0150] It should be noted that the foregoing explanations of the image synthesis method also apply to the image synthesis device in this embodiment.

[0151] In this embodiment, by acquiring a plurality of initial images, determining a plurality of target light source information respectively corresponding to the plurality of initial images, processing the corresponding plurality of initial images respectively according to the plurality of target light source information to obtain a plurality of images to be synthesized, and synthesizing the plurality of images to be synthesized to obtain a target image. Thus, corresponding optimization processing is realized according to the target light source information of the initial images. When the target image is synthesized based on the processed images to be synthesized, the quality of image synthesis can be effectively improved, and the representation accuracy of the relevant information of the plurality of initial images in the obtained target image can be improved.

[0152] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0153] Figure 8 A schematic block diagram of an exemplary electronic device that can be used to implement the image synthesis method of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0154] As Figure 8 shown, the device 800 includes a computing unit 801, which can execute various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0155] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as a keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as a disk, optical disc, etc.; and communication unit 809, such as a network card, modem, wireless communication transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0156] Computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit 801 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 801 executes the various methods and processes described above, such as executing an image synthesis method. For example, in some embodiments, executing the image synthesis method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by computing unit 801, one or more steps of the image synthesis method described above can be executed. Alternatively, in other embodiments, computing unit 801 can be configured to execute the image synthesis method in any other suitable way (e.g., by means of firmware).

[0157] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0158] The program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine, or entirely on the remote machine or server.

[0159] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0160] In order to provide interaction with a user, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0161] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected with each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), the Internet, and blockchain network.

[0162] A computer system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS"). The server can also be a server of a distributed system, or a server combined with blockchain.

[0163] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this is not limited herein.

[0164] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. An image synthesis method, comprising: Obtaining a plurality of initial images; Determining a plurality of target light source information respectively corresponding to the plurality of initial images; Determining a plurality of image description information of the plurality of initial images respectively based on the corresponding plurality of target light source information; Determining a plurality of image enhancement methods respectively corresponding to the corresponding plurality of target light source information according to the plurality of image description information; And Obtaining a plurality of calibration images respectively corresponding to the plurality of target light source information, where the calibration images are images that can be used as images to be synthesized for image synthesis operations; Extracting a plurality of calibration image features from the corresponding plurality of calibration images respectively based on the plurality of image enhancement methods, where the calibration image features are image features that the calibration images meet the feature requirements of the image to be synthesized; And Enhancing a plurality of image features to be enhanced of the corresponding plurality of initial images respectively according to the plurality of calibration image features to obtain a plurality of images to be synthesized; and Synthesizing the plurality of images to be synthesized to obtain a target image.

2. The method according to claim 1, wherein The determining the plurality of target light source information respectively corresponding to the plurality of initial images includes: Determining a plurality of target image features respectively corresponding to the plurality of initial images; Respectively determining the plurality of target light source information according to the plurality of target image features.

3. The method according to claim 2, wherein, The respectively determining the plurality of target light source information according to the plurality of target image features includes: Obtaining a plurality of reference light source information pre-configured for the initial images; Determining a plurality of reference image features respectively corresponding to the plurality of reference light source information; Determining a reference image feature matching the target image feature from the plurality of reference image features; and Using the reference light source information corresponding to the matched reference image feature as the target light source information to obtain the plurality of target light source information.

4. The method according to claim 1, wherein The determining the plurality of image enhancement methods respectively corresponding to the corresponding plurality of target light source information according to the plurality of image description information includes: Determining a plurality of feature information to be enhanced respectively corresponding to the corresponding plurality of target light source information according to the plurality of image description information; Respectively determining the corresponding plurality of image enhancement methods according to the plurality of feature information to be enhanced.

5. The method according to claim 1, wherein, The enhancing a plurality of image features to be enhanced of the corresponding plurality of initial images respectively according to the plurality of calibration image features to obtain the plurality of images to be synthesized includes: Extracting a plurality of image features to be enhanced from the corresponding plurality of initial images respectively based on the plurality of image enhancement methods; Enhancing the corresponding plurality of image features to be enhanced respectively according to the plurality of calibration image features to obtain a plurality of target image features; and Describing the corresponding plurality of initial images according to the plurality of target image features to obtain the plurality of images to be synthesized.

6. The method according to claim 1, wherein The synthesizing the plurality of images to be synthesized to obtain a target image includes: Determining reference synthesis parameters according to the plurality of target light source information; Synthesizing the plurality of images to be synthesized according to the reference synthesis parameters to obtain the target image.

7. An image synthesis device, comprising: An obtaining module for obtaining a plurality of initial images; A determination module, configured to determine a plurality of target light source information respectively corresponding to the plurality of initial images; A first processing module, configured to process the corresponding plurality of initial images respectively according to the plurality of target light source information to obtain a plurality of images to be synthesized; And A second processing module, configured to synthesize the plurality of images to be synthesized to obtain a target image; The first processing module includes: A third determination sub-module, configured to determine a plurality of image description information of the plurality of initial images respectively based on the corresponding plurality of target light source information; A fourth determination sub-module, configured to determine a plurality of image enhancement methods respectively corresponding to the corresponding plurality of target light source information according to the plurality of image description information; and A processing sub-module, configured to perform enhancement processing on the corresponding plurality of initial images respectively according to the plurality of image enhancement methods to obtain the plurality of images to be synthesized; The processing sub-module is specifically configured to: Obtain a plurality of calibration images respectively corresponding to the plurality of target light source information, where the calibration images are images that can be used as images to be synthesized for image synthesis operations; Extract a plurality of calibration image features from the corresponding plurality of calibration images respectively based on the plurality of image enhancement methods, where the calibration image features are image features that the calibration images meet the feature requirements of the images to be synthesized; and Perform enhancement processing on the plurality of to-be-enhanced image features of the corresponding plurality of initial images respectively according to the plurality of calibration image features to obtain the plurality of images to be synthesized.

8. The apparatus according to claim 7, wherein The determination module includes: A first determination sub-module, configured to determine a plurality of target image features respectively corresponding to the plurality of initial images; A second determination sub-module, configured to respectively determine the plurality of target light source information according to the plurality of target image features.

9. The device according to claim 8, wherein, The second determination sub-module is specifically configured to: Obtain a plurality of reference light source information pre-configured for the initial images; Determine a plurality of reference image features respectively corresponding to the plurality of reference light source information; Determine the reference image features matching the target image features from the plurality of reference image features; And Use the reference light source information corresponding to the matching reference image features as the target light source information to obtain the plurality of target light source information.

10. The apparatus according to claim 7, wherein The fourth determination sub-module is specifically configured to: Determine a plurality of to-be-enhanced feature information respectively corresponding to the corresponding plurality of target light source information according to the plurality of image description information; Determine the corresponding plurality of image enhancement methods respectively according to the plurality of to-be-enhanced feature information.

11. The apparatus according to claim 7, wherein, The processing sub-module is further configured to: Extract a plurality of to-be-enhanced image features from the corresponding plurality of initial images respectively based on the plurality of image enhancement methods; Perform enhancement processing on the corresponding plurality of to-be-enhanced image features respectively according to the plurality of calibration image features to obtain a plurality of target image features; And Perform description processing on the corresponding plurality of initial images according to the plurality of target image features to obtain the plurality of images to be synthesized.

12. The device according to claim 7, wherein, The second processing module is specifically configured to: Determine reference synthesis parameters according to the plurality of target light source information; Synthesize the plurality of images to be synthesized according to the reference synthesis parameters to obtain the target image.

13. An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

15. A computer program product, comprising a computer program which, when executed by a processor, implements the steps of the method according to any one of claims 1-6.

Images