Method, apparatus, device, and storage medium for determining a fused image

Abstract

Embodiments of the present application disclose a method, apparatus, device, and storage medium for determining a fused image. Related embodiments can be applied to fields such as maps, intelligent transportation, artificial intelligence, etc., to improve the efficiency of image fusion. The method of the embodiments of the present application includes: obtaining the first kernel energy value of the first image to be fused and the first kernel energy value of the second image to be fused based on the first kernel energy function, obtaining the second kernel energy value in the first image to be fused and the second kernel energy value of the second image to be fused based on the second kernel energy function, obtaining the third kernel energy value of the first image to be fused and the third kernel energy value of the second image to be fused based on the third kernel energy function, and obtaining the target fused image based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the second image to be fused.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of image data processing, and in particular, to a method, apparatus, device, and storage medium for determining a fused image. Background Art

[0002] In an industrial scenario, when photographing large objects, it is desired to obtain an image in which all areas of the object are clear. However, due to the limitation of the depth of field of the camera lens, it is impossible to focus on all areas simultaneously, resulting in only some areas being clear and other areas being blurred in the captured photo. The multi-focus image fusion technology can fuse multiple images with different focused areas in the same scene into a fully clear image, thereby effectively solving this problem and improving the information utilization rate of the image. However, how to accurately identify the clear areas in the image is a key issue in the multi-focus image fusion algorithm.

[0003] In existing traditional multi-focus image fusion algorithms, a regional energy function combined with multiple different operators is usually used to calculate the regional energy for image fusion. However, the calculation of the regional energy function combined with multiple different operators is complex, the decision conditions are designed complexly, and at the same time, the influence between multiple operators needs to be balanced, resulting in an increased amount of calculation and a long calculation time, thereby reducing the efficiency of image fusion. Summary of the Invention

[0004] The embodiments of the present application provide a method, apparatus, device, and storage medium for determining a fused image, which are used to obtain corresponding energy values based on a first kernel energy function, a second kernel energy function, and a third kernel energy function, and use a regional energy operator composed of multi-kernel functions with different kernel sizes for a first image to be fused and a second image to be fused, so as to sense the mutual dependence relationship of pixels within different regional sizes, and the same operator is used, and the decision conditions of the target fusion strategy are simple, without considering the influence of multiple operators, and can greatly reduce the amount of calculation and shorten the calculation time, thereby improving the efficiency of image fusion.

[0005] On the one hand, the embodiments of the present application provide a method for determining a fused image, including:

[0006] Obtain a first image to be fused and a second image to be fused, where the first image to be fused and the second image to be fused are images obtained by focusing on different regions of the same target object;

[0007] Based on the first kernel energy function, perform energy calculation on the first image to be fused and the second image to be fused respectively, to obtain a first kernel energy value corresponding to each pixel point in the first image to be fused, and a first kernel energy value corresponding to each pixel point in the second image to be fused;

[0008] Based on the second kernel energy function, energy calculations are respectively performed on the first image to be fused and the second image to be fused, to obtain the second kernel energy value corresponding to each pixel point in the first image to be fused, and the second kernel energy value corresponding to each pixel point in the second image to be fused, where the kernel size of the second kernel energy function is larger than the kernel size of the first kernel energy function;

[0009] Based on the third kernel energy function, energy calculations are respectively performed on the first image to be fused and the second image to be fused, to obtain the third kernel energy value corresponding to each pixel point in the first image to be fused, and the third kernel energy value corresponding to each pixel point in the second image to be fused, where the kernel size of the third kernel energy function is larger than the kernel size of the second kernel energy function;

[0010] Based on the target scene corresponding to the target object, a target fusion strategy is determined, where the target fusion strategy includes a high-contrast fusion strategy and a smooth fusion strategy;

[0011] Based on the target fusion strategy, the first kernel energy values, the second kernel energy values, and the third kernel energy values corresponding to each pixel point in the first image to be fused, and the first kernel energy values, the second kernel energy values, and the third kernel energy values corresponding to each pixel point in the second image to be fused, the first image to be fused and the second image to be fused are fused to obtain a target fusion image.

[0012] On the other hand, the present application provides a determining device for a fused image, including:

[0013] An acquisition unit, configured to acquire the first image to be fused and the second image to be fused, where the first image to be fused and the second image to be fused are images obtained by focusing on different regions of the same target object;

[0014] A processing unit, configured to respectively perform energy calculations on the first image to be fused and the second image to be fused based on the first kernel energy function, to obtain the first kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value corresponding to each pixel point in the second image to be fused;

[0015] The processing unit is further configured to respectively perform energy calculations on the first image to be fused and the second image to be fused based on the second kernel energy function, to obtain the second kernel energy value corresponding to each pixel point in the first image to be fused, and the second kernel energy value corresponding to each pixel point in the second image to be fused, where the kernel size of the second kernel energy function is larger than the kernel size of the first kernel energy function;

[0016] The processing unit is further configured to calculate the energy of the first image to be fused and the second image to be fused respectively based on the third kernel energy function, so as to obtain the third kernel energy value corresponding to each pixel point in the first image to be fused and the third kernel energy value corresponding to each pixel point in the second image to be fused, where the kernel size of the third kernel energy function is larger than the kernel size of the second kernel energy function;

[0017] The determining unit is configured to determine a target fusion strategy based on the target scene corresponding to the target object, where the target fusion strategy includes a high-contrast fusion strategy and a smooth fusion strategy;

[0018] The processing unit is further configured to fuse the first image to be fused and the second image to be fused based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the second image to be fused, so as to obtain a target fusion image.

[0019] In a possible design, in an implementation manner of another aspect of the embodiments of the present application, the processing unit may specifically be configured to:

[0020] When the target fusion strategy is a high-contrast fusion strategy, fusing the first image to be fused and the second image to be fused based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the second image to be fused, to obtain a target fusion image, including:

[0021] Based on the first fusion weight, perform weighted summation on the first kernel energy value, the second kernel energy value, and the third kernel energy value corresponding to each pixel point in the first image to be fused, so as to obtain the first total energy value corresponding to each pixel point in the first image to be fused;

[0022] Based on the first fusion weight, perform weighted summation on the first kernel energy value, the second kernel energy value, and the third kernel energy value corresponding to each pixel point in the second image to be fused, so as to obtain the second total energy value corresponding to each pixel point in the second image to be fused;

[0023] For each pixel point, select the larger energy value from the first total energy value and the second total energy value, and use the pixel point corresponding to the larger energy value as the target pixel point to obtain the target fusion image.

[0024] In a possible design, in an implementation manner of another aspect of the embodiments of the present application, the processing unit may specifically be configured to:

[0025] Obtain a first mask image based on the first kernel energy value corresponding to each pixel point in the first image to be fused and the first kernel energy value corresponding to each pixel point in the second image to be fused;

[0026] Obtain a second mask image based on the second kernel energy value corresponding to each pixel point in the first image to be fused and the second kernel energy value corresponding to each pixel point in the second image to be fused;

[0027] Obtain a third mask image based on the third kernel energy value corresponding to each pixel point in the first image to be fused and the third kernel energy value corresponding to each pixel point in the second image to be fused;

[0028] Obtain a target fused image based on the second fusion weight, the first mask image, the second mask image, and the third mask image.

[0029] In a possible design, in an implementation manner of another aspect of the embodiments of the present application, the processing unit may specifically be used for:

[0030] For each pixel point, select the energy value with the larger value from the first kernel energy value of the first image to be fused and the first kernel energy value of the second image to be fused;

[0031] For each pixel point, use the pixel point corresponding to the energy value with the larger value as the first mask pixel point to obtain the first mask image.

[0032] In a possible design, in an implementation manner of another aspect of the embodiments of the present application, the processing unit may specifically be used for:

[0033] For each pixel point, select the energy value with the larger value from the second kernel energy value of the first image to be fused and the second kernel energy value of the second image to be fused;

[0034] For each pixel point, use the pixel point corresponding to the energy value with the larger value as the second mask pixel point to obtain the second mask image.

[0035] In a possible design, in an implementation manner of another aspect of the embodiments of the present application, the processing unit may specifically be used for:

[0036] For each pixel point, select the energy value with the larger value from the third kernel energy value of the first image to be fused and the third kernel energy value of the second image to be fused;

[0037] For each pixel point, use the pixel point corresponding to the energy value with the larger value as the third mask pixel point to obtain the third mask image.

[0038] In a possible design, in an implementation manner of another aspect of the embodiments of the present application, the processing unit may specifically be used for

[0039] The processing unit is further configured to calculate the energy of the third image to be fused and the target fused image respectively based on the first kernel energy function, so as to obtain the first kernel energy value corresponding to each pixel point in the third image to be fused, and the first kernel energy value corresponding to each pixel point in the target fused image;

[0040] The processing unit is further configured to calculate the energy of the third image to be fused and the target fused image respectively based on the second kernel energy function, so as to obtain the second kernel energy value corresponding to each pixel point in the third image to be fused, and the second kernel energy value corresponding to each pixel point in the target fused image;

[0041] The processing unit is further configured to calculate the energy of the first image to be fused and the second image to be fused respectively based on the third kernel energy function, so as to obtain the third kernel energy value corresponding to each pixel point in the first image to be fused, and the third kernel energy value corresponding to each pixel point in the second image to be fused;

[0042] The processing unit is further configured to fuse the third image to be fused and the target fused image based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the third image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the target fused image, so as to obtain a fused image.

[0043] In a possible design, in an implementation manner of another aspect of this application embodiment,

[0044] The determining unit is further configured to determine the first kernel size, the second kernel size, and the third kernel size based on the image sizes of the first image to be fused and the second image to be fused, and the object density of the target object, where the first kernel size is smaller than the second kernel size, and the second kernel size is smaller than the third kernel size;

[0045] The determining unit is further configured to determine the first kernel energy function based on the first kernel size;

[0046] The determining unit is further configured to determine the second kernel energy function based on the second kernel size;

[0047] The determining unit is further configured to determine the third kernel energy function based on the third kernel size.

[0048] Another aspect of this application provides a computer device, including: a memory, a processor, and a bus system;

[0049] Wherein, the memory is used to store programs;

[0050] The processor is configured to implement the methods in the above aspects when executing the programs in the memory;

[0051] A bus system is used to connect a memory and a processor to enable communication between the memory and the processor.

[0052] Another aspect of the present application provides a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to execute the methods in the above aspects.

[0053] From the above technical solutions, it can be seen that the embodiments of the present application have the following beneficial effects:

[0054] By obtaining the first kernel energy value of the first image to be fused and the first kernel energy value of the second image to be fused based on the first kernel energy function, obtaining the second kernel energy value in the first image to be fused and the second kernel energy value of the second image to be fused based on the second kernel energy function, obtaining the third kernel energy value of the first image to be fused and the third kernel energy value of the second image to be fused based on the third kernel energy function, and obtaining the target fused image based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the first image to be fused, the first kernel energy value, the second kernel energy value, and the third kernel energy value corresponding to each pixel point in the second image to be fused. In this way, on the one hand, based on the first kernel energy function, the second kernel energy function, and the third kernel energy function, a regional energy operator composed of multi-kernel functions with different kernel sizes can be used for the first image to be fused and the second image to be fused to obtain corresponding energy values and sense the mutual dependence relationship of pixels within different regional sizes. On the other hand, on the basis of using the same operator, a suitable target fusion strategy is matched for different target scenarios. The decision conditions of the target fusion strategy are simple and do not need to consider the influence of multiple operators, which can greatly reduce the amount of calculation and shorten the calculation time, thereby improving the efficiency of image fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a schematic architecture diagram of an image data control system in an embodiment of the present application;

[0056] Figure 2 is a flowchart of an embodiment of a method for determining a fused image in an embodiment of the present application;

[0057] Figure 3 is another flowchart of an embodiment of a method for determining a fused image in an embodiment of the present application;

[0058] Figure 4 is another flowchart of an embodiment of a method for determining a fused image in an embodiment of the present application;

[0059] Figure 5 is another flowchart of an embodiment of a method for determining a fused image in an embodiment of the present application;

[0060] Figure 6 is another flowchart of the method for determining a fused image in an embodiment of the present application;

[0061] Figure 7 is another flowchart of the method for determining a fused image in an embodiment of the present application;

[0062] Figure 8 is another flowchart of the method for determining a fused image in an embodiment of the present application;

[0063] Figure 9 is another flowchart of the method for determining a fused image in an embodiment of the present application;

[0064] Figure 10 is a schematic diagram of the principle process of the method for determining a fused image in an embodiment of the present application;

[0065] Figure 11 is a schematic diagram of an embodiment of the apparatus for determining a fused image in an embodiment of the present application;

[0066] Figure 12 is a schematic diagram of an embodiment of a computer device in an embodiment of the present application. Detailed implementation manners

[0067] The embodiments of the present application provide a method, an apparatus, a device and a storage medium for determining a fused image. Based on a first kernel energy function, a second kernel energy function and a third kernel energy function, a regional energy operator composed of multi-kernel functions with different kernel sizes is used for a first image to be fused and a second image to be fused to obtain corresponding energy values, so as to sense the mutual dependence relationship of pixels within different region sizes, and the same operator is used. The decision condition of the target fusion strategy is simple, without considering the influence of multiple operators, which can greatly reduce the calculation amount and shorten the calculation time, thereby improving the efficiency of image fusion.

[0068] The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of the present application are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "correspond to" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0069] It is understood that in the specific embodiments of the present application, data such as sample reaction data sets are involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards of relevant countries and regions.

[0070] It is understood that the method for determining the fused image disclosed in the present application is specifically related to the Intelligent Vehicle Infrastructure Cooperative Systems (IVICS). The following further introduces the Intelligent Vehicle Infrastructure Cooperative Systems. The Intelligent Vehicle Infrastructure Cooperative Systems, abbreviated as the vehicle-road collaborative system, is a development direction of the Intelligent Transportation System (ITS). The vehicle-road collaborative system uses advanced wireless communication and new-generation Internet technologies to comprehensively implement dynamic real-time information interaction between vehicles and between vehicles and roads, and on the basis of the collection and fusion of all-time and all-space dynamic traffic information, conducts active vehicle safety control and road collaborative management, fully realizing the effective collaboration of people, vehicles, and roads, ensuring traffic safety, improving traffic efficiency, and thus forming a safe, efficient, and environmentally friendly road traffic system.

[0071] It is understood that the method for determining the fused image disclosed in the present application is also related to Artificial Intelligence (AI) technology. The following further introduces the Artificial Intelligence technology. Artificial Intelligence is to use a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, a theory, method, technology, and application system that perceives the environment, acquires knowledge, and uses knowledge to obtain the best results. In other words, Artificial Intelligence is a comprehensive technology of computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial Intelligence also studies the design principles and implementation methods of various intelligent machines, enabling the machines to have the functions of perception, reasoning, and decision-making.

[0072] Artificial Intelligence technology is an interdisciplinary subject, involving a wide range of fields, including both hardware-level technologies and software-level technologies. The basic technologies of Artificial Intelligence generally include technologies such as sensors, dedicated Artificial Intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of Artificial Intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0073] Secondly, Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between humans and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, the research in this field will involve natural language, that is, the language people use in daily life, so it has a close connection with the research of linguistics. Natural language processing technologies usually include text processing, semantic understanding, machine translation, robot question answering, knowledge graph, and other technologies.

[0074] Secondly, Machine Learning (ML) is an interdisciplinary subject that involves multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications cover all fields of artificial intelligence. Machine learning and deep learning usually include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rote learning.

[0075] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in multiple fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless, autonomous driving, drones, robots, smart healthcare, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0076] It should be understood that the method for determining the fused image provided in this application can be applied to various scenarios, including but not limited to artificial intelligence, maps, intelligent transportation, cloud technology, satellite science, etc., for fusing images obtained by focusing on different regions of the same target object to obtain a fused image, which can be applied to scenarios such as industrial image analysis, remote synchronous operation, and intelligent retrieval.

[0077] To solve the above problems, this application proposes a method for determining a fused image, which is applied to Figure 1 the reaction data control system shown in Figure 1 , Figure 1 which is a schematic architecture diagram of the reaction data control system in the embodiment of this application. As shown in Figure 1As shown, the server obtains the first image to be fused and the second image to be fused provided by the terminal device. Based on the first kernel energy function, it obtains the first kernel energy value of the first image to be fused and the first kernel energy value of the second image to be fused. Based on the second kernel energy function, it obtains the second kernel energy value in the first image to be fused and the second kernel energy value of the second image to be fused. Based on the third kernel energy function, it obtains the third kernel energy value of the first image to be fused and the third kernel energy value of the second image to be fused. Based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the second image to be fused, it obtains the target fusion image. Through the above method, on the one hand, based on the first kernel energy function, the second kernel energy function, and the third kernel energy function, a regional energy operator composed of multi-kernel functions with different kernel sizes can be used for the first image to be fused and the second image to be fused to obtain the corresponding energy values and sense the interdependence of pixels within different regional sizes. On the other hand, on the basis of using the same operator, a suitable target fusion strategy is matched for different target scenarios. The decision conditions of the target fusion strategy are simple and do not need to consider the influence of multiple operators, which can greatly reduce the calculation amount and shorten the calculation time, thereby improving the efficiency of image fusion.

[0078] It can be understood that Figure 1 only one type of terminal device is shown. In actual scenarios, more types of terminal devices can participate in the data processing process. Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, etc. The specific quantity and types depend on the actual scenario and are not specifically limited here. Additionally, Figure 1 one server is shown, but in actual scenarios, multiple servers can also participate. Especially in the scenario of multi-model training interaction, the number of servers depends on the actual scenario and is not specifically limited here.

[0079] It should be noted that in this embodiment, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms. The terminal device and the server can be directly or indirectly connected through wired or wireless communication methods. The terminal device and the server can be connected to form a blockchain network, which is not limited in this application.

[0080] Combined with the above introduction, the method for determining the fused image in this application will be introduced below. Please refer to Figure 2 In one embodiment of the method for determining the fused image in the embodiments of this application, it includes:

[0081] In step S101, a first image to be fused and a second image to be fused are obtained, where the first image to be fused and the second image to be fused are images obtained by focusing on different regions of the same target object;

[0082] In this embodiment, due to the limitation of the depth of field of the camera lens, it is impossible to focus on all regions simultaneously during the process of photographing the target object, resulting in only some regions being clear and other regions being blurred in the captured photo. Therefore, in order to fuse multiple images with different focusing regions obtained by photographing the target object in the same scene into a fully clear image, thereby improving the information utilization rate of the image, the first image to be fused and the second image to be fused can be obtained first.

[0083] Among them, the first image to be fused and the second image to be fused are multiple images with different focusing regions obtained by photographing the same target object in the same scene. The target object can specifically be industrial objects such as robotic arms, drivers, etc., and can also be in other forms, such as a cardboard box, a book, or a desk, etc., which is not specifically limited here.

[0084] Specifically, in the same scene (such as a machine processing scene), through a photographing device (such as a camera or a scanner, etc.), different focusings on the same target object (such as a robotic arm, a driver, etc.) are performed to collect multiple images with different focusing regions, that is, the first image to be fused and the second image to be fused.

[0085] In step S102, based on the first kernel energy function, energy calculations are respectively performed on the first image to be fused and the second image to be fused to obtain the first kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value corresponding to each pixel point in the second image to be fused;

[0086] In this embodiment, after obtaining the first image to be fused and the second image to be fused, the first kernel energy function can be called to perform energy calculations on each pixel point in the first image to be fused and the second image to be fused respectively, and the first kernel energy value corresponding to each pixel point in the first image to be fused and the first kernel energy value corresponding to each pixel point in the second image to be fused can be obtained.

[0087] Among them, the first kernel energy function can specifically be expressed as a Gaussian blur function of a small kernel, or it can also be expressed as other functions, such as a Laplacian kernel function, and no specific limitation is made here. The Gaussian blur function of the small kernel refers to a Gaussian blur function with a relatively small size of the Gaussian convolution kernel or Gaussian template. The Gaussian blur function of the small kernel considers pixels in a small range, is friendly to details and small regions, but is sensitive to noise. When there is local highlighting, it is prone to the phenomenon of halos. The small kernel can usually be set to 5, or it can also be set to other smaller sizes such as 3, and no specific limitation is made here.

[0088] Specifically, as Figure 10 shown, after obtaining the first image to be fused and the second image to be fused, the first kernel energy function can be called (such as the use of the small kernel energy function shown in Figure 10), and the following formulas (1) and (2) can be used to calculate the energy of each pixel point in the first image to be fused respectively, so as to obtain the first kernel energy value corresponding to each pixel point in the first image to be fused, and calculate the energy of each pixel point in the second image to be fused, so as to obtain the first kernel energy value corresponding to each pixel point in the second image to be fused:

[0089]

[0090] G(m,n)=[G S (m,n),G M (m,n),G L (m,n)] (2);

[0091] Among them, (m, n) represents a pixel point in the first image to be fused or the second image to be fused, G S (m,n) represents the first kernel energy value corresponding to (m, n), G M (m,n) represents the second kernel energy value corresponding to (m, n), G L (m,n) represents the third kernel energy value corresponding to (m, n), σ S is a parameter corresponding to the kernel size of the first kernel energy function, and one kernel size corresponds to one σ.

[0092] In step S103, based on the second kernel energy function, the energy of the first image to be fused and the second image to be fused is calculated respectively, and the second kernel energy value corresponding to each pixel point in the first image to be fused and the second kernel energy value corresponding to each pixel point in the second image to be fused are obtained, where the kernel size of the second kernel energy function is larger than the kernel size of the first kernel energy function;

[0093] In this embodiment, after obtaining the first image to be fused and the second image to be fused, the second kernel energy function can be called to calculate the energy of each pixel in the first image to be fused and the second image to be fused respectively, so as to obtain the second kernel energy value corresponding to each pixel in the first image to be fused and the second kernel energy value corresponding to each pixel in the second image to be fused.

[0094] Among them, the second kernel energy function can specifically be expressed as a Gaussian blur function of the medium kernel, and can also be expressed as other functions, such as the Laplace kernel function, which is not specifically limited here. The Gaussian blur function of the medium kernel refers to a Gaussian blur function with a medium-sized Gaussian convolution kernel or Gaussian template. The Gaussian blur function of the medium kernel can resist noise to a certain extent and is friendly to the edge details of the image. The kernel size of the second kernel energy function is larger than the kernel size of the first kernel energy function, that is, the kernel size of the medium kernel is larger than that of the small kernel. The medium kernel can usually be set to 21, or can also be set to other medium sizes such as 18, which is not specifically limited here.

[0095] Specifically, as Figure 10 shown, after obtaining the first image to be fused and the second image to be fused, the second kernel energy function (such as the use of the medium kernel energy function shown in Figure 10) can be called, and the above formula (2) and the following formula (3) are used to calculate the energy of each pixel in the first image to be fused respectively, so as to obtain the second kernel energy value corresponding to each pixel in the first image to be fused, and to calculate the energy of each pixel in the second image to be fused, so as to obtain the second kernel energy value corresponding to each pixel in the second image to be fused:

[0096]

[0097] Among them, (m, n) represents a pixel in the first image to be fused or the second image to be fused, G S (m,n) represents the first kernel energy value corresponding to (m, n), G M (m,n) represents the second kernel energy value corresponding to (m, n), G L (m,n) represents the third kernel energy value corresponding to (m, n), σ M is a parameter corresponding to the kernel size of the second kernel energy function, and one kernel size corresponds to one σ.

[0098] In step S104, based on the third kernel energy function, the energy of the first image to be fused and the second image to be fused is calculated respectively, so as to obtain the third kernel energy value corresponding to each pixel in the first image to be fused and the third kernel energy value corresponding to each pixel in the second image to be fused, where the kernel size of the third kernel energy function is larger than the kernel size of the second kernel energy function;

[0099] In this embodiment, after obtaining the first image to be fused and the second image to be fused, the third kernel energy function can be called to calculate the energy of each pixel in the first image to be fused and the second image to be fused respectively, so as to obtain the third kernel energy value corresponding to each pixel in the first image to be fused and the third kernel energy value corresponding to each pixel in the second image to be fused.

[0100] Among them, the third kernel energy function can specifically be expressed as a Gaussian blur function of a large kernel, and can also be expressed as other functions, such as a Laplacian kernel function, which is not specifically limited here. The Gaussian blur function of a large kernel refers to a Gaussian blur function with a relatively large size of the Gaussian convolution kernel or Gaussian template. The Gaussian blur function of a large kernel is robust to noise, considers more regional pixels, and can adapt to local highlight situations, but the edge details of the fused image are relatively blurred. The kernel size of the third kernel energy function is larger than the kernel size of the second kernel energy function, that is, the kernel size of the large kernel is larger than that of the medium kernel. The large kernel can usually be set to 41, or can also be set to other larger sizes such as 37, which is not specifically limited here.

[0101] Specifically, as Figure 10 shown, after obtaining the first image to be fused and the second image to be fused, the third kernel energy function (such as the use of the large kernel energy function shown in Figure 10) can be called, Formula (2) above, and the following Formula (4) to calculate the energy of each pixel in the first image to be fused respectively, so as to obtain the third kernel energy value corresponding to each pixel in the first image to be fused, and calculate the energy of each pixel in the second image to be fused, so as to obtain the third kernel energy value corresponding to each pixel in the second image to be fused:

[0102]

[0103] Among them, (m, n) represents a pixel in the first image to be fused or the second image to be fused, G S (m,n) represents the first kernel energy value corresponding to (m, n), G M (m,n) represents the second kernel energy value corresponding to (m, n), G L (m,n) represents the third kernel energy value corresponding to (m, n), σ L is a parameter corresponding to the kernel size of the third kernel energy function, and one kernel size corresponds to one σ.

[0104] In step S105, based on the target scene corresponding to the target object, determine the target fusion strategy, where the target fusion strategy includes a high-contrast fusion strategy and a smooth fusion strategy;

[0105] In this embodiment, based on the first image to be fused and the second image to be fused, the target scene corresponding to the target object in the image can be obtained. Then, based on the mapping relationship between the target scene and the target fusion strategy, the target fusion strategy can be quickly indexed, so that subsequently, based on the obtained target fusion strategy, the first image to be fused and the second image to be fused can be fused to better and more quickly obtain a fused image that meets the requirements of the target scene.

[0106] Among them, the target fusion strategies include a high-contrast fusion strategy and a smoothing fusion strategy. It can be understood that the high-contrast fusion strategy corresponds to a high-contrast scene, and the smoothing fusion strategy corresponds to a scene where highlights need to be suppressed. The high-contrast fusion strategy can fuse an image with high contrast, that is, the edge details are clear, without obvious blurred adhesion or artifacts, but there may be areas with high-brightness halos in some extreme cases. The smoothing fusion strategy can fuse a relatively smooth image, that is, without blurred adhesion or artifacts, and the high-brightness halo areas will be smoothed, but the edge detail contrast is not as high as that of the first strategy.

[0107] Specifically, as Figure 10 shown, to select the fusion strategy, it can specifically be by receiving the application requirements and application scenarios (such as a high-contrast scene where the image is relatively sharp, or a scene where highlights need to be suppressed, etc.) uploaded or selected by the terminal device for the first image to be fused and the second image to be fused. Furthermore, the fusion strategy can be selected according to the requirements and scenarios. If a high-contrast image needs to be fused, the high-contrast fusion strategy is selected. If a fused image with suppressed highlights is needed, the smoothing fusion strategy is selected.

[0108] In step S106, based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the second image to be fused, the first image to be fused and the second image to be fused are fused to obtain a target fused image.

[0109] In this embodiment, after the target fusion strategy is selected, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the second image to be fused are used to fuse the first image to be fused and the second image to be fused according to the decision conditions in the target fusion strategy to obtain a target fused image.

[0110] Specifically, as Figure 10As shown, after selecting the target fusion strategy, substitute the first core energy value, the second core energy value, the third core energy value corresponding to each pixel point in the first image to be fused, and the first core energy value, the second core energy value, the third core energy value corresponding to each pixel point in the second image to be fused into the target fusion strategy. According to the decision conditions in the target fusion strategy, fuse the first image to be fused and the second image to be fused. Specifically, for each pixel point, select the pixel point with the largest energy value in the first image to be fused or the second image to be fused as the output pixel point to obtain the fused image.

[0111] In the embodiments of the present application, a method for determining a fused image is provided. Through the above method, on the one hand, based on the first core energy function, the second core energy function, and the third core energy function, a regional energy operator composed of multi-core functions with different core sizes can be used for the first image to be fused and the second image to be fused to obtain corresponding energy values, and the mutual dependence relationship of pixels in different regional sizes can be felt. On the other hand, on the basis of using the same operator, a suitable target fusion strategy is matched for different target scenarios. The decision conditions of the target fusion strategy are simple and do not need to consider the influence of multiple operators, which can greatly reduce the amount of calculation and shorten the calculation time, thereby improving the efficiency of image fusion.

[0112] Optionally, on the basis of the corresponding embodiments above Figure 2 In another optional embodiment of the method for determining a fused image provided by the embodiments of the present application, as Figure 3 shown, when the target fusion strategy is a high-contrast fusion strategy, step S106 fuses the first image to be fused and the second image to be fused based on the target fusion strategy, the first core energy value, the second core energy value, the third core energy value corresponding to each pixel point in the first image to be fused, and the first core energy value, the second core energy value, the third core energy value corresponding to each pixel point in the second image to be fused to obtain the target fused image, including:

[0113] In step S301, based on the first fusion weight, perform weighted summation on the first core energy value, the second core energy value, and the third core energy value corresponding to each pixel point in the first image to be fused to obtain the first total energy value corresponding to each pixel point in the first image to be fused;

[0114] In step S302, based on the first fusion weight, perform weighted summation on the first core energy value, the second core energy value, and the third core energy value corresponding to each pixel point in the second image to be fused to obtain the second total energy value corresponding to each pixel point in the second image to be fused;

[0115] In step S303, for each pixel, select the energy value with the larger value from the first total energy value and the second total energy value, and use the pixel corresponding to the larger energy value as the target pixel to obtain the target fusion image.

[0116] In this embodiment, when the selected target fusion strategy is the high-contrast fusion strategy, first, according to the first fusion weight, perform weighted summation on the first core energy value, the second core energy value, and the third core energy value corresponding to each pixel in the first image to be fused to obtain the first total energy value corresponding to each pixel in the first image to be fused. At the same time, according to the first fusion weight, perform weighted summation on the first core energy value, the second core energy value, and the third core energy value corresponding to each pixel in the second image to be fused to obtain the second total energy value corresponding to each pixel in the second image to be fused. Then, according to the decision condition of the high-contrast fusion strategy, fuse the first image to be fused and the second image to be fused, that is, for each pixel, select the energy value with the larger value from the first total energy value and the second total energy value, and use the pixel corresponding to the larger energy value as the target pixel to output the fused target fusion image.

[0117] Among them, the first fusion weight is set according to actual application requirements and can usually be set to 1, without specific limitation here.

[0118] Specifically, based on the first fusion weight, use the following formula (5) to perform weighted summation on the first core energy value, the second core energy value, and the third core energy value corresponding to each pixel in the first image to be fused to obtain the first total energy value corresponding to each pixel in the first image to be fused:

[0119] E A (m,n) = αG S (m,n) + βG M (m,n) + γG L (m,n) (5);

[0120] Among them, E A (m,n) represents the first total energy value corresponding to each pixel in the first image to be fused, (m, n) represents a pixel in the first image to be fused, G S (m,n) represents the first core energy value corresponding to (m, n), G M (m,n) represents the second core energy value corresponding to (m, n), G L(m,n) represents the third nuclear energy value corresponding to (m, n). α is the first fusion weight corresponding to the first nuclear energy function, β is the first fusion weight corresponding to the second nuclear energy function, and γ is the first fusion weight corresponding to the third nuclear energy function. Usually, setting α, β, and γ to 1, 1, and 1 respectively can achieve a good fusion effect. The specific values can be set according to actual application requirements and are not specifically limited here.

[0121] Based on the first fusion weight, the following formula (6) is used to perform a weighted sum of the first nuclear energy value, the second nuclear energy value, and the third nuclear energy value corresponding to each pixel point in the second image to be fused, obtaining the second total energy value corresponding to each pixel point in the second image to be fused:

[0122] E B (m,n) = αG S (m,n) + βG M (m,n) + γG L (m,n) (6);

[0123] Among them, E B (m,n) represents the second total energy value corresponding to each pixel point in the second image to be fused, (m, n) represents a pixel point in the second image to be fused, G S (m,n) represents the first nuclear energy value corresponding to (m, n), G M (m,n) represents the second nuclear energy value corresponding to (m, n), G L (m,n) represents the third nuclear energy value corresponding to (m, n). α is the first fusion weight corresponding to the first nuclear energy function, β is the first fusion weight corresponding to the second nuclear energy function, and γ is the first fusion weight corresponding to the third nuclear energy function. Usually, setting α, β, and γ to 1, 1, and 1 respectively can achieve a good fusion effect. The specific values can be set according to actual application requirements and are not specifically limited here.

[0124] Furthermore, since the first nuclear energy function, the second nuclear energy function, and the third nuclear energy function use the same operator, in decision-making, this embodiment can use the following simple decision condition (7) to obtain the target fusion image:

[0125]

[0126] Among them, F(m, n) represents the target fusion image, (m, n) represents each target pixel point in the target fusion image, A(m, n) represents the first image to be fused, that is, the current pixel point comes from the first image to be fused, B(m, n) represents the second image to be fused, that is, the current pixel point comes from the second image to be fused, E A represents E A(m,n), that is, the first total energy value corresponding to each pixel point in the first image to be fused, E B denotes E B (m,n), that is, the second total energy value corresponding to each pixel point in the second image to be fused.

[0127] For example, for each pixel point, such as the E corresponding to pixel point F(0,0) A (0,0) The first total energy value is 0.9 and E B (0,0) The second total energy value is 0.7. Compare the E corresponding to this pixel point A (0,0) The first total energy value and E B (0,0) The second total energy value. It can be seen that the first total energy value E A (0,0) is greater than the second total energy value E B (0,0), that is, E A ≥E B , then use the larger energy value E A (0,0) corresponding A(0,0) as the target pixel point. Similarly, obtain other target pixel points to obtain the target fused image.

[0128] Optionally, based on the above Figure 2 corresponding embodiment, in another optional embodiment of the method for determining a fused image provided by the embodiments of the present application, as Figure 4 shown, when the target fusion strategy is a smoothing fusion strategy, step S106 fuses the first image to be fused and the second image to be fused based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the second image to be fused, to obtain a target fused image, including:

[0129] In step S401, based on the first kernel energy value corresponding to each pixel point in the first image to be fused and the first kernel energy value corresponding to each pixel point in the second image to be fused, obtain a first mask image;

[0130] In step S402, based on the second kernel energy value corresponding to each pixel point in the first image to be fused and the second kernel energy value corresponding to each pixel point in the second image to be fused, obtain a second mask image;

[0131] In step S403, based on the third kernel energy value corresponding to each pixel point in the first image to be fused and the third kernel energy value corresponding to each pixel point in the second image to be fused, obtain a third mask image;

[0132] In step S404, a target fusion image is obtained based on the second fusion weight, the first mask image, the second mask image, and the third mask image.

[0133] In this embodiment, when the selected target fusion strategy is a smoothing fusion strategy, based on the first kernel energy value corresponding to each pixel point in the first image to be fused and the first kernel energy value corresponding to each pixel point in the second image to be fused, the first mask image is obtained, and based on the second kernel energy value corresponding to each pixel point in the first image to be fused and the second kernel energy value corresponding to each pixel point in the second image to be fused, the second mask image is obtained, and based on the third kernel energy value corresponding to each pixel point in the first image to be fused and the third kernel energy value corresponding to each pixel point in the second image to be fused, the third mask image is obtained. Then, weighted fusion is performed based on the second fusion weight, the first mask image, the second mask image, and the third mask image to obtain the target fusion image.

[0134] Among them, the first mask image, the second mask image, and the third mask image are grayscale mask images formed by 0 and 1.

[0135] Specifically, when the selected target fusion strategy is a smoothing fusion strategy, for each pixel point, the energy value with the larger value is selected from the first kernel energy value of the first image to be fused and the first kernel energy value of the second image to be fused, and the pixel point corresponding to the energy value with the larger value is used as the first mask pixel point to fuse the first image to be fused and the second image to be fused into the first mask image M S (m,n). Similarly, for each pixel point, the energy value with the larger value is selected from the second kernel energy value of the first image to be fused and the second kernel energy value of the second image to be fused, and the pixel point corresponding to the energy value with the larger value is used as the second mask pixel point to fuse the first image to be fused and the second image to be fused into the second mask image M M (m,n), and for each pixel point, the energy value with the larger value is selected from the second kernel energy value of the first image to be fused and the second kernel energy value of the second image to be fused, and the pixel point corresponding to the energy value with the larger value is used as the third mask pixel point to fuse the first image to be fused and the second image to be fused into the third mask image M L (m,n).

[0136] Further, the following formulas (8) and (9) can be used to calculate the target mask image based on the second fusion weight, the first mask image, the second mask image, and the third mask image:

[0137] M(m,n) = μM S (m,n) + νM M (m,n) + λML (m,n) (8);

[0138] μ + ν + λ = 1 (9);

[0139] Among them, μ is the second fusion weight corresponding to the first nuclear energy function, that is, the weight of the Mask output by the first nuclear energy function, ν is the second fusion weight corresponding to the second nuclear energy function, that is, the weight of the Mask output by the second nuclear energy function, and λ is the second fusion weight corresponding to the third nuclear energy function, that is, the weight of the Mask output by the third nuclear energy function. Usually, setting μ, ν, and λ to 1 / 3, 1 / 3, and 1 / 3 respectively can achieve a good fusion effect. The specific values can be set according to actual application requirements and are not specifically limited here. It can be understood that when attention to details needs to be considered, the value of μ can be set as a larger weight, and when anti-noise is considered, the value of λ can be set as a larger weight.

[0140] Furthermore, the following formula (10) can be used to convert the calculated target mask image into a target fusion vector:

[0141] F(m,n) = M(m,n) × A(m,n) + (1 - M(m,n)) × B(m,n) (10);

[0142] Among them, A(m,n) represents the first image to be fused, that is, the current pixel point comes from the first image to be fused, and B(m,n) represents the second image to be fused, that is, the current pixel point comes from the second image to be fused.

[0143] Optionally, based on the above Figure 4 corresponding embodiment, in another optional embodiment of the method for determining a fused image provided by the embodiments of the present application, as Figure 5 shown, step S102 obtains the first mask image based on the first nuclear energy value corresponding to each pixel point in the first image to be fused and the first nuclear energy value corresponding to each pixel point in the second image to be fused, including:

[0144] In step S501, for each pixel point, select the energy value with a larger value from the first nuclear energy value of the first image to be fused and the first nuclear energy value of the second image to be fused;

[0145] In step S502, for each pixel point, use the pixel point corresponding to the energy value with a larger value as the first mask pixel point to obtain the first mask image.

[0146] In this embodiment, when the selected target fusion strategy is the smooth fusion strategy, for each pixel point, the larger energy value can be selected from the first kernel energy value of the first image to be fused and the first kernel energy value of the second image to be fused, and the pixel point corresponding to the larger energy value is used as the first mask pixel point, so as to fuse the first image to be fused and the second image to be fused into a first mask image M composed of 0s and 1s that is convenient for computer recognition and processing. S (m,n), so that the target fusion image can be obtained more quickly based on the acquired first mask image in the subsequent process, thereby improving the efficiency of obtaining the target fusion image to a certain extent.

[0147] Specifically, the following formula (11) can be used. For each pixel point, the first kernel energy value of the first image to be fused and the first kernel energy value of the second image to be fused are compared pairwise, and the larger energy value is selected from the first kernel energy value of the first image to be fused and the first kernel energy value of the second image to be fused as the first mask pixel point, so as to fuse the first image to be fused and the second image to be fused into the first mask image M. S (m,n):

[0148]

[0149] where M S (m,n) represents the first mask image, and G S A represents the first kernel energy value G S (m,n) of the first image to be fused, and G S B represents the first kernel energy value G S (m,n) of the second image to be fused.

[0150] For example, for a pixel point (1,1), the first kernel energy value of G S A (1,1) is 0.79 and the first kernel energy value of G S B (1,1) is 0.74. Comparing the first kernel energy value of the pixel point corresponding to G S A (1,1) and the first kernel energy value of G S B (1,1), it can be seen that the first kernel energy value G S A (1,1) is greater than the first kernel energy value G S B (1,1), that is, G S A ≥G S B, then the energy value G with a larger value S A (1, 1) corresponding pixel point A(1, 1) is converted to 1 as the first mask pixel point. Similarly, for a pixel point (2, 1), the corresponding G S A (2, 1) the first kernel energy value is 0.69 and G S B (2, 1) the first kernel energy value is 0.73. For the G corresponding to this pixel point S A (2, 1) the first kernel energy value and G S B (2, 1) the first kernel energy value is compared. It can be seen that the first kernel energy value G S A (2, 1) is less than the first kernel energy value G S B (2, 1), then the energy value G with a larger value S B (2, 1) corresponding pixel point B(2, 1) is converted to 0 as the first mask pixel point. Furthermore, other first mask pixel points can be obtained to obtain a grayscale image composed of 0s and 1s, that is, the first mask image.

[0151] Optionally, based on the above Figure 4 corresponding embodiment, in another optional embodiment of the method for determining the fused image provided by the embodiments of the present application, as Figure 6 shown, step S103 obtains a second mask image based on the second kernel energy value corresponding to each pixel point in the first image to be fused and the second kernel energy value corresponding to each pixel point in the second image to be fused, including:

[0152] In step S601, for each pixel point, select the energy value with a larger value from the second kernel energy value of the first image to be fused and the second kernel energy value of the second image to be fused;

[0153] In step S601, for each pixel point, use the pixel point corresponding to the energy value with a larger value as the second mask pixel point to obtain the second mask image.

[0154] In this embodiment, when the selected target fusion strategy is the smooth fusion strategy, for each pixel point, the energy value with a larger value can be selected from the second kernel energy value of the first image to be fused and the second kernel energy value of the second image to be fused, and the pixel point corresponding to the energy value with a larger value is used as the second mask pixel point to fuse the first image to be fused and the second image to be fused into a second mask image M composed of 0s and 1s for easy computer recognition and processing M(m,n) so that the subsequent target fusion image can be more quickly obtained based on the acquired second mask image, thereby improving the efficiency of obtaining the target fusion image to a certain extent.

[0155] Specifically, the following formula (12) can be used. For each pixel point, the second kernel energy value of the first image to be fused and the second kernel energy value of the second image to be fused are compared pairwise, and the larger energy value is selected from the second kernel energy value of the first image to be fused and the second kernel energy value of the second image to be fused as the second mask pixel point, so as to fuse the first image to be fused and the second image to be fused into the second mask image M M (m,n):

[0156]

[0157] where M M (m,n) represents the second mask image, G M A represents the second kernel energy value G M (m,n) of the first image to be fused, and G M B represents the second kernel energy value G M (m,n) of the second image to be fused.

[0158] For example, for a pixel point (1,2), the second kernel energy value of G M A (1,2) is 0.89 and the second kernel energy value of G M B (1,2) is 0.76. Comparing the second kernel energy value of G M A (1,2) corresponding to this pixel point and the second kernel energy value of G M B (1,2), it can be seen that the second kernel energy value G M A (1,2) is greater than the second kernel energy value G M B (1,2), that is, G M A ≥G M B , then the pixel point A(1,2) corresponding to the larger energy value G M A (1,2) is converted to 1 as the second mask pixel point. Similarly, for a pixel point (0,2), the second kernel energy value of G M A (0,2) is 0.71 and the second kernel energy value of G M B(0,2) The nuclear energy value of the first person is 0.75, and the G corresponding to this pixel point M A (0,2) The nuclear energy value of the second and G M B (0,2) Compare the nuclear energy value of the second, and it can be known that the nuclear energy value of the second G M A (0,2) is less than the nuclear energy value of the second G M B (0,2), then take the pixel point B(0,2) corresponding to the larger energy value G M B (0,2) and convert it to 0 as the second mask pixel point, and then other second mask pixel points can be obtained to obtain a grayscale image composed of 0 and 1, that is, the second mask image.

[0159] Optionally, on the basis of the above Figure 4 corresponding embodiment, in another optional embodiment of the method for determining the fused image provided by the embodiment of the present application, as Figure 7 shown, step S104 obtains the third mask image based on the third nuclear energy value corresponding to each pixel point in the first image to be fused and the third nuclear energy value corresponding to each pixel point in the second image to be fused, including:

[0160] In step S701, for each pixel point, select the energy value with the larger value from the third nuclear energy value of the first image to be fused and the third nuclear energy value of the second image to be fused;

[0161] In step S702, for each pixel point, use the pixel point corresponding to the larger energy value as the third mask pixel point to obtain the third mask image.

[0162] In this embodiment, when the selected target fusion strategy is the smooth fusion strategy, for each pixel point, the energy value with the larger value can be selected from the third nuclear energy value of the first image to be fused and the third nuclear energy value of the second image to be fused, and the pixel point corresponding to the larger energy value is used as the third mask pixel point to fuse the first image to be fused and the second image to be fused into a third mask image M M (m,n) composed of 0 and 1 for easy computer recognition and processing, so that the target fusion image can be obtained more quickly based on the obtained third mask image, thereby improving the efficiency of obtaining the target fusion image to a certain extent.

[0163] Specifically, the following formula (13) can be adopted. For each pixel, the third kernel energy values of the first image to be fused and the second image to be fused are compared pairwise, and the larger energy value is selected from the third kernel energy values of the first image to be fused and the second image to be fused as the third mask pixel, so as to fuse the first image to be fused and the second image to be fused into the third mask image M M (m,n):

[0164]

[0165] where M L (m,n) represents the third mask image, and G L A represents the third kernel energy value G L (m,n) of the first image to be fused, and G L B represents the third kernel energy value G L (m,n) of the third image to be fused.

[0166] For example, the third kernel energy value of G L A (0,3) corresponding to a pixel point (0,3) is 0.92, and the third kernel energy value of G L B (0,3) is 0.84. The third kernel energy value of G L A (0,3) corresponding to this pixel point and the third kernel energy value of G L B (0,3) are compared. It can be seen that the third kernel energy value G L A (0,3) is greater than the third kernel energy value G L B (0,3), that is, G M A ≥G M B . Then, the pixel point A(0,3) corresponding to the larger energy value G L A (0,3) is converted to 1 as the third mask pixel. Similarly, the third kernel energy value of G L A (2,2) corresponding to a pixel point (2,2) is 0.61, and the third kernel energy value of G L B (2,2) of the first person is 0.72. The third kernel energy value of G L A (2,2) corresponding to this pixel point and the third kernel energy value of G L BCompare with the third nuclear energy value at (2, 2), it can be known that the third nuclear energy value G L A (2, 2) is less than the third nuclear energy value G L B (2, 2), then take the energy value G with the larger value L B (2, 2) corresponding pixel point B(2, 2) is converted to 0 as the third mask pixel point, and then other third mask pixel points can be obtained to obtain a grayscale image composed of 0 and 1, that is, the third mask image.

[0167] Optionally, based on the above Figure 2 corresponding embodiment, in another optional embodiment of the method for determining the fused image provided by the embodiments of the present application, as Figure 8 shown, when the third image to be fused is obtained, after step S106, the method further includes:

[0168] In step S801, based on the first nuclear energy function, energy calculations are respectively performed on the third image to be fused and the target fused image to obtain the first nuclear energy value corresponding to each pixel point in the third image to be fused, and the first nuclear energy value corresponding to each pixel point in the target fused image;

[0169] In this embodiment, when two or more images to be fused are obtained, they can be fused in order. First, fuse the first image to be fused with the second image to be fused into a target fused image, and then fuse the target fused image with the third fused image, and so on, until all images are fused, so as to finally obtain a fused image. Therefore, after the third image to be fused and the target fused image are obtained, the first nuclear energy function can be called to perform energy calculations on each pixel point in the third image to be fused and the target fused image respectively, and the first nuclear energy value corresponding to each pixel point in the third image to be fused, and the first nuclear energy value corresponding to each pixel point in the target image to be fused can be obtained.

[0170] Specifically, after the third image to be fused and the target fused image are obtained, the first nuclear energy function (such as the small nuclear energy function shown in FIG. 10) can be called, and as shown in the above formulas (1) and (2), energy calculations are respectively performed on each pixel point in the third image to be fused to obtain the first nuclear energy value corresponding to each pixel point in the third image to be fused, and energy calculations are performed on each pixel point in the target fused image to obtain the first nuclear energy value corresponding to each pixel point in the target fused image.

[0171] In step S802, based on the second kernel energy function, energy calculations are respectively performed on the third image to be fused and the target fused image, to obtain the second kernel energy value corresponding to each pixel point in the third image to be fused, and the second kernel energy value corresponding to each pixel point in the target fused image;

[0172] In this embodiment, after obtaining the third image to be fused and the target fused image, the second kernel energy function can be called to perform energy calculations on each pixel point in the third image to be fused and the target fused image respectively, so as to obtain the second kernel energy value corresponding to each pixel point in the third image to be fused, and the second kernel energy value corresponding to each pixel point in the target fused image.

[0173] Specifically, after obtaining the third image to be fused and the target fused image, the second kernel energy function (such as the middle kernel energy function shown in FIG. 10) can be called, and by using the above formulas (2) and (3), energy calculations are respectively performed on each pixel point in the third image to be fused, so as to obtain the second kernel energy value corresponding to each pixel point in the third image to be fused, and energy calculations are performed on each pixel point in the target fused image, so as to obtain the second kernel energy value corresponding to each pixel point in the target fused image.

[0174] In step S803, based on the third kernel energy function, energy calculations are respectively performed on the third image to be fused and the target fused image, to obtain the third kernel energy value corresponding to each pixel point in the first image to be fused, and the third kernel energy value corresponding to each pixel point in the second image to be fused;

[0175] In this embodiment, after obtaining the third image to be fused and the target fused image, the third kernel energy function can be called to perform energy calculations on each pixel point in the third image to be fused and the target fused image respectively, so as to obtain the third kernel energy value corresponding to each pixel point in the third image to be fused, and the third kernel energy value corresponding to each pixel point in the target fused image.

[0176] Specifically, after obtaining the third image to be fused and the target fused image, the third kernel energy function (such as the large kernel energy function shown in FIG. 10) can be called, and by using the above formulas (2) and (4), energy calculations are respectively performed on each pixel point in the third image to be fused, so as to obtain the third kernel energy value corresponding to each pixel point in the third image to be fused, and energy calculations are performed on each pixel point in the target fused image, so as to obtain the third kernel energy value corresponding to each pixel point in the target image to be fused.

[0177] In step S804, based on the target fusion strategy, the first kernel energy values, second kernel energy values, and third kernel energy values corresponding to each pixel point in the third image to be fused, and the first kernel energy values, second kernel energy values, and third kernel energy values corresponding to each pixel point in the target fusion image, the third image to be fused and the target fusion image are fused to obtain a fused image.

[0178] In this embodiment, after selecting the target fusion strategy, the first kernel energy values, second kernel energy values, and third kernel energy values corresponding to each pixel point in the third image to be fused, and the first kernel energy values, second kernel energy values, and third kernel energy values corresponding to each pixel point in the target fusion image are used to fuse the third image to be fused and the target fusion image according to the decision conditions in the target fusion strategy, so as to obtain a fused image.

[0179] Specifically, as Figure 10 shown, after selecting the target fusion strategy, the first kernel energy values, second kernel energy values, and third kernel energy values corresponding to each pixel point in the third image to be fused, and the first kernel energy values, second kernel energy values, and third kernel energy values corresponding to each pixel point in the target fusion image are substituted into the target fusion strategy. According to the decision conditions in the target fusion strategy, the third image to be fused and the target fusion image are fused. Specifically, for each pixel point, the pixel point with the largest energy value in the third image to be fused or the target fusion image is selected as the output pixel point to obtain a fused image.

[0180] Optionally, on the basis of the corresponding embodiment above, in another optional embodiment of the method for determining a fused image provided by the embodiments of the present application, as Figure 2 shown, before step S102 calculates the energy of the first image to be fused and the second image to be fused respectively based on the first kernel energy function to obtain the first kernel energy values corresponding to each pixel point in the first image to be fused and the first kernel energy values corresponding to each pixel point in the second image to be fused, the method further includes: Figure 8 In step S901, based on the image sizes of the first image to be fused and the second image to be fused, and the object density of the target object, the first kernel size, the second kernel size, and the third kernel size are determined, where the first kernel size is smaller than the second kernel size, and the second kernel size is smaller than the third kernel size;

[0181] In step S902, based on the first kernel size, a first kernel energy function is determined;

[0182] In step S903, based on the second kernel size, a second kernel energy function is determined;

[0183] In step S904, based on the third kernel size, a third kernel energy function is determined;

[0184] In step S904, based on the third core size, a third core energy function is determined.

[0185] In this embodiment, after obtaining the first image to be fused and the second image to be fused, in order to better obtain a suitable core size, so as to obtain a matching core energy function to better perform energy calculation on the first image to be fused and the second image to be fused, the accuracy and efficiency of the fused image can be improved to a certain extent.

[0186] Among them, the image size refers to the pixel size of the first image to be fused and the second image to be fused, which is usually 2560*2560, and the object density of the target object is used to represent the compactness of the object structure of the target object.

[0187] Specifically, after obtaining the first image to be fused and the second image to be fused, the image size of the first image to be fused and the second image to be fused can be measured, that is, the image size. At the same time, the object density values of the target objects (such as robotic arms, drivers, books, etc.) in the first image to be fused and the second image to be fused can be obtained by traversing the density list.

[0188] Furthermore, it can be understood that the higher the density of the photographed target object, that is, the higher the compactness of the object structure, the smaller the selected core size. Therefore, according to the image size and the object density of the target object, the small core size, that is, the first core size, can be selected from the preset small core size range, and the medium core size, that is, the second core size, can be selected from the preset medium core size range, and the large core size, that is, the third core size, can be selected from the preset large core size range. Then, based on the first core size, a first core energy function is determined; based on the second core size, a second core energy function is determined; based on the third core size, a third core energy function is determined.

[0189] The determination device of the fused image in the present application will be described in detail below. Please refer to Figure 11 , Figure 11 which is a schematic diagram of an embodiment of the determination device of the fused image in the embodiment of the present application. The determination device 20 of the fused image includes:

[0190] An acquisition unit 201, configured to acquire a first image to be fused and a second image to be fused, where the first image to be fused and the second image to be fused are images obtained by focusing on different regions of the same target object;

[0191] A processing unit 202, configured to perform energy calculation on the first image to be fused and the second image to be fused respectively based on the first core energy function, to obtain a first core energy value corresponding to each pixel point in the first image to be fused, and a first core energy value corresponding to each pixel point in the second image to be fused;

[0192] The processing unit 202 is further configured to calculate the energy of the first image to be fused and the second image to be fused respectively based on the second kernel energy function, so as to obtain the second kernel energy value corresponding to each pixel point in the first image to be fused and the second kernel energy value corresponding to each pixel point in the second image to be fused, where the kernel size of the second kernel energy function is larger than that of the first kernel energy function;

[0193] The processing unit 202 is further configured to calculate the energy of the first image to be fused and the second image to be fused respectively based on the third kernel energy function, so as to obtain the third kernel energy value corresponding to each pixel point in the first image to be fused and the third kernel energy value corresponding to each pixel point in the second image to be fused, where the kernel size of the third kernel energy function is larger than that of the second kernel energy function;

[0194] The determining unit 203 is configured to determine a target fusion strategy based on the target scene corresponding to the target object, where the target fusion strategy includes a high-contrast fusion strategy and a smoothing fusion strategy;

[0195] The processing unit 202 is further configured to fuse the first image to be fused and the second image to be fused based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the second image to be fused, so as to obtain a target fusion image.

[0196] Optionally, based on the corresponding embodiment above Figure 11 In another embodiment of the device for determining a fusion image provided by the embodiment of the present application, the processing unit 202 may specifically be configured to:

[0197] When the target fusion strategy is a high-contrast fusion strategy, fusing the first image to be fused and the second image to be fused based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the second image to be fused, to obtain a target fusion image, including:

[0198] Performing weighted summation on the first kernel energy value, the second kernel energy value, and the third kernel energy value corresponding to each pixel point in the first image to be fused based on a first fusion weight, so as to obtain a first total energy value corresponding to each pixel point in the first image to be fused;

[0199] Based on the first fusion weight, perform a weighted sum of the first kernel energy value, the second kernel energy value, and the third kernel energy value corresponding to each pixel point in the second image to be fused, to obtain the second total energy value corresponding to each pixel point in the second image to be fused;

[0200] For each pixel point, select the energy value with the larger value from the first total energy value and the second total energy value, and use the pixel point corresponding to the energy value with the larger value as the target pixel point, so as to obtain the target fused image.

[0201] Optionally, based on the above Figure 11 corresponding embodiment, in another embodiment of the fused image determination device provided by the embodiments of the present application, the processing unit 202 may specifically be used for:

[0202] Based on the first kernel energy value corresponding to each pixel point in the first image to be fused and the first kernel energy value corresponding to each pixel point in the second image to be fused, obtain the first mask image;

[0203] Based on the second kernel energy value corresponding to each pixel point in the first image to be fused and the second kernel energy value corresponding to each pixel point in the second image to be fused, obtain the second mask image;

[0204] Based on the third kernel energy value corresponding to each pixel point in the first image to be fused and the third kernel energy value corresponding to each pixel point in the second image to be fused, obtain the third mask image;

[0205] Based on the second fusion weight, the first mask image, the second mask image, and the third mask image, obtain the target fused image.

[0206] Optionally, based on the above Figure 11 corresponding embodiment, in another embodiment of the fused image determination device provided by the embodiments of the present application, the processing unit 202 may specifically be used for:

[0207] For each pixel point, select the energy value with the larger value from the first kernel energy value of the first image to be fused and the first kernel energy value of the second image to be fused;

[0208] For each pixel point, use the pixel point corresponding to the energy value with the larger value as the first mask pixel point, so as to obtain the first mask image.

[0209] In a possible design, in an implementation manner on the other hand of the embodiments of the present application, the processing unit may specifically be used for:

[0210] For each pixel point, select the energy value with the larger value from the second kernel energy value of the first image to be fused and the second kernel energy value of the second image to be fused;

[0211] For each pixel, the pixel corresponding to the larger energy value is used as the second masked pixel to obtain a second masked image.

[0212] Optionally, based on the above Figure 11 In another embodiment of the fusion image determination device provided by the embodiments of the present application, on the basis of the corresponding embodiment, the processing unit 202 may specifically be used for:

[0213] For each pixel, select the larger energy value from the third kernel energy value of the first image to be fused and the third kernel energy value of the second image to be fused;

[0214] For each pixel, the pixel corresponding to the larger energy value is used as the third masked pixel to obtain a third masked image.

[0215] In a possible design, in an implementation manner on the other hand of the embodiments of the present application, the processing unit may specifically be used for

[0216] The processing unit 202 is further configured to calculate the energy of the third image to be fused and the target fusion image respectively based on the first kernel energy function, to obtain the first kernel energy value corresponding to each pixel in the third image to be fused, and the first kernel energy value corresponding to each pixel in the target fusion image;

[0217] The processing unit 202 is further configured to calculate the energy of the third image to be fused and the target fusion image respectively based on the second kernel energy function, to obtain the second kernel energy value corresponding to each pixel in the third image to be fused, and the second kernel energy value corresponding to each pixel in the target fusion image;

[0218] The processing unit 202 is further configured to calculate the energy of the third image to be fused and the target fusion image respectively based on the third kernel energy function, to obtain the third kernel energy value corresponding to each pixel in the first image to be fused, and the third kernel energy value corresponding to each pixel in the second image to be fused;

[0219] The processing unit 202 is further configured to fuse the third image to be fused and the target fusion image based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel in the third image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel in the target fusion image, to obtain a fusion image.

[0220] Optionally, based on the above Figure 11 In another embodiment of the fusion image determination device provided by the embodiments of the present application, on the basis of the corresponding embodiment,

[0221] The determination unit 203 is further configured to determine a first kernel size, a second kernel size, and a third kernel size based on the image sizes of the first image to be fused and the second image to be fused, and the object density of the target object, where the first kernel size is smaller than the second kernel size, and the second kernel size is smaller than the third kernel size;

[0222] The determination unit 203 is further configured to determine a first kernel energy function based on the first kernel size;

[0223] The determination unit 203 is further configured to determine a second kernel energy function based on the second kernel size;

[0224] The determination unit 203 is further configured to determine a third kernel energy function based on the third kernel size.

[0225] Another aspect of the present application provides a schematic diagram of another computer device, as Figure 12 shown, Figure 12 is a schematic diagram of the structure of a computer device provided by an embodiment of the present application. The computer device 300 may vary greatly due to different configurations or performances, and may include one or more central processing units (CPUs) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage devices) for storing application programs 331 or data 332. Among them, the memory 320 and the storage media 330 may be transient storage or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the computer device 300. Further, the central processor 310 may be configured to communicate with the storage media 330 and execute a series of instruction operations in the storage media 330 on the computer device 300.

[0226] The computer device 300 may further include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or, one or more operating systems 333, such as Windows Server TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM and so on.

[0227] The above computer device 300 is further configured to execute the steps in the Figures 2 to 9 corresponding embodiment.

[0228] Another aspect of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the method described in the embodiments as Figures 2 to 9 shown are implemented.

[0229] Another aspect of the present application provides a computer program product containing a computer program, and when the computer program is executed by a processor, the steps in the method described in the embodiments as Figures 2 to 9 shown are implemented.

[0230] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0231] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0232] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0233] In addition, the functional units in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0234] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

Claims

1. A method for determining a fused image, characterized in that, it includes: Obtain a first image to be fused and a second image to be fused, wherein the first image to be fused and the second image to be fused are images obtained by focusing on different regions of the same target object; Based on a first kernel energy function, calculate the energy of the first image to be fused and the second image to be fused respectively, to obtain the first kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value corresponding to each pixel point in the second image to be fused, wherein the first kernel energy function includes a first Gaussian blur function, and the first Gaussian blur function is used to focus on the pixels in the first region range; Based on a second kernel energy function, calculate the energy of the first image to be fused and the second image to be fused respectively, to obtain the second kernel energy value corresponding to each pixel point in the first image to be fused, and the second kernel energy value corresponding to each pixel point in the second image to be fused, wherein the kernel size of the second kernel energy function is larger than the kernel size of the first kernel energy function, the second kernel energy function includes a second Gaussian blur function, the second Gaussian blur function is used to focus on the edge details of the image, and the size of the Gaussian template of the second Gaussian blur function is larger than the size of the Gaussian template of the first Gaussian blur function; Based on a third kernel energy function, calculate the energy of the first image to be fused and the second image to be fused respectively, to obtain the third kernel energy value corresponding to each pixel point in the first image to be fused, and the third kernel energy value corresponding to each pixel point in the second image to be fused, wherein the kernel size of the third kernel energy function is larger than the kernel size of the second kernel energy function, the third kernel energy function includes a third Gaussian blur function, the third Gaussian blur function is used to focus on the pixels in the second region range, the size of the Gaussian template of the third Gaussian blur function is larger than the size of the Gaussian template of the second Gaussian blur function, and the second region range is larger than the first region range; Based on the target scene corresponding to the target object, determine a target fusion strategy, wherein the target fusion strategy includes a high-contrast fusion strategy and a smooth fusion strategy; Based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the second image to be fused, fuse the first image to be fused and the second image to be fused to obtain a target fused image.

2. The method according to claim 1, characterized in that, When the target fusion strategy is the high-contrast fusion strategy, fusing the first image to be fused and the second image to be fused based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the second image to be fused to obtain a target fusion image, including: Based on a first fusion weight, performing weighted summation on the first kernel energy value, the second kernel energy value, and the third kernel energy value corresponding to each pixel point in the first image to be fused to obtain a first total energy value corresponding to each pixel point in the first image to be fused; Based on the first fusion weight, performing weighted summation on the first kernel energy value, the second kernel energy value, and the third kernel energy value corresponding to each pixel point in the second image to be fused to obtain a second total energy value corresponding to each pixel point in the second image to be fused; For each pixel point, selecting the energy value with the larger value from the first total energy value and the second total energy value, and using the pixel point corresponding to the energy value with the larger value as the target pixel point to obtain the target fusion image.

3. The method according to claim 1, wherein, When the target fusion strategy is the smooth fusion strategy, fusing the first image to be fused and the second image to be fused based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the second image to be fused to obtain a target fusion image, including: Based on the first kernel energy value corresponding to each pixel point in the first image to be fused and the first kernel energy value corresponding to each pixel point in the second image to be fused, obtaining a first mask image; Based on the second kernel energy value corresponding to each pixel point in the first image to be fused and the second kernel energy value corresponding to each pixel point in the second image to be fused, obtaining a second mask image; Based on the third kernel energy value corresponding to each pixel point in the first image to be fused and the third kernel energy value corresponding to each pixel point in the second image to be fused, obtaining a third mask image; Based on a second fusion weight, the first mask image, the second mask image, and the third mask image, obtaining the target fusion image.

4. The method according to claim 3, wherein, The obtaining a first mask image based on the first kernel energy value corresponding to each pixel point in the first image to be fused and the first kernel energy value corresponding to each pixel point in the second image to be fused includes: For each pixel point, selecting the energy value with the larger value from the first kernel energy value of the first image to be fused and the first kernel energy value of the second image to be fused; For each pixel, the pixel corresponding to the larger energy value is used as the first masked pixel to obtain the first masked image.

5. The method according to claim 3, wherein, the obtaining the second masked image based on the second kernel energy value corresponding to each pixel in the first image to be fused and the second kernel energy value corresponding to each pixel in the second image to be fused includes: For each pixel, select the larger energy value from the second kernel energy value of the first image to be fused and the second kernel energy value of the second image to be fused; For each pixel, the pixel corresponding to the larger energy value is used as the second masked pixel to obtain the second masked image.

6. The method according to claim 3, wherein, the obtaining the third masked image based on the third kernel energy value corresponding to each pixel in the first image to be fused and the third kernel energy value corresponding to each pixel in the second image to be fused includes: For each pixel, select the larger energy value from the third kernel energy value of the first image to be fused and the third kernel energy value of the second image to be fused; For each pixel, the pixel corresponding to the larger energy value is used as the third masked pixel to obtain the third masked image.

7. The method according to claim 1, wherein, when the third image to be fused is obtained, the method further includes: Based on the first kernel energy function, perform energy calculations on the third image to be fused and the target fused image respectively to obtain the first kernel energy value corresponding to each pixel in the third image to be fused and the first kernel energy value corresponding to each pixel in the target fused image; Based on the second kernel energy function, perform energy calculations on the third image to be fused and the target fused image respectively to obtain the second kernel energy value corresponding to each pixel in the third image to be fused and the second kernel energy value corresponding to each pixel in the target fused image; Based on the third kernel energy function, perform energy calculations on the third image to be fused and the target fused image respectively to obtain the third kernel energy value corresponding to each pixel in the first image to be fused and the third kernel energy value corresponding to each pixel in the second image to be fused; Based on the target fusion strategy, the first kernel energy value, second kernel energy value, third kernel energy value corresponding to each pixel in the third image to be fused, and the first kernel energy value, second kernel energy value, third kernel energy value corresponding to each pixel in the target fused image, fuse the third image to be fused and the target fused image to obtain a fused image.

8. The method according to claim 1, wherein, Before calculating the energy of the first image to be fused and the second image to be fused respectively based on the first kernel energy function to obtain the first kernel energy value corresponding to each pixel point in the first image to be fused and the first kernel energy value corresponding to each pixel point in the second image to be fused, the method further includes: Determine a first kernel size, a second kernel size, and a third kernel size based on the image sizes of the first image to be fused and the second image to be fused, and the object density of the target object, where the first kernel size is smaller than the second kernel size, and the second kernel size is smaller than the third kernel size; Determine the first kernel energy function based on the first kernel size; Determine the second kernel energy function based on the second kernel size; Determine the third kernel energy function based on the third kernel size.

9. An apparatus for determining a fused image Characterized in that It includes: An acquisition unit, configured to acquire a first image to be fused and a second image to be fused, where the first image to be fused and the second image to be fused are images obtained by focusing on different regions of the same target object; A processing unit, configured to calculate the energy of the first image to be fused and the second image to be fused respectively based on the first kernel energy function to obtain the first kernel energy value corresponding to each pixel point in the first image to be fused and the first kernel energy value corresponding to each pixel point in the second image to be fused, where the first kernel energy function includes a first Gaussian blur function, and the first Gaussian blur function is used to focus on the pixels in the first region range; The processing unit is further configured to calculate the energy of the first image to be fused and the second image to be fused respectively based on the second kernel energy function to obtain the second kernel energy value corresponding to each pixel point in the first image to be fused and the second kernel energy value corresponding to each pixel point in the second image to be fused, where the kernel size of the second kernel energy function is larger than the kernel size of the first kernel energy function, the second kernel energy function includes a second Gaussian blur function, the second Gaussian blur function is used to focus on the edge details of the image, and the size of the Gaussian template of the second Gaussian blur function is larger than the size of the Gaussian template of the first Gaussian blur function; The processing unit is further configured to calculate the energy of the first image to be fused and the second image to be fused respectively based on the third kernel energy function to obtain the third kernel energy value corresponding to each pixel point in the first image to be fused and the third kernel energy value corresponding to each pixel point in the second image to be fused, where the kernel size of the third kernel energy function is larger than the kernel size of the second kernel energy function, the third kernel energy function includes a third Gaussian blur function, the third Gaussian blur function is used to focus on the pixels in the second region range, the size of the Gaussian template of the third Gaussian blur function is larger than the size of the Gaussian template of the second Gaussian blur function, and the second region range is larger than the first region range; A determination unit, configured to determine a target fusion strategy based on a target scenario corresponding to the target object, where the target fusion strategy includes a high-contrast fusion strategy and a smooth fusion strategy; The processing unit is further configured to fuse the first image to be fused and the second image to be fused based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the second image to be fused, to obtain a target fusion image.

10. The apparatus according to claim 9, wherein, The processing unit is specifically configured to: When the target fusion strategy is a high-contrast fusion strategy, fusing the first image to be fused and the second image to be fused based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the first image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the second image to be fused, to obtain a target fusion image, including: Based on a first fusion weight, performing weighted summation on the first kernel energy value, the second kernel energy value, and the third kernel energy value corresponding to each pixel point in the first image to be fused, to obtain a first total energy value corresponding to each pixel point in the first image to be fused; Based on a first fusion weight, performing weighted summation on the first kernel energy value, the second kernel energy value, and the third kernel energy value corresponding to each pixel point in the second image to be fused, to obtain a second total energy value corresponding to each pixel point in the second image to be fused; For each pixel point, selecting the energy value with the larger value from the first total energy value and the second total energy value, and using the pixel point corresponding to the energy value with the larger value as the target pixel point, so as to obtain a target fusion image.

11. The apparatus according to claim 9, wherein, The processing unit is specifically configured to: Based on the first kernel energy value corresponding to each pixel point in the first image to be fused and the first kernel energy value corresponding to each pixel point in the second image to be fused, obtain a first mask image; Based on the second kernel energy value corresponding to each pixel point in the first image to be fused and the second kernel energy value corresponding to each pixel point in the second image to be fused, obtain a second mask image; Based on the third kernel energy value corresponding to each pixel point in the first image to be fused and the third kernel energy value corresponding to each pixel point in the second image to be fused, obtain a third mask image; Based on a second fusion weight, the first mask image, the second mask image, and the third mask image, obtain a target fusion image.

12. The apparatus according to claim 11, wherein, The processing unit is specifically configured to: For each pixel point, select the energy value with the larger value from the first kernel energy value of the first image to be fused and the first kernel energy value of the second image to be fused; For each pixel point, use the pixel point corresponding to the energy value with the larger value as the first mask pixel point, so as to obtain a first mask image.

13. The apparatus according to claim 11, It is characterized in that The processing unit is specifically configured to: For each pixel point, select the energy value with the larger value from the second kernel energy value of the first image to be fused and the second kernel energy value of the second image to be fused; For each pixel point, use the pixel point corresponding to the larger energy value as the second masked pixel point to obtain a second masked image.

14. The apparatus according to claim 11, It is characterized in that The processing unit is specifically configured to: For each pixel point, select the energy value with the larger value from the third kernel energy value of the first image to be fused and the third kernel energy value of the second image to be fused; For each pixel point, use the pixel point corresponding to the larger energy value as the third masked pixel point to obtain a third masked image.

15. The apparatus according to claim 9, It is characterized in that The processing unit is specifically configured to: Based on the first kernel energy function, perform energy calculations on the third image to be fused and the target fused image respectively, to obtain the first kernel energy value corresponding to each pixel point in the third image to be fused, and the first kernel energy value corresponding to each pixel point in the target fused image; Based on the second kernel energy function, perform energy calculations on the third image to be fused and the target fused image respectively, to obtain the second kernel energy value corresponding to each pixel point in the third image to be fused, and the second kernel energy value corresponding to each pixel point in the target fused image; Based on the third kernel energy function, perform energy calculations on the third image to be fused and the target fused image respectively, to obtain the third kernel energy value corresponding to each pixel point in the first image to be fused, and the third kernel energy value corresponding to each pixel point in the second image to be fused; Based on the target fusion strategy, the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the third image to be fused, and the first kernel energy value, the second kernel energy value, the third kernel energy value corresponding to each pixel point in the target fused image, fuse the third image to be fused and the target fused image to obtain a fused image.

16. The apparatus according to claim 9, It is characterized in that The determination unit is further configured to determine a first kernel size, a second kernel size, and a third kernel size based on the image sizes of the first image to be fused and the second image to be fused, and the object density of the target object, wherein the first kernel size is smaller than the second kernel size, and the second kernel size is smaller than the third kernel size; The determination unit is further configured to determine a first kernel energy function based on the first kernel size; The determination unit is further configured to determine a second kernel energy function based on the second kernel size; The determination unit is further configured to determine a third kernel energy function based on the third kernel size.

17. A computer device, comprising a memory, a processor, and a bus system, the memory storing a computer program, It is characterized in that When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8; The bus system is used to connect the memory and the processor to enable communication between the memory and the processor.

18. A computer-readable storage medium, on which a computer program is stored, It is characterized in that When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

19. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images