Image processing method, device, apparatus and storage medium

Abstract

An image processing method, device, equipment and storage medium are provided in the embodiments of the present application. The method comprises: collecting a sequence of exposure images under a target exposure condition, the sequence of exposure images comprising a plurality of exposure images; determining a reference exposure image from the plurality of exposure images; calculating an image difference parameter between a target exposure image and the reference exposure image in the sequence of exposure images, the target exposure image being an exposure image other than the reference exposure image in the sequence of exposure images; determining a plurality of fusion exposure images in the target exposure image according to the image difference parameter; and obtaining a denoising image according to the plurality of fusion images. The image processing method provided in the present scheme reduces the workload of human participation when obtaining the denoising image, and improves the efficiency of obtaining the denoising image. On the other hand, the quality of the denoising image is also effectively improved, the fitting degree of the denoising image and the real scene is improved, and thus the effect of network denoising is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an image processing method, apparatus, device, and storage medium. Background Technology

[0002] Image noise refers to unnecessary or redundant interference information present in image data. The presence of noise seriously affects image quality, therefore, effective image noise removal is essential.

[0003] Currently, with the widespread application of deep learning, deep learning-based image algorithms are also widely used in existing cameras and mobile phones. In denoising applications, deep learning-based denoising is mainly divided into supervised and unsupervised denoising schemes. Supervised denoising schemes require collecting noisy and noise-free images to form image pairs, and then using these image pairs for network training. Unsupervised denoising, on the other hand, mainly designs denoising based on the correlation of noise within the image, without needing to collect noise-free images. In terms of performance, supervised learning-based denoising is far superior to supervised image denoising, but constructing its data pairs is more complex. Summary of the Invention

[0004] This application provides an image processing method that can effectively improve the efficiency and performance of acquiring denoised images in the prior art.

[0005] In a first aspect, this application provides an image processing method, the method comprising:

[0006] A sequence of exposure images is acquired under the target exposure conditions, the sequence of exposure images including multiple frames of exposure images;

[0007] A reference exposure image is determined from the multiple exposure images;

[0008] Calculate the image difference parameter between the target exposure image and the reference exposure image in the exposure image sequence, wherein the target exposure image is any other exposure image in the exposure image sequence other than the reference exposure image;

[0009] Based on the image difference parameters, determine the multiple frames of fusionable exposure images in the target exposure image;

[0010] A denoised image is obtained from the multi-frame fused image.

[0011] In some embodiments of this application, determining the reference exposure image among the multiple exposure images includes:

[0012] The confidence level of any one of the multiple exposure images is calculated as the reference exposure image, resulting in multiple confidence levels.

[0013] The reference exposure image is determined from the multiple exposure images based on the multiple confidence levels.

[0014] In some embodiments of this application, the step of calculating the confidence level of any one of the multiple exposure images as a reference exposure image to obtain multiple confidence levels includes:

[0015] Each of the multiple exposure images is taken as the first exposure image, and the image similarity between the first exposure image and the second exposure image is calculated to obtain the image similarity set of the first exposure image. The image similarity set includes multiple image similarities, and the second exposure image is the other exposure image in the exposure image sequence except for the first exposure image.

[0016] After calculating the image similarity set corresponding to each of the multiple exposed images, multiple image similarity sets are obtained;

[0017] Based on the multiple image similarity sets, the credibility of any one of the multiple exposure images as the reference exposure image is determined, resulting in multiple credibility values.

[0018] In some embodiments of this application, the image difference parameters include motion difference parameters and brightness difference parameters, and calculating the image difference parameters between the target exposure image and the reference exposure image in the exposure image sequence includes:

[0019] A first difference image is obtained by determining the first difference image between the reference exposure image and the target exposure image;

[0020] The local variance of the first difference image is calculated using the target variance window to obtain a local variance set. A local variance set includes multiple local variances, and one first difference image corresponds to one local variance set. The local variance is the motion difference parameter.

[0021] In some embodiments of this application, calculating the image difference parameter between the target exposure image and the reference exposure image in the exposure image sequence further includes:

[0022] Denoise is applied to the reference exposure image and the target exposure image respectively to obtain a first denoised image of the reference exposure image and a second denoised image of the target exposure image;

[0023] Each of the first denoised image and the plurality of second denoised images is determined as a plurality of second difference images;

[0024] The local brightness mean of the multiple second difference images is calculated using the target brightness mean calculation window to obtain multiple local brightness mean sets. The local brightness mean is the brightness difference parameter. Each local brightness mean set includes multiple local brightness mean values, and one second difference image corresponds to one local brightness mean set.

[0025] In some embodiments of this application, the local variance set is multiple, and the step of determining the multi-frame fusionable exposure image in the target exposure image based on the image difference parameter includes:

[0026] The maximum local variance in each of the multiple local variance sets is determined to obtain multiple maximum local variances for different local variance sets.

[0027] Determine the minimum value among the plurality of maximum local variances as the initial local variance threshold;

[0028] Based on the initial local variance threshold, determine the local variance threshold;

[0029] Based on the plurality of maximum local variances and the local variance threshold, a plurality of third exposure images with no motion difference between them and the reference exposure image are determined in the target exposure image;

[0030] The multi-frame fusionable exposure image is determined from the plurality of third exposure images.

[0031] In some embodiments of this application, determining the multi-frame fusionable exposure image among the plurality of third exposure images includes:

[0032] Determine a set of multiple target local brightness mean values ​​corresponding to the multiple third exposure images;

[0033] The maximum local brightness mean value in each of the multiple target local brightness mean value sets is determined respectively, resulting in multiple maximum local brightness mean values ​​for different local brightness mean value sets;

[0034] The minimum value among the plurality of maximum local brightness averages is determined as the initial local brightness average threshold;

[0035] Based on the initial local brightness mean threshold, the local brightness mean threshold is determined;

[0036] Based on the plurality of maximum local brightness averages and the local brightness average threshold, the multi-frame fusionable exposure image is determined from the plurality of third exposure images.

[0037] Secondly, embodiments of this application provide an image processing apparatus, the apparatus comprising:

[0038] An exposure image acquisition module is used to acquire a sequence of exposure images under target exposure conditions, wherein the exposure image sequence includes multiple frames of exposure images;

[0039] A reference exposure image determination module is used to determine a reference exposure image among the multiple exposure images;

[0040] An image difference determination module is used to calculate the image difference parameter between a target exposure image and a reference exposure image in the exposure image sequence, wherein the target exposure image is any other exposure image in the exposure image sequence other than the reference exposure image.

[0041] The image fusion determination module is used to determine multiple frames of fusionable exposure images in the target exposure image based on the image difference parameters.

[0042] An image fusion module is used to obtain a denoised image based on the multi-frame fusionable images.

[0043] In some embodiments of this application, the reference exposure image determination module is specifically used for:

[0044] The confidence level of any one of the multiple exposure images is calculated as the reference exposure image, resulting in multiple confidence levels.

[0045] The reference exposure image is determined from the multiple exposure images based on the multiple confidence levels.

[0046] In some embodiments of this application, the reference exposure image determination module is specifically used to: take any exposure image in the multi-frame exposure images as the first exposure image, calculate the image similarity between the first exposure image and the second exposure image, and obtain the image similarity set of the first exposure image, wherein the image similarity set includes multiple image similarities, and the second exposure image is other exposure images in the exposure image sequence other than the first exposure image;

[0047] After calculating the image similarity set corresponding to each of the multiple exposed images, multiple image similarity sets are obtained;

[0048] Based on the multiple image similarity sets, the credibility of any one of the multiple exposure images as the reference exposure image is determined, resulting in multiple credibility values.

[0049] In some embodiments of this application, the image difference parameters include motion difference parameters and brightness difference parameters, and the fused image determination module is specifically used for:

[0050] A first difference image is obtained by determining the first difference image between the reference exposure image and the target exposure image;

[0051] The local variance of the first difference image is calculated using the target variance window to obtain a local variance set. A local variance set includes multiple local variances, and one first difference image corresponds to one local variance set. The local variance is the motion difference parameter.

[0052] In some embodiments of this application, the fused image determination module is specifically used for:

[0053] Denoise is applied to the reference exposure image and the target exposure image respectively to obtain a first denoised image of the reference exposure image and a second denoised image of the target exposure image;

[0054] Each of the first denoised image and the plurality of second denoised images is determined as a plurality of second difference images;

[0055] The local brightness mean of the multiple second difference images is calculated using the target brightness mean calculation window to obtain multiple local brightness mean sets. The local brightness mean is the brightness difference parameter. Each local brightness mean set includes multiple local brightness mean values, and one second difference image corresponds to one local brightness mean set.

[0056] In some embodiments of this application, the local variance set is multiple, and the fused image determination module is specifically used for:

[0057] The maximum local variance in each of the multiple local variance sets is determined to obtain multiple maximum local variances for different local variance sets.

[0058] Determine the minimum value among the plurality of maximum local variances as the initial local variance threshold;

[0059] Based on the initial local variance threshold, determine the local variance threshold;

[0060] Based on the plurality of maximum local variances and the local variance threshold, a plurality of third exposure images with no motion difference between them and the reference exposure image are determined in the target exposure image;

[0061] The multi-frame fusionable exposure image is determined from the plurality of third exposure images.

[0062] In some embodiments of this application, the fused image determination module is specifically used for:

[0063] Determine a set of multiple target local brightness mean values ​​corresponding to the multiple third exposure images;

[0064] The maximum local brightness mean value in each of the multiple target local brightness mean value sets is determined respectively, resulting in multiple maximum local brightness mean values ​​for different local brightness mean value sets;

[0065] The minimum value among the plurality of maximum local brightness averages is determined as the initial local brightness average threshold;

[0066] Based on the initial local brightness mean threshold, the local brightness mean threshold is determined;

[0067] Based on the plurality of maximum local brightness averages and the local brightness average threshold, the multi-frame fusionable exposure image is determined from the plurality of third exposure images.

[0068] Thirdly, this application provides an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the image processing method as described in any of the first aspects.

[0069] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps of the image processing method as described in any of the first aspects.

[0070] Fifthly, embodiments of this application provide a chip that includes a processor coupled to a transceiver of a terminal device, for executing the technical solution provided in the first aspect of embodiments of this application.

[0071] In a sixth aspect, embodiments of this application provide a chip system including a processor for supporting a terminal device in implementing the functions involved in the first aspect above, such as generating or processing an exposure image involved in the image processing method provided in the first aspect above.

[0072] In one possible design, the aforementioned chip system also includes a memory for storing program instructions and data necessary for the terminal. The chip system can be composed of chips or may include chips and other discrete components.

[0073] In a seventh aspect, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the computer to execute the image processing method provided in the first aspect.

[0074] This application provides an image processing method, apparatus, device, and storage medium. The method includes: acquiring an exposure image sequence under target exposure conditions, the exposure image sequence including multiple exposure images; determining a reference exposure image among the multiple exposure images; calculating image difference parameters between a target exposure image and the reference exposure image in the exposure image sequence, the target exposure image being any other exposure image in the exposure image sequence besides the reference exposure image; determining multiple fusionable exposure images in the target exposure image based on the image difference parameters; and obtaining a denoised image from the multiple fusionable images. The image processing method proposed in this solution reduces the amount of manual work involved in obtaining the denoised image, thus improving the efficiency of acquiring the denoised image. Furthermore, it effectively improves the quality of the denoised image, enhancing its fit with the real scene, thereby improving the effectiveness of network denoising. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1 This is a schematic diagram of an embodiment of the image processing system provided in this application;

[0077] Figure 2 This is a schematic flowchart of an embodiment of the image processing method provided in this application;

[0078] Figure 3 This is a schematic flowchart of an embodiment of determining a reference exposure image provided in this application;

[0079] Figure 4 This is a schematic flowchart of an embodiment for calculating image difference parameters in this application.

[0080] Figure 5 This is a schematic flowchart of another embodiment for calculating image difference parameters in this application.

[0081] Figure 6 This is a schematic flowchart of an embodiment of determining multiple fused images provided in this application;

[0082] Figure 7 This is a schematic flowchart illustrating the process of determining fusionable exposure images provided in an embodiment of this application;

[0083] Figure 8 A complete flowchart of an embodiment of the image processing method provided in this application is shown.

[0084] Figure 9 This is a schematic diagram of an embodiment of the image processing apparatus provided in this application.

[0085] Figure 10 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0086] Figure 11 This is a schematic diagram of the terminal structure provided in the embodiments of this application;

[0087] Figure 12 This is a schematic diagram of a server structure provided in an embodiment of this application. Detailed Implementation

[0088] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0089] In the following description, specific embodiments of this application will be illustrated with reference to steps and symbols performed by one or more computers, unless otherwise stated. Therefore, these steps and operations will be referred to several times as being performed by a computer, and computer execution as referred to herein includes operations by a computer processing unit representing electronic signals of data in a structured format. This operation transforms the data or maintains it at a location in the computer's memory system, which can be reconfigured or otherwise alter the operation of the computer in a manner well known to those skilled in the art. The data structure maintained by the data is the physical location of the memory, which has specific characteristics defined by the data format. However, the principles of this application are described in the foregoing text, which is not intended to be limiting, and those skilled in the art will understand that many of the steps and operations described below can also be implemented in hardware.

[0090] The terms "module" or "unit" as used herein can be considered as software objects executing on the computing system. The different components, modules, engines, and services described herein can be considered as implementation objects on the computing system. The apparatus and methods described herein are preferably implemented in software, but can also be implemented in hardware, both of which are within the scope of this application.

[0091] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0092] Please refer to Figure 1 This application also provides an image processing system, such as Figure 1 As shown, the image processing system includes a computing device 100, which integrates the image processing apparatus provided in this application. In this embodiment, the computing device 100 may be a terminal device or a server.

[0093] In this embodiment, when the computing device 100 is a server, the server can be an independent server, a server network, or a server cluster. For example, the server described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing. In this embodiment, communication between the server and the client can be achieved through any communication method, including but not limited to mobile communication based on the 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), and Worldwide Interoperability for Microwave Access (WiMAX), or computer network communication based on the TCP / IP Protocol Suite (TCP / IP) and User Datagram Protocol (UDP).

[0094] It is understood that when the computing device 100 used in the embodiments of this application is a terminal device, the terminal device can be a device that includes both receiving hardware and transmitting hardware, that is, a device with receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such a terminal device may include: cellular or other communication devices, which have a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. Specifically, the computing device 100 may be a desktop terminal or a mobile terminal, and may specifically be one of a mobile phone, tablet computer, laptop computer, etc.

[0095] The terminal devices involved in the embodiments of this application can also be devices that provide voice and / or data connectivity to users, handheld devices with wireless connectivity, or other processing devices connected to a wireless modem. Examples include mobile phones (or "cellular" phones) and computers with mobile terminals, such as portable, pocket-sized, handheld, computer-embedded, or vehicle-mounted mobile devices that exchange voice and / or data with a wireless access network. Examples include Personal Communication Service (PCS) phones, cordless phones, Session Initiation Protocol (SIP) phones, Wireless Local Loop (WLL) stations, Personal Digital Assistants (PDAs), and other devices.

[0096] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include those that are more specific to this application. Figure 1 The number of computing devices shown, or the network connectivity of computing devices, for example... Figure 1 Only one computing device is shown in the image. It is understood that the image processing system may also include one or more other computing devices, and / or one or more other computing devices that are networked with computing device 100, without being limited here.

[0097] In addition, such as Figure 1 As shown, the image processing system may also include a memory 200 for storing the exposed images.

[0098] In this embodiment of the application, the memory 200 can be a cloud memory. Cloud storage is a new concept that is extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology and distributed storage file system functions to bring together a large number of storage devices of various types in the network (storage devices are also called storage nodes) through application software or application interfaces to work together to provide data storage and business access functions to the outside world.

[0099] Currently, the storage method of storage systems is as follows: Logical volumes are created. During the creation of a logical volume, physical storage space is allocated to each logical volume. This physical storage space may consist of a single storage device or the disks of several storage devices. Clients store data on a logical volume, which means storing the data on the file system. The file system divides the data into many parts, each part being an object. Each object contains not only the data but also additional information such as a data identifier (ID, ID entity). The file system writes each object to the physical storage space of that logical volume and records the storage location information of each object. Therefore, when a client requests access to data, the file system can allow the client to access the data based on the storage location information of each object.

[0100] The process by which a storage system allocates physical storage space to a logical volume is as follows: the physical storage space is pre-divided into strips according to the capacity estimate of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of Redundant Array of Independent Disks (RAID). A logical volume can be understood as a strip, thus allocating physical storage space to the logical volume.

[0101] It should be noted that, Figure 1 The schematic diagram of the image processing system shown is merely an example. The image processing system and scenario described in this application are for the purpose of more clearly illustrating the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of image processing systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0102] The solutions provided in this application involve technologies such as Artificial Intelligence (AI), Computer Vision (CV), and Machine Learning (ML), which are specifically illustrated through the following embodiments:

[0103] AI, or Artificial Intelligence, refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, Artificial Intelligence is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine capable of reacting in a manner similar to human intelligence. Artificial Intelligence studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0104] AI technology is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0105] Computer vision (CV) is the science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes for target recognition, tracking, and measurement, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include adversarial perturbation generation, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0106] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0107] Please see Figure 2 , Figure 2 The image processing method provided in this application embodiment includes the following steps 201 to 205:

[0108] 201. Acquire a sequence of exposure images under the target exposure conditions.

[0109] The image processing method provided in this application relies on the acquired real images when constructing denoised images. Therefore, it is first necessary to acquire an exposure image sequence including multiple exposure images; and this exposure image sequence needs to be acquired under certain exposure conditions.

[0110] Specifically, an Automatic Exposure Control (AEC) module can be used for metering to determine the exposure parameters for image acquisition. The metering method can be set according to the actual situation and is not limited here. After determining the exposure parameters, to avoid brightness changes caused by switching between different frames, the selected exposure parameters need to be locked during actual image acquisition, and exposure and image acquisition should be performed under these exposure parameters. In one specific embodiment, the acquired exposure image sequence may include 200-300 consecutive frames.

[0111] 202. Determine the reference exposure image from the multi-frame exposure images.

[0112] The denoised images provided in this application are mainly obtained by fusing multiple actually detected images. Therefore, it is necessary to determine the fusible exposure images from the acquired exposure image sequence. Before determining the fusible exposure images, it is also necessary to determine a reference exposure image; the fusible exposure images are then determined based on the reference exposure image.

[0113] like Figure 3 As shown, in some embodiments, determining a reference exposure image among multiple exposure images may include steps 301-304:

[0114] 301. Using any one of the multiple exposure images as the first exposure image, calculate the image similarity between the first exposure image and the second exposure image to obtain the image similarity set of the first exposure image.

[0115] 302. After calculating the image similarity set corresponding to each of the multiple exposed images, multiple image similarity sets are obtained.

[0116] In some embodiments, any exposure image from a multi-frame exposure image sequence can be used as the first exposure image, and other exposure images in the exposure image sequence besides the first exposure image can be used as the second exposure image. The similarity between the first and second exposure images is then calculated to obtain an image similarity set for the first exposure image. This image similarity set typically includes multiple image similarities. By using any exposure image from a multi-frame exposure image sequence as the first exposure image, multiple image similarity sets can be obtained.

[0117] 303. Based on multiple image similarity sets, determine the credibility of any one of the multiple exposure images as the reference exposure image, and obtain multiple credibility values.

[0118] 304. Determine the reference exposure image from multiple exposure images based on multiple confidence levels.

[0119] Specifically, the average similarity of multiple images in an image similarity set can be calculated to obtain an average similarity, which is the confidence level. After calculating the confidence level for each image similarity set, the confidence levels can be sorted in ascending order; and the highest confidence level among the confidence levels can be determined as the target confidence level. The exposure image corresponding to the target confidence level is the baseline exposure image.

[0120] In other embodiments of this application, the difference between the first exposed image and the second exposed image can also be calculated; the smaller the difference, the higher the similarity between the two. Therefore, when calculating the difference between the first exposed image and the second exposed image, after calculating the average difference value of multiple difference values, it is necessary to determine the minimum value, rather than the maximum value, of the average difference values. The first exposed image corresponding to the minimum average difference value is the reference exposed image.

[0121] 203. Calculate the image difference parameters between the target exposure image and the reference exposure image in the exposure image sequence.

[0122] 204. Determine the multi-frame fusionable exposure images in the target exposure image based on image difference parameters.

[0123] 205. Obtain a denoised image from multiple fused images.

[0124] After determining the reference exposure image, other exposure images in the exposure image sequence that have little or no difference from the reference exposure image can be identified. These images are then superimposed to obtain a noise-free image (or a denoised image). This application designs denoising based on the correlation of noise within the image, specifically by superimposing noisy data frames to obtain a noise-free image, thus achieving a better-performing denoised image. The specific process for determining the image difference parameters between the target exposure image and the reference exposure image will be described in subsequent embodiments and is not limited here.

[0125] This application provides an image processing method, which includes: acquiring an exposure image sequence under target exposure conditions, the exposure image sequence including multiple exposure images; determining a reference exposure image among the multiple exposure images; calculating image difference parameters between a target exposure image and the reference exposure image in the exposure image sequence, the target exposure image being any other exposure image in the exposure image sequence besides the reference exposure image; determining multiple fusionable exposure images in the target exposure image based on the image difference parameters; and obtaining a denoised image from the multiple fusionable images. The image processing method proposed in this solution reduces the amount of manual work involved in obtaining the denoised image, thus improving the efficiency of acquiring the denoised image. On the other hand, it also effectively improves the quality of the denoised image, enhancing its fit with the real scene, thereby improving the effect of network denoising.

[0126] In some embodiments, the image difference parameters may include both motion difference parameters and brightness difference parameters, such as... Figure 4 As shown, calculating the image difference parameters between the target exposed image and the reference exposed image in an exposed image sequence can include:

[0127] 401. Determine the first difference image between the reference exposure image and the target exposure image to obtain the first difference image.

[0128] 402. Calculate the local variance of multiple first difference images using the target variance window to obtain the local variance set.

[0129] When calculating motion differences, it is first necessary to determine the first difference image between the reference exposure image and the target exposure image. This can be achieved by pairing the reference exposure image with each of the remaining exposure images in the exposure image sequence, and then calculating the difference value between each exposure image and the reference exposure image. Since there can be one or more target exposure images, there can also be multiple first difference images when there are multiple target exposure images, with one first difference image corresponding to one target exposure image.

[0130] After obtaining multiple first difference images, a target variance window is needed to calculate the local variance of each of the multiple first difference images. The local variance is the motion difference parameter. The size of the target variance window is typically smaller than the size of the first difference images, so that the calculated variance is local. A single first difference image includes multiple local variances, which together form a local variance set; multiple first difference images can yield multiple local variance sets. In the embodiments of this application, the size of the target variance window can be set according to actual needs, for example, 5x5; other sizes are also possible in other embodiments, and are not limited here.

[0131] In the above embodiments, the initial local variance of the first difference image is actually calculated using the target variance window. After calculating an initial local variance set including multiple initial local variances for a first difference image, denoising processing is required on the multiple initial local variances in the initial local variance set to eliminate pixels with large deviations in the initial local variances and avoid abnormal points in the first difference image affecting the calculation of local variances. The final local variance, i.e., the final motion difference parameter, is obtained after denoising. In some embodiments, methods such as median filters can be used to denoise the initial local variances, which is not limited here. In this application, motion differences (including small brightness differences) in images are detected based on local variance, making the processing of exposure images generated under motion (or small brightness changes) more robust.

[0132] It should be noted that, in the above embodiments, the calculated motion difference parameters actually include a portion of smaller brightness changes; when calculating the brightness difference parameters subsequently, the brightness changes of larger areas are mainly calculated.

[0133] 403. Denoise the reference exposure image and the target exposure image respectively to obtain the first denoised image after denoising the reference exposure image and multiple second denoised images after denoising the target exposure image.

[0134] 404. Determine multiple second difference images between the first denoised image and multiple second denoised images respectively.

[0135] When calculating the brightness difference parameter, it is necessary to first denoise the reference exposure image and the target exposure image, and then use the denoised images to determine the brightness difference. This is because pixel abrupt changes have a significant impact on brightness variations, so it is necessary to remove pixels with abrupt changes before calculating the average brightness; to avoid pixels with abrupt changes affecting the calculated average brightness.

[0136] 405. Calculate the local brightness mean of multiple second difference images using the target brightness mean calculation window to obtain multiple local brightness mean sets.

[0137] After obtaining multiple second difference images, it is necessary to determine the brightness difference parameter based on the second difference images. Specifically, a preset mean calculation window can be used to calculate the local brightness mean of each second difference image. Similar to the motion difference parameter, the brightness mean calculated here is the local brightness mean. One second difference image corresponds to one set of local brightness means, and one set of local brightness means includes multiple local brightness means; that is, one second difference image corresponds to multiple local brightness means. Moreover, the initial local brightness mean calculated using the target brightness mean calculation window is the initial local brightness mean, which needs to be further filtered and denoised to obtain the final local brightness mean, i.e., the brightness difference parameter. In some embodiments, the size of the target brightness mean calculation window can be set according to actual needs, such as 3*3, and is not limited here. In this application, brightness difference detection in images based on local brightness mean makes the processing of exposure images generated under large brightness changes more robust.

[0138] It should be noted that when actually calculating motion difference parameters and brightness difference parameters, they are calculated separately and independently, and there is no order in their calculation processes.

[0139] like Figure 5 The diagram shown is a schematic flowchart of an embodiment of determining image difference parameters provided in this application. Figure 5 In this process, a base frame and a target frame (i.e., the current frame) form an image pair. Then, the image difference parameters between the two frames are calculated. For the motion difference parameters, the diff value between the base and target frames is directly calculated, resulting in the first difference image. Next, the Local Std module is used to calculate the initial local variance of the first difference image, obtaining an initial local variance set including multiple initial local variances. A median filter is then used to filter the initial local variances, yielding the final set of local variances, i.e., the motion difference parameters. Figure 5 In the process, the LocalStd Process module is used to detect motion and small changes in brightness. It includes several functional modules such as determining the difference image (diff), determining the initial local variance (Local Std), and filtering the initial local variance (median filter).

[0140] For the brightness difference parameter, firstly, the reference exposure image and the target exposure image need to be denoised separately to obtain a first denoised image and a second denoised image. Next, the diff value between the first and second denoised images is calculated, i.e., the second difference image. Then, the Local Mean module is used to determine the initial local brightness mean of the second difference image, obtaining an initial set of local brightness mean values. Similarly, median filtering is applied to the local brightness mean values ​​in the set of local brightness mean values ​​to obtain a filtered set of local brightness mean values, i.e., the brightness difference parameter. Figure 5 In the process, the Local Mean Process module is used to detect brightness changes over a large area. It includes several functional modules such as determining the difference image (diff), determining the initial local brightness mean (Local Mean), and filtering the initial local brightness mean (medianfilter).

[0141] In the aforementioned embodiments, motion difference parameters and brightness difference parameters were calculated separately. It is also necessary to determine the images that can be fused based on these parameters. For example... Figure 6 As shown, determining the multi-frame fusionable images in the target exposure image based on image difference parameters can include:

[0142] 601. Determine the maximum local variance in each of the multiple local variance sets to obtain multiple maximum local variances for different local variance sets.

[0143] 602. Determine the minimum value among multiple maximum local variances as the initial local variance threshold.

[0144] 603. Determine the local variance threshold based on the initial local variance threshold.

[0145] In one specific embodiment, the target exposure image can be first filtered based on motion difference parameters, and then filtered based on brightness difference parameters to finally determine the multi-frame images that can be fused from the target exposure image.

[0146] Specifically, since each local variance set contains multiple local variances, the local variance with the largest value in each set can be determined, resulting in multiple maximum local variances (Stdi); one local variance set corresponds to one maximum local variance. These maximum local variances are then sorted in ascending order, resulting in multiple sorted maximum local variances. The minimum value among these sorted maximum local variances needs to be determined as the initial local variance threshold (Std_min). However, the initial local variance threshold is not the final selection criterion for the motion difference parameter; the final local variance threshold needs to be determined based on the initial local variance threshold.

[0147] In one embodiment, since the initial local variance threshold is already the minimum among multiple maximum local variances, an adjustable parameter thres1 needs to be added to the initial local variance threshold Std_min to increase the initial local variance threshold and obtain the final local variance threshold, as follows:

[0148] Std_min = MIN(Std i )

[0149] Std_thres=Std_min*(1+thres1)

[0150] Where Stdi is the maximum local variance in each set of local variances, Std_min is the minimum value among multiple maximum local variances, i.e., the initial local variance threshold; and Std_thres is the local variance threshold obtained based on the initial local variance threshold. In a specific embodiment, the adjustable range of the adjustable parameter thres1 can be 0-15%.

[0151] 604. Based on multiple maximum local variances and local variance thresholds, determine multiple third exposure images in the target exposure image that have no motion difference from the reference exposure image.

[0152] Based on multiple maximum local variances and local variance thresholds, determining multiple third exposure images in the target exposure image that have no motion difference from the reference exposure image can include:

[0153] Among multiple maximum local variances, the target maximum local variance is determined: the maximum local variance is less than the local variance threshold, and the exposure image corresponding to the target maximum local variance is determined, which is the third exposure image.

[0154] 605. Determine the set of average local brightness values ​​of multiple targets corresponding to multiple third exposure images.

[0155] 606. Determine the maximum local brightness mean in each of the multiple target local brightness mean sets to obtain multiple maximum local brightness mean values ​​for different local brightness mean sets.

[0156] In the aforementioned embodiments, only multiple third-exposure images without motion differences were selected. Further filtering of these images is needed to determine those without brightness differences. Images without both motion and brightness differences can be fused. Similar to the process of determining the third-exposure images, when filtering using the average brightness value, multiple sets of target local brightness averages corresponding to the multiple third-exposure images can be determined first. Then, the maximum local brightness average value (Meani) in each target local brightness average value set can be determined, resulting in multiple maximum local brightness averages.

[0157] 607. Determine the minimum value among multiple maximum local brightness averages as the initial local brightness average threshold.

[0158] 608. Determine the local average brightness threshold based on the initial local average brightness threshold.

[0159] Similarly, multiple maximum local brightness averages can be sorted in ascending order, and the minimum value among them can be determined as the initial local brightness average threshold. An adjustable parameter `thres2` is then added to the initial local brightness average threshold to obtain the final local brightness average threshold. The details are as follows:

[0160] Mean_min=MIN(Mean i )

[0161] Mean_thres=Mean_min*(1+thres2)

[0162] Among them, Mean i Mean_min is the minimum local brightness mean among the maximum local brightness mean values ​​in each set of local brightness mean values, which is also the initial local brightness mean threshold. Mean_thres is the adjusted local brightness mean threshold. The adjustable parameter threshold2 can be set from 0% to 15%. It should be noted that the adjustable ranges of threshold1 and threshold2 can be the same or different; the specific values ​​of threshold1 and threshold2 can also be the same or different, depending on the actual needs.

[0163] 609. Based on multiple maximum local brightness averages and local brightness average thresholds, determine multiple frames of images that can be fused from multiple third exposure images.

[0164] After determining the local average brightness threshold, we can filter out the exposure images from multiple third exposure images where the maximum average brightness is less than the local average brightness threshold. These exposure images have motion difference parameters less than the local variance threshold and brightness difference parameters less than the local average brightness threshold. This indicates that the motion and brightness differences among these exposure images are relatively small, therefore they can be fused, resulting in a multi-frame fusionable exposure image.

[0165] It should be noted that when the image difference parameters include both motion difference parameters and brightness difference parameters, the target exposure image can be filtered first using the motion difference parameters, and then filtered again using the brightness difference parameters; as described above. In some other embodiments, the target exposure image can also be filtered first using the brightness difference parameters, and then filtered again using the motion difference parameters. That is, there is no fixed order for filtering using motion difference parameters and filtering using brightness difference parameters; the specific order can be set according to actual needs.

[0166] In the foregoing embodiments, the image difference parameters include both motion difference parameters and brightness difference parameters, therefore, they need to be used for filtering. In other embodiments, the image difference parameters may include only motion difference parameters or only brightness difference parameters. In this case, filtering based solely on motion difference parameters or solely on brightness difference parameters is sufficient to determine the images to be fused. The specific filtering process can be found in the foregoing embodiments and will not be repeated here.

[0167] like Figure 7 The diagram shown is a flowchart illustrating the process of determining fusionable exposure images according to an embodiment of this application. Figure 7 In this process, multiple image pairs (image pars) consisting of the baseline exposure image and the target exposure image need to have their motion difference parameters determined by the Local Std Process and their brightness difference parameters determined by the Local Mean Process. Then, a first filtering process (Std Sort) is performed based on the motion difference to obtain the third exposure image. This third exposure image then undergoes a second filtering process (MeanSort) based on the brightness difference parameters, ultimately resulting in multiple frames of fusionable exposure images. These images are then fused to obtain the fused denoised image. The fused denoised image can be combined with the baseline exposure image to form a noise pair, which can then be used as a training set for model training.

[0168] like Figure 8 The diagram shown is a complete flowchart of one embodiment of the image processing method provided in this application. Figure 8First, the exposure scene containing the desired exposure image sequence is selected, and the exposure parameters are determined using the automatic exposure control module. The exposure parameters need to be locked before acquiring the exposure image sequence. After acquiring multiple exposure images, one frame is selected as the base frame, and local motion compute is performed to calculate the image difference parameters. This results in multiple frames that can be fused, which are then fused to output the fused frame, which is the denoised image.

[0169] To facilitate better implementation of the image processing method provided in the embodiments of this application, the embodiments of this application also provide an apparatus based on the above-described image processing method. The meanings of the terms used are the same as in the above-described image processing method, and specific implementation details can be found in the descriptions within the method embodiments. For example... Figure 9 As shown, the image processing apparatus may include:

[0170] The exposure image acquisition module 901 is used to acquire a sequence of exposure images under target exposure conditions, the sequence of exposure images including multiple frames of exposure images;

[0171] The reference exposure image determination module 902 is used to determine a reference exposure image among multiple exposure images;

[0172] The image difference determination module 903 is used to calculate the image difference parameters between the target exposed image and the reference exposed image in the exposed image sequence. The target exposed image is any other exposed image in the exposed image sequence other than the reference exposed image.

[0173] The image fusion determination module 904 is used to determine multiple frames of fusionable exposure images in the target exposure image based on image difference parameters.

[0174] Image fusion module 905 is used to obtain a denoised image from multiple fused images.

[0175] This application provides an image processing apparatus. First, under target exposure conditions, an exposure image sequence is acquired, including multiple exposure images. A reference exposure image is determined from these multiple exposure images. Image difference parameters are calculated between the target exposure image and the reference exposure image in the exposure image sequence. The target exposure image is any exposure image in the sequence other than the reference exposure image. Multiple fusionable exposure images are determined from the target exposure image based on the image difference parameters. A denoised image is obtained from these multiple fusionable images. This proposed image processing method reduces the amount of manual work required to obtain a denoised image, thus improving the efficiency of acquiring the denoised image. Furthermore, it effectively improves the quality of the denoised image, enhancing its fit with the real scene, thereby improving the overall denoising effect of the network.

[0176] In some embodiments of this application, the reference exposure image determination module 902 is specifically used for:

[0177] The confidence level of any one of the multiple exposure images is calculated as the reference exposure image, resulting in multiple confidence levels. The reference exposure image is then determined from the multiple exposure images based on these multiple confidence levels.

[0178] In some embodiments of this application, the reference exposure image determination module 902 is specifically used for:

[0179] Each exposure image is taken as the first exposure image from multiple exposure images. The image similarity between the first exposure image and the second exposure image is calculated to obtain the image similarity set of the first exposure image. The image similarity set includes multiple image similarities. The second exposure image is the other exposure image in the exposure image sequence except for the first exposure image.

[0180] After calculating the image similarity set corresponding to each of the multiple exposure images, multiple image similarity sets are obtained. Based on the multiple image similarity sets, the credibility of any one of the multiple exposure images as the reference exposure image is determined, and multiple credibility values ​​are obtained.

[0181] In some embodiments of this application, the image difference parameters include motion difference parameters and brightness difference parameters, and the fused image determination module 904 can specifically be used for:

[0182] A first difference image is determined between the reference exposure image and the target exposure image to obtain the first difference image; the local variance of the first difference image is calculated using the target variance window to obtain a local variance set. A local variance set includes multiple local variances, and a first difference image corresponds to a local variance set. The local variance is the motion difference parameter.

[0183] In some embodiments of this application, the fused image determination module 904 may specifically be used for:

[0184] Denoise the reference exposure image and the target exposure image separately to obtain a first denoised image after denoising the reference exposure image and a second denoised image after denoising the target exposure image; determine multiple second difference images between the first denoised image and multiple second denoised images respectively;

[0185] The local brightness mean of multiple second difference images is calculated using the target brightness mean calculation window, resulting in multiple local brightness mean sets. The local brightness mean is the brightness difference parameter. Each local brightness mean set includes multiple local brightness mean values, and one second difference image corresponds to one local brightness mean set.

[0186] In some embodiments of this application, the local variance set is multiple, and the fused image determination module 904 can specifically be used for:

[0187] The maximum local variance in each of the multiple local variance sets is determined to obtain multiple maximum local variances for different local variance sets.

[0188] The minimum value among multiple maximum local variances is determined as the initial local variance threshold; based on the initial local variance threshold, the local variance threshold is determined; based on the multiple maximum local variances and the local variance threshold, multiple third exposure images with no motion difference between them and the reference exposure image are determined in the target exposure image; multiple frames of fusionable exposure images are determined from the multiple third exposure images.

[0189] In some embodiments of this application, the fused image determination module 904 may specifically be used for:

[0190] Multiple target local brightness mean sets corresponding to multiple third exposure images are determined; the maximum local brightness mean in each local brightness mean set in the multiple target local brightness mean sets is determined respectively, to obtain multiple maximum local brightness mean values ​​of different local brightness mean sets; the minimum value among the multiple maximum local brightness mean values ​​is determined as the initial local brightness mean threshold; a local brightness mean threshold is determined based on the initial local brightness mean threshold; and multiple frames of fusionable exposure images are determined from the multiple third exposure images based on the multiple maximum local brightness mean values ​​and the local brightness mean threshold.

[0191] This application also provides an electronic device, such as... Figure 10 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0192] The electronic device may include components such as a processor 1001 with one or more processing cores, a memory 1002 with one or more computer-readable storage media, a power supply 1003, and an input unit 1004. Those skilled in the art will understand that... Figure 10 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0193] The processor 1001 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 1002, and by calling data stored in the memory 1002, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, the processor 1001 may include one or more processing cores; preferably, the processor 1001 may integrate an application processor and a modem processor. The application processor mainly handles the operation of the storage medium, user interface, and application programs, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1001.

[0194] The memory 1002 can be used to store software programs and modules. The processor 1001 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002. The memory 1002 may mainly include a program storage area and a data storage area. The program storage area may store applications required for operating the storage medium and at least one function (such as sound playback function, image playback function, etc.); the data storage area may store data created according to the use of the electronic device. In addition, the memory 1002 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 1002 may also include a memory controller to provide the processor 1001 with access to the memory 1002.

[0195] The electronic device also includes a power supply 1003 that supplies power to various components. Preferably, the power supply 1003 can be logically connected to the processor 1001 through a power management storage medium, thereby enabling functions such as charging, discharging, and power consumption management through the power management storage medium. The power supply 1003 may also include one or more DC or AC power supplies, recharge storage media, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0196] The electronic device may also include an input unit 1004, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0197] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 1001 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 1002 according to the following instructions, and the processor 1001 runs the applications stored in the memory 1002, thereby realizing the steps in the above embodiment of the federated learning-based model training method.

[0198] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0199] Therefore, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, the computer program being loaded by a processor to execute any of the steps in the federated learning-based model training method provided in embodiments of this application.

[0200] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0201] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc. Since the computer program stored in the computer-readable storage medium can execute the steps of any of the federated learning-based model training methods provided in the embodiments of this application, it can achieve the beneficial effects that any of the federated learning-based model training methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0202] When the computing device in this application embodiment is a terminal device, this application embodiment also provides a terminal device, such as... Figure 11 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The terminal device can be any terminal device including mobile phones, tablets, personal digital assistants (PDAs), point-of-sale (POS) terminals, in-vehicle computers, etc. Taking a mobile phone as an example:

[0203] Figure 11 This diagram illustrates a partial structure of a mobile phone related to the terminal device provided in the embodiments of this application. (Reference) Figure 11The mobile phone includes components such as a radio frequency (RF) circuit 1110, a memory 1120, an input unit 1130, a display unit 1140, a sensor 1150, an audio circuit 1160, a wireless fidelity (WiFi) module 1170, a processor 1180, and a power supply 1190. Those skilled in the art will understand that... Figure 8 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0204] The following is combined Figure 11 A detailed introduction to each component of a mobile phone:

[0205] RF circuit 1110 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with processor 1180; additionally, it transmits uplink data to the base station. Typically, RF circuit 1110 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, RF circuit 1110 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0206] The memory 1120 can be used to store software programs and modules. The processor 1180 executes various functions and data processing of the mobile phone by running the software programs and modules stored in the memory 1120. The memory 1120 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 1120 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0207] The input unit 1130 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 1130 may include a touch panel 1131 and other input devices 1132. The touch panel 1131, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1131), and drive the corresponding connection devices according to a pre-set program. Optionally, the touch panel 1131 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 1180, and can receive and execute commands sent by the processor 1180. In addition, the touch panel 1131 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1131, the input unit 1130 may also include other input devices 1132. Specifically, other input devices 1132 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0208] Display unit 1140 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. Display unit 1140 may include display panel 1141, optionally configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar display panel 1141. Further, touch panel 1131 may cover display panel 1141. When touch panel 1131 detects a touch operation on or near it, it transmits the information to processor 1180 to determine the type of touch event. Subsequently, processor 1180 provides corresponding visual output on display panel 1141 based on the type of touch event. Although in Figure 11 In this embodiment, the touch panel 1131 and the display panel 1141 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 1131 and the display panel 1141 can be integrated to realize the input and output functions of the mobile phone.

[0209] The mobile phone may also include at least one sensor 1150, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1141 according to the ambient light level, and the proximity sensor can turn off the display panel 1141 and / or the backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0210] Audio circuit 1160, speaker 1161, and microphone 1162 provide an audio interface between the user and the mobile phone. Audio circuit 1160 converts received audio data into electrical signals and transmits them to speaker 1161, where speaker 1161 converts them into sound signals for output. On the other hand, microphone 1162 converts collected sound signals into electrical signals, which are received by audio circuit 1160, converted into audio data, and then processed by processor 1180 before being transmitted via RF circuit 1110 to, for example, another mobile phone, or the audio data can be output to memory 1120 for further processing.

[0211] Wi-Fi is a short-range wireless transmission technology. Through the Wi-Fi module 1170, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 11 Wi-Fi module 1170 is shown, but it is understood that it is not an essential component of a mobile phone and can be omitted as needed without changing the nature of the invention.

[0212] The processor 1180 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 1120, and calls data stored in the memory 1120 to perform various functions and process data, thereby providing overall monitoring of the phone. Optionally, the processor 1180 may include one or more processing units; optionally, the processor 1180 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into the processor 1180.

[0213] The mobile phone also includes a power supply 1190 (such as a battery) that supplies power to various components. Optionally, the power supply can be logically connected to the processor 1180 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0214] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0215] This application also provides a server; please refer to [link / reference]. Figure 12 , Figure 12 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 1200 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1222 (e.g., one or more processors) and memory 1232, and one or more storage media 1230 (e.g., one or more mass storage devices) for storing application programs 1242 or data 1244. The memory 1232 and storage media 1230 can be temporary or persistent storage. The program stored in the storage media 1230 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the server. Furthermore, the CPU 1222 may be configured to communicate with the storage media 1230 and execute the series of instruction operations in the storage media 1230 on the server 1200.

[0216] Server 1200 may also include one or more power supplies 1226, one or more wired or wireless network interfaces 1250, one or more input / output interfaces 1258, and / or one or more operating systems 1241, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0217] The steps in the federated learning-based model training method described in the above embodiments can be based on this... Figure 12 The structure of server 1200 is shown. For example, the central processing unit 1222 performs the following operations by calling instructions from memory 1232:

[0218] Acquire a sequence of exposure images under the target exposure conditions, which includes multiple exposure images; determine a reference exposure image from the multiple exposure images; calculate the image difference parameter between the target exposure image and the reference exposure image in the exposure image sequence, where the target exposure image is any other exposure image in the exposure image sequence besides the reference exposure image; determine multiple fusionable exposure images from the target exposure image based on the image difference parameter; and obtain a denoised image from the multiple fusionable images.

[0219] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0220] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0221] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or modules, and may be electrical, mechanical, or other forms.

[0222] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0223] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0224] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0225] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0226] The technical solutions provided in the embodiments of this application have been described in detail above. Specific examples have been used in the embodiments of this application to illustrate the principles and implementation methods of the embodiments of this application. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.

Claims

1. An image processing method, characterized in that, The method includes: A sequence of exposure images is acquired under the target exposure conditions, the sequence of exposure images including multiple frames of exposure images; A reference exposure image is determined from the multiple exposure images; Calculate the image difference parameter between the target exposure image and the reference exposure image in the exposure image sequence, wherein the target exposure image is any other exposure image in the exposure image sequence other than the reference exposure image; Based on the image difference parameters, determine the multiple frames of fusionable exposure images in the target exposure image; A denoised image is obtained from the multi-frame fusionable image; The step of determining the reference exposure image among the multiple exposure images includes: Each of the multiple exposure images is taken as the first exposure image, and the image similarity between the first exposure image and the second exposure image is calculated to obtain the image similarity set of the first exposure image. The image similarity set includes multiple image similarities, and the second exposure image is the other exposure image in the exposure image sequence except for the first exposure image. The average of multiple image similarities in the image similarity set of the first exposed image is calculated, and the average similarity is determined as the credibility of the first exposed image; The exposure image corresponding to the maximum confidence value among the multi-frame exposure images is determined as the reference exposure image.

2. The image processing method according to claim 1, characterized in that, Determining the reference exposure image among the multiple exposure images includes: The confidence level of any one of the multiple exposure images is calculated as the reference exposure image, resulting in multiple confidence levels. The reference exposure image is determined from the multiple exposure images based on the multiple confidence levels.

3. The image processing method according to claim 1, characterized in that, The image difference parameters include motion difference parameters and brightness difference parameters. Calculating the image difference parameters between the target exposed image and the reference exposed image in the exposed image sequence includes: A first difference image is obtained by determining the first difference image between the reference exposure image and the target exposure image; The local variance of the first difference image is calculated using the target variance window to obtain a local variance set. A local variance set includes multiple local variances, and one first difference image corresponds to one local variance set. The local variance is the motion difference parameter.

4. The image processing method according to claim 3, characterized in that, The calculation of the image difference parameter between the target exposure image and the reference exposure image in the exposure image sequence further includes: Denoise is applied to the reference exposure image and the target exposure image respectively to obtain a first denoised image of the reference exposure image and a second denoised image of the target exposure image; Each of the first denoised image and the plurality of second denoised images is determined as a plurality of second difference images; The local brightness mean of the multiple second difference images is calculated using the target brightness mean calculation window to obtain multiple local brightness mean sets. The local brightness mean is the brightness difference parameter. Each local brightness mean set includes multiple local brightness mean values, and one second difference image corresponds to one local brightness mean set.

5. The image processing method according to claim 4, characterized in that, The local variance set is multiple, and the step of determining the multi-frame fusionable exposure images in the target exposure image based on the image difference parameters includes: The maximum local variance in each of the multiple local variance sets is determined to obtain multiple maximum local variances for different local variance sets. Determine the minimum value among the plurality of maximum local variances as the initial local variance threshold; Based on the initial local variance threshold, determine the local variance threshold; Based on the plurality of maximum local variances and the local variance threshold, a plurality of third exposure images with no motion difference between them and the reference exposure image are determined in the target exposure image; The multi-frame fusionable exposure image is determined from the plurality of third exposure images.

6. The image processing method according to claim 5, characterized in that, The step of determining the multi-frame fusionable exposure image among the plurality of third exposure images includes: Determine a set of multiple target local brightness mean values ​​corresponding to the multiple third exposure images; The maximum local brightness mean value in each of the multiple target local brightness mean value sets is determined respectively, resulting in multiple maximum local brightness mean values ​​for different local brightness mean value sets; The minimum value among the plurality of maximum local brightness averages is determined as the initial local brightness average threshold; Based on the initial local brightness mean threshold, the local brightness mean threshold is determined; Based on the plurality of maximum local brightness averages and the local brightness average threshold, the multi-frame fusionable exposure image is determined from the plurality of third exposure images.

7. An image processing apparatus, characterized in that, The device includes: An exposure image acquisition module is used to acquire a sequence of exposure images under target exposure conditions, wherein the exposure image sequence includes multiple frames of exposure images; A reference exposure image determination module is used to determine a reference exposure image among the multiple exposure images; An image difference determination module is used to calculate the image difference parameter between a target exposure image and a reference exposure image in the exposure image sequence, wherein the target exposure image is any other exposure image in the exposure image sequence other than the reference exposure image. The image fusion determination module is used to determine multiple frames of fusionable exposure images in the target exposure image based on the image difference parameters. The image fusion module is used to obtain a denoised image based on the multi-frame fusionable images; The step of determining the reference exposure image among the multiple exposure images includes: Each of the multiple exposure images is taken as the first exposure image, and the image similarity between the first exposure image and the second exposure image is calculated to obtain the image similarity set of the first exposure image. The image similarity set includes multiple image similarities, and the second exposure image is the other exposure image in the exposure image sequence except for the first exposure image. The average of multiple image similarities in the image similarity set of the first exposed image is calculated, and the average similarity is determined as the credibility of the first exposed image; The exposure image corresponding to the maximum confidence value among the multi-frame exposure images is determined as the reference exposure image.

8. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the image processing method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to perform the steps of the image processing method according to any one of claims 1 to 6.

Images