Image processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image processing method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the comparison of a first image and a second image under a plurality of detection dimensions, obtaining an initial difference image corresponding to the detection dimensions, mapping a plurality of initial difference images to a unified scale space, generating a corresponding standard difference image, and carrying out the detection of the detection dimensions. And finally, fusing the plurality of standard difference images to obtain an output mask for indicating the image retouching area, and since difference information of a plurality of detection dimensions is combined and information fusion is carried out based on a unified scale space, the generated output mask can better represent the difference between the first image and the second image, and meanwhile, errors and noise are reduced, and the image retouching accuracy is improved. And the recognition precision and accuracy of the retouching area are improved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, comparing similar original and target images to determine whether a target image has been edited is a common comparison detection method that can be applied to various application scenarios such as photo editing and content recognition.

[0003] In existing technologies, the methods for identifying retouching areas by comparing the original image and the target image are usually based on pixel-by-pixel comparison and stitching consistency comparison. However, the above-mentioned methods in existing technologies have the problems of low accuracy and poor precision in identifying retouching areas. Summary of the Invention

[0004] This disclosure provides an image processing method, apparatus, electronic device, and storage medium to overcome the problems of low recognition accuracy and poor precision in the image retouching area.

[0005] In a first aspect, embodiments of this disclosure provide an image processing method, including:

[0006] A first image and a second image are acquired, and a multi-dimensional comparison detection is performed on the first image and the second image to generate at least two initial difference maps, wherein the at least two initial difference maps correspond to different detection dimensions; at least two standard difference maps are generated based on the at least two initial difference maps; the at least two standard difference maps are fused to obtain an output mask, wherein the output mask is used to indicate the retouching area of ​​the second image relative to the first image.

[0007] In a second aspect, embodiments of this disclosure provide an image processing apparatus, comprising:

[0008] The comparison module is used to acquire a first image and a second image, and perform multi-dimensional comparison detection on the first image and the second image to generate at least two initial difference maps, wherein the at least two initial difference maps correspond to different detection dimensions.

[0009] A mapping module is configured to generate at least two standard difference maps based on at least two initial difference maps;

[0010] A fusion module is used to fuse at least two of the standard difference maps to obtain an output mask, the output mask being used to indicate the retouching area of ​​the second image relative to the first image.

[0011] Thirdly, embodiments of this disclosure provide an electronic device, including: a processor and a memory;

[0012] The memory stores computer-executed instructions;

[0013] The processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the image processing method as described in the first aspect and various possible designs of the first aspect.

[0014] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the image processing method described in the first aspect and various possible designs of the first aspect.

[0015] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the image processing method described in the first aspect and various possible designs of the first aspect.

[0016] The image processing method, apparatus, electronic device, and storage medium provided in this embodiment acquire a first image and a second image, and perform multi-dimensional comparison detection on the first and second images to generate at least two initial difference maps, wherein the at least two initial difference maps correspond to different detection dimensions; at least two standard difference maps are generated based on the at least two initial difference maps; and the at least two standard difference maps are fused to obtain an output mask, which is used to indicate the retouching area of ​​the second image relative to the first image. By comparing the first image and the second image under multiple detection dimensions respectively, initial difference maps corresponding to the detection dimensions are obtained. Then, the multiple initial difference maps are mapped to a unified scale space to generate corresponding standard difference maps. Finally, the multiple standard difference maps are fused to obtain an output mask indicating the retouching area. Because the difference information of multiple detection dimensions is combined and information fusion is performed based on a unified scale space, the generated output mask can better represent the difference between the first image and the second image, while reducing errors and noise, and improving the recognition accuracy and precision of the retouching area. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is an application scenario diagram of the image processing method provided in the embodiments of this disclosure;

[0019] Figure 2 Flowchart of the image processing method provided in the embodiments of this disclosure Figure 1 ;

[0020] Figure 3 for Figure 2 A flowchart of a specific implementation of step S102 in the illustrated embodiment;

[0021] Figure 4 A schematic diagram illustrating the generation process of a standard difference map provided in this embodiment of the disclosure;

[0022] Figure 5 for Figure 2 A flowchart of another specific implementation of step S102 in the illustrated embodiment;

[0023] Figure 6 for Figure 2 A flowchart of another specific implementation of step S102 in the illustrated embodiment;

[0024] Figure 7 for Figure 2 A flowchart illustrating the specific implementation of step S103 in the illustrated embodiment;

[0025] Figure 8 Flowchart of the image processing method provided in the embodiments of this disclosure Figure 2 ;

[0026] Figure 9 for Figure 2 A flowchart illustrating the specific implementation of step S208 in the illustrated embodiment;

[0027] Figure 10 for Figure 2 A flowchart illustrating the specific implementation of step S209 in the illustrated embodiment;

[0028] Figure 11 This is a schematic diagram illustrating the execution process of an image processing method provided in an embodiment of the present disclosure;

[0029] Figure 12 This is a structural block diagram of an image processing apparatus provided in an embodiment of the present disclosure;

[0030] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure;

[0031] Figure 14 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0033] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0034] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0035] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0036] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0037] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0038] The application scenarios of the embodiments of this disclosure are explained below:

[0039] The image processing method provided in this disclosure can be applied to image content detection scenarios based on image comparison. More specifically, it can be applied to the image editing region detection function in an application with image editing capabilities. The executing entity in this embodiment can be a terminal device running the aforementioned application, a server deploying the server-side component corresponding to the aforementioned application, or other electronic devices performing similar functions. Specifically, when the executing entity is a terminal device, the terminal device executes the method provided in this embodiment by running the aforementioned application; when the executing entity is a server, the server-side component of the aforementioned image editing application can run partially or entirely on the server, executing the method provided in this embodiment on the server side, while the terminal device runs the client-side component of the application. Communication between the server and the terminal device is based on server-client communication, enabling the terminal device to obtain the execution result of the method provided in this embodiment and display it as needed.

[0040] In some embodiments, the terminal device or server can implement the image processing method provided in this disclosure by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be program-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system; they can be local applications, i.e., programs that need to be installed in the operating system to run, or small programs embedded in any APP, i.e., programs that run in a browser environment. In summary, the aforementioned computer-executable instructions can be any form of instruction, and the aforementioned computer programs can be any form of application, module, or plugin; the specific implementation can be configured as needed. Furthermore, in implementing the image processing method provided in this disclosure, the terminal device can execute the method by running computer-executable instructions or computer programs set locally, or by calling computer-executable instructions or computer programs set in an external server. In some embodiments, the server may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud storage, cloud communication, cloud database, cloud computing, cloud functions, network services, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms. Among these, cloud services may be interactive processing services that can be invoked by terminal devices.

[0041] Figure 1 This is an application scenario diagram of the image processing method provided in the embodiments of this disclosure, with reference to... Figure 1As shown in the figure, taking a terminal device as an example, the terminal device runs a target application with image editing function. The user inputs or selects the original image and the target image through the interactive interface of the target application, such as image P1 (original image) and image P2 (target image) shown in the figure. Then, by clicking the trigger control ("detect" button), the image processing method provided in this embodiment is invoked to detect image P2 with image P1 as the reference, and to identify the editing area in image P2, that is, the area in which the content of image P2 has changed relative to image P1; and to show it through display effects (dashed lines in the figure as an example), so that the user can accurately and efficiently determine the changed content and position of the target image relative to the original image.

[0042] In existing technologies, methods for identifying retouched regions by comparing the original and target images typically rely on pixel-by-pixel comparison and stitching consistency checks, performing the comparison within a single detection dimension. This limitation of a single detection dimension leads to poor accuracy. Further attempts have employed more complex methods for multi-dimensional comparison and detection of the original and target images to achieve better results. However, multi-dimensional comparison detection often requires configuring detection parameters for multiple dimensions. Since suitable parameters vary depending on the device and image source, fixed detection parameters and thresholds exhibit instability across different devices and implementation environments. Consequently, existing solutions suffer from low accuracy and poor precision in identifying retouched regions.

[0043] This disclosure provides an image processing method to solve the above-mentioned problems.

[0044] refer to Figure 2 , Figure 2 Flowchart of the image processing method provided in the embodiments of this disclosure Figure 1 The method of this embodiment can be applied to a terminal device or a server. In one possible implementation, for a terminal device executing the method provided in this embodiment, the terminal device can execute program code deployed locally and / or externally to implement the image processing method provided in this embodiment. In another possible implementation, the server can respond to a request from the terminal device by executing program code deployed locally and / or externally to implement the image processing method provided in this embodiment. For example, the server can be used to deploy functional services based on the image processing method provided in this embodiment, and the terminal device can execute the image processing method provided in this embodiment by accessing the server and calling the corresponding functional services. For example, the image processing method provided in this embodiment includes:

[0045] Step S101: Acquire the first image and the second image, and perform multi-dimensional comparison detection on the first image and the second image to generate at least two initial difference maps, wherein the at least two initial difference maps correspond to different detection dimensions.

[0046] Step S102: Generate at least two standard difference maps based on at least two initial difference maps.

[0047] Step S103: Fuse at least two standard difference maps to obtain an output mask, which is used to indicate the retouching area of ​​the second image relative to the first image.

[0048] refer to Figure 1 The illustrated application scenario diagram illustrates the image processing method provided in this embodiment, using a terminal device as the execution subject. For example, the terminal device loads or selects a first image and a second image by running a target application. The first image serves as the original image for comparison, while the second image is the image to be detected, serving as the target image for comparison. After obtaining the first and second images, the terminal device may optionally first perform initialization processing on them. This includes correcting the orientation of the first and / or second images based on EXIF ​​metadata and uniformly converting them to the RGB color space to lay the foundation for subsequent accurate comparison. Then, multi-dimensional comparison detection is performed on the first and second images, generating corresponding initial difference maps. Multi-dimensional comparison detection refers to comparing the first and second images from multiple detection dimensions. For example, each initial difference map corresponds to a different detection dimension. One detection dimension could be geometric deformation detection, which compares the geometric shapes of the same object in the first and second images. Another example is color difference detection, which compares the color differences of the same object in the first and second images, and so on. In the multi-dimensional comparison detection described above, the detection processes for each detection dimension are independent and can be executed in parallel or sequentially. After the multi-dimensional comparison detection, each detection dimension outputs a detection result, namely an initial difference map. The initial difference map consists of multiple difference points, which correspond to the pixels of the first and second images. In one possible implementation, the larger the difference point, the greater the difference in pixel value between the first and second images at that location, and vice versa. More specifically, the initial difference map can, for example, characterize regions of geometric deformation or color difference in the second image relative to the first image, and each initial difference map corresponds to a different detection dimension.

[0049] Specifically, in one possible implementation, the initial difference map includes a first initial difference map and a second initial difference map, and the specific implementation of step S101 includes:

[0050] Step S1011: Perform geometric difference detection on the first image and the second image to obtain a first initial difference map, which represents the geometric deformation of the second image relative to the first image;

[0051] Step S1012: Perform color difference detection on the first image and the second image to obtain a second initial difference map, which represents the color difference between the second image and the first image.

[0052] For example, geometric difference detection can be implemented using an optical flow estimation algorithm, such as a cyclic total pair domain transform algorithm, to calculate the pixel-level displacement field from the source image to the image under test, and then extract the Euclidean norm (i.e., amplitude) of the displacement vector to generate a grayscale difference map representing geometric deformation. Color difference detection can optionally convert the RGB image to a color difference space that better matches human visual perception. Then, for the first and second images, the color difference value is calculated pixel-by-pixel, which more accurately quantifies the color difference perceived by the human eye. Further, optionally, in this process, a small-kernel Gaussian filter can be applied to the first and second images, or a small-kernel Gaussian filter can be applied to the second initial difference map, to smooth the Gaussian noise introduced by JPEG compression. Specifically, a small-kernel Gaussian filter (e.g., 3×3 or 5×5, σ≈0‒1) is used without damaging the edge structure. Here, 3×3 / 5×5 is the kernel size (sliding window size) of the Gaussian filter, and σ (standard deviation) ≈0-1 is a parameter used to control the smoothness of the filter. This suppresses sporadic noise and compression artifacts, thereby improving the stability of subsequent boundary separation.

[0053] Furthermore, in one possible implementation, the initial difference map also includes a third initial difference map, and the specific implementation of step S101 includes:

[0054] Step S1013: Perform texture difference detection on the first image and the second image to obtain a third initial difference map. The third initial difference map represents the local texture difference between the second image and the first image.

[0055] For example, during the difference detection process of the first image and the second image, the image texture of the first image and the second image can be further detected to improve the detection accuracy. This step can be executed by calling the central processing unit (CPU) or graphics processing unit (GPU) of the terminal device. Specifically, under the GPU path, algorithms such as Sobel gradient, local binary patterns (LBP) or wavelet transform can be executed efficiently to improve computational efficiency; under the CPU path, multi-core parallel processing is used to improve efficiency. This step aims to capture changes in the local structure, edges and details of the image.

[0056] Furthermore, after obtaining the aforementioned multiple initial difference maps, a standardization mapping is performed on these initial difference maps, that is, at least two initial difference maps are mapped to a unified scale space, thereby converting the initial difference maps into corresponding standard difference maps, which is the execution process of step S102. Based on the previous embodiment steps, it is known that different initial difference maps, such as a first initial difference map representing geometric deformation, a second initial difference map representing color differences, and a third initial difference map representing local texture differences, can generate multiple initial difference maps with values ​​distributed over a relatively wide range and potentially different comparison coordinate systems after processing the first and second images using different image sources, algorithms, devices, and processing methods during the calculation process. The steps in this embodiment are used to map initial difference maps from different sources and with varying numerical distributions into a unified, comparable scale space to facilitate the subsequent generation of the output mask.

[0057] Specifically, in one possible implementation, the process of mapping the initial difference map to a fixed-size space and generating a corresponding standard difference map can be achieved using a pre-trained mapping model. That is, the initial difference map is input into the pre-trained mapping model, and the model's capabilities are used to generate a corresponding standard difference map. This implementation will not be elaborated further. In another possible implementation, such as... Figure 3 As shown, the specific implementation of step S102 includes:

[0058] Step S102A-1: Perform extreme value clamping on the initial difference map to obtain an extreme value-free difference map.

[0059] Step S102A-2: Linearly map the difference values ​​of the extremum difference map to the target numerical range to generate a standard difference map.

[0060] For example, taking the processing of one initial difference map as an example, firstly, extreme value clamping is performed on each difference value in the initial difference map, that is, the difference values ​​in the initial difference map that are greater than a preset value are fixed to the preset value, thereby obtaining an extreme value-removed difference map. For example, a percentile truncation strategy is used to clip the highest 0.1% extreme values ​​in the distribution of difference values ​​in the initial difference map to the value at the 99.9th percentile. That is, difference values ​​greater than the 99.9th percentile will be fixed to the difference value at the 99.9th percentile, thereby achieving the purpose of removing extreme values. Then, the difference values ​​in the extreme value-removed difference map are mapped to a preset target value range, such as [0, 255], thereby generating a standard difference map, in which the difference values ​​are all located between [0, 255]. In the steps of this embodiment, by normalizing the initial difference map, the influence of extreme points can be effectively reduced, and difference maps from different sources and data distributions can be unified into a comparable scale space for comparison.

[0061] Figure 4 This is a schematic diagram illustrating the generation process of a standard difference map provided in an embodiment of this disclosure. The following is in conjunction with... Figure 4 To further explain the above process, such as... Figure 4 As shown, for example, multi-dimensional comparison detection is performed on the first image and the second image to generate initial difference maps K1, K2, and K3, which respectively represent the geometric deformation, color difference, and local texture difference between the first image and the second image. Then, extreme value clamping is applied to the initial difference maps K1, K2, and K3 to obtain extreme value removal difference maps k1, k2, and k3. Referring to the figure, taking the initial difference map K1 as an example, the maximum value of the difference value in the first row of the initial difference map K1 is 'a'. In the first row of the extreme value removal difference map k1, values ​​exceeding 'b' are all set to 'b', i.e., clamped at 'b', thus achieving extreme value removal of the initial difference map. Then, the extreme value removal difference maps k1, k2, and k3 are mapped to the range [0, 255] to obtain the corresponding standard difference maps ks1, ks2, and ks3. As shown in the figure, taking the standard difference map ks1 as an example, the maximum value of the first row in the standard difference map ks1 is 255, which corresponds to b. This is equivalent to compressing the entire extremum difference map k1 into the numerical range of [0, 255], thereby effectively suppressing the dominant influence of a few outliers on the overall scale and improving the accuracy of judgment.

[0062] In another possible implementation, such as Figure 5 As shown, the specific implementation of step S102 includes:

[0063] Step S102B-1: Obtain the preset highlight upper limit value.

[0064] Step S102B-2: Obtain the scaling factor based on the ratio of the upper limit of the highlight value to the current highlight value of the initial difference map.

[0065] Step S102B-3: Adjust the difference values ​​of the initial difference map proportionally based on the scaling factor to generate the corresponding standard difference map.

[0066] For example, in some cases, different hardware or software libraries process images differently, resulting in differences in the numerical distribution of the initial difference map. To reduce these differences, this embodiment adjusts the initial difference map using a scaling factor. Specifically, taking the initial difference map corresponding to the texture channel (the third initial difference map) as an example, a preset brightness upper limit is first determined as the target anchor point. Specifically, when the GPU processes image textures, the brightness upper limit of the texture channel usually refers to the maximum brightness / numerical threshold allowed for a single color channel (such as R, G, B, or luminance channel) in the texture. Essentially, it is a numerical constraint or mapping upper limit for "overly bright" pixels in the texture. Then, a stable statistic in the initial difference map, such as the value of the 99.9% quantile, is selected as the current anchor point. The ratio of the current anchor point to the target anchor point is then calculated to determine the scaling factor, i.e., scaling factor c = target anchor point / current anchor point. Next, based on this scaling factor, the entire difference map is linearly adjusted to attenuate lower intensity segments, reducing the numerical distribution differences in the initial difference map caused by variations in different devices and processing methods. This is equivalent to performing a consistency calibration on the initial difference map. Through cross-device calibration, the differences generated during image processing by different devices can be eliminated, improving the accuracy of the results.

[0067] Of course, in another possible implementation, step S102 can be combined with Figure 3 and Figure 5 The corresponding two implementation steps together complete the step of generating the standard difference map. Specifically, as shown in the example... Figure 6 As shown, the specific implementation method of step S102 is as follows:

[0068] Step S1021: Perform extreme value clamping on the initial difference map to obtain an extreme value-free difference map.

[0069] Step S1022: Linearly map the difference values ​​of the extremum difference map to the target numerical range to generate the first difference map.

[0070] Step S1023: Obtain the preset highlight upper limit value, and obtain the scaling factor based on the ratio of the highlight upper limit value and the current highlight value of the first difference map.

[0071] Step S1024: Adjust the difference values ​​of the first difference map proportionally based on the scaling factor to generate the corresponding standard difference map.

[0072] For example, in this embodiment, the normalization process and consistency calibration are combined. As shown in the steps above, the initial difference map is first clamped to remove extreme values, then normalized based on the target numerical range, and then consistent calibration is performed based on the scaling factor to finally generate a standard difference map. After normalization and cross-device calibration, the method of this embodiment ensures the fairness and stability of subsequent fusion decisions and significantly improves the robustness and cross-platform reproducibility of the algorithm.

[0073] After the above steps, each initial difference map is mapped to obtain the corresponding standard difference map. Then, multiple standard difference maps are fused to obtain an output mask that indicates the retouching area of ​​the second image relative to the first image. The output mask is a binary mask composed of 0 and 1, for example, the area enclosed by 1 is used to refer to the retouching area.

[0074] In one possible implementation, such as Figure 7 As shown, the specific implementation of step S103 includes:

[0075] Step S1031: Based on the separation degree between the difference value and the background in the standard difference map, obtain the adaptive segmentation threshold.

[0076] Step S1032: Segment the standard difference map based on the adaptive segmentation threshold to generate the corresponding difference map mask.

[0077] Step S1033: Merge the difference map masks corresponding to at least two standard difference maps to obtain the output mask.

[0078] For example, unlike traditional methods that separate "significant differences" and background in an image using a fixed threshold, this embodiment generates an adaptive segmentation threshold by parsing a standard difference map and determining the segmentation boundary dynamically based on the separation degree between the difference values ​​and the background in the standard difference map. Specifically, the "maximum separation" threshold (i.e., the maximum inter-class variance) can be automatically determined using a large-scale algorithm based on the histogram corresponding to the standard difference map, serving as the adaptive segmentation threshold. A difference map mask corresponding to the standard difference map is then formed based on this adaptive segmentation threshold. The difference map mask is a binary mask composed of 0s and 1s. An OR operation is performed on multiple difference map masks to obtain the output mask. This method reduces the bias caused by subjective human thresholding and fixed thresholds. When the input standard difference map is a constant map, the algorithm is skipped, and a completely black or completely white (all 0s or all 1s) output mask is returned directly based on constant values, avoiding degraded output.

[0079] In this embodiment, a first image and a second image are acquired, and multi-dimensional comparison detection is performed on the first and second images to generate at least two initial difference maps, each corresponding to a different detection dimension. At least two standard difference maps are generated based on these initial difference maps. The at least two standard difference maps are then fused to obtain an output mask, which is used to indicate the retouching area of ​​the second image relative to the first image. By comparing the first and second images under multiple detection dimensions, initial difference maps corresponding to the detection dimensions are obtained. Then, these initial difference maps are mapped to a unified scale space to generate corresponding standard difference maps. Finally, these standard difference maps are fused to obtain an output mask indicating the retouching area. Because the difference information from multiple detection dimensions is combined and information fusion is performed based on a unified scale space, the generated output mask can better represent the differences between the first and second images, while reducing errors and noise, and improving the accuracy and precision of the retouching area recognition.

[0080] refer to Figure 8 , Figure 8 Flowchart of the image processing method provided in the embodiments of this disclosure Figure 2 This embodiment is in Figure 2 Based on the illustrated embodiment, step S103 is further refined, and a block processing step is added. This image processing method includes:

[0081] Step S201: Obtain the first image and the second image.

[0082] Step S202: Detect the system hardware environment and obtain the image size.

[0083] Step S203: Based on the system hardware environment and image size, if there is sufficient video memory resources, proceed to step S204; if there is insufficient video memory resources, proceed to step S205.

[0084] Step S204: Perform multi-dimensional comparison detection on the first image and the second image to generate at least two initial difference maps, and generate at least two standard difference maps based on the at least two initial difference maps.

[0085] Step S205: The first image and the second image are segmented into blocks, and based on the segmentation results, the first image and the second image are compared and detected in multiple detection dimensions to generate at least two initial difference maps consisting of multiple difference patches, each initial difference map corresponding to a different detection dimension.

[0086] For example, in this embodiment, before detecting the second image based on the first image, to avoid insufficient computing resources and time-consuming processes due to excessively large images, the system hardware environment and image size are first detected, and their matching degree is evaluated. If the GPU's video memory resources are sufficient, for example, meeting one or more conditions such as video memory exceeding a threshold, exceeding a preset multiple of the image size, or exceeding a preset percentage, then a multi-dimensional comparison detection is directly performed on the first and second images to generate an initial difference map. Furthermore, by mapping to a unified scale space, at least two standard difference maps are generated (i.e., step S204). The specific implementation process of this step can be found in [reference needed]. Figure 2 The relevant content in the illustrated embodiments will not be repeated here.

[0087] In another scenario, if the GPU's video memory resources are insufficient—for example, if the video memory does not meet any one or more of the following criteria: exceeding a threshold, exceeding a preset multiple of the image size, or exceeding a preset percentage—then the first and second images are segmented into multiple image blocks. These blocks are then compared across multiple detection dimensions to generate an initial difference map. This initial difference map consists of multiple difference blocks, the number of which is the same as the number of image blocks. For example, if the first and second images are each segmented into 9 image blocks for comparison, then the initial difference map generated under a specific detection dimension will also consist of 9 difference blocks. Each difference block corresponds to a pair of image blocks (at the same position in the first and second images), representing the difference between those pairs. Subsequently, a standardization mapping is performed based on the segmented initial difference map to generate a corresponding standard difference map. Specifically, this includes:

[0088] Step S206: Perform extreme value clamping on each difference block corresponding to each initial difference map to obtain the extreme value-free difference map block corresponding to each initial difference map.

[0089] Step S207: Linearly map the difference values ​​of the extremum-removed difference patches corresponding to each initial difference map to the target numerical range to obtain the standard extremum-removed difference patches corresponding to each initial difference map.

[0090] Step S208: Stitch together the standard extremum removal difference map blocks corresponding to the initial difference map to generate at least two standard difference maps.

[0091] For example, after obtaining the initial difference map, extreme value clamping and target value range determination are performed on each difference map block of the initial difference map. The specific implementation is similar to the steps of extreme value clamping and target value range determination for the initial difference map, and will not be repeated here. After this, the standard extreme value difference map blocks belonging to the same initial difference map are stitched together, and the stitching position relationship is consistent with the position relationship between the previous difference map blocks, thereby generating the standard difference map corresponding to each initial difference map.

[0092] Furthermore, in one possible implementation, such as Figure 9 As shown, the specific implementation of step S208 includes,

[0093] Step S2081: Stitch together the standard extreme value removal difference map blocks to generate the first stitched difference map;

[0094] Step S2082: Apply extreme value clamping to the first stitching difference map to obtain the corresponding second stitching difference map;

[0095] Step S2083: Linearly map the second stitched difference map to the target numerical range to generate the third stitched difference map.

[0096] Step S2084: Obtain the preset highlight upper limit value, and obtain the scaling factor based on the ratio of the highlight upper limit value and the current highlight value of the third stitching difference map.

[0097] Step S2085: Adjust the difference values ​​of the third stitched difference map proportionally based on the scaling factor to generate the corresponding standard difference map.

[0098] For example, in another possible implementation, after uniformly removing extrema and mapping the size space of each difference map patch to generate a standard extrema-removed difference map patch, the standard extrema-removed difference map patches belonging to the same initial difference map are first stitched together to form a corresponding first stitched difference map. This first stitched difference map is equivalent to the standard difference map generated in step S208. Afterward, with the first stitched difference map as the target, the step of mapping to a unified scale space is performed again, including extreme value clamping, mapping to the target numerical range, and scaling based on a scaling factor, sequentially generating a second stitched difference map, a third stitched difference map, and the final standard difference map. The specific implementation of each of the above steps is similar to... Figure 2The process of processing the initial difference map to generate the standard difference map, as described in the illustrated embodiment, is similar and will not be repeated here. Since dividing the initial difference map into blocks will result in scale differences between the different difference map blocks, the aforementioned secondary normalization process can unify the scale and threshold at the entire map level, balancing stability and detail preservation. In the above embodiment, steps S2084 and S2085 are optional steps; that is, in another possible implementation, the third stitched difference map can be output as the standard difference map. The specific implementation process will not be repeated here.

[0099] Step S209: Fuse at least two standard difference maps to obtain an output mask.

[0100] For example, after obtaining the standard difference maps, aligning them and then performing an OR operation can achieve the fusion of the standard difference maps to obtain the output mask. In another possible implementation, after fusion, the fusion result can be further denoised to improve the quality of the output mask. Specifically, in one possible implementation, such as... Figure 10 As shown, the specific implementation of step S209 includes:

[0101] Step S2091: Fuse at least two standard difference maps to obtain the first mask;

[0102] Step S2092: Obtain the number of pixels in the first mask and determine the corresponding scaling factor based on the number of pixels. The scaling factor is inversely proportional to the number of pixels.

[0103] Step S2093: Based on the scaling factor, downsample the first mask to obtain the second mask.

[0104] Step S2094: Based on the scaling factor, reduce the first noise filtering area corresponding to the first mask to obtain the second noise filtering area.

[0105] Step S2095: Perform noise filtering on the second mask based on the second noise filtering area to obtain the output mask.

[0106] For example, after obtaining the standard difference map through the above steps, the multiple standard difference maps are first fused to generate a first mask. This step is equivalent to the step in the previous embodiment of fusing multiple standard difference maps to generate an output mask, and the specific process will not be repeated. Next, the number of pixels in the first mask is obtained. The pixel data refers to the number of difference values ​​in the standard difference map, which is also the number of pixels in the first and second images. The larger the number of pixels, the larger the image. Optionally, if the number of pixels is greater than a preset threshold (i.e., the image is large), for example, greater than 8 million pixels, a scaling factor that is inversely proportional to the number of pixels in the first mask is determined. That is, the larger the number of pixels, the smaller the scaling factor. Specifically, the method for determining the scaling factor includes, for example, scaling factor s = sqrt(threshold / number of pixels), where sqrt() represents the square root. Then, the first mask is downsampled based on this scaling factor to obtain a smaller second mask. Next, a first noise filtering area is obtained. This first noise filtering area is the parameter used to identify noise points. When the connected area (number of pixels) on the image is greater than the first noise filtering area, the pixel corresponding to the connected area is considered a valid pixel; conversely, when the connected area (number of pixels) on the image is less than the first noise filtering area, it is considered a noise pixel. The first noise filtering area can be a preset value, which can be related to the size of the first image, the second image, or the standard difference map. Then, based on the scaling factor obtained in the previous step, the first noise filtering area is simultaneously reduced to obtain a second noise filtering area. Specifically, for example, the first noise filtering area is multiplied by the square of the scaling factor to obtain the second noise filtering area, thus matching the second noise filtering area with the second mask. Next, noise filtering is performed on the second mask based on the second noise filtering area, and the filtering result is upsampled using methods such as bilinear interpolation to restore the original resolution, obtaining the output mask. Through the above steps, the time-consuming high-cost connected component analysis directly on the high-resolution image is avoided, and the computational efficiency is improved by orders of magnitude while ensuring the noise reduction effect.

[0107] Figure 11 This is a schematic diagram illustrating the execution process of an image processing method provided in an embodiment of this disclosure. The following is in conjunction with... Figure 10 The above embodiments will be further described, such as... Figure 11As shown, for example, firstly, the first image, which serves as the original image, and the second image, which serves as the modified image, are subjected to EXIF ​​correction, color standardization, and size consistency processing. On the one hand, after processing, an initial image is generated, and the initial image is sent to the texture analysis unit, geometric analysis unit, and color analysis unit for multi-dimensional comparison detection. On the other hand, according to the system hardware environment, device adaptive judgment is performed to determine the image processing execution strategy, including several schemes such as CPU block processing, GPU block processing, and GPU processing of the complete image. Based on the optimal execution strategy determined by the adaptive judgment, subsequent multi-dimensional comparison detection is performed. Subsequently, based on the first initial difference map, second initial difference map, and third initial difference map output by the texture analysis template, geometric analysis unit, and color analysis unit, respectively, normalization and consistency calibration steps are performed to generate the first standard difference map, second standard difference map, and third standard difference map. Adaptive thresholding is then performed to automatically determine the segmentation threshold of each standard difference map, generating corresponding binarized masks, namely the first difference map mask, the second difference map mask, and the third difference map mask. Finally, the first difference map mask, the second difference map mask, and the third difference map mask are fused to generate the output mask.

[0108] In this embodiment, the implementation methods of steps S201, S204, and S209 are the same as those in this disclosure. Figure 2 The implementation methods of steps S101 and S103 in the illustrated embodiment are the same, and will not be described in detail here.

[0109] Corresponding to the image processing method in the above embodiments, Figure 12 This is a structural block diagram of an image processing apparatus provided in an embodiment of this disclosure. The method described in the above embodiments can be executed by this image processing apparatus, which can be implemented by software and / or hardware, and can be integrated into an electronic device with certain data processing capabilities. The electronic device may include, but is not limited to, mobile terminals with big data processing capabilities, as well as fixed terminals with big data processing capabilities such as desktop computers and supercomputers.

[0110] For ease of explanation, only the parts relevant to embodiments of this disclosure are shown. (Refer to...) Figure 12 The image processing device 3 includes:

[0111] The comparison module 31 is used to acquire a first image and a second image, and to perform multi-dimensional comparison detection on the first image and the second image to generate at least two initial difference maps, wherein the at least two initial difference maps correspond to different detection dimensions.

[0112] Mapping module 32 is used to generate at least two standard difference maps based on at least two initial difference maps;

[0113] The fusion module 33 is used to fuse at least two standard difference maps to obtain an output mask, which is used to indicate the retouching area of ​​the second image relative to the first image.

[0114] According to one or more embodiments of this disclosure, the initial difference map includes a first initial difference map and a second initial difference map. The comparison module 31 is specifically used for: performing geometric difference detection on the first image and the second image to obtain a first initial difference map, wherein the first initial difference map represents the geometric deformation of the second image relative to the first image; and performing color difference detection on the first image and the second image to obtain a second initial difference map, wherein the second initial difference map represents the color difference of the second image relative to the first image.

[0115] According to one or more embodiments of the present disclosure, the initial difference map further includes a third initial difference map. The comparison module 31 is further configured to: perform texture difference detection on the first image and the second image to obtain the third initial difference map, wherein the third initial difference map characterizes the local texture difference of the second image relative to the first image.

[0116] According to one or more embodiments of this disclosure, the mapping module 32 is specifically used to: clamp the initial difference map to extreme values ​​to obtain a difference map without extreme values; and linearly map the difference values ​​of the difference map without extreme values ​​to the target numerical range to generate a standard difference map.

[0117] According to one or more embodiments of this disclosure, when the mapping module 32 performs extreme value clamping on the initial difference map to obtain an extreme value-free difference map, it is specifically used to: perform extreme value clamping on each difference map block corresponding to the initial difference map to obtain the corresponding extreme value-free difference map block; when the mapping module 32 linearly maps the difference values ​​of the extreme value-free difference map to the target numerical range to generate a standard difference map, it is specifically used to: linearly map the difference values ​​of the extreme value-free difference map block to the target numerical range to obtain the corresponding standard extreme value-free difference map block; and stitch the standard extreme value-free difference map blocks together to generate a standard difference map.

[0118] According to one or more embodiments of this disclosure, when the mapping module 32 stitches together the standard extremum removal difference map blocks to generate a standard difference map, it is specifically used to: stitch together the standard extremum removal difference map blocks to generate a first stitched difference map; perform extreme value clamping on the first stitched difference map to obtain a corresponding second stitched difference map; and linearly map the second stitched difference map to the target numerical range to generate a standard difference map.

[0119] According to one or more embodiments of this disclosure, when generating at least two standard difference maps based on at least two initial difference maps, the mapping module 32 is specifically configured to: obtain a preset highlight upper limit value; obtain a scaling factor based on the ratio of the highlight upper limit value to the current highlight value of the initial difference map; and adjust the difference value of the initial difference map proportionally based on the scaling factor to generate the corresponding standard difference map.

[0120] According to one or more embodiments of this disclosure, the fusion module 33 is specifically used to: obtain an adaptive segmentation threshold based on the separation degree between the difference value and the background in the standard difference map; segment the standard difference map based on the adaptive segmentation threshold to generate a corresponding difference map mask; and merge the difference map masks corresponding to at least two standard difference maps to obtain an output mask.

[0121] According to one or more embodiments of this disclosure, the fusion module 33 is specifically configured to: fuse at least two standard difference maps to obtain a first mask; obtain the number of pixels in the first mask and determine a corresponding scaling factor based on the number of pixels, wherein the scaling factor is inversely proportional to the number of pixels; downsample the first mask according to the scaling factor to obtain a second mask; reduce the first noise filtering area corresponding to the first mask according to the scaling factor to obtain a second noise filtering area; and perform noise filtering on the second mask based on the second noise filtering area to obtain an output mask.

[0122] The comparison module 31, mapping module 32, and fusion module 33 are connected sequentially. The image processing device 3 provided in this embodiment can execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0123] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, such as... Figure 13 As shown, the electronic device 4 includes:

[0124] Processor 41, and memory 42 communicatively connected to processor 41;

[0125] Memory 42 stores instructions executed by the computer;

[0126] The processor 41 executes computer execution instructions stored in the memory 42 to achieve, for example, Figures 2-11 The image processing method in the illustrated embodiment.

[0127] Optionally, the processor 41 and the memory 42 are connected via a bus 43.

[0128] For relevant instructions, please refer to the corresponding text. Figures 2-11 The relevant descriptions and effects of the steps in the corresponding embodiments are understood, and will not be elaborated on here.

[0129] This disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement this disclosure. Figures 2-11 The image processing method provided in any of the corresponding embodiments.

[0130] This disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements this disclosure. Figures 2-11 The image processing method provided in any of the corresponding embodiments.

[0131] To implement the above embodiments, this disclosure also provides an electronic device.

[0132] refer to Figure 14 The diagram illustrates a structural schematic of an electronic device 900 suitable for implementing embodiments of the present disclosure. The electronic device 900 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablet computers, portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 14 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0133] like Figure 14 As shown, the electronic device 900 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the electronic device 900. The processing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0134] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic device 900 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 14An electronic device 900 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0135] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by a processing device 901, it performs the functions defined in the methods of embodiments of this disclosure.

[0136] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0137] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0138] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0139] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0140] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0141] The units or modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units or modules do not necessarily limit the specific unit itself.

[0142] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0143] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0144] In a first aspect, according to one or more embodiments of the present disclosure, an image processing method is provided, comprising:

[0145] Acquire a first image and a second image, and perform multi-dimensional comparison detection on the first image and the second image to generate at least two initial difference maps, wherein each initial difference map corresponds to a different detection dimension; generate at least two standard difference maps based on the at least two initial difference maps; fuse the at least two standard difference maps to obtain an output mask, which is used to indicate the retouching area of ​​the second image relative to the first image.

[0146] According to one or more embodiments of this disclosure, the initial difference map includes a first initial difference map and a second initial difference map. Multi-dimensional comparison detection is performed on the first image and the second image to generate at least two initial difference maps, including: performing geometric difference detection on the first image and the second image to obtain a first initial difference map, the first initial difference map representing the geometric deformation of the second image relative to the first image; and performing color difference detection on the first image and the second image to obtain a second initial difference map, the second initial difference map representing the color difference of the second image relative to the first image.

[0147] According to one or more embodiments of the present disclosure, the initial difference map further includes a third initial difference map, and further includes: performing texture difference detection on the first image and the second image to obtain the third initial difference map, wherein the third initial difference map characterizes the local texture difference of the second image relative to the first image.

[0148] According to one or more embodiments of this disclosure, generating at least two standard difference maps based on at least two initial difference maps includes: clamping the initial difference maps to obtain extreme value-free difference maps; and linearly mapping the difference values ​​of the extreme value-free difference maps to a target numerical range to generate standard difference maps.

[0149] According to one or more embodiments of this disclosure, extreme value clamping is applied to an initial difference map to obtain an extreme value-removed difference map, including: applying extreme value clamping to each difference map block corresponding to the initial difference map to obtain a corresponding extreme value-removed difference map block; linearly mapping the difference values ​​of the extreme value-removed difference map to a target numerical range to generate a standard difference map, including: linearly mapping the difference values ​​of the extreme value-removed difference map blocks to the target numerical range to obtain a corresponding standard extreme value-removed difference map block; and stitching together the standard extreme value-removed difference map blocks to generate a standard difference map.

[0150] According to one or more embodiments of this disclosure, stitching together standard extremum-removed difference maps to generate a standard difference map includes: stitching together standard extremum-removed difference maps to generate a first stitched difference map; applying extreme value clamping to the first stitched difference map to obtain a corresponding second stitched difference map; and linearly mapping the second stitched difference map to a target numerical range to generate a standard difference map.

[0151] According to one or more embodiments of this disclosure, generating at least two standard difference maps based on at least two initial difference maps includes: obtaining a preset highlight upper limit value; obtaining a scaling factor based on the ratio of the highlight upper limit value to the current highlight value of the initial difference map; and adjusting the difference value of the initial difference map proportionally based on the scaling factor to generate the corresponding standard difference map.

[0152] According to one or more embodiments of this disclosure, fusing at least two standard difference maps to obtain an output mask includes: obtaining an adaptive segmentation threshold based on the separation degree between the difference values ​​and the background in the standard difference maps; segmenting the standard difference maps based on the adaptive segmentation threshold to generate corresponding difference map masks; and merging the difference map masks corresponding to at least two standard difference maps to obtain an output mask.

[0153] According to one or more embodiments of this disclosure, fusing at least two standard difference maps to obtain an output mask includes: fusing at least two standard difference maps to obtain a first mask; obtaining the number of pixels in the first mask and determining a corresponding scaling factor based on the number of pixels, wherein the scaling factor is inversely proportional to the number of pixels; downsampling the first mask according to the scaling factor to obtain a second mask; reducing the first noise filtering area corresponding to the first mask according to the scaling factor to obtain a second noise filtering area; and performing noise filtering on the second mask based on the second noise filtering area to obtain an output mask.

[0154] Secondly, according to one or more embodiments of the present disclosure, an image processing apparatus is provided, comprising:

[0155] The comparison module is used to acquire a first image and a second image, and to perform multi-dimensional comparison detection on the first image and the second image to generate at least two initial difference maps, wherein each initial difference map corresponds to a different detection dimension.

[0156] The mapping module is used to generate at least two standard difference maps based on at least two initial difference maps.

[0157] The fusion module is used to fuse at least two standard difference maps to obtain an output mask, which is used to indicate the retouching area of ​​the second image relative to the first image.

[0158] According to one or more embodiments of this disclosure, the initial difference map includes a first initial difference map and a second initial difference map. The comparison module is specifically used for: performing geometric difference detection on the first image and the second image to obtain the first initial difference map, wherein the first initial difference map represents the geometric deformation of the second image relative to the first image; and performing color difference detection on the first image and the second image to obtain the second initial difference map, wherein the second initial difference map represents the color difference of the second image relative to the first image.

[0159] According to one or more embodiments of the present disclosure, the initial difference map further includes a third initial difference map. The comparison module is further configured to: perform texture difference detection on the first image and the second image to obtain the third initial difference map, wherein the third initial difference map characterizes the local texture difference of the second image relative to the first image.

[0160] According to one or more embodiments of this disclosure, the mapping module is specifically used for: clamping extreme values ​​on an initial difference map to obtain an extreme value-free difference map; and linearly mapping the difference values ​​of the extreme value-free difference map to a target numerical range to generate a standard difference map.

[0161] According to one or more embodiments of this disclosure, when the mapping module performs extreme value clamping on the initial difference map to obtain an extreme value-free difference map, it is specifically used to: perform extreme value clamping on each difference map block corresponding to the initial difference map to obtain the corresponding extreme value-free difference map block; when the mapping module linearly maps the difference values ​​of the extreme value-free difference map to the target numerical range to generate a standard difference map, it is specifically used to: linearly map the difference values ​​of the extreme value-free difference map block to the target numerical range to obtain the corresponding standard extreme value-free difference map block; and stitch the standard extreme value-free difference map blocks together to generate a standard difference map.

[0162] According to one or more embodiments of this disclosure, when the mapping module stitches together the standard extremum removal difference map blocks to generate a standard difference map, it is specifically used to: stitch together the standard extremum removal difference map blocks to generate a first stitched difference map; perform extreme value clamping on the first stitched difference map to obtain a corresponding second stitched difference map; and linearly map the second stitched difference map to the target numerical range to generate a standard difference map.

[0163] According to one or more embodiments of this disclosure, when the mapping module generates at least two standard difference maps based on at least two initial difference maps, it is specifically used to: obtain a preset highlight upper limit value; obtain a scaling factor based on the ratio of the highlight upper limit value and the current highlight value of the initial difference map; and adjust the difference value of the initial difference map proportionally based on the scaling factor to generate the corresponding standard difference map.

[0164] According to one or more embodiments of this disclosure, the fusion module is specifically used for: obtaining an adaptive segmentation threshold based on the separation degree between the difference value and the background in the standard difference map; segmenting the standard difference map based on the adaptive segmentation threshold to generate a corresponding difference map mask; and merging the difference map masks corresponding to at least two standard difference maps to obtain an output mask.

[0165] According to one or more embodiments of this disclosure, the fusion module is specifically configured to: fuse at least two standard difference maps to obtain a first mask; obtain the number of pixels in the first mask and determine a corresponding scaling factor based on the number of pixels, wherein the scaling factor is inversely proportional to the number of pixels; downsample the first mask according to the scaling factor to obtain a second mask; reduce the first noise filtering area corresponding to the first mask according to the scaling factor to obtain a second noise filtering area; and perform noise filtering on the second mask based on the second noise filtering area to obtain an output mask.

[0166] Thirdly, according to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one processor and a memory;

[0167] The memory stores the instructions that the computer executes;

[0168] At least one processor executes computer execution instructions stored in memory, causing at least one processor to perform the image processing method as described in the first aspect above and various possible designs of the first aspect.

[0169] Fourthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, which stores computer-executable instructions that, when executed by a processor, implement the image processing method as described in the first aspect and various possible designs of the first aspect.

[0170] Fifthly, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the image processing method as described in the first aspect and various possible designs of the first aspect.

[0171] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0172] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0173] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An image processing method, characterized by, include: A first image and a second image are acquired, and a multi-dimensional comparison detection is performed on the first image and the second image to generate at least two initial difference maps, wherein the at least two initial difference maps correspond to different detection dimensions. Generate at least two standard difference maps based on at least two initial difference maps; At least two of the standard difference maps are fused to obtain an output mask, which is used to indicate the retouched area of ​​the second image relative to the first image.

2. The method according to claim 1, characterized in that, The initial difference map includes a first initial difference map and a second initial difference map. The step of performing multi-dimensional comparison detection on the first image and the second image to generate at least two initial difference maps includes: Geometric difference detection is performed on the first image and the second image to obtain a first initial difference map, which represents the geometric deformation of the second image relative to the first image; Color difference detection is performed on the first image and the second image to obtain a second initial difference map, which represents the color difference between the second image and the first image.

3. The method according to claim 2, characterized in that, The initial difference map also includes a third initial difference map, and further includes: Texture difference detection is performed on the first image and the second image to obtain a third initial difference map, which represents the local texture difference of the second image relative to the first image.

4. The method according to claim 1, characterized in that, The generation of at least two standard difference maps based on at least two initial difference maps includes: Extreme value clamping is applied to the initial difference map to obtain an extreme value-free difference map; The difference values ​​in the extremum removal difference map are linearly mapped to the target numerical range to generate a standard difference map.

5. The method according to claim 4, characterized in that, The process of clamping the initial difference map to obtain a difference map without extreme values ​​includes: Extreme value clamping is applied to each difference block corresponding to the initial difference map to obtain the corresponding extreme value-free difference map block; The step of linearly mapping the difference values ​​of the extremum removal difference map to the target numerical range to generate a standard difference map includes: The difference values ​​of the extremum removal difference map are linearly mapped to the target numerical range to obtain the corresponding standard extremum removal difference map. The standard extremum difference map blocks are stitched together to generate a standard difference map.

6. The method according to claim 5, characterized in that, The step of stitching together the standard extremum difference map blocks to generate a standard difference map includes: The standard extreme value difference map blocks are stitched together to generate a first stitched difference map; Extreme value clamping is applied to the first splicing difference map to obtain the corresponding second splicing difference map; The second stitched difference map is linearly mapped to the target numerical range to generate a standard difference map.

7. The method according to claim 1, characterized in that, The generation of at least two standard difference maps based on at least two initial difference maps includes: Get the preset highlight limit value; The scaling factor is obtained based on the ratio of the upper limit of the highlight value to the current highlight value of the initial difference map; The difference values ​​of the initial difference map are proportionally adjusted based on the scaling factor to generate a corresponding standard difference map.

8. The method according to claim 1, characterized in that, The process of fusing at least two of the standard difference maps to obtain an output mask includes: Based on the separation degree between the difference value and the background in the standard difference map, an adaptive segmentation threshold is obtained; The standard difference map is segmented based on the adaptive segmentation threshold to generate a corresponding difference map mask; The output mask is obtained by merging the difference map masks corresponding to at least two of the standard difference maps.

9. The method according to claim 1, characterized in that, The process of fusing at least two of the standard difference maps to obtain an output mask includes: At least two of the aforementioned standard difference maps are fused to obtain a first mask; The number of pixels in the first mask is obtained, and a corresponding scaling factor is determined based on the number of pixels, wherein the scaling factor is inversely proportional to the number of pixels; Based on the scaling factor, the first mask is downsampled to obtain the second mask; Based on the scaling factor, the first noise filtering area corresponding to the first mask is reduced to obtain the second noise filtering area; The second mask is subjected to noise filtering based on the second noise filtering area to obtain the output mask.

10. An electronic device, characterized in that, include: Processor and memory; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the image processing method as described in any one of claims 1 to 9.

Images