Image processing method, apparatus, device, medium, and program product

Abstract

Embodiments of the present application provide an image processing method, device, equipment, medium and program product, which can be applied in the fields or scenarios of cloud technology, artificial intelligence, intelligent platform, application software, block chain, face identification, data authentication, etc., and include: performing feature extraction on a to-be-processed image to obtain a first feature map, and performing feature extraction on a frequency domain feature map of the to-be-processed image to obtain a second feature map; performing self-enhancement processing on the first feature map and the second feature map to obtain a first self-enhanced feature map and a second self-enhanced feature map; determining a first mutual-enhanced feature map according to the first self-enhanced feature map and a corresponding spatial attention map thereof, and determining a second mutual-enhanced feature map according to the second self-enhanced feature map and a corresponding spatial attention map thereof; and determining an image processing result of the to-be-processed image according to the first mutual-enhanced feature map and the second mutual-enhanced feature map. Through the embodiments of the present application, the distinguishability of representation can be improved, and thus the accuracy of the image processing result can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to image processing methods, image processing apparatus, computer equipment, computer-readable storage media, and computer program products. Background Technology

[0002] The internet is rife with fake images. While these fake images have fueled the growth of the entertainment and cultural exchange industries, they also pose significant security risks. For example, fake images can be used to bypass system security checks, thereby enabling attacks on systems. Before processing fake images, different types of features, such as image steganalysis, are typically defined manually. Neural networks are then used to perform tasks like image classification and identification based on these manually defined features. However, these methods rely on the subjective factors of the processor and have weak distinguishability, resulting in low accuracy in the image processing results. Summary of the Invention

[0003] This application provides an image processing method, apparatus, device, medium, and program product that can improve the distinguishability of the representation, thereby improving the accuracy of the image processing results.

[0004] In a first aspect, this application provides an image processing method, the method comprising:

[0005] A first feature map is obtained by performing feature extraction on the image to be processed, and a frequency domain feature map of the image to be processed is obtained. A second feature map is obtained by performing feature extraction on the frequency domain feature map.

[0006] The first feature map is subjected to self-enhancement processing to obtain a first self-enhancement feature map, and the second feature map is subjected to self-enhancement processing to obtain a second self-enhancement feature map;

[0007] The first self-enhancing feature map and the second self-enhancing feature map are fused together, and the spatial attention map corresponding to the first self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map are determined based on the fused feature map obtained by the fusion process.

[0008] Based on the first self-enhancing feature map and the spatial attention map corresponding to the first self-enhancing feature map, a first mutual-enhancing feature map is determined, and based on the second self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map, a second mutual-enhancing feature map is determined.

[0009] The image processing result of the image to be processed is determined based on the first mutually enhanced feature map and the second mutually enhanced feature map.

[0010] Secondly, this application provides an image processing apparatus, the apparatus comprising:

[0011] The acquisition module is used to extract features from the image to be processed to obtain a first feature map, and to acquire the frequency domain feature map of the image to be processed, and to extract features from the frequency domain feature map to obtain a second feature map.

[0012] The processing module is used to perform self-enhancement processing on the first feature map to obtain a first self-enhancement feature map, and to perform self-enhancement processing on the second feature map to obtain a second self-enhancement feature map.

[0013] The above-mentioned processing module is further configured to perform a fusion process on the first self-enhancing feature map and the second self-enhancing feature map, and determine the spatial attention map corresponding to the first self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map based on the fused feature map obtained by the fusion process.

[0014] The aforementioned processing module is further configured to determine a first mutual enhancement feature map based on the first self-enhancing feature map and the spatial attention map corresponding to the first self-enhancing feature map, and to determine a second mutual enhancement feature map based on the second self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map.

[0015] The classification module is used to determine the image processing result of the image to be processed based on the first mutual enhancement feature map and the second mutual enhancement feature map.

[0016] Thirdly, this application provides a computer device, including: a processor, a storage device, and a communication interface, wherein the processor, the communication interface, and the storage device are interconnected, wherein the storage device stores executable program code, and the processor is used to call the executable program code to implement the image processing method described above.

[0017] Fourthly, this application provides a computer-readable storage medium storing a computer program, the computer program including program instructions that are executed by a processor to implement the image processing method described above.

[0018] Fifthly, this application provides a computer program product, which includes a computer program or computer instructions, which are executed by a processor to implement the image processing method described above.

[0019] This application first extracts the image to be processed for dual-stream feature processing, and transforms the feature dimensions of the image to be processed into a more fine-grained frequency domain feature map. Subsequently, image processing is performed based on the image to be processed and its frequency domain feature map, improving the diversity of data dimensions. Then, self-enhancement processing is applied to the first feature map obtained from the image to be processed and the second feature map obtained from the frequency domain feature map, resulting in a first self-enhanced feature map and a second self-enhanced feature map. Self-enhanced processing is data augmentation within single-stream features, providing the possibility of uncovering more subtle artifacts hidden in the image to be processed or the frequency domain feature map, improving the distinguishability of the representation. By fusing the first and second self-enhanced feature maps, a dual-stream feature map that integrates the image to be processed and the frequency domain image can be obtained. The first self-enhancing feature map and the second self-enhancing feature map are used to enhance the representational ability of the features. Then, based on the fused feature map obtained by the fusion processing, the spatial attention maps corresponding to the first self-enhancing feature map and the second self-enhancing feature map are determined respectively. The spatial attention maps represent the importance of spatial location information in the image. In the process of determining the first self-enhancing feature map and the second self-enhancing feature map, since the first self-enhancing feature map (or the second self-enhancing feature map) is based on the first self-enhancing feature map (or the second self-enhancing feature map) and the spatial attention map used to calculate the spatial enhancement part, that is, the spatial dimension is enhanced on the basis of the first self-enhancing feature map (or the second self-enhancing feature map), which further enhances the distinguishability of the representation. This application then performs image processing based on the first self-enhancing feature map and the second self-enhancing feature map, which improves the accuracy of the image processing results. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the architecture of an image processing system provided in an exemplary embodiment of this application;

[0022] Figure 2 This is a schematic flowchart of an image processing method provided in an exemplary embodiment of this application;

[0023] Figure 3A This is a schematic diagram of an image preprocessing process provided in an exemplary embodiment of this application;

[0024] Figure 3B This is a schematic diagram of a process for obtaining a mutually enhanced feature map provided in an exemplary embodiment of this application;

[0025] Figure 4This is a schematic flowchart of another image processing method provided in an exemplary embodiment of this application;

[0026] Figure 5A This is a schematic diagram of a process for obtaining a self-enhanced feature map provided in an exemplary embodiment of this application;

[0027] Figure 5B This is a structural diagram and processing flowchart of an image processing model provided in an exemplary embodiment of this application;

[0028] Figure 5C This application provides a structural diagram and flowchart of an image processing model comprising multiple combined modules.

[0029] Figure 6 This is a schematic block diagram of an image processing apparatus provided in an exemplary embodiment of this application;

[0030] Figure 7 This is a schematic block diagram of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0032] It should be noted that the terms "first," "second," etc., used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature specified with "first" or "second" may explicitly or implicitly include at least one of those features.

[0033] The embodiments of the present invention can be applied to various scenarios such as cloud computing, cloud IoT, cloud gaming, artificial intelligence, vehicle scenarios, smart transportation, and assisted driving. Several typical application scenarios will be introduced below.

[0034] Cloud computing refers to the delivery and usage model of IT infrastructure, meaning obtaining necessary resources in an on-demand and easily scalable manner through a network. In a broader sense, cloud computing also refers to the delivery and usage model of services, meaning obtaining necessary services in an on-demand and easily scalable manner through a network. These services can be IT and software related, internet-related, or other services. Cloud computing is a product of the convergence of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing. Driven by the development of the internet, real-time data streams, the diversification of connected devices, and the demands of search services, social networks, mobile commerce, and open collaboration, cloud computing has developed rapidly. Unlike previous parallel and distributed computing, the emergence of cloud computing will fundamentally revolutionize the entire internet model and enterprise management model. This application can store various feature maps and image processing results corresponding to sample images used for model training on a cloud server. When different data are needed, they can be directly obtained from the cloud server, which greatly improves the data acquisition speed.

[0035] Cloud IoT aims to connect the information sensed and commands received by traditional IoT devices to the Internet, truly achieving networking. It also enables massive data storage and computation through cloud computing technology. Due to the nature of IoT, which involves connecting things to each other and sensing the current operating status of each "object" in real time, a large amount of data information is generated in this process. How to aggregate this information and how to sift out useful information from the massive amount of data to support decision-making for future development have become key issues affecting the development of IoT. As a result, IoT cloud based on cloud computing and cloud storage technology has become a powerful support for IoT technology and applications.

[0036] Intelligent Traffic Systems (ITS), also known as Intelligent Transportation Systems, effectively integrate advanced technologies (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operations research, artificial intelligence, etc.) into transportation, service control, and vehicle manufacturing. This strengthens the connection between vehicles, roads, and users, forming a comprehensive transportation system that ensures safety, improves efficiency, enhances the environment, and conserves energy. This application applies to intelligent transportation systems and can provide functions including facial recognition and identity verification. After identity verification, operators are allowed to use its traffic monitoring services, ensuring the security of the intelligent transportation system.

[0037] Intelligent Vehicle Infrastructure Cooperative Systems (IVICS) are a development direction of Intelligent Transportation Systems (ITS). IVICS utilizes advanced wireless communication and next-generation Internet technologies to implement comprehensive, real-time dynamic information exchange between vehicles and infrastructure. Based on the collection and fusion of dynamic traffic information across all times and spaces, it conducts active vehicle safety control and cooperative road management, fully realizing effective collaboration between people, vehicles, and roads. This ensures traffic safety, improves traffic efficiency, and ultimately forms a safe, efficient, and environmentally friendly road traffic system.

[0038] Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and cryptographic algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying platform, a platform product service layer, and an application service layer.

[0039] The underlying blockchain platform can include processing modules such as user management, basic services, smart contracts, and operational monitoring. The user management module is responsible for managing the identity information of all blockchain participants, including maintaining public and private key generation (account management), key management, and maintaining the correspondence between user real identities and blockchain addresses (access management). Furthermore, under authorization, it monitors and audits transactions of certain real identities and provides risk control rule configuration (risk control audit). The basic services module is deployed on all blockchain node devices to verify the validity of business requests. After consensus is reached on valid requests, they are recorded in storage. For a new business request, the basic services first perform interface adaptation parsing and authentication (interface adaptation), and then encrypt the business information using a consensus algorithm (consensus management). The blockchain process involves several layers of data processing. The first layer, after encryption, transmits the encrypted data to the shared ledger (network communication) and stores it. The second layer, the third layer, is responsible for contract registration, issuance, triggering, and execution. Developers can define contract logic using a programming language and publish it to the blockchain (contract registration). Based on the contract terms, the blockchain triggers execution by calling keys or other events, completing the contract logic. It also provides functions for contract upgrades and cancellations. The third layer, the operations monitoring layer, is primarily responsible for deployment, configuration modification, contract settings, cloud adaptation, and real-time visualization of the product's operational status, such as alarms, network monitoring, and monitoring of node device health. The platform's product service layer provides basic capabilities and implementation frameworks for typical applications. Developers can leverage these capabilities and add business characteristics to implement business logic on the blockchain. The application service layer provides blockchain-based application services to business participants. This application can be combined with blockchain technology to generate more intelligent services.

[0040] This application first proposes a method for identifying forged images. It begins by manually defining different types of features, such as image steganalysis features. Then, it utilizes support vector machines or neural networks to perform processing on the image to be processed, such as image classification and image identification, based on the manually defined features. Since certain regions of forged images (such as the eyeballs and teeth in facial images) may contain specific forgery traces, these traces can be detected and classified. While this method can achieve the purpose of identifying and classifying forged images, it relies on the subjective factors of the processor and has weak distinguishability, resulting in low accuracy of the image processing results. Furthermore, the processing flow is too manual, requiring a large amount of prior knowledge, leading to poor generalization ability.

[0041] The aforementioned methods primarily focus on the transformation of the data input space (i.e., processing of input data), implicitly influencing representations through different forms of supervision, but rarely address the representation learning process, failing to fully exploit subtle artifacts (i.e., specific forgery traces) generated during image forgery. Based on this, this application optimizes and improves the above methods, focusing on the representation learning process. It progressively amplifies the representational differences between positive and negative samples through progressive representation enhancement, thereby enhancing the distinguishability of representations. This application proposes a forgery detection method based on progressive representation enhancement. First, a progressive representation enhancement framework is designed to explicitly enhance the distinguishability of representations. Through input enhancement, the original image is converted into a fine-grained frequency domain image, and then both the original image and the frequency domain image are fed into a symmetrical two-stream network. Next, for the features extracted from each convolutional block at the same level in the two-stream network, a self-enhancement module enhances the noise and channel dimensions of the intra-stream features, and a mutual enhancement module enhances the spatial dimensions of the cross-stream features. Data augmentation offers the possibility of uncovering more subtle artifacts hidden in the frequency domain. This method can enhance the distinguishability of the representation and improve the generalization and robustness of face-swapping detection.

[0042] Deepfake technology has driven the emerging development of the entertainment and cultural exchange industries, but it also poses a significant potential threat to facial security. During the face replacement process, abnormal artifacts appear in local areas of the forged facial image. These artifacts affect the high-level features extracted by the neural network, which can be used to distinguish between real and forged facial images. Therefore, this application can be applied to forged face detection products to improve facial verification technology and enhance the security of facial recognition payment, identity authentication, and other services. This application helps judicial authorities to verify the authenticity of evidence forged using Deepfakes-related technologies. On multimedia platforms, the widespread dissemination of face-replacement videos has led to a decline in media credibility and increased susceptibility to misleading information. This application can help multimedia platforms screen videos, adding prominent labels (e.g., "made by Deepfakes") to detected forged videos, ensuring the credibility of the video content and maintaining public trust. In conclusion, this application can be effectively applied to products such as facial verification, judicial verification tools, and image / video authentication.

[0043] It is understood that in the specific embodiments of this application, data such as images to be processed are involved. When the above embodiments of this application are applied to specific products or technologies, the collection, use and processing of related data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0044] This application will be specifically illustrated through the following embodiments:

[0045] Please see Figure 1 , Figure 1 This is a schematic diagram of the architecture of an image processing system provided in an exemplary embodiment of this application. For example... Figure 1 As shown, the image processing system may specifically include a terminal device 101 and a server 102. The terminal device 101 and the server 102 are connected via a network, such as a local area network (LAN), a wide area network (WAN), or the mobile internet. The user operates on the browser or client application of the terminal device 101, performing image processing operations on various data. The server 102 can respond to this operation, providing various image processing services to the user.

[0046] Specifically, terminal device 101 can acquire the image to be processed and send it to server 102; server 102 can perform image processing operations on the received image to be processed to obtain the image processing result; server 102 sends the image processing result to terminal device 101 so that the operator can view the image processing result on terminal device 101.

[0047] Terminal equipment 101 is also referred to as terminal, user equipment (UE), access terminal, user unit, mobile device, user terminal, wireless communication equipment, user agent, or user device. Terminal equipment can be smart home appliances, handheld devices with wireless communication capabilities (such as smartphones and tablets), computing devices (such as personal computers (PCs), in-vehicle terminals, smart voice interaction devices, wearable devices, or other smart devices, but is not limited to these).

[0048] Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0049] It is understood that the system architecture diagrams described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. For example, the image processing method provided in the embodiments of this application can be executed not only by server 102, but also by other servers or server clusters that are different from server 102 and can communicate with terminal device 101 and / or server 102; it can also be executed by terminal device 101. For example, terminal device 101 acquires an image to be processed, and an image processing program integrated in terminal device 101 (or integrated in the application of terminal device 101) performs image processing operations on the received image to be processed to obtain the image processing result of the image to be processed; terminal device 101 sends the image processing result to server 102 so that terminal device 101 can retrieve the image processing result from server 102 at any time, thereby improving data acquisition and processing efficiency. It is understood by those skilled in the art that the number of terminal devices and servers mentioned above is merely illustrative. According to business implementation needs, any number of terminal devices and servers can be configured. Furthermore, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems. In subsequent embodiments, "client" will refer to the aforementioned terminal device 101, and "server" will refer to the aforementioned server 102; further details will not be repeated in subsequent embodiments.

[0050] Please see Figure 2 , Figure 2 This is a schematic flowchart of an image processing method provided in an exemplary embodiment of this application, applied to a client (referring to...). Figure 1 Taking the image processing model in the terminal device 101 as an example, the method may include the following steps:

[0051] S201. Perform feature extraction on the image to be processed to obtain a first feature map, and obtain a frequency domain feature map of the image to be processed, and perform feature extraction on the frequency domain feature map to obtain a second feature map.

[0052] In this embodiment, the image to be processed is the image for which image processing operations are to be performed (e.g., a forged image). The image processing model obtains the frequency domain feature map of the image to be processed, which can transform the feature dimension of the image to be processed into fine-grained frequency domain features. Then, image processing is performed based on the image to be processed and its frequency domain feature map, improving the feature representation capability. The image processing model then performs feature extraction on the image to be processed to obtain a first feature map, and on the frequency domain feature map to obtain a second feature map. The feature map can better represent the feature information of the image, and subsequent image processing is based on this feature map (including the first and second feature maps).

[0053] In one embodiment, the acquisition of the frequency domain feature map of the image to be processed can be achieved according to the following steps.

[0054] (1) Convert the image to be processed into a pixel format to obtain the image after pixel format conversion.

[0055] In one embodiment, the image to be processed can be a red-green-blue image (RGB image). RGB images are generated based on the RGB color mode, according to the color definition recognized by the human eye, that is, by varying the red (R), green (G), and blue (B) color channels and superimposing them to obtain various colors. The image after pixel format conversion can be a luminance-chrominance image (YCbCr image), where Y refers to the luminance component, Cb refers to the blue chrominance component, and Cr refers to the red chrominance component. By converting the RGB image to be processed into a pixel format, the pixel format converted image, i.e., the YCbCr image, can be obtained.

[0056] (2) Divide the image after pixel format conversion into M region images, and determine the global spectrum of each region image in the M region images, where M is a positive integer greater than 1.

[0057] In one embodiment, the image processing model can uniformly divide the pixel-formatted converted image into M region images, each region image being a part of the pixel-formatted converted image. For example, if the size of the pixel-formatted converted image is 16*16 units, and M is 4, then the pixel-formatted converted image is uniformly divided into 4 region images, each region image being 8*8 units in size. The global spectrogram of each of the M region images is determined, that is, each region image is converted into a corresponding spectrogram, and the spectrogram corresponding to that region image is the global spectrogram.

[0058] In one embodiment, the method of dividing the pixel-format converted image into M region images can be that the image processing model extracts M region images from the pixel-format converted image through a non-overlapping sliding window; the method of determining the global spectrum of each region image in the M region images can be that each region image is converted into a global spectrum by Fourier transform (e.g., discrete cosine Fourier transform, fast Fourier transform, etc.).

[0059] (3) Obtain N local spectrum sets from M global spectrum maps. Each local spectrum set includes M local spectrum maps with matching frequency bands. Each local spectrum map is a part of one of the global spectrum maps in the M global spectrum maps. The global spectrum maps corresponding to different local spectrum maps included in each local spectrum map set are different. N is a positive integer greater than 1.

[0060] In this embodiment, each local spectrum set includes M local spectrum maps that match the frequency bands (i.e., have the same frequency band range). Each local spectrum map is a part of one of the M global spectrum maps, and the global spectrum maps corresponding to different local spectrum maps in each local spectrum set are different. The above steps can be understood as follows: each global spectrum map includes N local spectrum maps corresponding to N frequency band ranges; M local spectrum maps of the same frequency band range are extracted from the M global spectrum maps, and these M extracted local spectrum maps are used as a local spectrum set, resulting in a total of N local spectrum sets (corresponding to N frequency band ranges).

[0061] In one embodiment, it is assumed that the size of the image after pixel format conversion is 16*16 units, and the size of each region image is 8*8 units (i.e., M is 4 and N is 64). The above method also obtains 64 local spectrogram sets from 4 global spectrograms, and each local spectrogram set includes 4 local spectrograms with matching frequency bands.

[0062] (4) Each of the M local spectrograms in each local spectrogram set is spliced ​​together to obtain N spliced ​​spectrograms.

[0063] In one embodiment, it is assumed that the size of the image after pixel format conversion is 16*16 units, and the size of each region image is 8*8 units (i.e., M is 4 and N is 64). The above method is to stitch together the four local spectrograms included in each local spectrogram set according to the order of the frequency band range in the global spectrogram, so as to obtain 64 stitched spectrograms.

[0064] (5) Determine the frequency domain feature map of the image to be processed based on the N spliced ​​spectrograms.

[0065] In one embodiment, after obtaining N stitched spectrograms, these N stitched spectrograms can be used as frequency domain feature maps of the image to be processed. Alternatively, convolutional kernels can be used to compress the N stitched spectrograms by channel dimension, and the compressed N stitched spectrograms can be used as frequency domain feature maps of the image to be processed.

[0066] In one embodiment, it is assumed that the size of the image after pixel format conversion is 16*16 units, and the size of each region image is 8*8 units (i.e., M is 4, N is 64). Since the image to be processed is first converted to pixel format, and then the converted image is converted into a frequency domain feature map, each frequency domain feature map can correspond to three channels. Taking each stitched spectrogram as one channel, we get 3*64 channels of data. Then, a 1*1 convolution kernel is used to compress the number of channels to the target value (e.g., 64), which is used as the frequency domain feature map of the image to be processed.

[0067] Please see Figure 3A , Figure 3A This is a flowchart illustrating the process of obtaining the frequency domain image of the image to be processed, as provided in an embodiment of this application. The method is integrated into an input image enhancement module. The input image enhancement module first converts the image to be processed (i.e., an RGB image) into a luminance-chrominance image (i.e., a YCbCr image). The input image enhancement module then determines multiple region images from the luminance-chrominance image using a sliding window. The input image enhancement module then performs a Fourier transform (e.g., Discrete Cosine Fourier Transform, Fast Fourier Transform, etc.) on each region image to obtain multiple global spectrograms corresponding to the multiple region images. The input image enhancement module then obtains multiple local spectrogram sets corresponding to multiple frequency ranges from the multiple global spectrograms and stitches the local spectrograms included in each local spectrogram set together to obtain multiple stitched spectrograms. Finally, the input image enhancement module uses the multiple stitched spectrograms as the frequency domain feature map of the image to be processed.

[0068] S202. Perform self-enhancement processing on the first feature map to obtain a first self-enhancement feature map, and perform self-enhancement processing on the second feature map to obtain a second self-enhancement feature map.

[0069] In this embodiment, the image processing model obtains a first self-enhanced feature map and a second self-enhanced feature map by performing self-enhancing processing on the first feature map and the second feature map. This self-enhancing processing can enhance the noise dimension and channel dimension of the features of the first feature map and the second feature map to achieve the effect of enhancing the differential features. The specific implementation processes of performing self-enhancing processing on the first feature map to obtain the first self-enhanced feature map, and performing self-enhancing processing on the second feature map to obtain the second self-enhanced feature map, will be described in detail in subsequent embodiments and will not be repeated in this embodiment.

[0070] S203. The first self-enhancing feature map and the second self-enhancing feature map are fused together, and the spatial attention map corresponding to the first self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map are determined based on the fused feature map obtained by the fusion process.

[0071] In this embodiment, the fusion of the first self-enhanced feature map and the second self-enhanced feature map is performed to obtain dual-stream feature information that combines the image to be processed (e.g., an RGB image) and the frequency domain image, thereby improving the feature representation capability. Then, the image processing model determines the spatial attention map corresponding to the first self-enhanced feature map and the spatial attention map corresponding to the second self-enhanced feature map using the fused feature map obtained from the fusion process.

[0072] In one embodiment, the above-mentioned fusion processing of the first self-enhancing feature map and the second self-enhancing feature map, and the determination of the spatial attention map corresponding to the first self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map based on the fused feature map obtained by the fusion processing, can be implemented according to the following steps.

[0073] (1) The first self-enhanced feature map and the second self-enhanced feature map are concatenated to obtain the concatenated feature map.

[0074] In one embodiment, the image processing model can be stitched together according to channels. For example, it can be stitched together in the order of the first self-enhanced feature map to the second self-enhanced feature map, or it can be stitched together in the order of the second self-enhanced feature map to the first self-enhanced feature map, and finally a stitched feature map is obtained.

[0075] (2) Perform convolution and normalization on the spliced ​​feature map to obtain the fused feature map.

[0076] In one embodiment, the image processing model can use a 1*1 convolution kernel to convolve the concatenated feature map; the sigmoid function (an activation function) can be used to normalize the concatenated feature map after convolution. After processing, a fused feature map that integrates the dual-stream feature information of the RGB image and the frequency domain image can be obtained.

[0077] (3) The fused feature map is split by channel, and a part of the fused feature map obtained by splitting is determined as the spatial attention map corresponding to the first self-enhancing feature map, and the other part of the fused feature map obtained by splitting is determined as the spatial attention map corresponding to the second self-enhancing feature map.

[0078] In one embodiment, the image processing model can split the fused feature map by channel to obtain a spatial attention map corresponding to a first self-enhanced feature map and a spatial attention map corresponding to a second self-enhanced feature map. For example, when the splicing order is from the first self-enhanced feature map to the second self-enhanced feature map, the first part of the fused feature map obtained by splitting by channel is determined as the spatial attention map corresponding to the first self-enhanced feature map; when the splicing order is from the second self-enhanced feature map to the first self-enhanced feature map, the second part of the fused feature map obtained by splitting by channel is determined as the spatial attention map corresponding to the second self-enhanced feature map.

[0079] S204. Determine the first mutual reinforcement feature map based on the first self-reinforcing feature map and the spatial attention map corresponding to the first self-reinforcing feature map, and determine the second mutual reinforcement feature map based on the second self-reinforcing feature map and the spatial attention map corresponding to the second self-reinforcing feature map.

[0080] In this embodiment, the spatial attention map is used to characterize the varying importance of spatial location information in an image. By obtaining the spatial attention map corresponding to the first self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map, the most spatially important parts of the image to be processed and the frequency domain image are identified for processing, thereby improving the accuracy of the image processing results. Since the first self-enhancing feature map is obtained by the image processing model based on the first self-enhancing feature map and the intermediate feature map representing the spatial enhancement part, that is, spatial dimension enhancement is performed on the basis of the first self-enhancing feature map, thus enhancing the distinguishability of the representation.

[0081] In one embodiment, the determination of the first mutual enhancement feature map based on the first self-enhancing feature map and the spatial attention map corresponding to the first self-enhancing feature map can be achieved according to the following steps.

[0082] (1) Perform operations on the first self-enhanced feature map and the spatial attention map corresponding to the first self-enhanced feature map to obtain the intermediate feature map.

[0083] In one embodiment, the image processing model can perform a matrix-based bitwise multiplication operation between the first self-enhanced feature map and the spatial attention map corresponding to the first self-enhanced feature map to obtain an intermediate feature map.

[0084] (2) Use the second channel weighting parameter to weight the intermediate feature map to obtain the weighted intermediate feature map.

[0085] In one embodiment, the image processing model can weight the intermediate feature map with a second channel weighting parameter to obtain a weighted intermediate feature map. The second channel weighting parameter can be a learnable channel weight, which can be automatically generated during model training using training samples.

[0086] (3) Perform operations on the weighted intermediate feature map and the first self-enhancing feature map to obtain the first mutual-enhancing feature map.

[0087] In one embodiment, the image processing model can perform a matrix-based bitwise addition operation between the weighted intermediate feature map and the first self-enhanced feature map to obtain a first mutually enhanced feature map. That is, spatial dimension enhancement is performed on the first self-enhanced feature map, thereby improving the distinguishability of the representation.

[0088] It should be noted that this embodiment requires feature enhancement processing on the image to be processed and its frequency domain feature map. After processing, the dual-stream features corresponding to the image to be processed (i.e., the features corresponding to the RGB image and the features corresponding to the frequency domain image) are obtained. Through the above steps (1)-(3), the first mutual enhancement feature map can be determined. At the same time, this application also needs to determine the second mutual enhancement feature map based on the second self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map. The specific implementation steps are as follows.

[0089] (1) Perform operations on the second self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map to obtain the intermediate feature map corresponding to the second self-enhancing feature map.

[0090] (2) Use the second channel weighting parameter to weight the intermediate feature map corresponding to the second self-enhanced feature map to obtain the weighted intermediate feature map.

[0091] (3) Perform operations on the weighted intermediate feature map and the second self-enhancing feature map to obtain the second mutual-enhancing feature map.

[0092] Please refer to the relevant description of the first mutually reinforcing feature map determination step (i.e., steps (1)-(3)) in S204 above) in this application for the above steps. This embodiment will not repeat the above steps.

[0093] Please see Figure 3B , Figure 3BThis is a flowchart illustrating the process of determining a first mutually enhanced feature map based on a first self-enhanced feature map, as provided in an embodiment of this application. This method can be integrated into a mutual enhancement module. The mutual enhancement module first merges the first and second self-enhanced feature maps. The mutual enhancement module then performs convolution and normalization processing on the merged feature map, followed by splitting it to obtain the spatial attention map corresponding to the first self-enhanced feature map (i.e., the first spatial attention map in the figure) and the spatial attention map corresponding to the second self-enhanced feature map (i.e., the second spatial attention map in the figure). The mutual enhancement module then performs a matrix-corresponding bitwise multiplication operation on the first self-enhanced feature map and the spatial attention map corresponding to the first self-enhanced feature map (wherein, the matrix-corresponding bitwise multiplication operation is performed on the first self-enhanced feature map). The first self-enhanced feature map is obtained by performing matrix-based bitwise multiplication on the feature maps corresponding to multiple channels of the merged feature map (i.e., the "multi-layer" process referred to in the figure). Then, the second self-enhanced feature map is obtained by performing matrix-based bitwise multiplication on the spatial attention maps corresponding to the first and second self-enhanced feature maps. Finally, the mutual enhancement module performs matrix-based bitwise addition on the first self-enhanced feature map and its corresponding intermediate feature map to obtain the first mutual enhancement feature map, and performs matrix-based bitwise addition on the second self-enhanced feature map and its corresponding intermediate feature map to obtain the second mutual enhancement feature map.

[0094] S205. Determine the image processing result of the image to be processed based on the first mutual enhancement feature map and the second mutual enhancement feature map.

[0095] In this embodiment, data augmentation provides the possibility of uncovering more subtle artifacts hidden in the frequency domain. The noise enhancement in the self-enhancement module increases the robustness of the image processing model network to input disturbances such as noise addition and blurring, and dynamically focuses on different channels through the channel attention mechanism. The mutual enhancement module comprehensively utilizes the shared spatial information of the two streams, including the image to be processed and the frequency domain image, to enhance the spatial features of each stream. That is, it fuses the dual-stream features of the image to be processed and the frequency domain image into the first mutual enhancement feature map and the second mutual enhancement feature map, which greatly improves its feature representation capability. Furthermore, the first and second mutual enhancement feature maps are obtained after feature enhancement processing, which improves the distinguishability of the representation. Based on the first and second mutual enhancement feature maps, the image processing results such as image classification and image identification of the image to be processed can be determined, thereby improving the accuracy of the image processing results.

[0096] In one embodiment, the image processing result of determining the image to be processed based on the first mutually enhanced feature map and the second mutually enhanced feature map can be achieved according to the following steps.

[0097] (1) The first and second mutual enhancement feature maps are concatenated along the channel dimension.

[0098] (2) Call the classifier to perform image processing on the first and second mutually enhanced feature maps after splicing, and obtain the image processing result of the image to be processed.

[0099] In one embodiment, when the image processing is image classification processing, by calling the classifier to process the stitched first and second mutual enhancement feature maps, multiple probability values ​​corresponding to multiple categories can be obtained. Then, the category corresponding to the highest probability value among the multiple probability values ​​is taken as the classification result of the image to be processed.

[0100] In summary, the beneficial effects of this application are as follows: First, this application extracts the image to be processed for dual-stream feature processing and transforms the feature dimensions of the image to be processed into a more fine-grained frequency domain feature map. Subsequently, image processing is performed based on the image to be processed and its frequency domain feature map, improving the diversity of data dimensions. Then, self-enhancement processing is applied to the first feature map obtained from the image to be processed and the second feature map obtained from the frequency domain feature map, resulting in a first self-enhanced feature map and a second self-enhanced feature map. Self-enhanced processing is data enhancement processing performed within single-stream features. Data enhancement provides the possibility of discovering more subtle artifacts hidden in the image to be processed or the frequency domain feature map, improving the distinguishability of the representation. By fusing the first and second self-enhanced feature maps, dual-stream feature information that integrates the image to be processed and the frequency domain image can be obtained, improving... The system enhances the representational ability of features; then, based on the fused feature map obtained through fusion processing, it determines the spatial attention maps corresponding to the first self-enhancing feature map and the second self-enhancing feature map, respectively. The spatial attention maps represent the importance of spatial location information in the image. In the process of determining the first and second mutual-enhancing feature maps, since the first mutual-enhancing feature map (or the second mutual-enhancing feature map) is based on the first self-enhancing feature map (or the second mutual-enhancing feature map) and the spatial attention map used to calculate the spatial enhancement part, that is, the spatial dimension is enhanced on the basis of the first self-enhancing feature map (or the second mutual-enhancing feature map), which further enhances the distinguishability of the representation. Based on the first and second mutual-enhancing feature maps, this application then performs image processing to determine the image processing results such as image classification and image identification of the image to be processed, thereby improving the accuracy of the image processing results.

[0101] This application also provides a specific method for obtaining the frequency domain feature map of the image to be processed. Through operations such as pixel format conversion, region division, and acquisition of channel data based on frequency bands, a frequency domain feature map corresponding to multiple frequency band ranges is finally obtained, providing a reference for the application and business scenarios of this application. This application also proposes to perform operations such as feature map stitching, convolution processing, normalization processing, and spatial attention map splitting in sequence during the process of determining the spatial attention map corresponding to the first self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map, and provides a variety of specific implementation methods to improve the flexibility of image processing. The self-enhancing module of this application can increase the robustness of the image processing model network to input perturbations such as noise addition and blurring by enhancing noise, and dynamically pay attention to different channels through the channel attention mechanism. The mutual enhancement module comprehensively utilizes the spatial information shared by the two streams, including the image to be processed and the frequency domain image, to enhance the spatial features of each stream respectively. That is, it fuses the dual-stream features of the image to be processed and the frequency domain image into the first mutual enhancement feature map and the second mutual enhancement feature map, which greatly improves its feature representation capability.

[0102] Please see Figure 4 , Figure 4 This is a schematic flowchart of an image processing method provided in an exemplary embodiment of this application, applied to a client (referring to...). Figure 1 Taking the image processing model in the terminal device 101 as an example, the method may include the following steps:

[0103] S401. Perform feature extraction on the image to be processed to obtain a first feature map, and obtain a frequency domain feature map of the image to be processed, and perform feature extraction on the frequency domain feature map to obtain a second feature map.

[0104] The specific implementation of step S401 is described in the relevant description of step S201 in the foregoing embodiments, and will not be repeated here.

[0105] In one embodiment, the image to be processed may be obtained from video data. Before obtaining the frequency domain feature map of the image to be processed, performing feature extraction on the image to be processed to obtain a first feature map, and performing feature extraction on the frequency domain feature map to obtain a second feature map, the following steps are also included.

[0106] (1) Obtain video data and perform image extraction operation on the video data to obtain multiple first images; wherein, the image extraction operation is to extract multiple frames of images from the video data.

[0107] (2) Perform target detection operation on multiple first images, and generate multiple second images based on the target detection results. Each of the multiple second images includes a target object corresponding to a target detection operation; wherein, the target detection operation is to detect the target object in the first image.

[0108] (3) Use multiple second images as images to be processed.

[0109] For example, when the video data is facial video, after acquiring the facial video, the image processing model can use software such as OpenCV to sample the facial video at equal intervals (e.g., sampling at 50 frames per second, resulting in 50 first images) to obtain the first image. Then, facial detection technology is used to define the region containing the face in the first image, and this region is expanded by a target factor (e.g., 1.2 times) to ensure that the cropping result includes the entire face and part of the surrounding background region, thus avoiding incomplete cropping and improving the accuracy of facial recognition. It should be noted that when multiple faces exist in a certain first image of the video data, all detected faces in the first image can be saved, increasing the richness of the data and further improving the accuracy of facial recognition. Through the above method, various types of raw data can be processed, expanding the scope of application.

[0110] This application requires performing self-enhancement processing on a first feature map to obtain a first self-enhanced feature map, and performing self-enhanced processing on a second feature map to obtain a second self-enhanced feature map. Specifically, the image processing model needs to perform the following two steps: the first step is to perform noise enhancement processing on the first feature map to obtain a noise-enhanced feature map corresponding to the first feature map; the second step is to perform channel enhancement processing on the noise-enhanced feature map to obtain a first self-enhanced feature map corresponding to the first feature map. The first step can be implemented based on the methods provided in S402-S405 of the subsequent embodiments, and the second step can be implemented based on the methods provided in S406-S408 of the subsequent embodiments.

[0111] S402. Denoise the first feature map to obtain the denoised feature map.

[0112] In this embodiment, images are often degraded during generation and transmission due to interference and influence from various noises (such as electrical noise, mechanical noise, channel noise, etc.), which adversely affects subsequent image processing. Therefore, in order to facilitate higher-level image processing of the image to be processed and to ensure the effect of image processing, the image processing model needs to perform denoising processing on the first feature map.

[0113] In one embodiment, spatial domain filtering, transform domain filtering, partial differential equations, variational methods, morphological noise filters, and other methods can be used to denoise the first feature map to obtain a denoised feature map. This application will use mean filtering to denoise the first feature map.

[0114] S403. Perform calculations on the first feature map and the denoised feature map to obtain the noise feature map.

[0115] In this embodiment of the application, the first feature map is mixed with noise. By performing calculations on the first feature map and the denoised feature map, the noise part present in the first feature map can be obtained, that is, the noise feature map.

[0116] In one embodiment, the first feature map can be subtracted from the denoised feature map by corresponding matrix elements to obtain the noise feature map.

[0117] S404. Normalize the noise feature map using a normalization function, and then weight the normalized noise feature map using noise weighting parameters to obtain a weighted noise feature map.

[0118] In one embodiment, the image processing model can use the sigmoid function to normalize the noise feature map. After normalization, the variables in the noise feature map can be mapped to the range [0,1]. The sigmoid function is monotonically increasing and symmetrical about (0,0.5), and its rate of change is relatively slow at both ends. Therefore, by using the sigmoid function, the magnitude of noise enhancement can be dynamically adjusted. Then, the normalized noise feature map is weighted using noise weighting parameters to obtain a weighted noise feature map.

[0119] The noise weighting parameter can be a learnable channel weight, which can be automatically generated during model training using training samples. The noise weighting parameter can adjust the weights of the normalized noise feature maps corresponding to each layer (that is, the feature maps corresponding to multiple channels obtained after feature extraction of the image to be processed through convolutional blocks), and finally obtain the weighted noise feature map.

[0120] S405. Perform calculations on the weighted noise feature map and the first feature map to obtain the noise enhancement feature map corresponding to the first feature map.

[0121] In one embodiment, the image processing model can perform a matrix-based bitwise addition operation between the weighted noise feature map and the first feature map to obtain a noise-enhanced feature map corresponding to the first feature map. That is, feature enhancement is performed based on the first feature map, further expanding the influence of noise on the first feature map and enhancing the distinguishability of the representation.

[0122] In one embodiment, since the effect of noise enhancement is limited in high-level features (i.e., the dimension of a certain channel of the feature map is high), and the image processing effect may even deteriorate at higher dimensions, the image processing model can adjust the processing steps of the self-enhancement model according to the channel features of the feature map (including feature maps of multiple channels) after processing the same convolutional block. The specific implementation is as follows.

[0123] (1) Obtain the feature dimension of the first feature map. The feature dimension is the dimension of the feature map of the corresponding channel of the first feature map.

[0124] (2) When the feature dimension meets the dimension threshold, perform channel enhancement processing on the noise enhancement feature map to obtain the first self-enhanced feature map corresponding to the first feature map. The dimension threshold can be preset.

[0125] The above method avoids noise enhancement processing on feature maps with high feature dimensions (that is, skipping steps S402-S405 and directly proceeding to steps S406-S408), thereby reducing the amount of computation and improving processing efficiency while ensuring the image processing effect.

[0126] The methods provided in S402-S405 of this application embodiment can achieve the purpose of noise enhancement processing on the first feature map to obtain a noise-enhanced feature map corresponding to the first feature map. The following will describe the steps of performing channel enhancement processing on the noise-enhanced feature map to obtain a first self-enhanced feature map corresponding to the first feature map.

[0127] S406. Perform pooling processing on the noise enhancement feature map, and determine the weighting parameters of the first channel based on the pooling processing results.

[0128] In this embodiment, the image processing model can perform pooling on the noise enhancement feature map, and then determine the weighting parameters of the first channel based on the pooling result. Pooling can reduce the feature dimension of the noise enhancement feature map, thereby improving image processing speed.

[0129] In one embodiment, the image processing model can employ multiple pooling methods for pooling processing, and then perform calculations on the resulting pooling results to obtain the first channel weighting parameters. For example, the image processing model performs global uniform pooling and global max pooling on the noise-enhanced feature map (which includes feature maps corresponding to multiple channels). Then, the result of the global uniform pooling is added to the result of the global max pooling, and the added result (which also corresponds to multiple results for the feature maps of multiple channels) is processed by a multilayer perceptron (e.g., a neural network) to finally obtain the first channel weighting parameters.

[0130] S407. The noise enhancement feature map is weighted according to the weighting parameters of the first channel to obtain the weighted noise enhancement feature map.

[0131] The first channel weighting parameter can be determined by the image processing model pooling the noise enhancement feature map and then based on the pooling result. Alternatively, it can be a learnable channel weight that is automatically generated during model training using training samples.

[0132] S408. Perform calculations on the weighted noise enhancement feature map and the noise enhancement feature map to obtain the first self-enhancing feature map corresponding to the first feature map.

[0133] In one embodiment, the image processing model can perform a matrix-based bitwise addition operation between the weighted noise enhancement feature map and the noise enhancement feature map to obtain a first self-enhanced feature map corresponding to the first feature map. That is, noise enhancement is performed on the basis of the noise enhancement feature map, further expanding the influence of noise on the noise enhancement feature map and enhancing the distinguishability of the representation.

[0134] The methods provided in S406-S408 of this application embodiment can achieve channel enhancement processing on the noise enhancement feature map to obtain the first self-enhanced feature map corresponding to the first feature map. Please refer to... Figure 5A , Figure 5A This is a flowchart illustrating the process of determining a first self-enhanced feature map and a second self-enhanced feature map based on a first feature map and a second feature map, as provided in this embodiment. This method is integrated into a self-enhancing module. Since the processing methods for the first and second feature maps are similar, this embodiment will use the processing of the first feature map by the self-enhancing model as an example. The self-enhancing module first calls a noise enhancement module to perform noise enhancement processing on the first feature map. Then, it performs a matrix-based bitwise addition operation between the noise-enhanced first feature map and the first feature map to obtain the noise-enhanced feature map corresponding to the first feature map. The self-enhancing module then calls a channel attention module to perform channel enhancement processing on the noise-enhanced feature map corresponding to the first feature map. Finally, it performs a matrix-based bitwise addition operation between the channel-enhanced noise-enhanced feature map and the noise-enhanced feature map to obtain the first enhanced feature information.

[0135] In the noise enhancement module of the self-enhancing model, the noise enhancement module first performs denoising on the first feature map to obtain a denoised feature map; then, it performs matrix-based bitwise addition on the first feature map and the denoised feature map to obtain a noise feature map; the noise enhancement module then normalizes the noise feature map and uses noise weighting parameters to perform matrix-based bitwise multiplication on the normalized noise feature map of each layer (including normalized noise feature maps corresponding to multiple channels) to obtain a weighted noise feature map (that is, the first feature map after noise enhancement as described above).

[0136] It should be noted that this embodiment requires self-enhancement processing on the first feature map to obtain a first self-enhancement feature map. Through the above steps S402-S408, the first self-enhancement feature map can be determined. At the same time, this application also requires self-enhancement processing on the second feature map to obtain a second self-enhancement feature map. The specific implementation steps are as follows.

[0137] (1) Denoise the second feature map to obtain the denoised feature map.

[0138] (2) Perform calculations on the second feature map and the denoised feature map to obtain a noise feature map.

[0139] (3) The noise feature map is normalized using a normalization function, and the normalized noise feature map is weighted using noise weighting parameters to obtain the weighted noise feature map.

[0140] (4) Perform calculations on the weighted noise feature map and the second feature map to obtain the noise enhancement feature map corresponding to the second feature map.

[0141] (5) Perform pooling processing on the noise enhancement feature map, and determine the weighting parameters of the first channel based on the pooling processing results.

[0142] (6) The noise enhancement feature map corresponding to the second feature map is weighted according to the weighting parameters of the first channel to obtain the weighted noise enhancement feature map.

[0143] (7) Perform calculations on the weighted noise enhancement feature map and the noise enhancement feature map to obtain the second self-enhancing feature map corresponding to the second feature map.

[0144] Please refer to the description of the first self-enhancing feature map determination step in this application (i.e., the relevant description of steps S402-S408 above, which will not be repeated in this embodiment).

[0145] S409. The first self-enhancing feature map and the second self-enhancing feature map are fused together, and the spatial attention map corresponding to the first self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map are determined based on the fused feature map obtained by the fusion process.

[0146] S410. Determine the first mutual reinforcement feature map based on the first self-reinforcing feature map and the spatial attention map corresponding to the first self-reinforcing feature map, and determine the second mutual reinforcement feature map based on the second self-reinforcing feature map and the spatial attention map corresponding to the second self-reinforcing feature map.

[0147] S411. Determine the image processing result of the image to be processed based on the first mutually enhanced feature map and the second mutually enhanced feature map.

[0148] The specific implementation of steps S409-S411 is described in the relevant description of steps S203-S205 in the foregoing embodiments, and will not be repeated here.

[0149] In one embodiment, the image processing model proposed in this application is obtained by training an initial model based on training samples. The training process of the model is described in S201-S205 and S401-S411 of this application, and will not be elaborated upon here. The trained image processing model can be used to infer forged facial images or forged facial videos.

[0150] For facial images, face detection techniques such as Multi-task Convolutional Neural Networks (MTCNN) can be used to crop the facial region from the image, and then enlarge this region by a factor of 1.2. Next, all cropped facial images are fed into a trained image processing model. The model will output a probability value indicating whether the facial image is a fake (e.g., a face-swapped image). The higher the probability value, the greater the probability that the face in the image has been replaced.

[0151] For facial videos, the probability of a facial video being forged can be calculated using the following method, with the specific steps as follows.

[0152] (1) Determine multiple facial images based on facial videos.

[0153] (2) Determine multiple probability values ​​based on multiple facial images.

[0154] (3) Determine the image processing result of the facial video based on the probability mean of multiple probability values.

[0155] Specifically, software such as OpenCV can be used to sample video frames of the facial video at equal intervals (the interval can be adjusted). Then, the facial region in the facial image is cropped using the method described above, and enlarged by a factor of 1.2 centered on this region. Next, each frame of the facial image is fed into a trained image processing model. The model will output multiple probability values ​​to indicate whether each frame of the facial image is a forged facial image (e.g., a face-swapped image). The forgery probability values ​​of each frame are averaged to obtain the forgery probability value of the facial video. If the forgery probability value of the facial video reaches a threshold (e.g., 0.8), then the facial video is determined to be forged.

[0156] In one embodiment, the method for calculating the forgery probability value of the facial video described above can also be replaced by a support vector machine or a multilayer perceptron. The forgery probability of each frame of facial image is input into the support vector machine or multilayer perceptron, and finally the forgery probability value of the output facial video is obtained.

[0157] Please see Figure 5B , Figure 5B This is a flowchart illustrating the structure and processing flow of an image processing model provided in an embodiment of this application. The image processing model includes an input image enhancement module, a self-enhancement module, and a mutual enhancement module. For the internal processing procedures of the input image enhancement module, the self-enhancement module, and the mutual enhancement module, please refer to the relevant descriptions of each module in the foregoing embodiments (i.e.,...). Figure 3A , Figure 3B and Figure 5A The relevant descriptions will not be repeated in this embodiment. This embodiment mainly describes the interaction process between the input image enhancement module, the self-enhancement module, and the mutual enhancement module.

[0158] The input image enhancement module outputs the image to be processed and its corresponding frequency domain feature map. The image processing model extracts features from the image to be processed and its corresponding frequency domain feature map using convolutional blocks, obtaining a first feature map and a second feature map. These first and second feature maps are then used as input to the self-enhancement module. The self-enhancement module acquires the first and second feature maps and performs image processing through its noise enhancement module, channel attention module, etc., to obtain a first self-enhancement feature map corresponding to the first feature map and a second self-enhancement feature map corresponding to the second feature map. These are then used as input to the mutual enhancement module. The mutual enhancement module acquires the first and second self-enhancement feature maps and performs mutual enhancement processing to obtain a first mutual enhancement feature map corresponding to the first self-enhancement feature map and a second mutual enhancement feature map corresponding to the second self-enhancement feature map. The image processing model then feeds the first and second mutual enhancement feature maps into the classifier and outputs the results (including real and fake data).

[0159] In one embodiment, the self-enhancing module and the mutual-enhancing module can be a combined module. When the image processing model processes the image to be processed and the frequency domain feature map based on multiple sets of convolutional blocks (each set of convolutional blocks corresponds to one processing of the image to be processed and the frequency domain feature map), the image processing model can include multiple combined modules including the self-enhancing module and the mutual-enhancing module (or, the image processing model calls the combined module including the self-enhancing module and the mutual-enhancing module multiple times) to process it.

[0160] In one embodiment, for convolutional blocks other than the last set, the dual-stream features obtained after processing by the mutual enhancement model are fed into one set of convolutional blocks in each of the two streams; for the last set of convolutional blocks, the dual-stream features obtained after processing by the mutual enhancement model are concatenated along the channel dimension and then fed into the classifier. Please refer to [link to relevant documentation]. Figure 5C , Figure 5C This application provides a structural diagram and flowchart of an image processing model comprising multiple combined modules. Figure 5C As shown, there are n sets of convolutional blocks (corresponding to n combination modules). The image processing model calls the first set of convolutional blocks to process the image to be processed and the frequency domain feature map, obtaining two feature maps. These two feature maps are then fed into the first combination module (including a self-enhancing module and a mutual-enhancing module), outputting two mutually enhanced feature maps. The image processing model then calls the second set of convolutional blocks to process the two mutually enhanced feature maps, obtaining two more feature maps. These two feature maps are then fed into the second combination module, again outputting two mutually enhanced feature maps. This process is repeated until the last set of convolutional blocks (the nth convolutional block in the figure, corresponding to the nth combination module) is processed. Finally, the two mutually enhanced feature maps output are fed into the classifier for image classification. This method allows for processing with multiple convolutional blocks, improving the image processing efficiency.

[0161] In summary, the beneficial effects of this application are as follows: This application provides a method for obtaining images to be processed from video data, and expands the region during the cropping of target objects to avoid incomplete cropping results; when multiple target objects exist in the video data, each target object is saved separately, improving data richness and the accuracy of image processing results. Through the above method, various types of raw data can be processed, expanding the applicability. When this application is applied to video data, the forgery probability value of each frame in the video data is obtained by inputting it into a support vector machine (or a multilayer perceptron, or using the probability mean), thus improving the accuracy of image processing results.

[0162] This application also proposes denoising multiple feature maps during the processing, facilitating higher-level image processing and ensuring the quality of image processing. Furthermore, it proposes performing a matrix-based bitwise addition operation between the weighted noise feature map and the first feature map, i.e., feature enhancement based on the first feature map, further expanding the influence of noise on the first feature map and enhancing the discriminability of the representation. This application also proposes that in the self-enhancement module, noise enhancement is not performed on feature maps with high feature dimensions; instead, the channel attention module is directly called for channel enhancement, reducing computational load and improving processing efficiency while ensuring image processing quality. Finally, this application proposes combining the self-enhancement module and the mutual enhancement module as a combined module, repeatedly calling the combined module based on multiple convolutional blocks for image processing, thereby improving the image processing effect.

[0163] Please see Figure 6 , Figure 6 This is a schematic block diagram of an image processing apparatus provided in an embodiment of this application. Specifically, the image processing apparatus may include:

[0164] The acquisition module 601 is used to extract features from the image to be processed to obtain a first feature map, and to acquire the frequency domain feature map of the image to be processed, and to extract features from the frequency domain feature map to obtain a second feature map.

[0165] The processing module 602 is used to perform self-enhancement processing on the first feature map to obtain a first self-enhancement feature map, and to perform self-enhancement processing on the second feature map to obtain a second self-enhancement feature map.

[0166] The processing module 602 is further configured to perform a fusion process on the first self-enhancing feature map and the second self-enhancing feature map, and determine the spatial attention map corresponding to the first self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map based on the fused feature map obtained by the fusion process.

[0167] The aforementioned processing module 602 is further configured to determine a first mutual enhancement feature map based on the first self-enhancing feature map and the spatial attention map corresponding to the first self-enhancing feature map, and to determine a second mutual enhancement feature map based on the second self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map.

[0168] The classification module 603 is used to determine the image processing result of the image to be processed based on the first mutual enhancement feature map and the second mutual enhancement feature map.

[0169] Optionally, when acquiring the frequency domain feature map of the image to be processed, the acquisition module 601 is specifically used for:

[0170] The above image to be processed is converted to a pixel format to obtain a pixel-formatted image.

[0171] The image after the above pixel format conversion is divided into M region images, and the global spectrogram of each of the above M region images is determined, where M is a positive integer greater than 1;

[0172] N local spectrum sets are obtained from M global spectrum maps. Each local spectrum set includes M local spectrum maps with matching frequency bands. Each local spectrum map is a part of one of the M global spectrum maps. The global spectrum maps corresponding to different local spectrum maps included in each local spectrum set are different. N is a positive integer greater than 1.

[0173] Each local spectrogram set contains M local spectrograms, which are then stitched together to obtain N stitched spectrograms.

[0174] The frequency domain feature map of the image to be processed is determined based on the above N spliced ​​spectrograms.

[0175] Optionally, when the processing module 602 performs self-enhancement processing on the first feature map to obtain the first self-enhancement feature map, it is specifically used for:

[0176] The first feature map is subjected to noise enhancement processing to obtain the noise-enhanced feature map corresponding to the first feature map;

[0177] The above noise enhancement feature map is subjected to channel enhancement processing to obtain the first self-enhanced feature map corresponding to the first feature map.

[0178] Optionally, when the processing module 602 performs noise enhancement processing on the first feature map to obtain a noise-enhanced feature map corresponding to the first feature map, it is specifically used for:

[0179] The first feature map is denoised to obtain a denoised feature map.

[0180] The first feature map and the denoised feature map are processed to obtain a noise feature map.

[0181] The noise feature map above is normalized using a normalization function, and the normalized noise feature map is weighted using noise weighting parameters to obtain a weighted noise feature map.

[0182] The weighted noise feature map and the first feature map are processed to obtain the noise enhancement feature map corresponding to the first feature map.

[0183] Optionally, when the processing module 602 performs channel enhancement processing on the noise enhancement feature map to obtain the first self-enhancing feature map corresponding to the first feature map, it is specifically used for:

[0184] The above noise enhancement feature map is subjected to pooling processing, and the weighting parameters of the first channel are determined based on the pooling processing results;

[0185] The noise enhancement feature map is weighted according to the weighting parameters of the first channel to obtain the weighted noise enhancement feature map.

[0186] The weighted noise enhancement feature map and the noise enhancement feature map are processed together to obtain the first self-enhancing feature map corresponding to the first feature map.

[0187] Optionally, when the processing module 602 fuses the first self-enhancing feature map and the second self-enhancing feature map, and determines the spatial attention map corresponding to the first self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map based on the fused feature map obtained from the fusion process, it is specifically used for:

[0188] The first self-enhanced feature map and the second self-enhanced feature map are concatenated to obtain a concatenated feature map;

[0189] The above-mentioned spliced ​​feature maps are subjected to convolution and normalization processing to obtain fused feature maps;

[0190] The above fused feature map is split by channel. One part of the split fused feature map is determined as the spatial attention map corresponding to the first self-enhancing feature map, and the other part of the split fused feature map is determined as the spatial attention map corresponding to the second self-enhancing feature map.

[0191] Optionally, when the processing module 602 determines the first mutual enhancement feature map based on the first self-enhancing feature map and the spatial attention map corresponding to the first self-enhancing feature map, it specifically performs the following:

[0192] The first self-enhanced feature map and the spatial attention map corresponding to the first self-enhanced feature map are processed to obtain an intermediate feature map;

[0193] The intermediate feature map is weighted using the second channel weighting parameter to obtain the weighted intermediate feature map;

[0194] The weighted intermediate feature map and the first self-enhancing feature map are processed to obtain the first mutually enhanced feature map.

[0195] It should be noted that the functions of each functional module of the image processing device in this application embodiment can be specifically implemented according to the methods in the above method embodiments. The specific implementation process can be referred to the relevant descriptions in the above method embodiments, which will not be repeated here.

[0196] Please see Figure 7 , Figure 7 This is a schematic block diagram of a computer device provided in an embodiment of this application. As shown in the figure, the smart terminal in this embodiment may include: a processor 701, a storage device 702, and a communication interface 703. The processor 701, the storage device 702, and the communication interface 703 can interact with each other.

[0197] The aforementioned storage device 702 may include volatile memory, such as random-access memory (RAM); the storage device 702 may also include non-volatile memory, such as flash memory, solid-state drive (SSD), etc.; the aforementioned storage device 702 may also include a combination of the above types of memory.

[0198] The processor 701 described above may be a central processing unit (CPU). In one embodiment, the processor 701 may also be a graphics processing unit (GPU). Alternatively, the processor 701 may be a combination of a CPU and a GPU. In one embodiment, the storage device 702 is used to store program instructions, which the processor 701 can invoke to perform the following operations:

[0199] The first feature map is obtained by extracting features from the image to be processed, and the frequency domain feature map of the image to be processed is obtained. The second feature map is obtained by extracting features from the frequency domain feature map.

[0200] The first feature map is subjected to self-enhancement processing to obtain a first self-enhancement feature map, and the second feature map is subjected to self-enhancement processing to obtain a second self-enhancement feature map;

[0201] The first self-enhancing feature map and the second self-enhancing feature map are fused together, and the spatial attention map corresponding to the first self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map are determined based on the fused feature map obtained by the fusion process.

[0202] Based on the first self-enhancing feature map and the spatial attention map corresponding to the first self-enhancing feature map, a first mutual-enhancing feature map is determined, and based on the second self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map, a second mutual-enhancing feature map is determined.

[0203] The image processing result of the image to be processed is determined based on the first mutually enhanced feature map and the second mutually enhanced feature map.

[0204] Optionally, when the processor 701 is used to acquire the frequency domain feature map of the image to be processed, it is specifically used for:

[0205] The above image to be processed is converted to a pixel format to obtain a pixel-formatted image.

[0206] The image after the above pixel format conversion is divided into M region images, and the global spectrogram of each of the above M region images is determined, where M is a positive integer greater than 1;

[0207] N local spectrum sets are obtained from M global spectrum maps. Each local spectrum set includes M local spectrum maps with matching frequency bands. Each local spectrum map is a part of one of the M global spectrum maps. The global spectrum maps corresponding to different local spectrum maps included in each local spectrum set are different. N is a positive integer greater than 1.

[0208] Each local spectrogram set contains M local spectrograms, which are then stitched together to obtain N stitched spectrograms.

[0209] The frequency domain feature map of the image to be processed is determined based on the above N spliced ​​spectrograms.

[0210] Optionally, when the processor 701 performs self-enhancement processing on the first feature map to obtain a first self-enhancement feature map, it is specifically used for:

[0211] The first feature map is subjected to noise enhancement processing to obtain the noise-enhanced feature map corresponding to the first feature map;

[0212] The above noise enhancement feature map is subjected to channel enhancement processing to obtain the first self-enhanced feature map corresponding to the first feature map.

[0213] Optionally, when the processor 701 performs noise enhancement processing on the first feature map to obtain a noise-enhanced feature map corresponding to the first feature map, it is specifically used for:

[0214] The first feature map is denoised to obtain a denoised feature map.

[0215] The first feature map and the denoised feature map are processed to obtain a noise feature map.

[0216] The noise feature map above is normalized using a normalization function, and the normalized noise feature map is weighted using noise weighting parameters to obtain a weighted noise feature map.

[0217] The weighted noise feature map and the first feature map are processed to obtain the noise enhancement feature map corresponding to the first feature map.

[0218] Optionally, when the processor 701 performs channel enhancement processing on the noise enhancement feature map to obtain the first self-enhanced feature map corresponding to the first feature map, it is specifically used for:

[0219] The above noise enhancement feature map is subjected to pooling processing, and the weighting parameters of the first channel are determined based on the pooling processing results;

[0220] The noise enhancement feature map is weighted according to the weighting parameters of the first channel to obtain the weighted noise enhancement feature map.

[0221] The weighted noise enhancement feature map and the noise enhancement feature map are processed together to obtain the first self-enhancing feature map corresponding to the first feature map.

[0222] Optionally, when the processor 701 performs fusion processing on the first self-enhanced feature map and the second self-enhanced feature map, and determines the spatial attention map corresponding to the first self-enhanced feature map and the spatial attention map corresponding to the second self-enhanced feature map based on the fused feature map obtained from the fusion processing, it is specifically used for:

[0223] The first self-enhanced feature map and the second self-enhanced feature map are concatenated to obtain a concatenated feature map;

[0224] The above-mentioned spliced ​​feature maps are subjected to convolution and normalization processing to obtain fused feature maps;

[0225] The above fused feature map is split by channel. One part of the split fused feature map is determined as the spatial attention map corresponding to the first self-enhancing feature map, and the other part of the split fused feature map is determined as the spatial attention map corresponding to the second self-enhancing feature map.

[0226] Optionally, when the processor 701 determines the first mutual enhancement feature map based on the first self-enhancing feature map and the spatial attention map corresponding to the first self-enhancing feature map, it specifically performs the following:

[0227] The first self-enhanced feature map and the spatial attention map corresponding to the first self-enhanced feature map are processed to obtain an intermediate feature map;

[0228] The intermediate feature map is weighted using the second channel weighting parameter to obtain the weighted intermediate feature map;

[0229] The weighted intermediate feature map and the first self-enhancing feature map are processed to obtain the first mutually enhanced feature map.

[0230] In specific implementation, the processor 701, storage device 702, and communication interface 703 described in the embodiments of this application can execute the embodiments of this application. Figure 2 or Figure 4 The implementation methods described in the relevant embodiments of the provided image processing method can also be used to execute the embodiments of this application. Figure 6 The implementation methods described in the relevant embodiments of the provided image processing apparatus will not be repeated here.

[0231] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, and systems can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and other division methods may exist in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0232] Furthermore, it should be noted that this application embodiment also provides a computer-readable storage medium storing a computer program executed by the aforementioned image processing apparatus. This computer program includes program instructions, which, when executed by a processor, enable the execution of the aforementioned image processing apparatus. Figure 2 , Figure 4 The methods described in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same methods will also not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments related to this application, please refer to the description of the method embodiments of this application. As an example, program instructions can be deployed on a computer device, or executed on multiple computer devices located in one location, or executed on multiple computer devices distributed across multiple locations and interconnected via a communication network. These multiple computer devices distributed across multiple locations and interconnected via a communication network can constitute a blockchain system.

[0233] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned... Figure 2 , Figure 4 The methods described in the corresponding embodiments are therefore not repeated here.

[0234] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0235] The above-disclosed embodiments are merely some of the embodiments of this application, and should not be construed as limiting the scope of this application. Those skilled in the art can understand that implementing all or part of the above embodiments and making equivalent changes in accordance with the claims of this application still fall within the scope of the invention.

Claims

1. An image processing method, characterized in that, The method includes: A first feature map is obtained by performing feature extraction on the image to be processed, and a frequency domain feature map of the image to be processed is obtained. A second feature map is obtained by performing feature extraction on the frequency domain feature map. The first feature map is subjected to self-enhancement processing to obtain a first self-enhancement feature map, and the second feature map is subjected to self-enhancement processing to obtain a second self-enhancement feature map; The first self-enhancing feature map and the second self-enhancing feature map are fused together, and the spatial attention map corresponding to the first self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map are determined based on the fused feature map obtained by the fusion process. A first mutual enhancement feature map is determined based on the first self-enhancing feature map and the spatial attention map corresponding to the first self-enhancing feature map, and a second mutual enhancement feature map is determined based on the second self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map; The image processing result of the image to be processed is determined based on the first mutual enhancement feature map and the second mutual enhancement feature map.

2. The method according to claim 1, characterized in that, The process of obtaining the frequency domain feature map of the image to be processed includes: The image to be processed is converted to a pixel format to obtain a pixel-formatted image. The image after pixel format conversion is divided into M region images, and the global spectrogram of each region image in the M region images is determined, where M is a positive integer greater than 1; N local spectrum sets are obtained from M global spectrum maps. Each local spectrum set includes M local spectrum maps with matching frequency bands. Each local spectrum map is a part of one of the M global spectrum maps. The global spectrum maps corresponding to different local spectrum maps included in each local spectrum set are different. N is a positive integer greater than 1. Each local spectrogram set contains M local spectrograms, which are then stitched together to obtain N stitched spectrograms. The frequency domain feature map of the image to be processed is determined based on the N spliced ​​spectrograms.

3. The method according to claim 1, characterized in that, The step of performing self-enhancement processing on the first feature map to obtain the first self-enhancement feature map includes: The first feature map is subjected to noise enhancement processing to obtain a noise-enhanced feature map corresponding to the first feature map. The noise enhancement feature map is subjected to channel enhancement processing to obtain the first self-enhanced feature map corresponding to the first feature map.

4. The method according to claim 3, characterized in that, The step of performing noise enhancement processing on the first feature map to obtain a noise-enhanced feature map corresponding to the first feature map includes: The first feature map is denoised to obtain a denoised feature map. The first feature map and the denoised feature map are processed to obtain a noise feature map. The noise feature map is normalized using a normalization function, and then the normalized noise feature map is weighted using noise weighting parameters to obtain a weighted noise feature map. The weighted noise feature map and the first feature map are processed to obtain the noise enhancement feature map corresponding to the first feature map.

5. The method according to claim 3 or 4, characterized in that, The step of performing channel enhancement processing on the noise enhancement feature map to obtain the first self-enhanced feature map corresponding to the first feature map includes: The noise enhancement feature map is subjected to pooling processing, and the weighting parameters of the first channel are determined based on the pooling processing result; The noise enhancement feature map is weighted according to the weighting parameters of the first channel to obtain a weighted noise enhancement feature map. The weighted noise enhancement feature map and the noise enhancement feature map are processed to obtain the first self-enhancing feature map corresponding to the first feature map.

6. The method according to any one of claims 1-4, characterized in that, The step of fusing the first self-enhancing feature map and the second self-enhancing feature map, and determining the spatial attention map corresponding to the first self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map based on the fused feature map obtained by the fusion process, includes: The first self-enhanced feature map and the second self-enhanced feature map are concatenated to obtain a concatenated feature map; The spliced ​​feature map is subjected to convolution and normalization processing to obtain a fused feature map; The fused feature map is split by channel, and a portion of the split fused feature map is determined as the spatial attention map corresponding to the first self-enhancing feature map, while the other portion of the split fused feature map is determined as the spatial attention map corresponding to the second self-enhancing feature map.

7. The method according to any one of claims 1-4, characterized in that, The step of determining the first mutual enhancement feature map based on the first self-enhancing feature map and the spatial attention map corresponding to the first self-enhancing feature map includes: The first self-enhanced feature map and the spatial attention map corresponding to the first self-enhanced feature map are processed to obtain an intermediate feature map; The intermediate feature map is weighted using the second channel weighting parameter to obtain a weighted intermediate feature map; The weighted intermediate feature map and the first self-enhancing feature map are processed to obtain the first mutually enhanced feature map.

8. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to extract features from the image to be processed to obtain a first feature map, and to acquire a frequency domain feature map of the image to be processed, and to extract features from the frequency domain feature map to obtain a second feature map; The processing module is used to perform self-enhancement processing on the first feature map to obtain a first self-enhancement feature map, and to perform self-enhancement processing on the second feature map to obtain a second self-enhancement feature map; The processing module is further configured to fuse the first self-enhancing feature map and the second self-enhancing feature map, and determine the spatial attention map corresponding to the first self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map based on the fused feature map obtained by the fusion process. The processing module is further configured to determine a first mutual enhancement feature map based on the first self-enhancing feature map and the spatial attention map corresponding to the first self-enhancing feature map, and to determine a second mutual enhancement feature map based on the second self-enhancing feature map and the spatial attention map corresponding to the second self-enhancing feature map; The classification module is used to determine the image processing result of the image to be processed based on the first mutual enhancement feature map and the second mutual enhancement feature map.

9. A computer device, characterized in that, include: The processor, the communication interface, and the storage device are interconnected, wherein the storage device stores executable program code, and the processor is used to call the executable program code to implement the image processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions, which are executed by a processor to implement the image processing method as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, The computer program product includes a computer program or computer instructions, which, when executed by a processor, are used to implement the image processing method as described in any one of claims 1 to 7.

Images