Image processing method, device and equipment

Abstract

The present application provides an image processing method, device and equipment, which relates to the field of artificial intelligence technology and can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and assisted driving. The method includes: obtaining a target object image; performing a frequency domain transformation on the target object image to obtain a target frequency domain image corresponding to the target object image; calling a preset privacy model, and performing noise perturbation on the target frequency domain image based on the target privacy allocation parameters corresponding to each pixel point in the target frequency domain image to obtain a target noisy image corresponding to the target frequency domain image; calling a target image recognition model to perform image recognition on the target noisy image to obtain an image recognition result; the preset privacy model is a model loss obtained by performing image recognition on a sample noisy image corresponding to a sample object image based on the target image recognition model, and back-propagation training is performed on the initial privacy model. The present application can effectively protect image privacy.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image processing method, device and equipment. Background Art

[0002] Image recognition technology is used in many fields such as finance, justice, security, education and medical care due to its high recognition accuracy and real-time prediction capabilities. However, many images directly display the private information of the object, such as the object's gender, identity or approximate age, which introduces the risk of privacy leakage. Therefore, it is necessary to process the image accordingly to achieve privacy protection. However, the existing image processing methods have too much computational complexity and consume a lot of computing resources, which seriously increases the image recognition delay and affects the real-time prediction performance. Alternatively, it is necessary to introduce multiple parties to participate, increase the number of credit objects, and increase the risk of privacy leakage. Therefore, it is necessary to provide a reliable and effective solution to solve the problems existing in existing image processing. Summary of the invention

[0003] The present application provides an image processing method, apparatus and device that can optimize image recognition precision, accuracy and efficiency and effectively protect image privacy.

[0004] In one aspect, the present application provides an image processing method, the method comprising:

[0005] Acquire a target object image;

[0006] Performing frequency domain transformation on the target object image to obtain a target frequency domain image corresponding to the target object image;

[0007] Calling a preset privacy model, performing noise perturbation on the target frequency domain image based on the target privacy allocation parameters corresponding to each pixel point in the target frequency domain image, to obtain a target noisy image corresponding to the target frequency domain image, wherein the target privacy allocation parameters indicate the degree of noise addition on the corresponding pixel point in the target frequency domain image;

[0008] Calling a target image recognition model to perform image recognition on the target noisy image to obtain an image recognition result;

[0009] Among them, the preset privacy model is a model loss obtained by performing image recognition on a sample noisy image corresponding to a sample object image based on the target image recognition model, and a model obtained by back-propagation training of the initial privacy model, and the sample noisy image is obtained by performing noise perturbation on a sample frequency domain image corresponding to the sample object image by the initial privacy model.

[0010] Another aspect provides an image processing device, the device comprising:

[0011] Image acquisition module: used to acquire the image of the target object;

[0012] Frequency domain transformation module: used to perform frequency domain transformation on the target object image to obtain a target frequency domain image corresponding to the target object image;

[0013] Noise perturbation module: used to call a preset privacy model, perform noise perturbation on the target frequency domain image based on the target privacy allocation parameter corresponding to each pixel point in the target frequency domain image, and obtain a target noisy image corresponding to the target frequency domain image, wherein the target privacy allocation parameter indicates the degree of noise addition to the corresponding pixel point in the target frequency domain image;

[0014] Image recognition module: used to call the target image recognition model to perform image recognition on the target noisy image to obtain an image recognition result;

[0015] Among them, the preset privacy model is a model loss obtained by performing image recognition on a sample noisy image corresponding to a sample object image based on the target image recognition model, and a model obtained by back-propagation training of the initial privacy model, and the sample noisy image is obtained by performing noise perturbation on a sample frequency domain image corresponding to the sample object image by the initial privacy model.

[0016] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method as described above.

[0017] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored in the storage medium, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the image processing method as described above.

[0018] On the other hand, a terminal is provided, which includes a processor and a memory, and the device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method as described above.

[0019] On the other hand, a server is provided, the server includes a processor and a memory, the device includes a processor and a memory, the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method as described above.

[0020] On the other hand, a computer program product or a computer program is provided, which comprises computer instructions, and when the computer instructions are executed by a processor, the image processing method as described above is implemented.

[0021] The image processing method, apparatus, device, storage medium, terminal, server and computer program product provided by this application have the following technical effects:

[0022] The technical solution of the present application obtains a target object image and performs a frequency domain transformation on the target object image to obtain a target frequency domain image corresponding to the target object image, and then calls a preset privacy model to perform noise perturbation on the target frequency domain image based on the target privacy allocation parameters corresponding to each pixel in the target frequency domain image to obtain a target noisy image corresponding to the target frequency domain image, wherein the target privacy allocation parameters indicate the degree of noise addition to the corresponding pixel in the target frequency domain image; and then calls a target image recognition model to perform image recognition on the target noisy image to obtain an image recognition result. The preset privacy model is a model loss obtained by performing image recognition on a sample noisy image corresponding to a sample object image based on the target image recognition model, and a model obtained by back-propagation training of the initial privacy model, and the sample noisy image is obtained by performing noise perturbation on the sample frequency domain image corresponding to the sample object image by the initial privacy model. In this way, the image can be transformed into the frequency domain, and perturbed in the frequency domain based on a learnable privacy model. Adaptive noise can be added to each pixel to optimize image recognition precision and accuracy while effectively protecting image privacy. In addition, the image encryption process has low requirements for data calculations, significantly reducing resource usage and computational complexity, and effectively improving image encryption processing efficiency and recognition speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application;

[0025] Figure 2 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0026] Figure 3 It is a schematic diagram of the image processing principle for a target object image provided by an embodiment of the present application;

[0027] Figure 4is a flowchart of another image processing method provided in an embodiment of the present application;

[0028] Figure 5 is a flowchart of another image processing method provided in an embodiment of the present application;

[0029] Figure 6 is a flowchart of an image processing method provided by an embodiment of the present application;

[0030] Figure 7 It is a flowchart of a privacy model training method provided in an embodiment of the present application;

[0031] Figure 8 It is a flowchart of another privacy model training method provided in an embodiment of the present application;

[0032] Fig. 9 is a schematic diagram of a framework of an image processing device based on differential privacy provided in an embodiment of the present application;

[0033] Fig.10 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0034] Fig.11 It is a hardware structure block diagram of an electronic device of an image processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0036] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.

[0037] Among them, machine learning (ML) is a multi-disciplinary interdisciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0038] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing so that computer processing becomes an image that is more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0039] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, Internet of Vehicles, smart transportation, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role. This application mainly involves technologies such as image processing, semantic understanding, machine learning and deep learning, which are specifically explained through the following embodiments.

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0041] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server comprising a series of steps or submodules is not necessarily limited to those steps or submodules clearly listed, but may include other steps or submodules that are not clearly listed or inherent to these processes, methods, products or devices.

[0042] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0043] Discrete Cosine Transform (DCT): A transform related to the Fourier transform, similar to the discrete Fourier transform, but using only real numbers. The discrete cosine transform uses the sum of cosine functions oscillating at different frequencies to express a finite sequence of data points. It is a widely used transform technique in signal processing and data compression. Discrete cosine transform compression, also known as block compression, compresses data as a collection of discrete DCT blocks. DCT has the property of "energy compression", that is, most of the energy will be concentrated in a small area.

[0044] Block Discrete Cosine Transform (BDCT): In actual image processing, the image is divided into blocks during the DCT transformation process, and then the image is subjected to DCT transformation and inverse transformation in each block, and then the blocks are merged.

[0045] Upsampling: It is to convert the image from a lower size [H, W, C] to a larger size [sH, sW, C], where s is the upsampling multiple. The transformation process from a small image to a large image is also called super-resolution reconstruction of the image.

[0046] Differential privacy (DP): is a privacy model and a method in cryptography that aims to maximize the accuracy of data queries when querying from a statistical database while minimizing the chance of identifying its records, that is, providing maximum privacy by minimizing the possibility of personal records being identified. It introduces the concept of privacy budget (ε) to represent the degree of privacy guaranteed by the randomization algorithm for a specific database. Any two databases that differ by only one row are adjacent.

[0047] See also Figure 1 , Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application, such as Figure 1 As shown, the application environment may include at least a server 01 and a terminal 02. In actual applications, the server 01 and the terminal 02 may be directly or indirectly connected via wired or wireless communication to achieve interaction between the terminal 02 and the server 01, and this application does not limit this.

[0048] In the embodiment of the present application, server 01 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Specifically, the server may include physical devices, which may specifically include network communication submodules, processors, and memories, etc., and may also include software running in physical devices, which may specifically include applications, etc.

[0049] Specifically, cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to achieve data computing, storage, processing and sharing. It distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called "cloud". Among them, artificial intelligence cloud service is generally also called AIaaS (AI as a Service, Chinese for "AI as a Service"). This is a mainstream service mode of artificial intelligence platform. Specifically, AIaaS platform will split several common AI services and provide independent or packaged services in the cloud. This service model is similar to opening an AI theme mall: all developers can access one or more artificial intelligence services provided by the platform through API interfaces. Some senior developers can also use the AI ​​framework and AI infrastructure provided by the platform to deploy and operate their own cloud artificial intelligence services.

[0050] In the embodiment of the present application, the terminal 02 may include a smart phone, a desktop computer, a tablet computer, a laptop computer, a digital assistant, an augmented reality (AR) / virtual reality (VR) device, an intelligent voice interaction device, a smart home appliance, a smart wearable device, a vehicle-mounted terminal device, an aircraft and other types of physical devices, and may also include software running in the physical device, such as an application program. Specifically, the terminal 02 is used to obtain the target object image, and perform frequency domain conversion and noise disturbance processing on the target object image to obtain the target noisy image, and send it to the server to trigger the server 01 to perform the corresponding image recognition operation. The server 01 can be used to receive the target noisy image sent by the terminal, and use the pre-deployed image recognition model to perform image recognition on it. During the image recognition process, feature comparison can be performed based on the pre-stored feature library to obtain the image recognition result. In some cases, the image recognition model can also be deployed on the terminal, and the terminal directly calls the image recognition model to perform image recognition on the target noisy image to obtain the corresponding image recognition result. Specifically, the server 01 can provide a training service for the initial privacy model to obtain a preset privacy model.

[0051] Please refer to Figure 3 , Figure 3The schematic diagram of the image processing principle for the target object image provided by the embodiment of the present application is shown. The target object image is input into the frequency domain conversion module for frequency domain conversion to obtain the target frequency domain image, and the target frequency domain image is subjected to noise disturbance processing using the preset privacy model to obtain the target noisy image, which is used as the input of the image recognition model. The target noisy image is subjected to feature extraction through the backbone network of the image recognition model to obtain image features, and image recognition is performed on it to obtain the corresponding image recognition result and model loss. The model loss can be used for back propagation training of the initial privacy model. Figure 3 As shown, the image recognition process includes determining the predicted probability (P) for each label (GT) based on the image features. n ), the probability of P2 is the largest, then the image feature belongs to the label corresponding to P2.

[0052] Furthermore, it is understandable that Figure 1 What is shown is only an application environment of an image processing method, and the application environment may include more or fewer nodes, and this application does not impose any limitation thereto.

[0053] The application environment involved in the embodiment of the present application, or the server 01 and the terminal 02 in the application environment, can be a distributed system formed by connecting a client and multiple nodes (any form of computing devices in the access network, such as a server, a user terminal) through network communication. The distributed system can be a blockchain system, which can provide the above-mentioned image processing services, model training services, data storage services, etc.

[0054] The following is an introduction to an image processing method of the present application based on the above application environment. The present application embodiment can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc. Please refer to Figure 2 , Figure 2 It is a flowchart of an image processing method provided by an embodiment of the present application. This specification provides method operation steps such as the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in the order shown in the embodiment or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment). Specifically, Figure 2 As shown, the method may include the following steps S201-S207.

[0055] S201: Acquire a target object image.

[0056] Specifically, the target object may be a living or non-living object, including but not limited to a person, an animal, an object or a place, etc., and accordingly, the target object image may be an image of the target object, such as a facial image, etc. In some embodiments, the target object image is obtained by preprocessing the image to be processed including the target object, and the preprocessing includes but is not limited to image size conversion, filtering, and color space conversion, etc. It should be noted that when the target object is a person, the facial image or other relevant privacy data is obtained with the authorization of the object. Accordingly, when the target object image is a facial image, before acquiring the target object image, an authorization permission request is sent and displayed, and after receiving the authorization permission operation of the object in response to the authorization permission request, the facial image is acquired.

[0057] S203: Perform frequency domain transformation on the target object image to obtain a target frequency domain image corresponding to the target object image.

[0058] The frequency domain transformation refers to converting the target object image from the spatial domain to the frequency domain, for example, converting the target object image from the RGB or YCbCr color space domain to the frequency domain. Figure 4 , S203 may include the following steps.

[0059] S2031: Performing frequency domain transformation on the target object image based on a preset frequency domain transformation method to obtain an initial frequency domain image corresponding to the target object image.

[0060] S2032: Remove target low-frequency information that meets a preset information amount condition from the initial frequency domain image to obtain a target frequency domain image.

[0061] Among them, the preset frequency domain transformation method may include but is not limited to Fourier transform, discrete image orthogonal transform, discrete cosine transform and Walsh-Hadamard transform. Specifically, each pixel point in the target object image is transformed from the spatial domain to the frequency domain using the preset frequency domain transformation method to generate a frequency domain representation of the target object image, and then obtain an initial frequency domain image. In the initial frequency domain image, the low-frequency information and high-frequency information in the target object image are distinguished based on the preset information amount condition to determine the target low-frequency information therein. The preset information amount condition can be determined based on the frequency domain transformation method, the accuracy of the image recognition model and the privacy protection requirements of the target object image. Exemplarily, when the preset frequency domain transformation method is discrete cosine transform, the target low-frequency information that meets the preset information amount condition can be an element whose element value in the frequency domain representation is greater than or equal to a preset element threshold, or it can also be an element corresponding to a DC component in the frequency domain representation. Correspondingly, removing the target low-frequency information that meets the preset information amount condition from the initial frequency domain image is specifically: removing the pixel points corresponding to the elements that meet the preset information amount condition from the initial frequency domain image, that is, deleting the pixel points corresponding to the elements whose element values ​​are greater than or equal to the preset element threshold, or deleting the pixel points corresponding to the DC component element. In this way, sensitive information is removed to achieve effective protection of image privacy, and differential privacy processing is performed in the frequency domain, which significantly reduces resource usage, significantly reduces performance degradation and improves anti-recovery capabilities while ensuring recognition accuracy.

[0062] In some embodiments, the target object image is transformed in the frequency domain based on the block discrete cosine transform (BDCT) mechanism. Figure 5 , S2031 may include the following steps.

[0063] S20311: Divide the target object image into a preset number of object image blocks based on preset block parameters.

[0064] S20312: Perform discrete cosine transformation on each object image block to obtain a frequency domain block corresponding to each object image block.

[0065] S20313: Based on the position of each object image block in the target object image, reorganize each frequency domain block to obtain an initial frequency domain image.

[0066] Among them, the preset block parameter represents the size of the image block, that is, the segmentation window size. The target object image is obtained by preprocessing the image to be processed including the target object. In a specific embodiment, the preprocessing process includes: converting the size of the image to be processed into a preset size, and converting it from RGB format to YCbCr format, and then adjusting the numerical range of the image pixel points to [-128, 127] to obtain the target object image. Specifically, the preset size is an integer multiple of the segmentation window size, and the preset number is the ratio of the target image size to the segmentation window size.

[0067] Specifically, the target object image is segmented based on a segmentation window set to a preset segmentation parameter to obtain a plurality of object image blocks. For each object image block, each pixel point therein is transformed from the spatial domain to the frequency domain based on a discrete cosine transform method, such as performing a normalized two-dimensional discrete cosine transform on each object image block to generate a frequency domain block of each object image block, wherein low-frequency information in the frequency domain block gathers toward the upper left corner, the first element [0,0] in the upper left corner is a DC component, and high-frequency information is concentrated toward the lower right corner.

[0068] Furthermore, the pixels at the same element position in each frequency domain block are concentrated in the same channel to obtain image channels with the same number of pixels in the frequency domain block. If the frequency domain block includes n pixels, n image channels are obtained, and each image channel includes a Y channel, a Cb channel, and a Cr channel. Then, the frequency domain blocks are sorted and reorganized according to the original position of each object image block in the target object image to obtain an initial frequency domain image. The first channel in the upper left corner of the initial frequency domain image is the DC component (DC) channel.

[0069] Furthermore, the pixel points corresponding to the DC component of each frequency domain block are removed from the initial frequency domain image, that is, the DC channel in the initial frequency domain image is deleted to remove the target low-frequency information and obtain the target frequency domain image. The DC component defines the basic tone of the entire object image block. The target low-frequency information is the main information in the target object image, such as the main information representing facial features in a facial image. It is understandable that in some cases, two or more channels including the DC component in the upper left corner of each frequency domain block can also be removed to remove more main information and further avoid privacy leakage.

[0070] In one possible implementation, before S20311, the target object image is also upsampled based on a preset sampling multiple that matches the preset blocking parameters to obtain an enlarged object image corresponding to the target object image; accordingly, S20311 specifically includes: dividing the enlarged object image into a preset number of object image blocks based on the preset blocking parameters.

[0071] The matching of the preset sampling multiple and the preset block parameter means that the enlarged object image is segmented based on the preset block parameter, and each object image block obtained corresponds to a pixel point in the target object image. For example, the preset block parameter is 8*8, and the size of the image segmentation window is 8 pixels*8 pixels, then the preset sampling multiple is 8 times. In this way, each block corresponds to a pixel point of the original image, which can avoid image information loss and ensure recognition accuracy.

[0072] In a specific embodiment, a BDCT with an image segmentation window size of 8*8 is used to perform 8-fold upsampling on a target object image of image dimension [W, H, 3], where W is the image width, H is the image height, and 3 represents the number of color channels; then, based on the 8*8 image segmentation window, the enlarged object image is segmented into [W, H, 3] 8*8 object image blocks, and a normalized two-dimensional DCT is applied to each object image block to convert each object image region into an 8*8 frequency domain block, wherein the element value of the element at the upper left corner of each frequency domain block is the largest, which is the DC component of the frequency domain block; elements at the same position in each frequency domain block are concentrated in the same channel, such as concentrating the elements corresponding to the DC components of each frequency domain block in one channel to form a DC channel, and the frequency domain blocks are arranged and reorganized according to the original positions of the object image blocks in the target object image, so as to convert the target object image of [H, W, 3] into an initial frequency domain image of [H, W, 8, 8, 3], which includes a total of (8*8)*3=192 channels.

[0073] Furthermore, in the reconstructed initial frequency domain image, the target low-frequency information, i.e., the main information, is concentrated in the DC channel. The DC channel is deleted, i.e., the three DC channels of YCbCr are deleted, and (8*8-1)*3=189 channels remain, and the target frequency domain image is obtained. The image dimension of the target frequency domain image is [H, W, 189].

[0074] S205: Calling a preset privacy model, performing noise perturbation on the target frequency domain image based on the target privacy allocation parameters corresponding to each pixel in the target frequency domain image, and obtaining a target noisy image corresponding to the target frequency domain image. The target privacy allocation parameter indicates the degree of noise addition to the corresponding pixel in the target frequency domain image. The larger the target privacy allocation parameter, the larger the corresponding privacy budget and the higher the degree of noise addition, that is, the greater the noise added to the pixel.

[0075] Specifically, the preset privacy model is a model loss obtained by performing image recognition on a sample noisy image corresponding to a sample object image based on a target image recognition model. The model is obtained by back-propagation training of the initial privacy model. The sample noisy image is obtained by performing noise perturbation on a sample frequency domain image corresponding to the sample object image by the initial privacy model. During the back-propagation training process, privacy allocation parameters are learned to obtain target privacy allocation parameters.

[0076] Specifically, the noise of each pixel is determined based on the target privacy allocation parameter of the pixel, and the noise is added to the target frequency domain image pixel by pixel to obtain the target noisy image.

[0077] S207: Calling the target image recognition model to perform image recognition on the target noisy image to obtain an image recognition result.

[0078] In the embodiment of the present application, an image feature library is pre-stored, and the image feature library stores pre-stored object features for feature comparison. The pre-stored object features are features obtained by extracting object features from historical noisy images using a target image recognition model. The historical noisy images are images obtained by each terminal after frequency domain conversion and noise processing of historical object images in the manner of steps S203 and S205. The target image recognition model is called to extract object features from the target object in the target noisy image to obtain corresponding target object features, and the target object features are compared with the pre-stored object features in the image feature library to obtain feature similarity with the pre-stored object features. If the feature similarity is greater than a preset similarity threshold, it is determined that the target object features match the pre-stored object features, and the corresponding object recognition results, such as object attributes, are obtained. Among them, the image recognition model can be constructed and trained based on the actual needs of the application scenario. For example, in the scenario of facial image recognition, the target image recognition model can be a facial image recognition model. Specifically, the number of input layer channels of the target image recognition model is consistent with the number of channels of the target frequency domain image, such as 189. In some embodiments, S201-S207 may be executed on the terminal side, or S201-S207 may be executed on the server side; in other embodiments, S201-S205 may be executed on the terminal side, and S207 may be executed on the server side.

[0079] In a specific embodiment, please refer to Figure 3 and Figure 6 The terminal is provided with an image sensor, and the aforementioned frequency domain conversion module and the preset privacy model are deployed in the terminal to encrypt the target object image. Taking the target object image as the object face image as an example, the specific process of image processing includes the following steps.

[0080] S301: Acquire a target facial image through an image sensor.

[0081] For example, the terminal obtains the object facial image of object A through an RGB sensor, and preprocesses it to obtain a target facial image.

[0082] S303: Encrypt the target facial image based on the frequency domain conversion module and the preset privacy model to obtain a facial noisy image.

[0083] Specifically, the target facial image is converted into a frequency domain image through a frequency domain conversion module, the DC component channel is removed to obtain a target frequency domain image, and the target frequency domain image is encrypted based on a preset privacy model to obtain a facial noisy image.

[0084] S305: Obtain facial features of the facial noisy image.

[0085] S307: Obtain feature similarity between the facial features and pre-stored facial features.

[0086] S309: If the feature similarity is greater than or equal to a preset similarity threshold, it is determined that the target object and the object corresponding to the pre-stored facial features are the same person.

[0087] S311: If the feature similarity is less than a preset similarity threshold, it is determined that the target object is different from the object corresponding to the pre-stored facial feature.

[0088] Specifically, the facial noisy image is feature extracted through the backbone network of the image recognition model to extract the facial features e of the facial noisy image, and compared with the facial features of the object B pre-stored in the preset feature library to obtain the feature similarity sigma; the obtained feature similarity sigma is compared with the preset similarity threshold th, if it is higher than th, it is considered that object A is the same as object B, otherwise it is different, and the facial features pre-stored in the preset feature library are traversed and compared until the obtained feature similarity sigma is higher than th; if no matching facial features are found after the traversal, it indicates that the facial features of object A do not exist. Among them, the image recognition model and the preset feature library can be deployed on the terminal, or can also be deployed on the server side. In this embodiment, if Figure 3 As shown, N is the total number of objects corresponding to the preset feature library, P n The predicted probability that the recognized facial feature a belongs to object n is represented, and GT is the corresponding label. The label is 1, indicating that the target object and object n are the same person, and the label is 0, indicating that they are different.

[0089] In the related art, homomorphic encryption and other methods are usually used to encrypt images, so as to calculate the encrypted image without decryption to obtain the encrypted calculation result, or to use the multi-party computing technology that introduces a multi-party protocol, so that the first party provides the image and the other party accesses the image database, so that the first party cannot learn more basic parameters of the database from the execution of the protocol, and the second party cannot know the input image or the result of the recognition process. However, the above homomorphic encryption method requires encryption operations to be performed at the terminal, the algorithm calculation amount is too large, and a large amount of computing resources are consumed, which seriously increases the image recognition delay and affects the real-time prediction performance. The multi-party protocol method requires the introduction of a third party, increases the number of trusted objects, and further increases the risk of privacy leakage. Based on the above technical solution, the image can be transformed into the frequency domain, and the perturbation can be performed in the frequency domain based on the learnable privacy model. Adaptive noise can be added to each pixel to optimize the image recognition precision and accuracy, while effectively protecting the image privacy. In addition, the image encryption process has low requirements for data calculation, significantly reduces resource occupation requirements and calculation amount, effectively improves the image encryption processing efficiency and recognition speed, and avoids affecting the real-time prediction performance.

[0090] Based on some or all of the above implementations, in some embodiments, the preset privacy model is a privacy model built based on a differential privacy mechanism. Differential privacy is a system that publicly shares data set information by describing group patterns in the data set while concealing individual information in the data set. The mechanism of differential privacy is that if the effect of any single replacement in the database is small enough, the query result cannot be used to infer the situation of any single individual, thereby protecting privacy. Existing image recognition protection methods based on differential privacy mainly add noise of the same parameters to the input image according to a given privacy budget. This random perturbation seriously affects the accuracy of image recognition. The technical solution of the present application, after converting the target object image to the frequency domain, constructs a privacy model with learnable privacy allocation parameters based on the differential privacy mechanism to perform noise perturbation on the target object image. Accordingly, the method may also include an initial privacy model construction step, specifically including: 1) obtaining a preset frequency domain image library, and performing statistics on the pixel values ​​of the pixels at the same position in each frequency domain image in the preset frequency domain image library to determine the sensitivity corresponding to each pixel position; 2) determining the image distance between each pair of frequency domain images in the preset frequency domain image library according to the sensitivity of each pixel position; 3) constructing an initial privacy model based on the differential privacy mechanism according to the sensitivity of each pixel position, the image distance and the preset noise distribution method.

[0091] In a specific embodiment, the preset frequency domain image library can be obtained based on the MS1M data set, where the dimension of each frequency domain image is [H, W, C], where C represents the number of channels, for example, 189. Based on the sensitivity obtained by statistics Characterizes the sensitivity of the element (pixel) at position [i,j,k] in the frequency domain image, where i corresponds to H, j corresponds to W, and k corresponds to C. is the minimum pixel value of the element at position [i,j,k] in the frequency domain image obtained by statistics, is the maximum value of the pixel of the element at position [i, j, k] in the frequency domain image obtained by statistics. Correspondingly, the distance between the elements at the same position in the frequency domain image is as follows, where d i,j,k (x1,x2) represents the distance between elements a1 and a2 at position [i,j,k] for two given frequency domain images X1 and X2, E i,j,k A set of elements representing all possible frequency domain images at the corresponding position [i, j, k], x1 represents the pixel value of element a1, and x2 represents the pixel value of element a2.

[0092]

[0093] Furthermore, the image distance between two frequency domain images is expressed as follows, where d(X1,X2) represents the image distance between two frequency domain images X1 and X2, R represents a set of real numbers of dimension H*W*C, and R H,W,C Characterizes the preset frequency domain image library. The image distance value is the maximum element distance between elements at the same position in the two images.

[0094]

[0095] Correspondingly, the privacy model based on the differential privacy mechanism is expressed as follows: Satisfying ∈-DP, for any frequency domain images X1 and X2, it is impossible to determine whether the noisy output E (noised image) is generated based on X1 or from X2, that is, it is impossible to determine which target object it is. Represent the preset privacy model, E is the noisy image obtained after noise perturbation, ∈ is the privacy budget, the smaller ∈ is, the more secure it is, and the correspondingly less effective information is obtained.

[0096]

[0097] Specifically, by adding noise disturbance, that is, adding a noise vector Y∈R H,W,C , any frequency domain image X∈R H,W,C can be protected by ∈-DP, where each element of Y i,j,k is an independent random variable. The scaling parameter corresponding to the preset noise distribution can be determined based on the sensitivity of each pixel position and the preset noise distribution mode, and the scaling parameter indicates the variance of the noise distribution. When the preset noise distribution follows the Laplace distribution, the scaling parameter Among them, Y represents the noise vector for the frequency domain image, σ i,j,k Characterizes the scaling parameter of the Laplace sampling distribution corresponding to the pixel point at position [i,j,k]. The scaling parameter is proportional to the variance of the Laplace distribution. The larger its value, the larger the variance, and the larger the corresponding noise added, ∈ i,j,k It is represented as the privacy budget allocated to the pixel at position [i, j, k], ∈ is the preset total privacy budget, and the smaller ∈ is, the stronger the privacy protection performance is.

[0098] Based on the above privacy model construction mechanism, before S205, a privacy model training step is also included. The following is a privacy model training method provided by this application, which is described in conjunction with the figure. Please refer to Figure 7 , comprising the following steps.

[0099] S401: Acquire a sample training set, where the sample training set includes sample object images and corresponding object labels.

[0100] Specifically, the sample training set includes a large number of sample object images, which are similar to the target object images described above and are not described in detail here. The object label may be an object attribute, an object position or an object category, etc., and may be set based on actual needs.

[0101] S403: Perform frequency domain transformation on the sample object image to obtain a sample frequency domain image corresponding to the target sample image.

[0102] Specifically, step S403 is similar to the aforementioned S203 and will not be described in detail here.

[0103] S405: calling the initial privacy model, and performing noise perturbation on the sample frequency domain image based on the initial privacy allocation parameters corresponding to each pixel point in the sample frequency domain image to obtain a sample noisy image.

[0104] Specifically, in the initial privacy model, for each pixel position in the frequency domain image of the preset dimension [H, W, C], the corresponding initial privacy allocation parameters are initialized and set respectively. The normalization layer (softmax layer) of the initial privacy model is used to normalize each initial privacy allocation parameter to obtain the allocation rate parameter corresponding to each initial privacy allocation parameter. The sum of all allocation rate parameters is 1, and the corresponding allocation rate parameter = ∈ i,j,k / ∈. Then, the allocation rate parameter corresponding to each pixel is multiplied by the preset total privacy budget to obtain the initial privacy budget of each pixel, and Calculate the scaling parameters of the Laplace noise corresponding to each pixel, based on the preset expected μ and the calculated σ i,j,k, generate the Laplace noise distribution of each pixel, and then randomly sample the Laplace noise distribution to obtain the noise corresponding to each pixel, generate a noise vector for the sample frequency domain image, and add the sampled noise vector to the sample frequency domain image pixel by pixel to obtain the sample noisy image.

[0105] S407: Calling the image recognition model to perform image recognition on the sample noisy image to obtain a sample image recognition result.

[0106] Specifically, step S407 is similar to the aforementioned S207 and will not be described in detail here.

[0107] S409: Calculate the loss based on the sample image recognition result and the corresponding object label to obtain the model loss.

[0108] S411: Perform back-propagation training on the initial privacy model based on the model loss to obtain a preset privacy model, where model parameters of the preset privacy model include target privacy allocation parameters corresponding to each pixel point.

[0109] For some examples, please refer to Figure 8 , S411 includes the following steps.

[0110] S4111: Update model parameters including initial privacy allocation parameters according to the model loss.

[0111] S4112: Update the model loss using the updated initial privacy model and image recognition model.

[0112] S4113: When the updated model loss satisfies the preset conditions, the updated initial privacy model is used as the preset privacy model, and the updated initial privacy allocation parameters are used as the target privacy allocation parameters.

[0113] Specifically, the loss is calculated based on the loss function of the image recognition model to obtain the model loss, and the model loss is reversely regressed to adjust the model parameters such as the initial privacy allocation parameters to obtain the updated initial privacy model and the updated privacy allocation parameters. The updated initial privacy model is used to repeat the above S405-S409 based on the updated privacy allocation parameters to obtain the updated model loss, and it is determined whether it meets the preset conditions. If it does, the current updated initial privacy model is the preset privacy model, and the updated initial privacy allocation parameters are fixed as the target privacy allocation parameters. If not, steps S4111 and S4112 are repeated until the obtained model loss meets the preset conditions. Among them, satisfying the preset conditions can be that the model loss is less than or equal to the preset loss, or the number of epochs for which the current updated model loss is obtained reaches the preset number of epochs.

[0114] Correspondingly, based on the above differential privacy mechanism, the aforementioned S205 may include the following steps.

[0115] S2051: Mapping the target privacy allocation parameters corresponding to each pixel point based on the preset total privacy budget to obtain the target privacy budget corresponding to each pixel point.

[0116] S2052: Determine the noise distribution corresponding to each pixel point based on the target privacy budget, and perform noise sampling on each noise distribution to obtain the noise information corresponding to each pixel point.

[0117] S2053: Based on each noise information, corresponding pixel points in the target frequency domain image are subjected to noise addition processing to obtain a target noisy image.

[0118] It can be understood that the method of obtaining the target noisy image is similar to the method of obtaining the sample noisy image mentioned above. First, the target privacy allocation parameter corresponding to each pixel is normalized by the normalization layer to obtain the allocation rate parameter of each pixel. The sum of all allocation rate parameters is 1, and the allocation rate parameter = ∈ i,j,k / ∈; Then, the allocation rate parameter corresponding to each pixel is multiplied by the preset total privacy budget to obtain the target privacy budget for each pixel. Using the noise sampling layer, the scaling parameter of the noise distribution of each pixel is calculated according to the target privacy budget to determine the size of the variance, and the noise distribution of each pixel is determined based on the preset expected value and the calculated scaling parameter. Each noise distribution is randomly sampled to obtain the noise corresponding to each pixel, that is, the overall noise vector corresponding to the target frequency domain image is obtained. The image generation layer is used to add noise to the target frequency domain image pixel by pixel, that is, to add the corresponding noise to each pixel to obtain the target noisy image. When the noise distribution follows the Laplace noise distribution, the scaling parameter In some embodiments, the preset privacy model includes a normalization layer, a noise sampling layer, and an image generation layer, wherein the normalization layer executes the above S2051, the noise sampling layer executes the above S2052, and the image generation layer executes the above S2053.

[0119] Please refer to Fig. 9 , Fig. 9 The schematic diagram of the framework of the image processing device based on differential privacy is shown. In the figure, the face image of the object is taken as an example. The RGB image X collected by the terminal RGB After YCbCr conversion and other preprocessing, it is input into the frequency domain conversion module, and image segmentation, discrete cosine transform and reorganization are performed based on block discrete cosine transform to obtain the initial frequency domain image, and after removing DC, the target frequency domain image X is obtained. Freq ; X FreqInput the preset privacy model, normalize and map the learnable privacy budget allocation parameters, obtain the privacy budget of each pixel, determine the noise distribution of each pixel in combination with the sensitivity, and then obtain the noise information of each pixel; perform noise processing on the target frequency domain image based on the noise information to generate the facial noisy image X Mask ; Input it into the image recognition model to obtain the image recognition result and model loss L fr .

[0120] The technical solution of this application transforms the image into the frequency domain, and performs differential perturbations on the image in the frequency domain based on the differential privacy mechanism and the learnable privacy budget scheme. It has low requirements for data calculation, high object recognition accuracy, and can ensure excellent irreversibility under black box attacks, effectively protecting image recognition privacy, and has the advantages of high recognition accuracy, fast inference recognition speed, and simple implementation, ensuring the recognition performance of the model. This method can be applied to various image recognition scenarios involving privacy protection, such as face-swiping payment scenarios.

[0121] The present application also provides an image processing device 800, such as Fig.10 As shown, Fig.10 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application is shown, and the device may include the following modules.

[0122] Image acquisition module 10: used to acquire the target object image.

[0123] The frequency domain transformation module 20 is used to perform frequency domain transformation on the target object image to obtain a target frequency domain image corresponding to the target object image.

[0124] Noise perturbation module 30: used to call the preset privacy model, and perform noise perturbation on the target frequency domain image based on the target privacy allocation parameters corresponding to each pixel point in the target frequency domain image to obtain a target noisy image corresponding to the target frequency domain image, wherein the target privacy allocation parameters indicate the degree of noise addition to the corresponding pixel points in the target frequency domain image.

[0125] Image recognition module 40: used to call the target image recognition model to perform image recognition on the target noisy image to obtain an image recognition result.

[0126] Among them, the preset privacy model is a model loss obtained by performing image recognition on a sample noisy image corresponding to a sample object image based on a target image recognition model, and is a model obtained by back-propagation training of the initial privacy model. The sample noisy image is obtained by performing noise perturbation on a sample frequency domain image corresponding to the sample object image by the initial privacy model.

[0127] In some embodiments, the noise addition disturbance module 30 may include the following submodules.

[0128] Allocation parameter mapping submodule: used to map the target privacy allocation parameters corresponding to each pixel point based on the preset total privacy budget, and obtain the target privacy budget corresponding to each pixel point.

[0129] Noise sampling submodule: It is used to determine the noise distribution corresponding to each pixel based on the target privacy budget, and perform noise sampling on each noise distribution to obtain the noise information corresponding to each pixel.

[0130] Noise processing submodule: used to perform noise processing on the corresponding pixel points in the target frequency domain image based on each noise information to obtain the target noisy image.

[0131] In some embodiments, the device may further include the following modules.

[0132] Sample training set acquisition module: used to obtain a sample training set before calling the preset privacy model and performing noise perturbation on the target frequency domain image based on the target privacy allocation parameters corresponding to each pixel in the target frequency domain image. The sample training set includes a sample object image and a corresponding object label.

[0133] Sample conversion module: used to perform frequency domain transformation on the sample object image to obtain the sample frequency domain image corresponding to the target sample image.

[0134] Sample noise perturbation module: used to call the initial privacy model, based on the initial privacy allocation parameters corresponding to each pixel in the sample frequency domain image, to perform noise perturbation on the sample frequency domain image to obtain a sample noisy image.

[0135] Sample image recognition module: used to call the image recognition model to perform image recognition on the sample noisy image to obtain the sample image recognition result.

[0136] Loss calculation module: used to calculate the loss based on the sample image recognition results and the corresponding object labels to obtain the model loss.

[0137] Back propagation training module: used to perform back propagation training on the initial privacy model based on the model loss to obtain a preset privacy model. The model parameters of the preset privacy model include the target privacy allocation parameters corresponding to each pixel point.

[0138] In some embodiments, the back-propagation training module may include the following sub-modules.

[0139] Model parameter update submodule: used to update the model parameters including the initial privacy allocation parameters according to the model loss.

[0140] Model loss update submodule: used to update the model loss using the updated initial privacy model and image recognition model.

[0141] Model determination submodule: used to use the updated initial privacy model as the preset privacy model and the updated initial privacy allocation parameters as the target privacy allocation parameters when the updated model loss meets the preset conditions.

[0142] In some embodiments, the frequency domain transform module 20 may include the following submodules.

[0143] Frequency domain transformation submodule: used to perform frequency domain transformation on the target object image based on a preset frequency domain transformation method to obtain an initial frequency domain image corresponding to the target object image.

[0144] Low-frequency information removal submodule: used to remove target low-frequency information that meets the preset information amount condition from the initial frequency domain image to obtain the target frequency domain image.

[0145] In some embodiments, the frequency domain transformation submodule may include the following units.

[0146] Image segmentation unit: used to segment the target object image into a preset number of object image blocks based on preset segmentation parameters.

[0147] Discrete cosine transform unit: used to perform discrete cosine transform on each object image block respectively to obtain the frequency domain block corresponding to each object image block.

[0148] Image reorganization unit: used to reorganize each frequency domain block based on the position of each object image block in the target object image to obtain an initial frequency domain image.

[0149] In some embodiments, the low-frequency information removal submodule includes a module specifically used for: removing pixel points corresponding to the DC component of each frequency domain block from the initial frequency domain image to remove the target low-frequency information and obtain the target frequency domain image.

[0150] In some embodiments, the frequency domain transformation submodule may further include an upsampling unit: used to upsample the target object image based on a preset sampling multiple matching the preset blocking parameters before dividing the target object image into a preset number of object image blocks based on the preset blocking parameters, so as to obtain an enlarged object image corresponding to the target object image.

[0151] Accordingly, the image segmentation unit may be specifically configured to segment the enlarged object image into a preset number of object image blocks based on preset segmentation parameters.

[0152] It should be noted that the above device embodiment and method embodiment are based on the same implementation method.

[0153] An embodiment of the present application provides an image processing device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method provided in the above method embodiment.

[0154] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0155] The method embodiments provided in the embodiments of the present application can be executed in electronic devices such as mobile terminals, computer terminals, servers or similar computing devices. Fig.11 is a hardware structure block diagram of an electronic device of an image processing method provided in an embodiment of the present application. Fig.11 As shown, the electronic device 900 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 910 (the processor 910 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 930 for storing data, and one or more storage media 920 (such as one or more mass storage devices) for storing application programs 923 or data 922. Among them, the memory 930 and the storage medium 920 can be short-term storage or permanent storage. The program stored in the storage medium 920 may include one or more modules, each of which may include a series of instruction operations in the electronic device. Furthermore, the central processing unit 910 can be configured to communicate with the storage medium 920 and execute a series of instruction operations in the storage medium 920 on the electronic device 900. The electronic device 900 may also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input and output interfaces 940, and / or, one or more operating systems 921, such as Windows Server TM , Mac OS X TM , Unix TM , LinuxTM, FreeBSDTM, etc.

[0156] The input / output interface 940 may be used to receive or send data via a network. The specific example of the network may include a wireless network provided by a communication provider of the electronic device 900. In one example, the input / output interface 940 includes a network adapter (Network Interface Controller, NIC), which may be connected to other network devices via a base station so as to communicate with the Internet. In one example, the input / output interface 940 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0157] It can be understood by those skilled in the art that Fig.11 The structure shown is only for illustration and does not limit the structure of the above electronic device. Fig.11 More or fewer components as shown, or with Fig.11 Different configurations shown.

[0158] An embodiment of the present application also provides a computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to an image processing method in a method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method provided by the above method embodiment.

[0159] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0160] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0161] It can be seen from the embodiments of the image processing method, device, equipment, server, terminal, storage medium and program product provided by the present application that the technical solution of the present application obtains the target object image and performs frequency domain transformation on the target object image to obtain the target frequency domain image corresponding to the target object image, and then calls the preset privacy model to perform noise perturbation on the target frequency domain image based on the target privacy allocation parameters corresponding to each pixel point in the target frequency domain image to obtain the target noisy image corresponding to the target frequency domain image, and the target privacy allocation parameters indicate the degree of noise addition to the corresponding pixel point in the target frequency domain image; then calls the target image recognition model to perform image recognition on the target noisy image to obtain the image recognition result. Among them, the preset privacy model is the model loss obtained by performing image recognition on the sample noisy image corresponding to the sample object image based on the target image recognition model, and the model obtained by back-propagation training of the initial privacy model, and the sample noisy image is the sample frequency domain image corresponding to the sample object image obtained by the initial privacy model. The noise perturbation. In this way, the image can be transformed into the frequency domain, and perturbed in the frequency domain based on a learnable privacy model. Adaptive noise can be added to each pixel to optimize image recognition precision and accuracy while effectively protecting image privacy. In addition, the image encryption process has low requirements for data calculations, significantly reducing resource usage and computational complexity, and effectively improving image encryption processing efficiency and recognition speed.

[0162] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of the present application are described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0163] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0164] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0165] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a target object image; Performing frequency domain transformation on the target object image to obtain a target frequency domain image corresponding to the target object image; Calling a preset privacy model, performing noise perturbation on the target frequency domain image based on the target privacy allocation parameters corresponding to each pixel point in the target frequency domain image, to obtain a target noisy image corresponding to the target frequency domain image, wherein the target privacy allocation parameters indicate the degree of noise addition on the corresponding pixel point in the target frequency domain image; Calling a target image recognition model to perform image recognition on the target noisy image to obtain an image recognition result; Among them, the preset privacy model is a model obtained by back-propagation training of the initial privacy model based on the model loss obtained by performing image recognition on the sample noisy image corresponding to the sample object image based on the target image recognition model before calling the preset privacy model and noising the target frequency domain image based on the target privacy allocation parameters corresponding to each pixel in the target frequency domain image; the sample noisy image is obtained by performing noise perturbation on the sample frequency domain image obtained by frequency domain conversion of the sample object image based on the initial privacy model based on the initial privacy allocation parameters corresponding to each pixel in the sample frequency domain image; the image loss is calculated based on the sample image recognition result obtained by image recognition of the sample noisy image and the object label corresponding to the sample noisy image; the model parameters of the preset privacy model include the target privacy allocation parameters corresponding to each pixel.

2. The method according to claim 1, characterized in that The calling of the preset privacy model and performing noise perturbation on the target frequency domain image based on the target privacy allocation parameters corresponding to each pixel point in the target frequency domain image to obtain a target noisy image corresponding to the target frequency domain image comprises: Based on the preset total privacy budget, mapping processing is performed on the target privacy allocation parameters corresponding to each pixel point respectively to obtain the target privacy budget corresponding to each pixel point; Determine the noise distribution corresponding to each pixel point based on the target privacy budget, and perform noise sampling on each noise distribution to obtain the noise information corresponding to each pixel point; Based on each noise information, noise processing is performed on the corresponding pixel points in the target frequency domain image to obtain the target noisy image.

3. The method according to claim 1, characterized in that Before calling the preset privacy model and performing noise perturbation on the target frequency domain image based on the target privacy allocation parameters corresponding to each pixel point in the target frequency domain image, the method further includes: Acquire a sample training set, wherein the sample training set includes sample object images and corresponding object labels; Performing frequency domain transformation on the sample object image to obtain a sample frequency domain image corresponding to the sample object image; Calling the initial privacy model, and performing noise perturbation on the sample frequency domain image based on the initial privacy allocation parameters corresponding to each pixel point in the sample frequency domain image, to obtain the sample noisy image; Calling the image recognition model to perform image recognition on the sample noisy image to obtain a sample image recognition result; Perform loss calculation based on the sample image recognition result and the corresponding object label to obtain the model loss; The initial privacy model is back-propagated and trained based on the model loss to obtain the preset privacy model, wherein the model parameters of the preset privacy model include the target privacy allocation parameters corresponding to each pixel point.

4. The method according to claim 3, characterized in that The performing back propagation training on the initial privacy model based on the model loss to obtain the preset privacy model comprises: updating model parameters including the initial privacy allocation parameters according to the model loss; Updating the model loss using the updated initial privacy model and the image recognition model; When the updated model loss satisfies a preset condition, the updated initial privacy model is used as the preset privacy model, and the updated initial privacy allocation parameter is used as the target privacy allocation parameter.

5. The method according to claim 1, characterized in that The performing frequency domain transformation on the target object image to obtain a target frequency domain image corresponding to the target object image comprises: Performing frequency domain transformation on the target object image based on a preset frequency domain transformation method to obtain an initial frequency domain image corresponding to the target object image; The target low-frequency information that meets a preset information amount condition is removed from the initial frequency domain image to obtain the target frequency domain image.

6. The method according to claim 5, characterized in that The performing frequency domain transformation on the target object image based on a preset frequency domain transformation method to obtain an initial frequency domain image corresponding to the target object image comprises: Segmenting the target object image into a preset number of object image blocks based on preset segmentation parameters; Performing discrete cosine transformation on each object image block respectively to obtain a frequency domain block corresponding to each object image block; Based on the position of each object image block in the target object image, each frequency domain block is reorganized to obtain the initial frequency domain image.

7. The method according to claim 5, characterized in that The removing target low-frequency information satisfying a preset information amount condition from the initial frequency domain image to obtain the target frequency domain image comprises: The pixel points corresponding to the direct current component of each frequency domain block are removed from the initial frequency domain image to remove the target low-frequency information and obtain the target frequency domain image.

8. The method according to claim 6, characterized in that Before segmenting the target object image into a preset number of object image blocks based on the preset segmentation parameters, the method further includes: Performing up-sampling processing on the target object image based on a preset sampling multiple matching the preset block parameter to obtain an enlarged object image corresponding to the target object image; The step of dividing the target object image into a preset number of object image blocks based on the preset block parameters comprises: The enlarged object image is divided into a preset number of object image blocks based on the preset block parameters.

9. An image processing device, characterized in that: The device comprises: Image acquisition module: used to acquire the image of the target object; Frequency domain transformation module: used for performing frequency domain transformation on the target object image to obtain a target frequency domain image corresponding to the target object image; Noise perturbation module: used to call a preset privacy model, perform noise perturbation on the target frequency domain image based on the target privacy allocation parameter corresponding to each pixel point in the target frequency domain image, and obtain a target noisy image corresponding to the target frequency domain image, wherein the target privacy allocation parameter indicates the degree of noise addition to the corresponding pixel point in the target frequency domain image; Image recognition module: used to call the target image recognition model to perform image recognition on the target noisy image to obtain the image recognition result; Among them, the preset privacy model is a model obtained by back-propagation training of the initial privacy model based on the model loss obtained by performing image recognition on the sample noisy image corresponding to the sample object image based on the target image recognition model before calling the preset privacy model and noising the target frequency domain image based on the target privacy allocation parameters corresponding to each pixel in the target frequency domain image; the sample noisy image is obtained by performing noise perturbation on the sample frequency domain image obtained by frequency domain conversion of the sample object image based on the initial privacy model based on the initial privacy allocation parameters corresponding to each pixel in the sample frequency domain image; the image loss is calculated based on the sample image recognition result obtained by image recognition of the sample noisy image and the object label corresponding to the sample noisy image; the model parameters of the preset privacy model include the target privacy allocation parameters corresponding to each pixel.

10. The device according to claim 9, characterized in that The noise disturbance module comprises: Allocation parameter mapping submodule: used to map the target privacy allocation parameters corresponding to each pixel point based on the preset total privacy budget, so as to obtain the target privacy budget corresponding to each pixel point; Noise sampling submodule: used to determine the noise distribution corresponding to each pixel point based on the target privacy budget, and perform noise sampling on each noise distribution to obtain the noise information corresponding to each pixel point; Noise processing submodule: used to perform noise processing on the corresponding pixel points in the target frequency domain image based on each noise information to obtain the target noisy image.

11. The device according to claim 9, characterized in that The device also includes: A sample training set acquisition module is used to acquire a sample training set before calling the preset privacy model and performing noise perturbation on the target frequency domain image based on the target privacy allocation parameters corresponding to each pixel point in the target frequency domain image, wherein the sample training set includes a sample object image and a corresponding object label; Sample conversion module: used for performing frequency domain transformation on the sample object image to obtain a sample frequency domain image corresponding to the sample object image; A sample noise perturbation module is used to call the initial privacy model, and based on the initial privacy allocation parameters corresponding to each pixel point in the sample frequency domain image, perform noise perturbation on the sample frequency domain image to obtain the sample noisy image; Sample image recognition module: used to call the image recognition model to perform image recognition on the sample noisy image to obtain a sample image recognition result; Loss calculation module: used to perform loss calculation according to the sample image recognition result and the corresponding object label to obtain the model loss; Back propagation training module: used to perform back propagation training on the initial privacy model based on the model loss to obtain the preset privacy model, wherein the model parameters of the preset privacy model include the target privacy allocation parameters corresponding to each pixel point.

12. The device according to claim 11, characterized in that The back propagation training module includes: A model parameter updating submodule: used for updating the model parameters including the initial privacy allocation parameters according to the model loss; A model loss updating submodule: used to update the model loss using the updated initial privacy model and the image recognition model; Model determination submodule: used to use the updated initial privacy model as the preset privacy model and the updated initial privacy allocation parameters as the target privacy allocation parameters when the updated model loss meets the preset conditions.

13. The device according to claim 9, characterized in that The frequency domain transformation module comprises: Frequency domain transformation submodule: used for performing frequency domain transformation on the target object image based on a preset frequency domain transformation method to obtain an initial frequency domain image corresponding to the target object image; The low-frequency information removal submodule is used to remove the target low-frequency information that meets the preset information amount condition from the initial frequency domain image to obtain the target frequency domain image.

14. The device according to claim 13, characterized in that The frequency domain transformation submodule comprises: An image segmentation unit: used for segmenting the target object image into a preset number of object image blocks based on preset segmentation parameters; Discrete cosine transform unit: used to perform discrete cosine transform on each object image block respectively to obtain a frequency domain block corresponding to each object image block; An image recombining unit is used to recombine each frequency domain block based on the position of each object image block in the target object image to obtain the initial frequency domain image.

15. The device according to claim 13, characterized in that The low-frequency information removal submodule is specifically used to remove the pixel points corresponding to the DC component of each frequency domain block from the initial frequency domain image to remove the target low-frequency information and obtain the target frequency domain image.

16. The device according to claim 14, characterized in that The device also includes: An upsampling unit: used for performing upsampling processing on the target object image based on a preset sampling multiple matching the preset block parameters before dividing the target object image into a preset number of object image blocks based on the preset block parameters, so as to obtain an enlarged object image corresponding to the target object image; The image segmentation unit is specifically configured to segment the enlarged object image into a preset number of object image blocks based on the preset segmentation parameters.

17. A computer device, comprising a processor and a memory, characterized in that: The memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method according to any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method according to any one of claims 1 to 8.

19. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the image processing method according to any one of claims 1 to 8.

20. A terminal, characterized in that: The terminal includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method according to any one of claims 1 to 8.

21. A server, characterized in that: The server includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method according to any one of claims 1 to 8.

Images