Method and system for image processing

Abstract

In the method and system for image processing provided in the application, first, first image data in a first color space is subjected to first-scale target convolution operation to obtain second image data, then the second image data is subjected to nonlinear operation to obtain third image data in a second color space, and then a desensitized image is generated based on the third image data, that is, in the color space conversion process in the embodiment of the application, the first-scale target convolution operation and the nonlinear operation are introduced, the sensitive information in the first image data in the original first color space can be desensitized through the first-scale target convolution operation and the nonlinear operation, so that the desensitized image data is obtained based on the third image data in the second color space, and the simple confusion of the first image data in the original first color space is avoided, which makes it easy to be attacked by attackers, and is conducive to improving the security of the image data.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically to image processing methods and systems in the field of image processing. Background Technology

[0002] In the field of image processing, privacy protection is typically implemented for raw image data to enhance its security, such as for raw facial image data. Current technology involves randomly obfuscating the raw image data before encrypting it. However, this method of encrypting raw image data is relatively insecure and easily compromised by attackers, thus compromising the overall security of the image data. Summary of the Invention

[0003] This application provides an image processing method and system that can improve the security of image processing.

[0004] In a first aspect, an image processing method is provided, comprising: acquiring first image data in an original first color space; performing a nonlinear color space conversion on the first image data to obtain third image data in a second color space; performing a target convolution operation on the first image data at a first scale to obtain second image data; performing a nonlinear operation on the second image data to obtain third image data in the second color space, wherein the first scale is N*N, and N is a positive integer greater than or equal to 1; and generating desensitized image data based on the third image data.

[0005] In some embodiments, the nonlinear operation is a ReLU operation of a linear rectified function.

[0006] In some embodiments, N is 3.

[0007] In some embodiments, before performing a target convolution operation at a first scale on the first image data to obtain second image data, the method further includes: determining the target convolution operation and the target deconvolution operation according to a first loss function; wherein the first loss function includes color space difference loss and image reconstruction loss.

[0008] In some embodiments, the color space difference loss includes the loss between the predicted image data in the second color space and the actual image data in the first color space.

[0009] In some embodiments, the image reconstruction loss includes the loss between the predicted image data in the first color space obtained by performing convolution and nonlinear operations on the image data in the first color space and then performing deconvolution operations, and the actual image data in the first color space.

[0010] In some embodiments, generating the target desensitized image based on the third image data includes:

[0011] The third image data is subjected to a separable transformation to obtain the fourth image data;

[0012] The fourth image data is scaled to obtain the target desensitized image.

[0013] In some embodiments, performing a separable transformation on the third image data to obtain the fourth image data includes: performing a first discrete cosine transform on the third image data to obtain the fourth image data; wherein the objective basis function of the first discrete cosine transform is cos[(π / n)(m+1 / 2)k], where n is a floating-point number, m is the phase offset, and k is the frequency.

[0014] In some embodiments, before performing a discrete cosine transform on the third image data to obtain the fourth image data, the method further includes: determining the target basis function of the first discrete cosine transform according to a second loss function; wherein, the second loss function includes the loss between the image data obtained by performing a discrete Fourier transform on the image data in the second color space and then performing an inverse discrete Fourier transform on the image data in the second color space and the image data in the second color space.

[0015] In some embodiments, scaling the fourth image data to obtain the desensitized image data includes: scaling the fourth image data according to a target quantization table to obtain the desensitized image data; wherein the target quantization table is obtained by training a neural network model.

[0016] Secondly, this specification provides an image processing system, including at least one storage medium and at least one processor, the at least one storage medium storing at least one instruction set for processing images; the at least one processor is communicatively connected to the at least one storage medium, wherein when the image processing system is running, the at least one processor reads the at least one instruction set and implements the image processing method described in the first aspect of this specification.

[0017] As can be seen from the above technical solutions, the image processing method and system provided in this application embodiment first performs a first-scale target convolution operation on the original first image data in the first color space to obtain second image data. Then, a nonlinear operation is performed on the second image data to obtain third image data in the second color space. Then, a desensitized image is generated based on the third image data. In other words, in this application embodiment, a first-scale target convolution operation and a nonlinear operation are introduced in the color space conversion process. The sensitive information in the original first image data in the first color space can be desensitized through the first-scale target convolution operation and the nonlinear operation to obtain the third image data in the second color space. The desensitized image data is then obtained based on the third image data. This avoids the situation where the original first image data in the first color space is easily obfuscated by attackers, which is conducive to improving the security of image data.

[0018] Other functions of the image processing methods and systems provided in this application will be partially listed in the following description. The figures and examples described below will be readily apparent to those skilled in the art. The inventive aspects of the payment processing methods and systems provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic diagram of an image processing system provided according to an embodiment of this specification is shown;

[0021] Figure 2 A hardware schematic diagram of a computing device provided according to an embodiment of this specification is shown;

[0022] Figure 3 A schematic flowchart of an image processing method according to an embodiment of this specification is shown; and

[0023] Figure 4 A schematic diagram of the process for generating desensitized image data based on third image data, according to an embodiment of this specification, is shown. Detailed Implementation

[0024] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0025] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0026] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0027] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0028] With the development of machine vision technology, images are applied to various aspects of life. As images become more widely used, protecting the privacy information within them is paramount. For example, facial images can be used for facial payment, facial attendance, and facial recognition for entry, and the privacy information within these images can include the user's face. Similarly, user identity information captured in images also falls under the category of privacy information. Image processing can include steps such as acquisition, processing, storage, and transmission. Any of these steps can potentially lead to the leakage of privacy information. For instance, during transmission, when device A transmits image data to device B, the image data may be vulnerable to attack along the transmission path. Therefore, device A needs to desensitize the image data before transmission to prevent attacks. To protect privacy information in images, some solutions obfuscate the image data, such as by shifting rows and columns. However, this simple obfuscation is easily compromised by attackers and has low security capabilities. In other solutions, the image data corresponding to the acquired image can be input into a deep learning model to obtain desensitized image data. Since this method is based on a deep learning model for desensitization, it involves a large amount of computation, requires high processing power from the processor, and has a slow processing speed, for example, the processing time can be as long as 200ms to 2s. For example, the processor is a central processing unit (CPU) or a neural processing unit (NPU).

[0029] In this embodiment, the electronic device first performs a nonlinear color space conversion and a first-scale target convolution operation on the original first image data in the first color space to obtain second image data. Then, a nonlinear operation is performed on the second image data to obtain third image data in the second color space. The electronic device then generates a desensitized image based on the third image data. In other words, this embodiment introduces a first-scale target convolution operation and a nonlinear operation during the color space conversion process. This allows sensitive information in the original first image data in the first color space to be desensitized through the first-scale target convolution operation and the nonlinear operation, resulting in the third image data in the second color space. The desensitized image data is then obtained based on this third image data. This avoids the vulnerability to attackers caused by simply obfuscating the original first image data in the first color space, thus improving the security of image data and reducing the complexity of desensitization using deep learning models.

[0030] Figure 1This diagram illustrates an application scenario of an image processing system 100 according to an embodiment of this application. The image processing system 100 (hereinafter referred to as system 100) can be applied to scenarios involving image processing. System 100 may include an image acquisition device 110 and an image processing device 120. In some embodiments, system 100 may not include the image acquisition device 110. In some embodiments, system 100 may also include a network 130. In some embodiments, system 100 may also include a database 140.

[0031] Image acquisition device 110 acquires raw first image data in a first color space. For example, image acquisition device 110 can be an image sensor, such as a mobile phone, camera, or any device with shooting capabilities. For instance, image acquisition device 110 can acquire a first image in a first color space and send the first image data corresponding to the acquired first image in the first color space to image processing device 120. Alternatively, image acquisition device 110 can acquire video in a first color space and send the video data corresponding to the acquired video in the first color space to image processing device 120. Or, image acquisition device 110 can extract an image from the acquired video in the first color space at preset intervals and send the first image data corresponding to the extracted image to image processing device 120. In some embodiments, the first color space can be the RGB color space.

[0032] Image processing device 120 acquires first image data in a first color space. In some embodiments, acquiring the first image data in the first color space may include: the image processing device 120 receiving raw first image data in the first color space from the image acquisition device 110, or it may include: the image processing device 120 receiving raw video data in the first color space from the image acquisition device 110, and acquiring image data in the first color space from the video data in the first color space. The image processing device 120 may perform a first-scale target convolution operation and a nonlinear operation on the first image data to obtain third image data in a second color space, the second color space being different from the first color space. In some embodiments, the image processing device 120 may be a device capable of processing images, for example, it may be a CPU or a device including a CPU.

[0033] Network 130 can facilitate the exchange of information and / or data. For example... Figure 1As shown, the image acquisition device 110 is connected to the image processing device 120 via a network 130, and the image acquisition device 110 and the image processing device 120 transmit information and / or data to each other via the network 130. For example, the image acquisition device 110 can send first image data in a first color space to the image processing device 120 via the network 130, etc. In some embodiments, the network 130 can be any type of wired or wireless network, or a combination thereof. For example, the network 130 may include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, a near field communication (NFC) network, or a similar network. In some embodiments, the network 130 may include one or more network access points. For example, network 130 may include one or more wired or wireless network access points through which one or more components of image acquisition device 110, image processing device 120, and database 140 can connect to network 130 to exchange data and / or information.

[0034] Database 140 may store data and / or instructions. In some embodiments, database 140 may store image data, such as first image data and / or third image data, or image data generated during image processing. In some embodiments, database 140 may store data and / or instructions that image acquisition device 110 or image processing device 120 can execute or use to execute the payment processing method described in this specification. In some embodiments, database 140 may store data obtained from image acquisition device 110. Image processing device 120 may have access to database 140, and image processing device 120 may access data or instructions stored in database 140 via network 130. In some embodiments, database 140 may be directly connected to image acquisition device 110 and / or image processing device 120. If database 140 is connected to image acquisition device 110, image processing device 120 can obtain first image data in a first color space from the database. In some embodiments, database 140 may be part of image processing device 120. In some embodiments, database 140 may include mass storage, removable storage, volatile read-write memory, read-only memory (ROM), or similar content, or any combination thereof. Exemplary high-capacity storage may include non-transitory storage media such as hard disks, optical disks, and solid-state drives. Exemplary removable storage may include flash drives, floppy disks, optical disks, memory cards, zip disks, and magnetic tapes. Typical volatile read / write memory may include random access memory (RAM). Exemplary RAM may include dynamic RAM (DRAM), dual date rate synchronous dynamic RAM (DDR SDRAM), static RAM (SRAM), thyristor RAM (T-RAM), and zero-capacitance RAM (Z-RAM). Exemplary ROM may include mask ROM (MROM), programmable ROM (PROM), virtual programmable ROM (PEROM), electronically programmable ROM (EEPROM), optical disk (CD-ROM), and digital multifunction disk ROM. In some embodiments, database 140 may be implemented on a cloud platform. By way of example only, cloud platforms may include private clouds, public clouds, hybrid clouds, community clouds, distributed clouds, inter-cloud clouds, or similar, or any combination thereof.

[0035] Figure 2A hardware schematic diagram of a computing device 200 is shown. In some embodiments, the data or instructions for the image processing device 120 to perform an image processing method can be implemented on the computing device 200. That is, a portion of the hardware structure of the image processing device 120 can be the hardware structure shown for the computing device 300. The computing device 300 can perform the image processing methods described in the embodiments of this application. The image processing methods are described in other parts of the embodiments of this application.

[0036] like Figure 2 As shown, the computing device 200 may include at least one storage medium 230 and at least one processor 220. In some embodiments, the computing device 200 may also include a communication port 250 and an internal communication bus 210. In some embodiments, the computing device 200 may also include I / O components 260.

[0037] The internal communication bus 210 can connect different system components to enable data communication between components in the computing device 200, including the storage medium 230, processor 220, communication port 250, and I / O component 260. For example, the processor 220 can send data to the storage medium 230 or other hardware such as the I / O component 260 via the internal communication bus 210. In some embodiments, the internal communication bus 210 can be an Industry Standard (ISA) bus, an Extended Industry Standard (EISA) bus, a Video Electronics Standard (VESA) bus, an External Component Interconnect (PCI) bus, etc.

[0038] I / O component 260 can be used to input or output signals, data, or information. I / O component 260 supports input / output between computing device 200 and other components. In some embodiments, I / O component 260 may include input devices and output devices. Exemplary input devices may include a camera, keyboard, mouse, display screen, and microphone, or any combination thereof. Exemplary output devices may include a display device, a voice playback device (e.g., a speaker), a printer, a projector, or any combination thereof. Exemplary display devices may include a liquid crystal display (LCD), a light-emitting diode (LED) based display, a flat panel display, a curved display, a television device, a cathode ray tube (CRT), or any combination thereof.

[0039] Communication port 250 can be connected to a network to enable computing device 200 to communicate with the outside world. The connection can be wired, wireless, or a combination of both. Wired connections can include cables, fiber optic cables, or telephone lines, or any combination thereof. Wireless connections can include Bluetooth, Wi-Fi, WiMax, WLAN, ZigBee, mobile networks (e.g., 3G, 4G, or 5G), or any combination thereof. In some embodiments, communication port 250 can be a standardized port, such as RS232 or RS485. In some embodiments, communication port 250 can be a specially designed port.

[0040] Storage medium 230 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 232, a read-only storage medium (ROM) 234, or a random access storage medium (RAM) 236. Storage medium 230 also includes at least one instruction set stored in the data storage device. The instructions are computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that execute the payment processing methods provided in this specification.

[0041] At least one processor 220 can be communicatively connected to at least one storage medium 230 and a communication port 250 via an internal communication bus 210. The at least one processor 220 is used to execute the at least one instruction set described above. When the computing device 200 is running, the at least one processor 220 reads the at least one instruction set and executes the image processing method provided in this application embodiment according to the instructions of the at least one instruction set. The processor 220 can execute all the steps included in the image processing method. The processor 220 can be in the form of one or more processors. In some embodiments, the processor 220 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof. For illustrative purposes only, only one processor 220 is described in this specification for the computing device 200. However, it should be noted that the computing device 200 in this specification may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described in this specification, or they may be executed jointly by multiple processors. For example, if the processor 220 of the computing device 200 in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 220 (e.g., the first processor executes step A, the second processor executes step B, or the first and second processors jointly execute steps A and B).

[0042] The following describes the process by which the image processing device 120 acquires the target convolution operation and the target deconvolution operation in the embodiments of this application.

[0043] a) The image processing device 120 acquires an initial convolution operation, a nonlinear operation, an initial deconvolution operation, and a first sample set at a first scale. The first sample set includes L samples, and each of the L samples includes image data in a true first color space and image data in a true second color space. Wherein, a sample that includes image data in a true first color space and image data in a true second color space is the image data of the same image, and L is a positive integer.

[0044] b) The image processing device 120 determines a first loss function, which includes color space difference loss and image reconstruction loss.

[0045] In some embodiments, color space difference loss includes the loss of image data in the predicted second color space compared to image data in the true first color image.

[0046] For example, color space difference loss can be expressed as Among them, E i (x i Let x represent the predicted image data in the second color space for the i-th sample in the first sample set. i This represents the true image data in the first color space of the i-th sample, where m is a preset value. E i (·) represents the i-th convolution operation and nonlinear operation. For example, when i=1, E1(x1) means inputting the true first color space image data x1 of the first sample into the first convolution operation to obtain the image data after the first convolution operation, and then performing a nonlinear operation on the image data after the first convolution operation to obtain the predicted second color space image data of the first sample. For example, the initial convolution operation can be the first convolution operation. Then, the first color space difference loss is calculated. The value of i ranges from 1 to L.

[0047] In some embodiments, the color space difference loss can also be expressed by other formulas, such as The embodiments of this application do not impose any limitations on the way color space difference loss is represented.

[0048] In some embodiments, the image reconstruction loss includes the loss between the predicted image data in the first color space obtained by performing convolution and nonlinear operations on the image data in the first color space, and the image data of the real first color image.

[0049] For example, image reconstruction loss can be expressed as x i E represents the true first color space image data of the i-th sample. i (x i Let x be the true image data of the i-th sample in the first color space from the first sample set. i The image data in the second color space of the i-th sample is obtained by first performing the i-th convolution operation and then the nonlinear operation, D i (·) is for performing the i-th deconvolution operation, D i (E i (x iE1(x1) represents the predicted image data in the first color space obtained by performing the i-th deconvolution operation on the predicted second color space image data of the i-th sample. For example, when i=1, E1(x1) represents the predicted second color space image data of the first sample obtained by inputting the true first color space image data x1 of the first sample into the first convolution operation, obtaining the image data after the first convolution operation, and then performing a nonlinear operation on the image data after the first convolution operation. D1(E1(x1)) represents the predicted first color space image data obtained by performing the first deconvolution operation on the predicted second color space image data of the first sample. The initial convolution operation can be the first convolution operation, and the initial deconvolution operation can be the first deconvolution operation. Then, the image reconstruction loss of the first nonlinear color space conversion is calculated.

[0050] In some embodiments, the image reconstruction loss can also be expressed as other formulas, such as The embodiments of this application do not impose any limitations on the way image reconstruction loss is represented.

[0051] c) The image processing device 120 determines the target convolution operation and the target deconvolution operation based on the i-th loss of the first loss function.

[0052] In some embodiments, the i-th loss of the first loss function may consist of the color space difference loss of the i-th sample and the image reconstruction loss of the i-th sample. The image processing device 120 may adjust the i-th convolution operation and the i-th deconvolution operation according to the i-th loss of the first loss function to obtain the (i+1)-th convolution operation and the (i+1)-th deconvolution operation. The image processing device 120 determines the convolution operation obtained when the loss of the first loss function converges as the target convolution operation, and determines the deconvolution operation obtained when the loss of the first loss function converges as the target deconvolution operation. In some embodiments, the convergence of the loss of the first loss function may be: the absolute value of the i-th loss of the first loss function and the (i+1)-th loss of the first loss function is less than a preset value.

[0053] Combining the examples of color space difference loss and image reconstruction loss, the loss of the i-th sample in the first loss function can be expressed as: In other words, a loss can be calculated for a single sample. Based on the calculated loss, the i-th convolution operation is adjusted to obtain the (i+1)-th convolution operation, and the i-th deconvolution operation is adjusted to obtain the (i+1)-th deconvolution operation, and so on. When the loss of the first loss function converges, the convolution operation can be the target convolution operation, and when the loss of the first loss function converges, the deconvolution operation can be the target deconvolution operation.

[0054] Among them, the scale of the initial convolution operation, the scale of the initial deconvolution operation, the scale of the i-th convolution operation, the scale of the target convolution operation, and the scale of the target deconvolution operation are all the first scale. That is to say, the scale of any convolution operation used in the process of determining the target convolution operation and the target deconvolution operation is the same.

[0055] In some embodiments, the target convolution operation and nonlinear operation can form a color space encoder, which can be used for image data desensitization. The target deconvolution operation can be called a color space decoder. The color space decoder can be used to recover the privacy information of the image data.

[0056] In some embodiments, the nonlinear operation in the target convolution operation process for determining the first scale described above can be a Rectified Linear Unit (ReLU) function, for example, the ReLU function can be... Where x represents the image data after convolution. In some embodiments, a non-linear element can be introduced using the ReLU function to increase the security of privacy information in the image data.

[0057] In some embodiments, the nonlinear operation in the process of determining the target convolution operation of the first scale described above can also be other nonlinear functions, such as the sigmoid function. This application does not impose any restrictions on the linear function used for the nonlinear operation.

[0058] The above describes how the image processing device 120 determines the target convolution operation and the target deconvolution operation. In some embodiments, other devices can determine the target convolution operation and the target deconvolution operation and send the determined target convolution operation and the target deconvolution operation to the image processing device 120. The principle by which other devices determine the target convolution operation and the target deconvolution operation is similar to that of the image processing device 120, and will not be described in detail to avoid redundancy.

[0059] The process of obtaining the target basis function for determining the first discrete cosine transform in the embodiments of this application is described below.

[0060] a) The image processing device 120 determines the initial basis function cos[(π / n0)(m+1 / 2)k] of the discrete cosine transform and the second sample set. The second sample set includes P samples, each of which contains image data in the temporal domain of the true second color space; where n0 is a floating-point number, m is the phase offset, k is the frequency, and P is a positive integer.

[0061] In some embodiments, each of the P samples in the second sample set includes image data of the temporal true second color space of the same size, that is, the number of image pixels corresponding to the image data of the temporal true second color space included in each sample is the same.

[0062] b) The image processing device 120 determines a second loss function. The second loss function includes the loss between the image data of the second color space predicted in the time domain obtained by performing a discrete Fourier transform on the image data of the real second color space in the time domain and then performing an inverse discrete Fourier transform, and the image data of the real second color space in the time domain.

[0063] In some embodiments, the loss of the second loss function can be expressed as: Where j takes the value of a positive integer from 1 to P. j The image data in the second color space of the j-th sample in the second sample set is the true time-domain image data of the second color space. This means that the image data in the frequency domain predicted by performing a discrete cosine transform on the real image data in the second color space in the time domain is obtained. This represents the image data in the predicted second color space obtained by performing an inverse discrete cosine transform on the predicted image data in the second color space.

[0064] In some embodiments, the second sample set includes the same number of samples as the first sample set, that is, N = P. In some embodiments, the image processing device 120 can obtain the second sample set based on the first sample set. For example, the actual image data in the second color space in the first sample set can be used as the actual image data in the second color space in the second sample set.

[0065] In some embodiments, the second sample set may be a sample set independent of the first sample set, that is, the second sample set is not associated with the first sample set.

[0066] c) The image processing device 120 determines the target basis function of the first discrete cosine transform based on the j-th loss of the second loss function, for example, the determined target basis function is cos[(π / n)(m+1 / 2)k]. The image processing device 120 can adjust the basis functions of the j-th discrete cosine transform based on the j-th loss of the second loss function to obtain the basis functions of the (j+1)-th discrete cosine transform. The image processing device 120 determines the basis function of the convolution operation obtained when the loss of the second loss function converges as the target basis function of the first convolution operation. In some embodiments, the convergence of the loss of the second loss function can be achieved when the absolute value of the j-th loss of the second loss function and the (j+1)-th loss of the second loss function is less than a preset value.

[0067] In some embodiments, the number of basis functions of the j-th discrete cosine is equal to the number of pixels in the image corresponding to a sample in the second sample set.

[0068] In some embodiments, the number of target basis functions is equal to the number of pixels in the image corresponding to a sample in the second sample set. For example, if the number of pixels in the image data of the real second color space in the time domain included in each sample in the second sample set is 256*256, then the image processing device 120 can determine 256*256 target basis functions according to the above method. That is, in order to better transform the image data of the second color space in the time domain to the second color space in the frequency domain, the number of target basis functions of the first discrete cosine transform can be made consistent with the number of image pixels, thereby achieving better time-frequency transformation.

[0069] The above describes how the image processing device 120 determines the target basis function of the first discrete cosine transform. In some embodiments, other devices can determine the target basis function of the first discrete cosine transform and send the determined target basis function of the first discrete cosine transform to the image processing device 120. The principle by which other devices determine the target basis function of the first discrete cosine transform is similar to that of the image processing device 120. To avoid redundancy, it will not be described in detail.

[0070] The process of obtaining the target quantification table is described below.

[0071] a) The image processing device 120 determines the model structure of the neural network model and the third sample set, the third sample set including K samples, each of the K samples including the image data of the second color space in the frequency domain, the quantization table, and the image data of the second color space in the frequency domain quantized according to the quantization table, where K is a positive integer.

[0072] In some embodiments, the model structure of the neural network model can be UNET.

[0073] b) The image processing device 120 determines a third loss function. The third loss function includes the loss between the image data in the frequency domain true second color space after quantization according to the quantization table and the image data in the frequency domain true second color space.

[0074] In some embodiments, the third loss function is a mutual information calculation function. In some embodiments, the mutual information calculation function is Loss. s =H(W s (x s ),x s ), where x s W represents the image data in the second color space that is the true value in the frequency domain, where the s-th sample in the third sample set is included. s (x sH(W) represents the image data after quantization of the image data in the second color space, which includes the true frequency domain, based on the quantization table included in the s-th sample in the third sample set. s (x s ),x s The loss represents the difference between the quantized image data based on the s-th sample and the image data of the s-th sample in the true second color space, including the frequency domain. s Let represent the mutual information of the s-th sample, where s is a positive integer from 1 to K.

[0075] c) The image processing device 120 determines the target quantization table based on the s-th loss of the third loss function. The image processing device 120 can adjust the quantization table in the s-th sample based on the s-th loss of the third loss function to obtain the quantization table included in the (s+1)-th sample. In some embodiments, the image processing device 120 determines the quantization table obtained when the loss of the third loss function converges as the target quantization table. In some embodiments, the convergence of the loss of the third loss function can be defined as: the absolute value of the s-th loss of the third loss function and the (s+1)-th loss of the third loss function is less than a preset value.

[0076] In some embodiments, the image processing device 120 performs an average operation on the quantization tables included in the K samples to obtain a target quantization table.

[0077] In some embodiments, the number of samples included in the third sample set is the same as the number of samples included in the second sample set, that is, K = P. In some embodiments, the image processing device 120 can obtain image data in the frequency domain of the second color space by performing a first discrete cosine transform on the image data in the time domain of the second color space included in the second sample set.

[0078] In some embodiments, the third sample set may be a sample set independent of the second sample set.

[0079] The above describes the process by which the image processing device 120 determines the target convolution operation at the first scale, the target deconvolution operation at the first scale, and the target basis function of the first discrete cosine transform. The following describes the process of desensitizing the image data. Figure 3 A schematic flowchart of an image processing method 300 according to an embodiment of this application is shown. The image processing device 120 can execute the method 300 provided in the embodiment of this application. Specifically, the processor 320 in the image processing device 120 can read an instruction set stored in its local storage medium, and then execute the method 300 provided in the embodiment of this application according to the provisions of the instruction set. The method 300 may include:

[0080] S310, the image processing device 120 acquires the first image data in the original first color space.

[0081] In some embodiments, S310 includes: the image processing device 120 can receive first image data in the original first color space from the image acquisition device 110.

[0082] In some embodiments, the image acquisition device 110 can acquire a first image in the original first color space and send the first image data corresponding to the first image to the image processing device 120. The image processing device 120 receives the video data in the original first color space and obtains the first image data in the first color space from the video data in the first color space.

[0083] In some embodiments, the image acquisition device 110 is used to acquire video in a first color space. The image acquisition device 110 extracts an image from the video in the first color space at preset intervals of a number of frames, and sends the first image data corresponding to the extracted image to the image processing device 120. The image processing device 120 receives the first image data from the image acquisition device.

[0084] In some embodiments, the first image data includes a plurality of pixel values ​​in a first color space.

[0085] S320, the image processing device 120 performs a target convolution operation of the first scale on the first image data to obtain the second image data.

[0086] In S320, the image processing device 120 can perform a target convolution operation at a first scale on the first image data to obtain the second image data. That is, a pixel value in the second image data includes the pixel values ​​at a first scale surrounding the corresponding pixel value in the first image data. In other words, a pixel value in the second image data includes local information at a first scale in the first image data. In this way, privacy information in the first image data can be hidden, thereby improving security.

[0087] In some embodiments, the first image data includes a plurality of pixel values ​​in a first color space. S320 includes: the image processing device 120 performing a target convolution operation of a first scale on the plurality of pixel values ​​included in the first image data to obtain second image data. The second image data may also include a plurality of pixel values.

[0088] In some embodiments, the number of pixel values ​​included in the first image data may be the same as the number of pixel values ​​included in the image data of the first color space included in the samples in the first sample set. For example, if the number of pixel values ​​in the image data of the first color space included in the samples in the first sample set is 256*256, then the number of pixel values ​​included in the first image data may also be 256*256.

[0089] In some embodiments, the first scale can be N*N, where N is a positive integer greater than or equal to 1. In some embodiments, when N is greater than 1, during the target convolution operation of the first image data in the first color space at the first scale, other information from the local pixels can be relied upon. This can achieve the purpose of hiding the privacy information of the first image data in the first color space, thereby improving security. In some embodiments, N is 3. In this case, the image processing device 120 can use a 3*3 target convolution operation to process the first image data, thereby avoiding the high computational complexity when N is too large, and also ensuring that the second image data includes the local information in the first image data, thereby hiding the privacy information in the first image data.

[0090] S330, the image processing device 120 performs nonlinear operations on the second image data to obtain third image data in the second color space.

[0091] In S330, the image processing device 120 can convert the second image data in the first color space into the third image data in the second color space according to nonlinear operations. In other words, a nonlinear operator is introduced in the color space conversion process, which can better hide privacy information and improve security.

[0092] In some embodiments, the nonlinear operation is a ReLU operation using a linear rectified function. In some embodiments, the ReLU function can be... Where x represents the pixel values ​​included in the second image data.

[0093] In some embodiments, the second color space is different from the first color space. For example, the first color space is the RGB space, and the second color space is a color space different from the RGB space.

[0094] In some embodiments, the number of pixel values ​​included in the first image data is the same as the number of pixel values ​​included in the third image data.

[0095] In some embodiments, S320 and S330 may also be referred to as nonlinear color space conversion.

[0096] S340, the image processing device 120 generates desensitized image data based on the third image data.

[0097] In some embodiments, such as Figure 4 S340 shown includes: S341 and S342, wherein: S341: the image processing device 120 performs a separable transformation on the third image data to obtain the fourth image data; S342: the image processing device 120 performs a scale transformation on the fourth image data to obtain the target desensitized image.

[0098] In some embodiments, S341 and S342 can be understood as a process of compressing the third image data, such as performing Joint Photographic Experts Group (JPEG) compression on the second image.

[0099] In some embodiments, the separable transform may include the Discrete Cosine Transform (DCT) or the Discrete Fourier Transform (DFT).

[0100] In some embodiments, the image processing device 120 performs a separable transformation on the third image data to obtain fourth image data, including: performing a first discrete cosine transform on the third image data to obtain the fourth image data; wherein the target basis function of the first discrete cosine transform is cos[(π / n)(m+1 / 2)k], where n is a floating-point number, m is the phase offset, and k is the frequency. By using a floating-point number for the target basis function in the discrete cosine transform of the third image data, the image processing device 120 can perform a finer-grained transformation on the third image data, thereby hiding privacy information in the third image data and improving security. In some embodiments, the number of target basis functions of the first discrete cosine transform is equal to the number of pixel values ​​included in the third image data, and different target basis functions have different values ​​for n.

[0101] In some embodiments, the image processing device 120 performs scale transformation on the fourth image data to obtain a target desensitized image, including: the image processing device 120 performs scale transformation on the fourth image data according to a target quantization table to obtain desensitized image data.

[0102] In some embodiments, the image processing device 120 performs scale transformation on the fourth image data according to the target quantization table to obtain desensitized image data, including: the image processing device 120 performs scale transformation on the amplitude included in the fourth image data according to the target quantization table to obtain desensitized image data.

[0103] In some embodiments, since the target quantization table is obtained by training a neural network model, the image processing device can hide the privacy information in the fourth image data during the process of scaling the fourth image data according to the target quantization table to obtain the desensitized image. This results in higher security for the desensitized image data and avoids the problem of being vulnerable to attacks caused by simply scaling the fourth image data proportionally.

[0104] In summary, in the image processing method 300 and system 100 provided in this application embodiment, the image processing device 120 first performs a first-scale target convolution operation on the original first image data in the first color space to obtain second image data, and then performs a nonlinear operation on the second image data to obtain third image data in the second color space. That is, in this application embodiment, a first-scale target convolution operation and a nonlinear operation are introduced during the color space conversion process. The image processing device 120 can desensitize sensitive information in the original first image data in the first color space through the first-scale target convolution operation and the nonlinear operation, thereby obtaining the third image data in the second color space, which can hide some privacy information. Furthermore, in the process of performing a first discrete cosine transform on the third image data to obtain fourth image data, since the target basis function n is a floating-point number, the image processing device 120 can further perform fine-grained processing on the third image data, thereby further hiding the privacy information of the third image data. The image processing device 120 performs a scale transformation on the fourth image data to obtain desensitized image data. Since the target quantization table for the scale transformation is obtained based on a neural network model, it is not a simple scaling transformation according to a ratio. This further hides privacy information in the fourth image data, generating desensitized image data. The method provided in this embodiment avoids simply obfuscating the original first image data in the first color space, which makes it vulnerable to attacks, thus improving image data security. It also avoids the computational complexity caused by using a deep learning model to desensitize the original first color space. The method in this embodiment takes less than a few milliseconds to obtain desensitized data, thereby improving system performance.

[0105] In some embodiments, the image processing device 120 can restore the desensitized image data to obtain the original data in the first color space. This process is the reverse of generating the desensitized image data in method 300. Specifically, the reverse process includes: the image processing device 120 performing an inverse scale transform on the desensitized image data to obtain image data in the second color space of the frequency domain before the scale transform. Then, the image processing device 120 performs an inverse first discrete Fourier transform on the image data in the second color space to obtain image data in the second color space of the time domain. Finally, the image processing device 120 performs a target deconvolution operation on the image data in the second color space of the time domain to obtain the original image data in the first color space. In some embodiments, the desensitized image data can be restored by a device other than the image processing device 120. For example, the other device could be a cloud server. The image processing device 120 can send the desensitized image data to the cloud server, and the cloud server can restore the desensitized image data. The principle of the cloud server's restoration is the same as that of the image processing device 120's restoration. To avoid redundancy, this embodiment does not limit this approach.

[0106] In some embodiments, the image processing device 120 can perform a first discrete cosine transform on the fifth image data in the second color space to obtain the sixth image data; wherein the objective basis function of the first discrete cosine transform is cos[(π / n)(m+1 / 2)k], where n is a floating-point number, m is the phase offset, and k is the frequency. That is, performing the first discrete cosine transform can be independent of the third image data in S320. In this way, the image processing device 120 can perform fine-grained processing on the fifth image data in the second color space, thereby hiding the privacy information of the fifth image data. In some embodiments, the image processing device 120 can perform a discrete cosine transform on the seventh image data in the third color space, wherein the basis function of the discrete cosine transform is similar to cos[(π / n)(m+1 / 2)k], thereby the image processing device 120 can perform a discrete cosine transform on image data in any color space, thereby hiding the privacy information in image data in any color space. The third color space may be different from the first and second color spaces.

[0107] In some embodiments, the image processing device 120 can perform a scaling transformation on the eighth image data in the second color space of the frequency domain according to a target quantization table to obtain quantized ninth image data. The target quantization table can be obtained by training a neural network model, and therefore is not a simple scaling transformation according to a ratio, which can hide privacy information in the eighth image data, thereby improving security. That is, quantization according to the target quantization table can be performed without relying on the fourth image data after the first discrete cosine in method 300. In some embodiments, the image processing device 120 can perform a scaling transformation on the image data in the fourth color space of the frequency domain according to the target quantization table, thereby hiding privacy information in the image data in the fourth color space. In this way, the image processing device 120 can perform a scaling transformation on image data in any color space according to the target quantization table, thereby hiding privacy information in image data in any color space. The fourth color space may be different from the first color space and the second color space.

[0108] This specification, in another aspect, provides a non-transitory storage medium storing at least one set of executable instructions for image processing. When executed by a processor, the executable instructions instruct the processor to perform the steps of the image processing method 300 described herein. In some possible implementations, various aspects of the embodiments of this application can also be implemented as a program product comprising program code. When the program product is run on a computing device 300, the program code causes the computing device 300 to perform the image processing steps described herein. The program product for implementing the above method may employ a portable compact disc read-only memory (CD-ROM) containing program code and may run on the computing device 300. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system (e.g., processor 320). The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on computing device 300, partially on computing device 300, as a standalone software package, partially on computing device 300 and partially on a remote computing device, or entirely on a remote computing device.

[0109] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0110] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0111] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0112] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and to aid in understanding a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may extract some features as individual embodiments when reading this specification. That is, the embodiments in this specification can also be understood as an integration of multiple sub-embodiments. It is also valid when each sub-embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0113] Each patent, patent application, publication of the patent application, and other materials such as articles, books, specifications, publications, documents, articles, etc., cited herein may be incorporated by reference. The entire contents used for all purposes, except for any history of prosecution documents associated with it, that may be inconsistent with or conflict with this document, or that may have a limiting effect on the widest extent of the claims, are now or hereafter associated with this document. For example, in the event of any inconsistency or conflict between the description, definition, and / or use of terms associated with any of the included materials and the terms, description, definition, and / or used in connection with this document, the terms used herein shall prevail.

[0114] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. An image processing method, comprising: Obtain the first image data in the original first color space; Perform a target convolution operation of the first scale on the first image data to obtain the second image data; A nonlinear operation is performed on the second image data to obtain third image data in a second color space, wherein the first scale is N*N, and N is a positive integer greater than or equal to 1; and Based on the third image data, desensitized image data is generated; Before performing a target convolution operation of a first scale on the first image data to obtain the second image data, the method further includes: The target convolution operation and target deconvolution operation are determined according to a first loss function; wherein, the first loss function includes color space difference loss and image reconstruction loss; the color space difference loss includes: the loss between the predicted image data in the second color space and the actual image data in the first color space; the image reconstruction loss includes: the loss between the predicted image data in the first color space and the actual image data in the first color space obtained by performing convolution operation and the nonlinear operation on the image data in the first color space, and then performing deconvolution operation.

2. The method according to claim 1, wherein the nonlinear operation is a ReLU operation of a linear rectified function.

3. The method according to claim 1, wherein N is 3.

4. The method according to any one of claims 1 to 3, wherein generating the target desensitized image based on the third image data comprises: The third image data is subjected to a separable transformation to obtain the fourth image data; The fourth image data is scaled to obtain the target desensitized image.

5. The method according to claim 4, wherein performing a separable transformation on the third image data to obtain the fourth image data comprises: The third image data is subjected to a first discrete cosine transform to obtain the fourth image data; Wherein, the objective basis function of the first discrete cosine transform is ,in For floating-point numbers, For phase shift, For frequency.

6. The method according to claim 5, wherein before performing a discrete cosine transform on the third image data to obtain the fourth image data, the method further comprises: The target basis function of the first discrete cosine transform is determined based on the second loss function; The second loss function includes the loss between the image data obtained by performing a discrete Fourier transform on the image data in the second color space and then performing an inverse discrete Fourier transform on the image data in the second color space, and the image data in the second color space.

7. The method according to claim 4, wherein performing scale transformation on the fourth image data to obtain the desensitized image data comprises: The fourth image data is scaled according to the target quantization table to obtain the desensitized image data; The target quantization table is obtained by training a neural network model.

8. An image processing system, comprising: At least one storage medium storing at least one set of instructions for image processing; as well as At least one processor, communicatively connected to the at least one storage medium, reads the at least one set of instructions during runtime and executes any one of the methods claimed in claims 1-7.

Images