Image processing method, related apparatus and storage medium

Abstract

Embodiments of the present application relate to the field of computer vision, and provide an image processing method, related device and storage medium, the method comprising: acquiring a first image and a second image, wherein the first image and the second image are both digital images, the second image is acquired by a first device based on a third image, the third image is output by a second device based on the first image, the first image comprises at least one color region, and each pixel in the color region has the same value in the same channel of a same color space; acquiring at least one target color block from the second image for at least one color region of the first image, wherein the target color block corresponds to the color region one by one; acquiring a correction pixel of the target color block; and mapping each color region in the first image and each correction pixel respectively to obtain a target data set. The embodiments of the present application have low training data collection cost.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of computer vision, and more particularly to an image processing method, related apparatus and storage medium. BACKGROUND

[0002] Currently, in order to test the security of a machine learning model, an adversarial image is often generated. In some scenarios, the adversarial image needs to be materialized and then placed in a test environment, and then collected by an image collection device and input into the machine learning model to be tested. In the above process, since the digital adversarial image undergoes materialization by different physical devices, digital collection and other processes, if color changes occur in the middle, the image input into the machine learning model may not be consistent with the effect presented by the generated adversarial image, affecting the test results of the machine learning model. In order to reduce the color change of the adversarial image B presented after the adversarial image A is output by a physical device and then collected again, and improve the stability of the test results of the machine learning model, color transformation prediction needs to be performed on the adversarial image B, so that the image display effect of the adversarial image B is consistent with the target.

[0003] There are mainly two categories of existing color transformation prediction methods, the first category is a search-based method, and the second category is a fitting-based method.

[0004] The search-based method enumerates all color categories that can be output by a related device (such as a printer), and then uses a k-neighbor method to find the k closest colors for each pixel of the image, and uses the k colors and a preset kernel function to predict the color of the pixel after output by the device. The time complexity of the search-based method is proportional to the number of colors in the preset color table. When a large number of colors are selected, the precision is high but the running is slow; and when a small number of colors are selected, the running is fast but the precision is low. The search-based method has the disadvantage that it cannot balance between precision and operation speed.

[0005] The fitting-based method uses a machine learning method to fit a color transformation prediction model on the training data, and then uses the trained color transformation prediction model to predict the results of a new picture. The fitting-based method can be divided into two categories: a deep learning-based method and a device mechanism-based method.

[0006] The deep learning-based fitting method often needs a large number of training images. Although the prior art can use a picture block-based training method to improve the efficiency of the deep learning model in utilizing pictures, the amount of training image data required is still large, especially the method of supervised learning training requires paired training data. Collecting a large amount of training data means that the corresponding cost will be high, and it is often difficult to obtain paired data, i.e., one-to-one paired pre-transformation image and post-transformation image in terms of pixel accuracy.

[0007] Although the fitting method based on the device mechanism does not require a large number of training images, there is currently no modeling method for the internal mechanism of devices such as printers and projectors. SUMMARY

[0008] Embodiments of the present application provide an image processing method, related device and storage medium, which can reduce the collection cost of training data for training a color transformation prediction model, improve the color transformation prediction efficiency, reduce the color change of the adversarial image caused by the input and output processing of different physical devices, and reduce the influence on the stability of the adversarial image test.

[0009] In a first aspect, embodiments of the present application provide an image processing method, which comprises:

[0010] obtaining a first image and a second image, wherein the first image and the second image are both digital images, the second image is obtained based on a third image by a first device, the third image is output based on the first image by a second device, the first image comprises at least one color region, and the values of each pixel in the one color region in the same channel of a same color space are all the same;

[0011] obtaining at least one target color block from the second image based on at least one color region of the first image, wherein the target color block corresponds to the color region one by one;

[0012] obtaining a correction pixel of the target color block, wherein the correction pixel is estimated based on the values of each pixel of the target color block in each channel of a preset color space;

[0013] mapping each color region in the first image and each correction pixel respectively to obtain a target data set, wherein the color region and the correction pixel correspond to each other one by one, the target data set comprises a mapping relationship between the color region and the correction pixel, and the target data set is used for training a color transformation prediction model.

[0014] In a possible design, the target color block is sampled from the second image based on the associated color region in the first image.

[0015] In a possible design, the values of each channel of the preset color space of the correction pixel are determined based on the median of the values of each channel of the preset color space of each pixel of the target color block.

[0016] In a second aspect, embodiments of the present application provide an image processing method, which comprises:

[0017] obtaining a fourth image;

[0018] obtain a fifth image based on each pixel in the fourth image and a mapping relationship in a target data set, wherein each pixel in the fifth image corresponds to each pixel in the fourth image one by one, and each pixel in the fifth image is independently transformed from each pixel in the fourth image according to the mapping relationship;

[0019] The mapping relationship includes a mapping relationship between a color region and a correction pixel, and the target data set is obtained according to the method of the first aspect. The target data set is used to train a color transformation prediction model.

[0020] In one possible design, the fifth image is obtained based on the fourth image input by a preset color transformation prediction model;

[0021] The preset color transformation prediction model is trained based on the target data set, and the machine learning model obtains the fifth image based on each pixel in the fourth image.

[0022] In one possible design, the preset color transformation prediction model is a deep convolutional neural network model, the deep convolutional neural network model includes at least one convolutional layer, and the convolutional layer adopts a specific convolutional kernel to fit an independent mapping relationship between a single pixel in each color region in the first image and each correction pixel.

[0023] In one possible design, the preset color transformation prediction model is a gradient boosting decision tree model, wherein the number of gradient boosting decision trees in the gradient boosting decision tree model is consistent with the number of preset color channels of a preset color space in the target data set, and each gradient boosting decision tree independently fits an independent mapping relationship between a single pixel in each color region in the first image in a preset color channel and each correction pixel in the preset color channel.

[0024] In one possible design, the fourth image is a candidate adversarial image generated by a preset device;

[0025] The fifth image is obtained based on the fourth image and a forward mapping relationship in the target data set, is used to input a target model, and obtains an identification result; wherein the identification result is used to feed back to the preset device to determine whether the candidate adversarial image meets a preset condition.

[0026] In one possible design, the fourth image is a target adversarial image generated by a preset device;

[0027] The fifth image is obtained based on the fourth image and a reverse mapping relationship in the target data set, is used to input a target model, and is used to test the security of the target model.

[0028] In a third aspect, an embodiment of the present application provides an image processing method, which comprises:

[0029] obtaining a first candidate adversarial image, wherein the first candidate adversarial image is obtained based on an updated historical candidate adversarial image, and the historical candidate adversarial image comprises an original image;

[0030] performing color transformation processing on the first candidate adversarial image based on a forward mapping relationship in a target data set to obtain a second candidate adversarial image, wherein the target data set is obtained according to the method of the first aspect, the forward mapping relationship comprises a forward mapping relationship of a color region and a correction pixel, and the target data set is used to train a color transformation prediction model;

[0031] obtaining an identification similarity between the second candidate adversarial image and a target image;

[0032] if the identification similarity does not meet a preset condition, updating the first candidate adversarial image and the second candidate adversarial image until the identification similarity between the second candidate adversarial image and the target image meets the preset condition, and taking the first candidate adversarial image when the identification similarity meets the preset condition as a target adversarial image.

[0033] In one possible design, the adversarial attack comprises a targeted attack and a non-targeted attack.

[0034] In a case where the adversarial attack is the targeted attack, the preset condition comprises that the identification similarity is greater than a first preset value.

[0035] In a case where the adversarial attack is the non-targeted attack, the target image comprises the original image, and the preset condition comprises that the identification similarity is less than a second preset value.

[0036] In a fourth aspect, an embodiment of the present application provides an image processing method, which comprises:

[0037] obtaining a third candidate adversarial image, wherein the third candidate adversarial image is obtained based on an updated historical candidate adversarial image, and the historical candidate adversarial image comprises an original image;

[0038] obtaining an identification similarity between the third candidate adversarial image and a target image;

[0039] if the identification similarity does not meet a preset condition, updating the third candidate adversarial image until the identification similarity between the third candidate adversarial image and the target image meets the preset condition, and taking the third candidate adversarial image when the identification similarity meets the preset condition as a target adversarial image;

[0040] perform color transformation processing on the target adversarial image based on a reverse mapping relationship in a target data set, to obtain a source digital adversarial image, wherein the target data set is obtained according to the method of the first aspect, the reverse mapping relationship includes a reverse mapping relationship of a color region and a correction pixel, and the target data set is used to train a color transformation prediction model;

[0041] The source digital adversarial image is used to be transformed into the target adversarial image after being output by a physical device and being input processed.

[0042] In one possible design, the adversarial attack includes a targeted attack and a non-targeted attack.

[0043] When the adversarial attack is the targeted attack, the preset condition includes that the recognition similarity is greater than a first preset value.

[0044] When the adversarial attack is the non-targeted attack, the target image includes an original image, and the preset condition includes that the recognition similarity is less than a second preset value.

[0045] In a fifth aspect, an embodiment of the present application provides an image processing apparatus having a function of implementing the image processing method provided in the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, which can be software and / or hardware.

[0046] In one embodiment, the image processing apparatus includes:

[0047] An input and output module configured to acquire a first image and a second image, wherein the first image and the second image are both digital images, the second image is acquired by a first device based on a third image, the third image is output by a second device based on the first image, the first image includes at least one color region, and values of each pixel in the one color region in the same channel of a same color space are all the same.

[0048] A processing module configured to acquire at least one target color block from the second image based on at least one color region of the first image, wherein the target color block corresponds to the color region in a one-to-one manner.

[0049] The processing module is further configured to acquire a correction pixel of the target color block, wherein the correction pixel is estimated based on values of each pixel of the target color block in each channel of a preset color space; and

[0050] Map each color region in the first image to each of the correction pixels respectively to obtain a target data set, wherein the color regions and the correction pixels are one-to-one corresponding, and the target data set comprises a mapping relationship between the color regions and the correction pixels, and is used for training a color transformation prediction model.

[0051] In a sixth aspect, an embodiment of the present application provides an image processing apparatus having a function of implementing the image processing method provided in the second aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software comprises one or more modules corresponding to the function, and the module can be software and / or hardware.

[0052] In an embodiment, the image processing apparatus comprises:

[0053] An input and output unit configured to acquire a fourth image;

[0054] A processing unit configured to acquire a fifth image based on each pixel in the fourth image and a mapping relationship in a target data set, wherein each pixel in the fifth image corresponds to each pixel in the fourth image one-to-one, and each pixel in the fifth image is independently transformed from each pixel in the fourth image according to the mapping relationship.

[0055] The target data set is obtained by the apparatus of the third aspect.

[0056] In a seventh aspect, an embodiment of the present application provides a computer readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform the image processing method of any one of the first aspect to the fourth aspect.

[0057] In an eighth aspect, an embodiment of the present application provides a computing device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the image processing method of any one of the first aspect to the fourth aspect when executing the computer program.

[0058] Compared to existing technologies, in this embodiment, a mapping relationship is established between the color transformations of digital images (first image and second image) before and after physical device output and input processing operations (target dataset). Furthermore, instead of simply averaging the pixels of a color region after transformation as in existing technologies, the colors in this embodiment are robustly estimated as the original color after transformation. Therefore, compared to simple averaging in existing technologies, the color transformation prediction based on this mapping relationship is more robust, and the estimated color transformation results more closely match the actual transformation situation after physical device output and input processing, reducing noise and further improving the accuracy of the subsequently trained color transformation prediction model. In addition, when establishing the color transformation mapping relationship of digital images before and after physical device output and input processing operations, this application considers each pixel in the image independently. That is, a pixel after color transformation only establishes a mapping relationship with its corresponding pixel before color transformation, rather than all pixels in the image. This reduces the amount of data computation and noise interference during the calculation process, improving prediction efficiency and accuracy. In addition, this application provides an image processing method suitable for generating adversarial images. It takes into account that when the generated digital adversarial images are used to attack the target model, they need to go through the physical device output and input processing, which may cause color changes. When iteratively updating the digital adversarial images, the recognition result of the target model is used as a reference, so that the display effect of the directly generated digital adversarial images presented to the target model after going through the physical device output and input processing is consistent with the expectation, thereby improving the stability of the target model in attack testing of the generated adversarial images. Attached Figure Description

[0059] The objectives, features, and advantages of the embodiments of this application will become readily understood by referring to the accompanying drawings and the detailed description of the embodiments. Wherein:

[0060] Figure 1 This is a schematic diagram of an image processing system according to an embodiment of the image processing method in this application;

[0061] Figure 2 This is a schematic diagram of another image processing system according to the image processing method in the embodiments of this application;

[0062] Figure 3 This is a schematic diagram of another image processing system according to the image processing method in the embodiments of this application.

[0063] Figure 4 This is a schematic flowchart of an image processing method according to an embodiment of this application;

[0064] Figure 5This is a schematic diagram of the transformation and sampling process of the self-made color card in an embodiment of this application;

[0065] Figure 6 This is a flowchart illustrating another image processing method according to an embodiment of this application;

[0066] Figure 7 This is a schematic diagram illustrating the difference between the image predicted by color transformation and the actual transformed image in an embodiment of this application.

[0067] Figure 8 This is a schematic flowchart of an image processing method for generating adversarial images according to an embodiment of this application;

[0068] Figure 9 This is a comparison chart showing the effect of using a color transformation prediction model to generate physical-world adversarial images according to an embodiment of this application;

[0069] Figure 10 This is a schematic flowchart of another image processing method for generating adversarial images according to an embodiment of this application;

[0070] Figure 11 This is a schematic diagram of the structure of an image processing apparatus according to an embodiment of this application;

[0071] Figure 12 This is a schematic diagram of the structure of another image processing apparatus according to an embodiment of this application;

[0072] Figure 13 This is a schematic diagram of the structure of a computing device according to an embodiment of this application;

[0073] Figure 14 This is a schematic diagram of the structure of a mobile phone in one embodiment of this application;

[0074] Figure 15 This is a schematic diagram of a server structure in one embodiment of this application.

[0075] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0076] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects (e.g., the first image and the second image represent different images, and so on), and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The division of modules in the embodiments of this application is merely a logical division; in actual applications, there may be other division methods. For example, multiple modules may be combined into or integrated into another system, or some features may be omitted or not performed. Additionally, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interface, indirect coupling between modules, or electrical or other similar forms of communication connection, none of which are limited in the embodiments of this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed among multiple circuit modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of this application.

[0077] This application provides an image processing method, related apparatus, and storage medium, which can be applied to an image processing system. When the image processing method is implemented based on this image processing system, training data for training a color transformation prediction model can be obtained, or the data can be used to predict the color transformation of an image, or to generate adversarial images. These are described below:

[0078] When this image processing method is used to obtain training data for training a color transformation prediction model, the image processing system may include a first image processing device, a first device, and a second device. The first device is used to acquire data based on a third image to obtain a second image, and the second device is used to perform output processing based on the first image to obtain the third image. The first image processing device is at least used to acquire the first image and the second image, and to generate a target dataset based on the first image and the second image. The first image processing device may be an application that acquires the first image and the second image and generates a target dataset based on the first image and the second image, or a server with the application installed that acquires the first image and the second image and generates a target dataset based on the first image and the second image; the second device may be an image output device such as a printer or projector; and the first device may be a camera, a scanner, or a terminal device integrated with an image acquisition module.

[0079] When the image processing method is used to predict the color transformation of a digital image after it has undergone output and input processing by a physical device, the image processing system may include a second image processing device. This second image processing device is at least used to acquire a fourth image, perform color transformation prediction based on the fourth image, and obtain a fifth image. The second image processing device may be an application program that acquires the fourth image, performs color transformation prediction based on the fourth image, and obtains the fifth image, or a server that has installed the application program that acquires the fourth image, performs color transformation prediction based on the fourth image, and obtains the fifth image.

[0080] When the image processing method is used to generate adversarial images, the image processing system may include a third image processing device and a recognition device. The third image processing device and the recognition device may be deployed integratedly or separately. The image processing device is at least used to update candidate adversarial images and generate target adversarial images based on recognition results fed back by the recognition device. The recognition device is used to recognize the candidate adversarial images after color transformation processing to obtain recognition results. At least one recognition result (such as a recognition probability distribution) obtained by the recognition device can be used by the image processing device to iteratively update the candidate adversarial images, for example, to update the pixel values ​​of the adversarial images or adversarial perturbations. The image processing device may be an application for updating candidate adversarial images and generating target adversarial images, or a server with the application for updating candidate adversarial images and generating target adversarial images installed; the recognition device may be a recognition program for recognizing candidate adversarial images after color transformation processing to obtain recognition results, the recognition program being, for example, an image recognition model, and the recognition device may also be a terminal device with an image recognition model deployed.

[0081] The solutions provided in this application involve technologies such as Artificial Intelligence (AI), Computer Vision (CV), and Machine Learning (ML), which are specifically illustrated through the following embodiments:

[0082] AI, or Artificial Intelligence, refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, Artificial Intelligence is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine capable of reacting in a manner similar to human intelligence. Artificial Intelligence studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0083] AI technology is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0084] Computer vision (CV) is the science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes for target recognition, tracking, and measurement, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include adversarial perturbation generation, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0085] In existing technologies, color correction is often only applied to color errors caused during image acquisition. For example, Chinese patent application CN 111062876 A only corrects the color difference between the captured image and the real object, without considering that errors also occur after the digital image undergoes output and input processing, affecting the intended display effect of the digital image itself. Furthermore, when correcting the color difference between the digital image and the real object, Chinese patent application CN 111062876 A is based on a pre-established conventional neural network model. However, the fully connected layers in the neural network model treat all pixels in the image uniformly, which destroys the original spatial structure of the image. This is equivalent to considering all pixels in the entire image when correcting the color of a single pixel, introducing excessive noise into the processing, wasting resources, and resulting in low efficiency.

[0086] Compared with existing technologies, in this embodiment, a mapping relationship is established between the color transformations of digital images (first image and second image) before and after physical device output and input processing operations (target dataset). Furthermore, instead of simply averaging the pixels of a color region after transformation as in existing technologies, the colors in this embodiment are robustly estimated as the original color after transformation. Therefore, compared with the simple averaging in existing technologies, the color transformation prediction based on this mapping relationship is more robust, and the estimated color transformation results more closely match the actual transformation situation after physical device output and input processing, reducing noise and further improving the accuracy of the subsequently trained color transformation prediction model. In addition, when establishing the color transformation mapping relationship of digital images before and after physical device output and input processing operations, this application considers each pixel in the image independently. That is, a pixel after color transformation only establishes a mapping relationship with its corresponding pixel before color transformation, rather than all pixels in the image. This reduces the amount of data computation and noise interference during the calculation process, improving prediction efficiency and accuracy. In addition, this application provides an image processing method suitable for generating adversarial images. It takes into account that when the generated digital adversarial images are used to attack the target model, they need to go through the physical device output and input processing, which may cause color changes. When iteratively updating the digital adversarial images, the recognition result of the target model is used as a reference, so that the display effect of the directly generated digital adversarial images presented to the target model after going through the physical device output and input processing is consistent with the expectation, thereby improving the stability of the target model in attack testing of the generated adversarial images.

[0087] The image processing method provided in this application embodiment can be based on Figures 1-3 The image processing system shown is implemented.

[0088] In some implementations, reference is made to Figure 1 , can be based on Figure 1 The image processing system shown processes images to obtain a target dataset for training a color change model. This image processing system may include a server 01, a first device 02, and a second device 03.

[0089] The server 01 can be an image processing device, in which image processing programs can be deployed. The first device 02 can be a camera. The second device 03 can be a printer. The server 01 can generate a digitized first image itself or receive a digitized first image from an external source. After obtaining the first image, it can send the first image to the second device 03. The second device 03 can print out the first image to obtain a physical third image. The first device 02 can capture the third image to obtain a digitized second image, and then send the second image to the server 01. The server 01 can process the first and second images to generate a mapping relationship between the original colors and the reconstructed and transformed colors, thereby obtaining the target dataset.

[0090] In some implementation methods, reference can be made to Figure 2 , can be based on Figure 2 The image processing system shown processes digital images and predicts the color transformations of the digital images after processing by physical devices. This image processing system may include a server 04. The server 04 can generate a digitized fourth image itself or receive a digitized fourth image from an external source. After obtaining the fourth image, it can perform color transformations on the fourth image according to the mapping relationship in the target dataset to obtain a fifth image. In some embodiments, a color transformation prediction model can be trained in advance based on the mapping relationship in the target dataset, and then the fourth image can be input into the color transformation prediction model to obtain the fifth image.

[0091] In some implementations, reference is made to Figure 3 , can be based on Figure 3 The image processing system shown processes the original image to obtain an adversarial image that retains its attack testing performance even after physical device input and output processing. This image processing system may include a server 05 and a terminal device 06.

[0092] The server 05 can be an image processing device in which image processing programs, such as adversarial image generation programs, can be deployed.

[0093] The terminal device 06 can be a recognition device, which can be equipped with a recognition model, such as an image recognition model trained using machine learning methods. This image recognition model can be a face recognition model, a vehicle recognition model, or an animal recognition model, etc.

[0094] Server 05 can receive the original image from an external source, and then iteratively update the candidate adversarial image (fourth image) based on the original image to achieve the adversarial attack target. This candidate adversarial image is then color-transformed (fifth image) and sent to terminal device 06. Terminal device 06 can process the color-transformed candidate adversarial image (fifth image) using a recognition model to obtain a recognition result, such as a probability distribution, and then feed the recognition result back to server 05. Server 05 can determine the recognition similarity between the candidate adversarial image and the target image based on the recognition result, and determine whether the candidate adversarial image can achieve the attack target. If it can achieve the attack target, it is identified as the target adversarial image.

[0095] Alternatively, in some embodiments, server 05 can receive the original image from an external source, and then iteratively update candidate adversarial images based on the original image to achieve the adversarial attack target, and send the candidate adversarial images to terminal device 06. Terminal device 06 can process the candidate adversarial images using a recognition model to obtain recognition results, such as a recognition probability distribution, and then feed back the recognition results to server 01. Server 05 can determine the recognition similarity between the candidate adversarial image and the target image based on the recognition results, and determine whether the candidate adversarial image can achieve the attack target. If it can achieve the attack target, it is identified as the target adversarial image. Then, based on the inverse mapping relationship in the target dataset, server 05 can perform color transformation processing on the target adversarial image to obtain what kind of digital target adversarial image should be output to the physical device in order to present the target adversarial image to the recognition model.

[0096] It should be noted that the server involved in the embodiments of this application can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0097] The terminal devices involved in the embodiments of this application can be devices that provide voice and / or data connectivity to users, handheld devices with wireless connectivity, or other processing devices connected to a wireless modem. Examples include mobile phones (or "cellular" phones) and computers with mobile terminals, such as portable, pocket-sized, handheld, computer-embedded, or vehicle-mounted mobile devices that exchange voice and / or data with a wireless access network. Examples include Personal Communication Service (PCS) phones, cordless phones, Session Initiation Protocol (SIP) phones, Wireless Local Loop (WLL) stations, Personal Digital Assistants (PDAs), and other similar devices.

[0098] Reference Figure 4 , Figure 4 This is a schematic flowchart illustrating an image processing method provided in an embodiment of this application. The method can be... Figure 11 The image processing device shown is applied to a color reconstruction transformation modeling scenario, processing the acquired first and second images to obtain a target dataset for training a color transformation prediction model. The image processing method includes:

[0099] Step S110: Obtain the first image and the second image.

[0100] In this embodiment, the second image is obtained by reconstructing and transforming the first image. The reconstruction and transformation refers to obtaining another digital image (the second image) from a first digital image output by one physical device and then acquired by another physical device. The second image differs from the first image in that its colors have changed.

[0101] Reference Figure 1 The second image can be acquired by the first device based on the third image. The first device can be any image acquisition device, such as a camera, scanner, or terminal device with an integrated image acquisition module. The third image can be output by the second device based on the first image. The first image can be the most original image, and the second device can be an output device such as a printer, projector, monitor, or television.

[0102] Therefore, the reconstruction transformation in the embodiments of this application includes various methods, which may be one of the following examples:

[0103] 1. Digital image A, and its digital image B, which was printed and then scanned;

[0104] 2. Digital image A, and its digital image B, which was printed and then photographed;

[0105] 3. Digital image A, and digital image B, which was captured after being played on a television screen;

[0106] 4. Digital image A, and digital image B, which is projected onto a screen and then photographed.

[0107] In this embodiment, the purpose of acquiring the first image and the second image is to establish the color transformation relationship after the image has been reconstructed and transformed by a specific physical device, so as to predict the effect image presented after the digital image reconstruction and transformation. To more efficiently model the aforementioned color transformation relationship, in this embodiment, the first image can be set to include multiple color regions, and each color region is a pure color; the sizes of the multiple different color regions in the first image can be the same or different. For example, if the color of a color region A is blue, then all pixels in this color region A are the same and blue. Considering that in the field of computer vision, the color of a pixel can be quantitatively described by the values ​​of each color channel in a certain color space, it can be set that the values ​​of each pixel in any color region of the first image in the same channel of the same color space are the same.

[0108] Considering that the original image for which color transformation prediction is required may contain few colors, a first image can be generated based on the prediction requirements. That is, the first image includes all the colors in the original image, and each color is assigned a color region, with each color region including the same number of pixels.

[0109] Considering that the number of colors commonly used in practical applications is limited, and that existing technologies often use color charts (also known as color tables) as tools for unifying color standards, in order to establish a more unified and reusable target dataset and color transformation prediction model, in one possible design, a standard color chart (e.g., the Macbeth color chart) can be set as the first image. The standard color chart is then processed based on the desired modeling reconstruction transformation (first device and second device) to obtain the second image. An example of the second image can be seen in the figure.

[0110] Understandably, in some scenarios, users may expect the color variations in the model to be coarser or finer than those in the color table, i.e., different in color granularity. Therefore, to better suit user needs, in one possible design, the user can create the first image according to their actual needs. This first image is obtained by traversing each channel of a preset color space at preset numerical intervals, and each combination of color channel values ​​accessed during the traversal forms a color region.

[0111] In this design, we take a user-created first image (color chart) as an example. For ease of description, the color before transformation will be denoted as variable X in the following text.

[0112] By traversing each color channel in the RGB color space at intervals of 16, that is, uniformly sampling the colors of the R, G, and B channels in the RGB color gamut, 4096 colors can be obtained, which are the colors before transformation, denoted as variable X∈R. 3 The RGB values ​​of these colors are as follows:

[0113] (0,0,0),(0,0,16)...(0,0,240),(0,16,0),(0,16,16)...(0,16,240),(0,32,0)...(16,0,0)...(240,240,240).

[0114] On multiple color table templates of preset sizes (e.g., 1320 pixels x 880 pixels), 40-pixel x 40-pixel square color blocks are filled with the original color. These square color blocks are neatly arranged on the color table templates to obtain the user-made original color table (first image), such as... Figure 5 As shown in image a; then, a reconstruction transformation is performed to obtain the transformed color table (second image), as shown in image a. Figure 5 As shown in b. Although this application embodiment only provides one method for creating the color table before transformation, the acquisition of the color table before transformation is not limited to this method.

[0115] Step S120: Based on at least one color region of the first image, obtain at least one target color patch from the second image.

[0116] In this embodiment of the application, in order to model the mapping relationship between each color in the first image before and after reconstruction transformation, it is necessary to obtain a target color block for each color region, that is, the target color block corresponds one-to-one with the color region.

[0117] In this embodiment, the target color patch can be sampled from the second image based on the color region associated with the first image. Specifically, since the second image is obtained by reconstructing and transforming the first image, if the first image includes three color regions A, B, and C, then the second image should also include a color region A' with the same position and size as color region A, a color region B' with the same position and size as color region B, and a color region C' with the same position and size as color region C. Therefore, in order to establish the mapping relationship before and after each color reconstruction transformation, the target color patch can be obtained from each color region in the second image. For example, a sub-region image (target color patch) of a preset size can be extracted from the center region of each color region to sample the value of each channel of the sub-region image in the preset color space.

[0118] Step S130: Obtain the corrected pixels of the target color block.

[0119] In this embodiment of the application, the corrected pixel is estimated based on the numerical values ​​of each pixel of the target color block in each channel of a preset color space.

[0120] In this embodiment, considering that the color changes of pixels in the same color region may not be consistent after undergoing the same reconstruction transformation, for example, during the reconstruction transformation process, pixels at different positions in the same color region may be affected by factors such as uneven lighting, uneven paper quality, and uneven printing effects, resulting in differences in the colors exhibited by the pixels after reconstruction transformation. To minimize the influence of these factors, reduce noise in the target dataset, and reduce interference with subsequent modeling and prediction, a robust estimation method is used to estimate the corrected pixel based on the target color patch in one possible design. Specifically, the value of the corrected pixel in each channel of the preset color space can be determined by the median value of each pixel in each channel of the preset color space. Compared with the simple averaging in the prior art, the color transformation prediction based on this mapping relationship in this embodiment is more robust, the estimation result of the color transformation more closely matches the actual transformation situation after the physical device processes the color input and output, reduces noise, and is more helpful in improving the accuracy of the subsequently trained color transformation prediction model.

[0121] For example, pixel sampling is performed on the target color block to obtain the transformed color, denoted as variable Y∈R3. In this embodiment, a small square color block can be extracted from the center of the target color block, and the median of each pixel in the R, G, and B channels of this small square color block can be used to estimate the color value of the corresponding position in the second image in that channel (i.e., the corrected pixel).

[0122]

[0123]

[0124]

[0125] In the formula, Rest, Gest, and Best represent the estimated values ​​of the R, G, and B channels, respectively; Ri, Gi, and Bi represent the actual values ​​of the R, G, and B channels of the i-th sampled pixel, respectively; N is the total number of sampled pixels; and median(.) represents the median statistic for one-dimensional data.

[0126] In this embodiment of the application, after reconstruction and transformation based on the user's self-made color table, the sampled target color patch is as follows: Figure 5 As shown in c, the purpose of sampling is to ensure that the sampling points are all in the center of the target color block. Other sampling methods, such as random sampling, do not affect the effectiveness of the embodiments of this application.

[0127] In other possible designs, other robust statistical methods can be used to estimate the color of the target color patch. The key is to reduce noise by increasing the amount of data, that is, by performing robust estimation on multiple transformed colors (each pixel in the target color patch) of the same original color, to reduce annotation noise and improve the annotation accuracy of the transformed color. The key to robust estimation is to make the estimated value insensitive to noise outliers, with the aim of noise reduction, thereby avoiding noise introduced by factors such as uneven lighting, uneven paper quality, and uneven printing effects into the transformed image. Other robust estimation methods, such as quartile estimation, or conversion to other color spaces and then performing robust estimation in the new color space, do not affect the effectiveness of the embodiments of this application.

[0128] Step S140: Map each color region in the first image to each of the corrected pixels to obtain the target dataset.

[0129] The color regions and the correction pixels correspond one-to-one. The target dataset includes the mapping relationship between the color regions and the correction pixels. The target dataset is used to train the color transformation prediction model.

[0130] In this embodiment, the color table before transformation (first image) is created from the color X (multiple pure color regions) before transformation, while the color Y (corrected pixels) after transformation is estimated from the color table after transformation (second image). Furthermore, there is a one-to-one correspondence between the color table before transformation and the color table after transformation (each pixel in the first and second images). Therefore, in this embodiment, the color X before transformation and the color Y after transformation can be matched one-to-one according to this one-to-one correspondence to obtain a pair (X,Y), which is then integrated into a target dataset.

[0131] Steps S110 to S140 constitute the data acquisition section. The acquisition results in the pairing of the original color X and the transformed color Y. The original color X covers most of the original colors that need to be modeled, and the transformed color Y is the estimated value of the transformed color obtained from the acquisition. Next, the image color transformation can be predicted and the color transformation prediction model can be trained based on the mapping relationship between the original color X and the transformed color Y (i.e., the positive mapping relationship between the colors of the first image and the second image).

[0132] Understandably, in one possible design, a mapping relationship between the transformed color Y and the original color X can also be established (i.e., the inverse mapping relationship between the colors of the first image and the second image) so as to predict and reconstruct the original color based on an image.

[0133] It is understood that although the above embodiments only use the RGB color space as an example to introduce image transformation, pixel robust estimation, and the establishment of the target dataset, those skilled in the art can obtain the mapping relationship between the first image and the second image in any other color space according to actual needs, and the embodiments of this application do not limit this. In addition, in order to further increase the amount of data in the target dataset and improve the utilization efficiency of the first image and the second image, a target dataset can be generated for each color space in the prior art, or the first image and the second image can be converted from the first color space to the second color space before establishing the target dataset.

[0134] The image processing method for generating the target dataset in this application embodiment can improve the model's efficiency in utilizing training data by assuming that the color changes of each pixel in the transformed image are independent and identically distributed. At the same time, in the target dataset production stage, robust estimation of multiple transformed colors of the same pre-transformation color is performed to reduce annotation noise and improve the annotation accuracy of the transformed colors.

[0135] Reference Figure 6 , Figure 6 This is a schematic flowchart illustrating an image processing method provided in an embodiment of this application. The method can be... Figure 12 The image processing apparatus shown is executed in a color reconstruction transformation prediction scenario, processing the acquired fourth image to obtain a fifth image after color transformation of the fourth image. The image processing method includes:

[0136] Step S210: Obtain the fourth image.

[0137] In this embodiment of the application, the fourth image is a digital image, which may be generated by the image processing device itself, such as an adversarial image; or it may be sent to the image processing device by other devices, requiring the image processing device to perform color transformation prediction.

[0138] Step S220: Based on each pixel in the fourth image and the mapping relationship in the target dataset, obtain the fifth image.

[0139] In this embodiment, to improve color transformation prediction efficiency and ensure that the color transformation of each pixel is not affected by the overall image, each pixel in the fifth image corresponds one-to-one with each pixel in the fourth image, and each pixel in the fifth image is independently transformed from each pixel in the fourth image according to the mapping relationship. Specifically, assuming the pixel in the upper left corner of the fourth image is p1, the color of pixel p1 after reconstruction can be determined simply by looking up the mapping relationship of the color of pixel p1 (i.e., the value of each color channel in the preset color space) in the target dataset, and then the pixel p1 in the upper left corner of the fifth image is determined based on the color. It is understood that the fifth image and the fourth image have the same size, and each pixel in the fifth image is obtained by color transformation of the corresponding pixel in the fourth image.

[0140] In this embodiment of the application, in order to further reduce the noise of color transformation prediction, the target dataset is obtained according to the method described in the above embodiments, and the mapping relationship includes the mapping relationship between color regions and correction pixels.

[0141] To improve the efficiency and reusability of color transformation prediction, in one possible design, the fifth image is obtained by a preset color transformation prediction model based on the input of the fourth image; wherein, the preset color transformation prediction model is trained based on the target dataset, and the machine learning model obtains the fifth image based on each pixel in the fourth image.

[0142] This application provides two possible designs for illustrating how to build a preset machine learning model based on the target data model and perform color transformation prediction.

[0143] In design ①, the preset color transformation prediction model is a deep convolutional neural network model. The deep convolutional neural network model includes at least one convolutional layer, and the convolutional layer uses a specific convolutional kernel to fit the independent mapping relationship between individual pixels in each color region of the first image and each of the corrected pixels.

[0144] In design ①, a machine learning model f can be used. θTo fit the target dataset. Depending on the actual application scenario, this design can model the mapping from the color before transformation to the color after transformation by fitting X (first image) to Y (second image), or by fitting Y (second image) to X (first image) to model the mapping from the color after transformation to the color before transformation.

[0145] It is understood that using different machine learning methods for modeling and different learning methods for solving model parameters does not affect the effectiveness of the embodiments of this application. In the embodiments of this application, other neural networks can also be used to model the mapping from X to Y.

[0146] In this design, a four-layer deep convolutional neural network model can be built, with each layer using a 1×1 convolutional kernel to locally model pixel-to-pixel mappings. In this design, the number of channels in the four convolutional layers can be 768, 768, 768, and 3, respectively. Each convolutional layer can use group convolution with 3 groups to reduce the number of parameters, thereby reducing the model size and the number of parameters to be solved. Furthermore, the activation function for each convolutional layer can be tanh. Of course, in practical applications, the number of convolutional layers and channels is not limited to the above; this is just one example.

[0147] It should be noted that the key to this design is to use 1×1 convolution to locally model the pixel-to-pixel mapping in order to fit the independent mapping relationship between individual pixels in each color region of the first image and each of the corrected pixels, rather than all pixels in the image. This reduces the amount of data computation and noise interference during the computation process, and improves prediction efficiency and prediction accuracy.

[0148] In this design, the deep convolutional neural network model can be trained by solving the following objective function:

[0149]

[0150] Where l represents the L1 loss function, x i ,y i These are the values ​​of x and y for the i-th pair, respectively.

[0151] To improve the stability of model predictions, spectral normalization can be used to stabilize training in this design. The parameter for spectral normalization is set to λ, and the optimizer used for training can be Adam, BGD, SGD, or MBGD, with a learning rate set to 1e^(-λ / 2). -3 The weight decay rate can be set to 1e. -4 .

[0152] Once the model training is complete, it can be tested by inputting the image before transformation (fourth image) into the color transformation prediction model to obtain the transformed image (fifth image), or by inputting the transformed image (fourth image) into the color transformation prediction model to obtain the image before transformation (fifth image).

[0153] In this design, let the image before transformation (fourth image) and the image after transformation (fifth image) be Ix, Iy∈R, respectively. H ×W×3 Where H and W are the height and width of the image, respectively, a color transformation prediction model is used for prediction:

[0154]

[0155] Where i and j are the pixels in the i-th row and j-th column.

[0156] Without causing confusion, I can also be used in this design. y =F θ (I x ) to represent the fourth image I x The process of performing prediction processing. Specifically, for f... θ As an example of a convolutional neural network, the prediction calculation of the color transformation prediction model mentioned above can be efficiently implemented using current high-performance deep learning computing libraries (such as TensorFlow and PyTorch), thereby improving the computation speed.

[0157] It is understood that the color transformation prediction of the image in the embodiments of this application can be based on the RGB color space of the above embodiments, or it can be the Lab space, YUV space, or YC space which decouples luminance and chromaticity. b C r Space, HSV space.

[0158] In one possible design, the variables X and Y in the above embodiment can be transformed from the RGB color space to a color space decoupled by luminance and chromaticity, denoted by the transformation function f. trans (.), its inverse function is The key to this design lies in using colorimetry theory to preprocess the data, separating luminance and chromaticity to improve the efficiency of the color transformation prediction model in using the target dataset. Other possible designs that decouple luminance and chromaticity using different color spaces do not affect the effectiveness of the embodiments described in this application.

[0159] To predict the color transformation of an image in the chromaticity channel, after obtaining the first image data and the second image, we can first preprocess the images, that is, transform them. Let the colors in the transformed image be:

[0160] X*=f trans (X)

[0161] Y*=f trans (Y)

[0162] Next, we can perform extraction processing on the transformed image to extract the brightness and chromaticity components. Let f be the operation for extracting brightness. l Obtain the brightness before transformation and brightness after transformation Let f be the operation for extracting chroma. c Obtain the original chromaticity. and the transformed chromaticity

[0163]

[0164]

[0165] In this context, the subscript l represents the luminance dimension, and the subscript c represents the chroma dimension.

[0166] Next, the processed luminance and chromaticity are paired separately to obtain the luminance training dataset. Color training dataset

[0167] Through a machine learning model f θ To fit the chromaticity training dataset While maintaining the brightness training dataset Remaining unchanged, based on specific requirements in practice, fitting can be used... arrive To model the mapping from chromaticity before transformation to chromaticity after transformation, or by fitting... arrive To model the mapping from chromaticity after transformation to chromaticity before transformation.

[0168] Using different machine learning methods for modeling and different learning methods for solving model parameters does not affect the effectiveness of this design. In this design, a deep convolutional neural network is used for modeling. arrive Let's take the mapping as an example to introduce it.

[0169] In this design, the selected deep convolutional neural network model consists of four convolutional layers, each using a 1×1 convolutional kernel. The number of channels in these four convolutional layers are 64, 64, 64, and 2, respectively. Each layer can use group convolution with a group size of 3, and the activation function for each layer can be tanh.

[0170] In this design, the deep convolutional neural network model can be trained by solving the following objective function:

[0171]

[0172] Where l represents the L1 loss function, These are the i-th pair. and The value of .

[0173] To improve the stability of the color transformation prediction model, spectral normalization is used for training. The parameter for spectral normalization is set to λ, the optimizer used for training is Adam, and the learning rate is 1e-λ / 2. 3 The weight decay rate is 1e- 4 .

[0174] Once the model training is complete, it can be tested by inputting the image before transformation (fourth image) into the color transformation prediction model to obtain the transformed image (fifth image), or by inputting the transformed image (fourth image) into the color transformation prediction model to obtain the image before transformation (fifth image).

[0175] In this design, let the image before transformation (fourth image) and the image after transformation (fifth image) be Ix, Iy∈R, respectively. H ×W×3 Where H and W are the height and width of the image, respectively, a color transformation prediction model is used for prediction:

[0176]

[0177]

[0178]

[0179] Where i and j are the pixels in the i-th row and j-th column; [.] indicates a concatenation operation, and the superscript -1 indicates the inverse function.

[0180] Without causing confusion, I will also be used below. y =F θ (I x ) to represent the fourth image I x The process of performing prediction processing. Specifically, for f... θ As an example of a convolutional neural network, the prediction calculation of the color transformation prediction model mentioned above can be efficiently implemented using current high-performance deep learning computing libraries (such as TensorFlow and PyTorch), thereby improving the computation speed.

[0181] To verify the performance of this design, an experiment was conducted in this application embodiment to model the printing effect of an HP printer (model PageWide Managed MFP P77740dn) on Fengcai brand thick color inkjet printing paper.

[0182] For example, the first device uses the scanner from the HP Color LaserJet MFP M281fdw printer kit. The image is a randomly selected image from the COCO dataset. The result is as follows: Figure 7 As shown, Figure 7 b is the image before the transformation. Figure 7 'a' is the actual transformed image. Figure 7 c represents the result of color correction using color transformation preprocessing. Figure 7 d represents the result without color transformation preprocessing for color correction. It is evident that incorporating color transformation preprocessing improves prediction accuracy. Figure 7 The RGB value of c compared to Figure 7 d, closer to Figure 7 a.

[0183] In design ②, the preset color transformation prediction model is a gradient boosting decision tree model, wherein the number of gradient boosting decision trees in the gradient boosting decision tree model is consistent with the number of preset color channels in the preset color space of the target dataset, and each gradient boosting decision tree individually fits the independent mapping relationship between a single pixel in each color region of the first image in a preset color channel and each correction pixel in the preset color channel.

[0184] In this design, a gradient boosting decision tree can be used to fit the one-to-one matching color training dataset. While maintaining the brightness training dataset Remaining unchanged, based on specific requirements in practice, fitting can be used... arrive To model the mapping from chromaticity before transformation to chromaticity after transformation, or by fitting... arrive To model the mapping from chromaticity after transformation to chromaticity before transformation, the key to this design is that the color transformation prediction model only models chromaticity and not luminance.

[0185] Specifically, two gradient boosting decision trees can be selected to model the mapping of the two color pathways in groups. For the j-th group (j=1, 2), the model training iteratively performs the following steps, using t=1, ..., t=10 to represent the time step (round) of the iteration, with the maximum time step set to 100:

[0186] Step 1: Solve the following objective function

[0187]

[0188] Among them, we can first initialize Where l is the Huber loss; x i y i f represents the chromaticity values ​​of the i-th pair of x and y, respectively; t (.) represents the gradient boosting decision tree model at the current time step; The predicted chromaticity value is the value for the previous time step; the formula for Huber loss is:

[0189]

[0190] Where Ω is the regularization term, and its formula is:

[0191]

[0192] Where T is the number of leaf nodes corresponding to the gradient boosting decision tree, and ω is the L2 norm of the value corresponding to the leaf node.

[0193] In this design, γ can be set to 1, λ can be set to 0.1, and δ can be set to 0.5.

[0194] Step 2: Update the predicted chromaticity value

[0195]

[0196] After the color transformation prediction model is trained, a color transformation prediction test can be performed. This involves inputting the image before transformation into the color transformation prediction model to obtain the image after transformation, or inputting the image after transformation into the color transformation prediction model to obtain the image before transformation.

[0197] In this embodiment, let the new image before transformation and the image after transformation be Ix, Iy∈R, respectively. H×W×3 Where H and W are the height and width of the image, respectively, and a color transformation prediction model is used for prediction:

[0198]

[0199]

[0200]

[0201] Where i and j are the pixel chromaticity values ​​in the i-th row and j-th column; [.] indicates a concatenate operation, and the superscript -1 indicates an inverse function.

[0202] As can be seen from the above embodiments, the image processing method provided in this application is not limited to one color space and can be converted to one another. In order to systematically compare the processing effects of image color transformation prediction in different color spaces, this experiment uses gradient boosting decision tree as the color transformation prediction model and tests the prediction error on a test set of randomly sampled colors before transformation. The results are shown in Table 1. It can be seen that color space preprocessing (converting X and Y from RGB color space to a color space decoupled from luminance and chromaticity, such as Lab space, YUV space, YC_b C_r space, HSV space, etc.) can significantly improve the prediction accuracy of certain colors.

[0203] Table 1 shows the prediction error on the test set (random color chart).

[0204]

[0205] The image processing method for predicting image color transformation in this application embodiment considers each pixel in the image independently when establishing the color transformation mapping relationship of a digital image before and after the output and input processing operations of the physical device. That is, a pixel after color transformation only establishes a mapping relationship with the corresponding pixel before color transformation, rather than all pixels in the image. This reduces the amount of data calculation and noise interference in the calculation process, and improves prediction efficiency and prediction accuracy.

[0206] Reference Figure 8 , Figure 8 This is a schematic flowchart illustrating an image processing method provided in an embodiment of this application. The method can be... Figure 12 The image processing apparatus shown is executed in an adversarial image generation scenario. It can perform color transformation processing on candidate adversarial images, and a recognition device identifies the color-transformed image of the candidate adversarial image, determining whether the candidate adversarial image can be used as the target adversarial image based on the recognition result. The image processing method includes:

[0207] Step S310: Obtain the first candidate adversarial image.

[0208] In this embodiment, the first candidate adversarial image can be updated based on historical candidate adversarial images, which include the original image. When generating candidate adversarial images based on the original image, any adversarial attack method can be used, and this embodiment does not limit this.

[0209] In one possible design, the fourth image in the previous embodiment can be a first candidate adversarial image generated by a preset device; the fifth image (i.e., the second candidate adversarial image) is obtained based on the positive mapping relationship between the fourth image and the target dataset, and is used to input the target model to obtain the recognition result; wherein, the recognition result is used to feed back to the preset device to determine whether the candidate adversarial image meets the preset conditions.

[0210] Step S320: Based on the positive mapping relationship in the target dataset, perform color transformation processing on the first candidate adversarial image to obtain the second candidate adversarial image.

[0211] In this embodiment, the target dataset can be obtained using the methods described in steps S110-S140 of the above embodiments, and then the first candidate adversarial image can be processed according to the positive mapping relationship between color regions and correction pixels in the target dataset; alternatively, the target dataset can be modeled using machine learning to obtain a color transformation model, and then the color transformation prediction model can be concatenated with the adversarial image generation model to form an image processing model with color transformation and adversarial image generation functions. For example, the original image and the target image can be input into the image processing model to iteratively generate a target adversarial image that, after reconstruction transformation by a physical device, can present the same display effect in the target model as during generation.

[0212] It is understood that, in the embodiments of this application, any pixel in the second candidate adversarial image is also obtained by transforming the corresponding pixel in the first candidate adversarial image and the forward mapping relationship in the target dataset.

[0213] Step S330: Obtain the recognition similarity between the second candidate adversarial image and the target image.

[0214] In this embodiment, the recognition similarity between two images can be the feature recognition similarity between the two images, and the features can be obtained by the feature extractor of the target model. After obtaining the features of the two images, the recognition similarity can be calculated using any similarity or feature distance calculation method, such as cosine similarity, Euclidean distance, etc., and this embodiment does not limit this.

[0215] Step S340: If the recognition similarity does not meet the preset conditions, then update the first candidate adversarial image and the second candidate adversarial image until the recognition similarity between the second candidate adversarial image and the target image meets the preset conditions, and use the first candidate adversarial image when the recognition similarity meets the preset conditions as the target adversarial image.

[0216] In this embodiment, the specific content of the target image and preset conditions can be set according to the attack objective of the adversarial attack. Specifically, adversarial attacks include targeted attacks and untargeted attacks. An untargeted attack means that the recognition model's recognition result for the adversarial image is different from the recognition model's recognition result for the original image (i.e., the target image). A targeted attack can mean that the recognition model's recognition result for the adversarial image is a specific recognition result, which is the same as or not significantly different from the recognition model's recognition result for the target image.

[0217] Therefore, when the adversarial attack is a targeted attack, the embodiments of this application may require that the recognition similarity between the adversarial image and the target image is greater than a first preset value (e.g., 80%), that is, the preset condition can be that the recognition similarity is greater than the first preset value; when the adversarial attack is an untargeted attack, the embodiments of this application may require that the recognition similarity between the adversarial image and the target image (original image) is less than a second preset value (e.g., 30%), that is, the preset condition can be that the recognition similarity is less than the second preset value.

[0218] It is understood that although the embodiments of this application use numerical similarity as the update stopping condition, it is not limited to this. In other possible designs, the update iterations may reach a preset number, such as 100 times.

[0219] Next, we will use a specific example to illustrate how to combine color transformation with adversarial image generation to produce target adversarial images that are more stable in reconstructed transformations:

[0220] First, modeling arrive Mapping: F θ .

[0221] Let the model to be attacked be g, then construct an adversarial image generation function h(.) = g(F θ (.)).

[0222] The core idea behind constructing this adversarial image generation function is to first use F... θ The display effect of the candidate adversarial image after transformation is simulated, and then the prediction result of the transformed candidate adversarial image is given by g, thereby obtaining the supervision signal required for iteratively updating the candidate adversarial image.

[0223] Next, based on the input image I x Candidate adversarial images are obtained.

[0224] The candidate adversarial images can be generated using existing optimization-based adversarial image generation techniques. The following description uses a momentum-based adversarial image generation method as an example to illustrate the generation method. Assume the objective function of the momentum-based adversarial image generation method is:

[0225]

[0226]

[0227]

[0228] Where L(.) is any differentiable objective function describing the attack effect; |·|∞ is the infinite range; ∈ is the maximum possible perturbation value; ⊙ is the element-wise product of vectors; W is a mask matrix, and the domain of each element of W is {0, 1}. Elements marked as 1 mark the pixel positions that need to be optimized, and elements marked as 0 mark the pixel positions that do not need to be optimized.

[0229] The momentum and adversarial image in the iteration can be updated using the following formula:

[0230]

[0231] I = proj(I + W⊙α·sign(z))

[0232] Where z represents momentum (which can be initialized to 0); μ represents the rate of momentum decay. Let ||1| represent the gradient of the objective function relative to the model input; |·|1 is the L1 norm; I represents the variable representing the adversarial image in the iteration; proj represents projecting the variable back into the constraint range; α represents the step size of gradient descent; and sign is the sign function. During the above iterative update process, I = I0 can be initialized. x .

[0233] Thus, the target adversarial image that meets the preset conditions is obtained through iterative updates. Afterwards, it can be made into a physical entity to obtain an adversarial image of the physical world, so as to test the target model in the physical world.

[0234] To verify the performance of the adversarial images generated in the embodiments of this application, for example, this experiment uses the object detection system Faster-RCNN as the target model g to be attacked. The experiment uses a non-directional attack on the classification end of the RCNN of Faster-RCNN, and the reconstruction transformation adopts "print first, then scan", with "printing" as the method to create physical entities. Then, the attack effect of the physical world adversarial image is tested in the shooting scene.

[0235] The results are as follows Figure 9 As shown, Figure 9The attack results are shown in Figure 1, which compares the adversarial image generated using color transformation with the one generated without color transformation. Figure 9 'a' represents the result of an adversarial attack targeting the car category. Figure 9 The left image in the image was generated after color transformation, and its recognition probability was 0.61, which was significantly lower than the recognition probability of 0.81 of the adversarial image on the right that was not generated after color transformation. Figure 9 b represents the result of an adversarial attack targeting the bus category. Figure 9 The left image in the diagram was generated after color transformation, and its recognition probability is 0.81, which is significantly lower than the recognition probability of 1.0 of the adversarial image on the right that was not generated after color transformation. It can be seen that for adversarial image attacks in the physical world, the result of the prediction model with color transformation can significantly reduce the prediction confidence and improve the attack success rate.

[0236] In one possible design, a target adversarial image that can be input into the target model to achieve the attack objective can be generated first, and then the inverse mapping relationship in the target dataset can be used to obtain what kind of digital adversarial image should be acquired.

[0237] Reference Figure 10 , Figure 10 This is a schematic flowchart illustrating an image processing method provided in an embodiment of this application. The method can be... Figure 12 The image processing apparatus shown is applied to an adversarial image generation scenario. It uses a color transformation prediction model to predict the original digital adversarial image before transformation from a pre-generated target adversarial image. The image processing method includes:

[0238] Step S410: Obtain the third candidate adversarial image.

[0239] The third candidate adversarial image is obtained by updating the historical candidate adversarial image, which includes the original image. When generating the candidate adversarial image based on the original image, any adversarial attack method can be used, and this application embodiment does not limit this.

[0240] In one possible design, the fourth image in the above embodiments can be a target adversarial image generated by a preset device; the fifth image (source digital adversarial image) is obtained based on the inverse mapping relationship between the fourth image and the target dataset, and is used for the target model to test the security of the target model.

[0241] Step S420: Obtain the recognition similarity between the third candidate adversarial image and the target image.

[0242] In this embodiment, the recognition similarity between two images can be the feature recognition similarity between the two images, and the features can be obtained by the feature extractor of the target model. After obtaining the features of the two images, the recognition similarity can be calculated using any similarity or feature distance calculation method, such as cosine similarity, Euclidean distance, etc., and this embodiment does not limit this.

[0243] Step S430: If the recognition similarity does not meet the preset conditions, the third candidate adversarial image is updated until the recognition similarity between the third candidate adversarial image and the target image meets the preset conditions, and the third candidate adversarial image when the recognition similarity meets the preset conditions is used as the target adversarial image.

[0244] In this embodiment, the specific content of the target image and preset conditions can be set according to the attack objective of the adversarial attack. Specifically, adversarial attacks include targeted attacks and untargeted attacks. An untargeted attack means that the recognition model's recognition result for the adversarial image is different from the recognition model's recognition result for the original image (i.e., the target image). A targeted attack can mean that the recognition model's recognition result for the adversarial image is a specific recognition result, which is the same as or not significantly different from the recognition model's recognition result for the target image.

[0245] Therefore, when the adversarial attack is a targeted attack, the embodiments of this application may require that the recognition similarity between the adversarial image and the target image is greater than a first preset value (e.g., 80%), that is, the preset condition can be that the recognition similarity is greater than the first preset value; when the adversarial attack is an untargeted attack, the embodiments of this application may require that the recognition similarity between the adversarial image and the target image (original image) is less than a second preset value (e.g., 30%), that is, the preset condition can be that the recognition similarity is less than the second preset value.

[0246] It is understood that although the embodiments of this application use numerical similarity as the update stopping condition, it is not limited to this. In other possible designs, the update iterations may reach a preset number, such as 100 times.

[0247] Step S440: Based on the inverse mapping relationship in the target dataset, perform color transformation processing on the target adversarial image to obtain the source digital adversarial image.

[0248] In this embodiment, the target dataset can be obtained using the methods described in steps S110-S140 of the above embodiments, and then the target adversarial image can be processed according to the inverse mapping relationship between color regions and correction pixels in the target dataset; alternatively, machine learning can be used to model the inverse mapping relationship of the target dataset to obtain a color transformation model, and then the color transformation prediction model can be concatenated with the adversarial image generation model to form an image processing model with color transformation and adversarial image generation functions. For example, the original image and the target image can be input into the image processing model to iteratively generate a source digital adversarial image that, after reconstruction and transformation by physical devices, can present the same display effect in the target model as during generation.

[0249] It is understood that, in the embodiments of this application, any pixel in the source digital adversarial image is also obtained by transforming the inverse mapping relationship between the corresponding pixel in the target adversarial image and the target dataset.

[0250] The source digital adversarial image is used to transform into the target adversarial image after being processed by physical devices for output and input.

[0251] Next, we will use a specific example to show how to combine color transformation with adversarial image generation, creating another way to use adversarial images.

[0252] In this design, the color transformation prediction model models Y c * To X c * The mapping.

[0253] Let the model to be attacked be g.

[0254] Based on the original input image I x The update process is performed to obtain the target adversarial image. In this design, existing optimization-based adversarial image generation techniques can be used to generate the target adversarial image. The following section uses a momentum-based adversarial image generation method as an example to introduce the target adversarial image generation method. Assume the objective function of the momentum-based adversarial image generation method is:

[0255]

[0256]

[0257]

[0258] In this design, z = 0 and I = Ix can be initialized.

[0259] By iteratively solving the above objective function, the adversarial image of the target is obtained. Then, a color transformation prediction model f can be used. θ (.) is used to predict the target adversarial image before transformation:

[0260]

[0261] Finally, the target adversarial image before transformation can be transformed. Output through physical devices, such as printing, to obtain adversarial image entities of the physical world.

[0262] The image processing method for generating adversarial images in this application embodiment takes into account that the generated digital adversarial images need to undergo physical device output and input processing when attacking the target model, which may cause color changes. When iteratively updating the digital adversarial images, the recognition result of the target model inputting the color-changed adversarial images is used as a reference, so that the display effect of the directly generated digital adversarial images presented to the target model after undergoing physical device output and input processing is consistent with expectations, thereby improving the stability of the target model in attack testing of the generated adversarial images.

[0263] After introducing the method of the embodiments of this application, the following references are made. Figure 11 This application describes an image processing apparatus for acquiring a target dataset, which can also be applied to... Figure 1 The server 01 shown, the device 60 includes:

[0264] The input / output module 601 is configured to acquire a first image and a second image, wherein the first image and the second image are both digital images, the second image is acquired by a first device based on a third image, the third image is output by a second device based on the first image, and the first image includes at least one color region, wherein the values ​​of each pixel in the same channel of the same color space are the same.

[0265] The processing module 602 is configured to obtain at least one target color patch from the second image based on at least one color region of the first image, wherein the target color patch corresponds one-to-one with the color region;

[0266] The processing module 602 is further configured to acquire corrected pixels of the target color patch, wherein the corrected pixels are estimated based on the values ​​of each pixel of the target color patch in each channel of a preset color space; and

[0267] Each color region in the first image is mapped to each of the correction pixels to obtain a target dataset, wherein the color region and the correction pixel correspond one-to-one, and the target dataset includes the mapping relationship between the color region and the correction pixel. The target dataset is used to train a color transformation prediction model.

[0268] In some embodiments, the processing module 602 is further configured to estimate the corrected pixel based on the target color patch using a robust estimation method.

[0269] In some implementations, the target color patch is sampled from the second image based on a color region associated with the first image.

[0270] In some implementations, the values ​​of the corrected pixels in each channel of a preset color space are determined based on the median values ​​of the pixels of the target color block in each channel of the preset color space.

[0271] In some implementations, the first image is obtained by traversing each channel of a preset color space according to a preset numerical interval, and the combination of color channel values ​​visited each time during the traversal forms a color region.

[0272] The image processing apparatus for generating target datasets in this application embodiment can improve the efficiency of the model in utilizing training data by assuming that the color changes of each pixel in the transformed image are independent and identically distributed. At the same time, in the target dataset production stage, the annotation noise is reduced and the annotation accuracy of the transformed colors is improved by performing robust estimation on multiple transformed colors of the same pre-transformation color.

[0273] refer to Figure 12 This application describes an image processing apparatus for predicting color changes, which can also be applied to... Figure 2 The server 04 shown, the device 70 includes:

[0274] The input / output unit 701 is configured to acquire a fourth image;

[0275] The processing unit 702 is configured to obtain a fifth image based on each pixel in the fourth image and the mapping relationship in the target dataset, wherein each pixel in the fifth image corresponds one-to-one with each pixel in the fourth image, and each pixel in the fifth image is independently transformed from each pixel in the fourth image according to the mapping relationship;

[0276] The mapping relationship includes the mapping relationship between color regions and correction pixels. The target dataset is obtained by processing the aforementioned device 60 and is used to train a color transformation prediction model.

[0277] In some implementations, the fifth image is obtained by a preset machine learning model based on the fourth image as input;

[0278] The preset machine learning model is trained based on the target dataset, and the machine learning model obtains the fifth image based on each pixel in the fourth image.

[0279] In some implementations, the preset machine learning model is a deep convolutional neural network model, which includes at least one convolutional layer and uses a specific convolutional kernel to fit the independent mapping relationship between individual pixels in each color region of the first image and each of the corrected pixels.

[0280] In some implementations, the preset machine learning model is a gradient boosting decision tree model, wherein the number of gradient boosting decision trees in the gradient boosting decision tree model is consistent with the number of preset color channels in the preset color space of the target dataset, and each gradient boosting decision tree individually fits the independent mapping relationship between a single pixel in each color region of the first image in a preset color channel and each corrected pixel in the preset color channel.

[0281] The image processing apparatus for predicting image color transformation in this application embodiment considers each pixel in the image independently when establishing the color transformation mapping relationship of a digital image before and after the physical device output and input processing operation. That is, a pixel after color transformation only establishes a mapping relationship with the corresponding pixel before color transformation, rather than all pixels in the image. This reduces the amount of data calculation and noise interference in the calculation process, and improves prediction efficiency and prediction accuracy.

[0282] In some implementations, the fourth image is a candidate adversarial image generated by a preset device;

[0283] The fifth image is obtained based on the positive mapping relationship between the fourth image and the target dataset, and is used as input to the target model to obtain the recognition result; wherein, the recognition result is used to feed back to the first device to determine whether the candidate adversarial image meets the preset conditions.

[0284] In some implementations, the fourth image is a target adversarial image generated by a preset device;

[0285] The fifth image is obtained based on the inverse mapping relationship between the fourth image and the target dataset, and is used for the target model to test the security of the target model.

[0286] The image processing apparatus for generating adversarial images in this application embodiment takes into account that the generated digital adversarial images need to undergo physical device output and input processing when attacking the target model, which may cause color changes. When iteratively updating the digital adversarial images, the recognition result of the target model is used as a reference, so that the display effect of the directly generated digital adversarial images presented to the target model after undergoing physical device output and input processing is consistent with expectations, thereby improving the stability of the target model in attack testing of the generated adversarial images.

[0287] In some embodiments, the device 70 can also be used to perform the two image processing methods for generating adversarial images described in the above embodiments.

[0288] The image processing apparatus in the embodiments of this application has been described above from the perspective of modular functional entities. The server and terminal device executing the image processing method in the embodiments of this application are described below from the perspective of hardware processing.

[0289] It should be noted that in the embodiments of this application (including...) Figure 11 , Figure 12 In the embodiments shown, the physical devices corresponding to all transceiver modules can be transceivers, and the physical devices corresponding to all processing modules can be processors. Figure 11 , Figure 12 Each of the devices shown can have the following characteristics: Figure 13 The structure shown.

[0290] For example, Figure 11 The image generating apparatus 60 shown can have, for example, Figure 13 The structure shown, when Figure 11 The device shown has the following characteristics: Figure 13 When the structure shown is used, Figure 13 The processor and transceiver in the device can perform the same or similar functions as the processing module 602 and input / output module 601 provided in the aforementioned embodiment of the image generation apparatus 60. Figure 13 The central memory of the processor stores the computer programs that need to be called when executing the image processing method described above for generating the target dataset. (In this application embodiment) Figure 11 In the illustrated embodiment, the physical device corresponding to the input / output module 601 can be an input / output interface, and the physical device corresponding to the processing module 602 can be a processor.

[0291] For example, Figure 12 The image processing apparatus 70 shown may have, for example: Figure 13 The structure shown, when Figure 12 The device shown has the following characteristics: Figure 13 When the structure shown is used,Figure 13 The processor and transceiver in the image processing apparatus 70 can perform the same or similar functions as the processing unit 702 and input / output unit 701 provided in the aforementioned apparatus embodiment corresponding to the image processing apparatus 70. Figure 13 The central memory stores the computer programs that the processor needs to call when executing the image processing method described above for predicting color changes. (In this application embodiment) Figure 13 In the illustrated embodiment, the physical device corresponding to the input / output unit 701 can be an input / output interface, and the physical device corresponding to the processing unit 702 can be a processor.

[0292] This application also provides a terminal device, such as... Figure 14 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The terminal device can be any terminal device including mobile phones, tablets, personal digital assistants (PDAs), point-of-sale (POS) terminals, in-vehicle computers, etc. Taking a mobile phone as an example:

[0293] Figure 14 This diagram illustrates a partial structural representation of a mobile phone related to the terminal device provided in this embodiment. (Reference) Figure 14 The mobile phone includes components such as a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090. Those skilled in the art will understand that... Figure 14 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0294] The following is combined with Figure 14 A detailed introduction to each component of a mobile phone:

[0295] The RF circuit 1010 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 1080; additionally, it transmits uplink data to the base station. Typically, the RF circuit 1010 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the RF circuit 1010 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0296] The memory 1020 can be used to store software programs and modules. The processor 1080 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 1020. The memory 1020 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 1020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0297] The input unit 1030 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1031), and drive the corresponding connection devices according to a pre-set program. Optionally, the touch panel 1031 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1080, and can also receive and execute commands sent by the processor 1080. In addition, the touch panel 1031 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1031, the input unit 1030 may also include other input devices 1032. Specifically, other input devices 1032 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0298] The display unit 1040 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 1040 may include a display panel 1041, which may optionally be configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar display. Further, a touch panel 1031 may cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it transmits the information to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides corresponding visual output on the display panel 1041 based on the type of touch event. Although in Figure 14 In this embodiment, the touch panel 1031 and the display panel 1041 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.

[0299] The mobile phone may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1041 according to the ambient light level, and the proximity sensor can turn off the display panel 1041 and / or the backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0300] The audio circuit 1060, speaker 1061, and microphone 1062 provide an audio interface between the user and the mobile phone. The audio circuit 1060 converts the received audio data into electrical signals and transmits them to the speaker 1061, where the speaker 1061 converts them into sound signals for output. On the other hand, the microphone 1062 converts the collected sound signals into electrical signals, which are then received by the audio circuit 1060, converted into audio data, and then processed by the processor 1080 before being transmitted via the RF circuit 1010 to, for example, another mobile phone, or the audio data can be output to the memory 1020 for further processing.

[0301] WiFi is a short-range wireless transmission technology. Through the WiFi module 1070, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 14 The WiFi module 1070 is shown, but it is understood that it is not an essential component of a mobile phone and can be omitted as needed without changing the essence of the invention.

[0302] The processor 1080 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 1020 and calls data stored in the memory 1020 to perform various functions and process data, thereby providing overall monitoring of the phone. Optionally, the processor 1080 may include one or more processing units; optionally, the processor 1080 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into the processor 1080.

[0303] The mobile phone also includes a power supply 1090 (such as a battery) that supplies power to various components. Optionally, the power supply can be logically connected to the processor 1080 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0304] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0305] In this embodiment of the application, the processor 1080 included in the mobile phone also has a method flow for controlling the execution of the above-mentioned identification candidate adversarial image identification performed by the identification device.

[0306] This application also provides a server; please refer to [link / reference]. Figure 15 , Figure 15 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 1100 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1122 (e.g., one or more processors) and memory 1132, and one or more storage media 1130 (e.g., one or more mass storage devices) for storing application programs 1142 or data 1144. The memory 1132 and storage media 1130 may be temporary or persistent storage. The program stored in the storage media 1130 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the server. Furthermore, the CPU 1122 may be configured to communicate with the storage media 1130 and execute the series of instruction operations in the storage media 1130 on the server 1100.

[0307] Server 1100 may also include one or more power supplies 1120, one or more wired or wireless network interfaces 1150, one or more input / output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0308] The steps performed by the server in the above embodiments can be based on this Figure 15 The structure of server 1100 shown. For example, as in the above embodiment, by Figure 15 The steps performed by the image processing device 60 shown can be based on this Figure 15 The server structure is shown. For example, the central processing unit 1122 performs the following operations by calling instructions from memory 1132:

[0309] The first image and the second image are acquired through the input / output interface 1158. Both the first image and the second image are digital images. The second image is acquired by the first device based on the third image. The third image is output by the second device based on the first image. The first image includes at least one color region. The values ​​of each pixel in the same channel of the same color space are the same.

[0310] Based on at least one color region of the first image, at least one target color patch is obtained from the second image, wherein the target color patch corresponds one-to-one with the color region;

[0311] Obtain the corrected pixels of the target color block, wherein the corrected pixels are estimated based on the values ​​of each pixel of the target color block in each channel of a preset color space;

[0312] Each color region in the first image is mapped to each of the correction pixels to obtain a target dataset, wherein the color region and the correction pixel correspond one-to-one, and the target dataset includes the mapping relationship between the color region and the correction pixel, which is used to train a color transformation prediction model.

[0313] The target adversarial image can also be output through the input / output interface 1158 to materialize it, attack the target model in the physical world, and measure the security of the target model.

[0314] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0315] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0316] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or modules, and may be electrical, mechanical, or other forms.

[0317] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0318] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0319] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0320] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0321] The technical solutions provided in the embodiments of this application have been described in detail above. Specific examples have been used in the embodiments of this application to illustrate the principles and implementation methods of the embodiments of this application. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.

Claims

1. An image processing method, the method comprising: Acquire a first image and a second image, wherein the first image and the second image are both digital images, the second image is acquired by a first device based on a third image, the third image is output by a second device based on the first image, the first image includes at least one color region, and the values ​​of each pixel in the same channel of the same color space are the same. Based on at least one color region of the first image, at least one target color patch is obtained from the second image, wherein the target color patch corresponds one-to-one with the color region; Based on the target color patch, a robust estimation method is used to obtain the corrected pixel of the target color patch, wherein the corrected pixel is estimated based on the median value of each pixel of the target color patch in each channel of a preset color space; Each color region in the first image is mapped to each of the correction pixels to obtain a target dataset, wherein the color region and the correction pixel correspond one-to-one, and the target dataset includes the mapping relationship between the color region and the correction pixel. The target dataset is used to train a color transformation prediction model.

2. The method as described in claim 1, wherein, The first image is obtained by traversing each channel of a preset color space according to a preset numerical interval, and the combination of color channel values ​​visited each time during the traversal forms a color region.

3. An image processing method, the method comprising: Obtain the fourth image; Based on the pixels in the fourth image and the mapping relationship in the target dataset, a fifth image is obtained, wherein each pixel in the fifth image corresponds one-to-one with each pixel in the fourth image, and each pixel in the fifth image is independently transformed from each pixel in the fourth image according to the mapping relationship. ； The mapping relationship includes the mapping relationship between color regions and correction pixels. The target dataset is obtained according to the method described in claim 1 or 2, and the target dataset is used to train the color transformation prediction model.

4. An image processing method, comprising: Obtain a first candidate adversarial image, wherein the first candidate adversarial image is updated based on historical candidate adversarial images, and the historical candidate adversarial images include the original image; Based on the positive mapping relationship in the target dataset, the first candidate adversarial image is subjected to color transformation processing to obtain the second candidate adversarial image. The target dataset is obtained according to the method of claim 1 or 2. The positive mapping relationship includes the positive mapping relationship between color regions and correction pixels. The target dataset is used to train the color transformation prediction model. Obtain the recognition similarity between the second candidate adversarial image and the target image; If the recognition similarity does not meet the preset conditions, the first candidate adversarial image and the second candidate adversarial image are updated until the recognition similarity between the second candidate adversarial image and the target image meets the preset conditions, and the first candidate adversarial image when the recognition similarity meets the preset conditions is taken as the target adversarial image.

5. An image processing method, comprising: Obtain a third candidate adversarial image, wherein the third candidate adversarial image is updated based on historical candidate adversarial images, and the historical candidate adversarial images include the original image; Obtain the recognition similarity between the third candidate adversarial image and the target image; If the recognition similarity does not meet the preset conditions, the third candidate adversarial image is updated until the recognition similarity between the third candidate adversarial image and the target image meets the preset conditions, and the third candidate adversarial image when the recognition similarity meets the preset conditions is used as the target adversarial image. Based on the inverse mapping relationship in the target dataset, the target adversarial image is subjected to color transformation processing to obtain the source digital adversarial image. The target dataset is obtained according to the method according to claim 1 or 2. The inverse mapping relationship includes the inverse mapping relationship between color regions and correction pixels. The target dataset is used to train the color transformation prediction model. The source digital adversarial image is used to transform into the target adversarial image after being processed by physical devices for output and input.

6. An image processing apparatus, comprising: The input / output module is configured to acquire a first image and a second image, wherein the first image and the second image are both digital images, the second image is acquired by a first device based on a third image, the third image is output by a second device based on the first image, and the first image includes at least one color region, wherein the values ​​of each pixel in the same channel of the same color space are the same. The processing module is configured to obtain at least one target color patch from the second image based on at least one color region of the first image, wherein the target color patch corresponds one-to-one with the color region; The processing module is further configured to obtain corrected pixels of the target color patch using a robust estimation method, wherein the corrected pixels are estimated based on the median values ​​of each pixel of the target color patch in each channel of a preset color space; and Each color region in the first image is mapped to each of the correction pixels to obtain a target dataset, wherein the color region and the correction pixel correspond one-to-one, and the target dataset includes the mapping relationship between the color region and the correction pixel. The target dataset is used to train a color transformation prediction model.

7. An image processing apparatus, comprising: The input / output unit is configured to acquire a fourth image; The processing unit is configured to obtain a fifth image based on each pixel in the fourth image and the mapping relationship in the target dataset, wherein each pixel in the fifth image corresponds one-to-one with each pixel in the fourth image, and each pixel in the fifth image is independently transformed from each pixel in the fourth image according to the mapping relationship; The mapping relationship includes the mapping relationship between color regions and correction pixels. The target dataset is obtained by processing the apparatus according to claim 6, and the target dataset is used to train a color transformation prediction model.

8. A computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the method as claimed in claim 1 or 2, or the method as claimed in claim 3, 4 or 5.

9. A computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the method as claimed in claim 1 or 2, or the method as claimed in claim 3, 4, or 5.

Images