Image processing method and device

Abstract

The invention discloses an image processing method and device, and relates to the technical field of image processing. For the obtained original image and reference image with the same size, the image processing device can replace the image of the key coordinate area in the original image with the image of the key coordinate area in the reference image to obtain a first target image. The key coordinate region is at least partially overlapped with the recognition coordinate regions of at least n image recognition models in the m image recognition models, and the recognition coordinate regions are regions which need to be recognized emphatically when the image recognition models carry out image recognition, so that when the first target image is used as an adversarial sample to train the m image recognition models, the recognition precision of the m image recognition models is improved. The image recognition model can perform adversarial training on the key coordinate area, so that the problem that the adversarial training effect of the image recognition model is poor is solved.

Description

Image processing method and device Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and device. Background Art

[0002] With the development of deep neural networks, deep learning models based on these networks (such as face recognition models in image recognition models) have been widely used in various scenarios. However, attacks against deep learning models are also emerging. Among them, the main threat to deep learning models is adversarial sample attacks, in which attackers make minor modifications to the original samples (such as face images) input to the deep learning model to generate adversarial samples. These adversarial samples have minimal differences from the original samples in human visual perception, but can cause the deep learning model to output incorrect results with a high probability. Therefore, improving the robustness and generalization ability of deep learning models against adversarial samples is an urgent issue that needs to be addressed in the process of their widespread application.

[0003] In related technologies, adversarial samples can be used to perform adversarial training on deep learning models to improve the robustness and generalization ability of deep learning models against adversarial samples.

[0004] Summary of the Invention

[0005] This application provides an image processing method and device, the technical solution of which is as follows:

[0006] In one aspect, an image processing method is provided, the method comprising:

[0007] Get the original image and reference image of the same size;

[0008] Determining a key coordinate region; wherein the key coordinate region at least partially overlaps with recognition coordinate regions of at least n image recognition models among the m image recognition models; m and n are both integers and m≥n≥1, and the image recognition model is used to determine the similarity between the original image and the reference image based on features of the recognition coordinate region in the image;

[0009] The image of the key coordinate area in the original image is replaced by the image of the key coordinate area in the reference image to obtain a first target image.

[0010] Optionally, m>n>1.

[0011] Optionally, before determining the key coordinate area, the method further includes:

[0012] For one image recognition model among the m image recognition models:

[0013] performing similarity determination operations on a plurality of unit coordinate regions in sequence, wherein the size of the unit coordinate regions is smaller than the size of the reference image, and the plurality of unit coordinate regions have the same size and different positions; wherein the similarity determination operation on one of the plurality of unit coordinate regions comprises: replacing an image of the unit coordinate region in the original image with an image of the unit coordinate region in the reference image to obtain an intermediate image corresponding to the unit coordinate region; and determining, using the image recognition model, a similarity between the intermediate image corresponding to the unit coordinate region and the reference image to obtain a similarity corresponding to the unit coordinate region;

[0014] Among the similarities corresponding to the multiple unit coordinate areas, x unit coordinate areas with the greatest similarity are used as the recognition coordinate areas of the image recognition model, where x is an integer greater than or equal to 1 and is less than the number of the unit coordinate areas.

[0015] Optionally, the area ratio of the image of the key coordinate region in the original image is smaller than a ratio threshold.

[0016] Optionally, m>n>1, and the area ratio of the image of the key coordinate region in the original image is less than a ratio threshold; determining the key coordinate region includes:

[0017] Determine at least one target unit coordinate area among the plurality of unit coordinate areas; the target unit coordinate area belongs to the recognition coordinate areas of at least two image recognition models;

[0018] When the area ratio of the image of the at least one target unit coordinate region in the original image is greater than or equal to the ratio threshold, shrinking the at least one target unit coordinate region to obtain the key coordinate region;

[0019] When the area ratio of the image of the at least one target unit coordinate region in the original image is less than the ratio threshold, the at least one target unit coordinate region is determined as the key coordinate region.

[0020] Optionally, the method further includes:

[0021] updating the image of the key coordinate area of ​​the first target image to obtain a second target image;

[0022] The second target image is different from the reference image, and the similarity between the second target image and the reference image is greater than the similarity between the first target image and the reference image.

[0023] Optionally, updating the image of the key coordinate area of ​​the first target image to obtain the second target image includes:

[0024] Determining loss values ​​of the m image recognition models respectively, where for one of the image recognition models, the loss value of the image recognition model is determined based on features of the first target image and features of the reference image extracted by the image recognition model;

[0025] Based on the sum of the loss values ​​of the m image recognition models, the image of the key coordinate area of ​​the first target image is updated to obtain a second target image; the sum of the loss values ​​of the m image recognition models is negatively correlated with the similarity between the first target image and the reference image.

[0026] Optionally, for one of the image recognition models, the loss value of the image recognition model is: the mean square error between the features of the first target image extracted by the image recognition model and the features of the reference image.

[0027] Optionally, updating the image of the key coordinate area of ​​the first target image to obtain a second target image includes:

[0028] An image in the key coordinate region of the first target image is iteratively updated to obtain the second target image.

[0029] Optionally, the updating of the image of the key coordinate area of ​​the first target image is further achieved by the following formula:

[0030] Among them, X1 represents the first target image, X2 represents the image obtained by updating the key coordinate area of ​​the first target image X1, ∈ represents the learning rate, and L1 represents the sum of the loss values ​​of the m image recognition models; MASK represents a mask image, and the pixel values ​​of the image in the key coordinate area in the mask image are all 1, and the pixel values ​​of the images in other areas of the mask image except the key coordinate area are all 0.

[0031] In another aspect, an image processing apparatus is provided, the apparatus comprising:

[0032] An acquisition module, used to acquire an original image and a reference image of the same size;

[0033] a first determination module, configured to determine a key coordinate region; wherein the key coordinate region at least partially overlaps with recognition coordinate regions of at least n image recognition models among the m image recognition models; m and n are both integers and m≥n≥1; and the image recognition model is configured to determine a similarity between the original image and the reference image based on features of the recognition coordinate region in the image;

[0034] The replacement module is used to replace the image of the key coordinate area in the original image with the image of the key coordinate area in the reference image to obtain a first target image.

[0035] Optionally, m>n>1.

[0036] Optionally, the device further comprises:

[0037] a second determination module configured to perform a similarity determination operation on a plurality of unit coordinate regions in sequence for one of the m image recognition models; wherein the similarity determination operation on one of the plurality of unit coordinate regions comprises: replacing an image of the unit coordinate region in the original image with an image of the unit coordinate region in the reference image to obtain an intermediate image corresponding to the unit coordinate region; and determining, using the image recognition model, a similarity between the intermediate image corresponding to the unit coordinate region and the reference image to obtain a similarity corresponding to the unit coordinate region;

[0038] The selection module is configured to select the x unit coordinate areas with the largest similarities among the multiple unit coordinate areas as the recognition coordinate areas of the image recognition model, where x≥1 and x is less than the number of the unit coordinate areas.

[0039] Optionally, the area ratio of the image of the key coordinate region in the original image is smaller than a ratio threshold.

[0040] Optionally, m>n>1, and the area ratio of the image in the key coordinate region in the original image is less than a ratio threshold; the first determining module is configured to:

[0041] Determine at least one target unit coordinate area among the plurality of unit coordinate areas; the target unit coordinate area belongs to the recognition coordinate areas of at least two image recognition models;

[0042] When the area ratio of the image of the at least one target unit coordinate region in the original image is greater than or equal to the ratio threshold, shrinking the at least one target unit coordinate region to obtain the key coordinate region;

[0043] When the area ratio of the image of the at least one target unit coordinate region in the original image is less than the ratio threshold, the at least one target unit coordinate region is determined as the key coordinate region.

[0044] Optionally, the image processing device further includes:

[0045] an updating module, configured to update the image of the key coordinate area of ​​the first target image to obtain a second target image;

[0046] The second target image is different from the reference image, and the similarity between the second target image and the reference image is greater than the similarity between the first target image and the reference image.

[0047] Optionally, the update module is used to:

[0048] Determining loss values ​​of the m image recognition models respectively, where for one of the image recognition models, the loss value of the image recognition model is determined based on features of the first target image and features of the reference image extracted by the image recognition model;

[0049] Based on the sum of the loss values ​​of the m image recognition models, the image of the key coordinate area of ​​the first target image is updated to obtain a second target image; the sum of the loss values ​​of the m image recognition models is negatively correlated with the similarity between the first target image and the reference image.

[0050] Optionally, for one of the image recognition models, the loss value of the image recognition model is: the mean square error between the features of the first target image extracted by the image recognition model and the features of the reference image.

[0051] Optionally, the updating module is configured to iteratively update the image of the key coordinate region of the first target image to obtain the second target image.

[0052] Optionally, the updating module is configured to further implement the updating of the image of the key coordinate area of ​​the first target image by using the following formula:

[0053] Among them, X1 represents the first target image, X2 represents the image obtained by updating the key coordinate area of ​​the first target image X1, ∈ represents the learning rate, and L1 represents the sum of the loss values ​​of the m image recognition models; MASK represents a mask image, and the pixel values ​​of the image in the key coordinate area in the mask image are all 1, and the pixel values ​​of the images in other areas of the mask image except the key coordinate area are all 0.

[0054] On the other hand, an image processing device is provided, which includes a processor and a memory, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement the image processing method provided in the above aspect.

[0055] On the other hand, a non-transitory computer-readable storage medium is provided, wherein instructions are stored in the storage medium, and the instructions are loaded and executed by a processor to implement the image processing method provided in the above aspect.

[0056] On the other hand, a computer program product is provided, which includes computer instructions, and the instructions are loaded and executed by a processor to implement the image processing method provided in the above aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0058] FIG1 is a schematic diagram of generating adversarial samples provided by an embodiment of the present application;

[0059] FIG2 is a schematic diagram of the structure of an image processing system provided in an embodiment of the present application;

[0060] FIG3 is a flow chart of an image processing method provided by an embodiment of the present application;

[0061] FIG4 is a flowchart of another image processing method provided in an embodiment of the present application;

[0062] FIG5 is a flowchart of another image processing method provided in an embodiment of the present application;

[0063] FIG6 is a schematic diagram of an image processing apparatus for determining image similarity provided by an embodiment of the present application;

[0064] FIG7 is a schematic diagram of a first target image provided in an embodiment of the present application;

[0065] FIG8 is a flowchart of another image processing method provided in an embodiment of the present application;

[0066] FIG9 is a schematic diagram of updating a first target image provided by an embodiment of the present application;

[0067] FIG10 is a schematic structural diagram of an image processing device provided in an embodiment of the present application;

[0068] FIG11 is a schematic structural diagram of another image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0070] Currently, the main attack surface for deep learning models is adversarial attack, also known as adversarial attack. Adversarial attack involves using adversarial examples to attack (i.e., “trick”) a deep learning model, causing it to make inaccurate inferences.

[0071] For example, referring to Figure 1, for a face recognition model in a deep learning model, under normal circumstances, for a face image A input to the face recognition model, the face recognition model calculates that the similarity between face image A and a pre-stored face image B is very low (e.g., 0.12). Therefore, the face recognition model can determine that face image A and face image B are not the face images of the same person.

[0072] When an adversarial algorithm is used to alter face image A to obtain face image C, and face image C is input into the face model as an adversarial sample, the face recognition model calculates that face image C has a high similarity (for example, 0.78) with the pre-stored face image B, and the face recognition model determines that face image C and face image B are the same person. This effectively "tricks" the face recognition model.

[0073] Adversarial attacks can be categorized in various ways. Based on the attack environment, they can be divided into black-box attacks and white-box attacks. A black-box attack involves an attacker interacting with the model through its inputs and outputs to construct adversarial examples, without knowing the model's internal structure, training parameters, and defense methods. A white-box attack, on the other hand, involves an attacker possessing all the information about the model (i.e., the model's internal structure, training parameters, and defense methods) and controlling the model's inputs, even modifying the input data at the bit level, to launch an adversarial attack.

[0074] In order to improve the ability of deep learning models to resist adversarial attacks, adversarial samples can be used to perform adversarial training on deep learning models to improve the robustness and generalization ability of deep learning models to adversarial samples.

[0075] Among them, adversarial samples can be generated in three ways. The first is to obtain adversarial samples by mapping on the original samples. The second is to add small perturbations (i.e. noise) to the original samples to obtain adversarial samples. The third is to generate adversarial samples through models (such as generative adversarial network models).

[0076] However, the adversarial examples generated by the first method are easily detectable to the naked eye, resulting in poor training effectiveness. Regarding the second method, when training a model using adversarial examples, the goal is to enable the deep learning model to distinguish between the original and adversarial examples—that is, to identify the perturbations added to the original examples. However, because these perturbations are small, they are difficult for the deep learning model to identify during training, resulting in poor training effectiveness. Larger perturbations, however, fail to achieve the desired effect against the attack. Regarding the third method, since it involves training the model used to generate the adversarial examples, the implementation process is more complex.

[0077] FIG2 is a schematic diagram of the structure of an image processing system provided in an embodiment of the present application. Referring to FIG2 , the system includes: a server 110 and a terminal 120. A wired or wireless communication connection is established between the server 110 and the terminal 120. Optionally, the server 110 can be an independent physical server, or a server cluster or distributed system consisting of multiple physical servers. The terminal 120 can be a personal computer (PC), tablet computer, smartphone, wearable device, intelligent robot, or other terminal with data calculation, processing, and storage capabilities.

[0078] In an embodiment of the present application, the terminal 120 in the system can be used to obtain an original image and a reference image for image processing by the server 110, and send the original image and the reference image to the server 110. The server 110 can then process the original image based on the reference image, and use the target image obtained after processing as a training sample (for example, as an adversarial sample) to train the image recognition model.

[0079] Furthermore, the system can also be a system capable of performing specific image processing tasks (e.g., image classification tasks). Accordingly, the terminal device 120 in the system can be used to obtain an original image to be identified and send the original image to be identified to the server 110. The server 110 pre-stores a trained image recognition model. After the server 110 obtains the original image to be identified, it can input the original image into the image recognition model to identify the original image and output a recognition result. The server 110 can then send the recognition result to the terminal 120.

[0080] It is also understandable that the above-mentioned image processing system may also include only the server 110, and the server 110 may directly obtain the original image and reference image to be processed. Alternatively, the original image and reference image may be pre-stored in the server 110. The server 110 may directly perform image processing and other operations based on the original image and reference image.

[0081] FIG3 is a flowchart of an image processing method provided by an embodiment of the present application. The method can be applied to an image processing device, which can be the server 110 shown in FIG2 . Referring to FIG3 , the method includes:

[0082] Step 101: Acquire an original image and a reference image of the same size.

[0083] The image processing device may obtain a pre-stored original image and reference image, or may receive an original image and reference image sent by another device (e.g., a terminal). The original image and reference image may have the same size but different image content. For example, the original image and reference image may both be facial images.

[0084] In an embodiment of the present application, the original image may be an image to be detected by an image recognition model when performing an image recognition task. The image processing device may identify the original image by calculating the similarity between the original image and a reference image. For example, if the image processing device determines that the calculated similarity is greater than a pre-set similarity threshold, the original image and the reference image may be determined to be the same image.

[0085] Step 102: Determine the key coordinate area.

[0086] In an embodiment of the present application, m image recognition models are pre-stored in the image processing device. The image processing device can select recognition coordinate areas of at least n image recognition models from the recognition coordinate areas of the m image recognition models based on pre-set selection conditions, and select a key coordinate area therefrom. The key coordinate area at least partially overlaps with the recognition coordinate areas of the at least n image recognition models, that is, the overlapping area of ​​the recognition coordinate areas of the at least n image recognition models to which the key coordinate area belongs. Both m and n are integers, and m≥n≥1.

[0087] When recognizing an image, the image recognition model can perform image recognition based on the image features of the detection coordinate area in the image. The detection coordinate area includes at least the recognition coordinate area (also called the sensitive area of ​​the image recognition model). The image of the recognition coordinate area can significantly characterize the features of the image of the detection coordinate area (also called the entire image). Based on this, for the original image and the reference image acquired by the image processing device, the m image recognition models can determine the similarity between the original image and the reference image based only on the features of the recognition coordinate area in the original image and the reference image.

[0088] For example, when the face recognition model in the image recognition model recognizes a face image, since the facial features in the face can significantly represent the overall characteristics of the face, the recognition coordinate area of ​​the face recognition model can be the facial features area in the face image.

[0089] Optionally, the recognition coordinate areas of the m image recognition models may be coordinate areas predetermined and stored by the image processing device, or the recognition coordinate areas may be areas determined by the image processing device by using the m image recognition models to recognize the image.

[0090] Step 103: Replace the image of the key coordinate area in the original image with the image of the key coordinate area in the reference image to obtain a first target image.

[0091] In an embodiment of the present application, the first target image can be used to train any one of the m image recognition models (e.g., adversarial training). It is understandable that since the images of the key coordinate areas in the first target image and the reference image are the same, if the image recognition model is used to extract the features of the first target image and the reference image, the similarity calculated based on the features is higher. The probability that the two images are the same image in the recognition results of the image recognition model is also higher, but the two are essentially two different images. Based on this, the first target image can be used as an adversarial sample to perform adversarial training on any one of the above-mentioned m image recognition models.

[0092] Optionally, the image processing device may directly output the first target image. For example, the image processing device may output the first target image as an adversarial sample to a device for adversarial training of an image recognition model. Alternatively, the image processing device may be capable of adversarial training of an image recognition model. Accordingly, the image processing device may not output the first target image and may instead conduct adversarial training on the image recognition model based on the first target image.

[0093] Furthermore, the image processing device may perform image recognition using an image recognition model obtained through adversarial training.

[0094] It is understood that in order to increase the similarity between the adversarial sample image and the reference image, the image processing device may further process and optimize the first target image, thereby reducing the difference between the processed image and the reference image, thereby achieving a better training effect.

[0095] Based on the above analysis, it can be known that by processing the original image using the image processing method provided in the embodiment of the present application, an adversarial sample for adversarial training of the model can be generated. Among them, the operation of replacing the image of the key coordinate area in the original image in the present application can also be called masking the original image, and the key coordinate area can also be called the mask area. Since the mask area of ​​the first target image is determined based on the key coordinate area when at least n image recognition models among m image recognition models recognize the image, and the image of the mask area is the same as the image of the corresponding coordinate area in the reference image, therefore, when the first target image obtained by this method is used as an adversarial sample to conduct adversarial training on the image recognition model, the image recognition model can be subjected to adversarial training on the key coordinate area, and the training effect is better.

[0096] In summary, embodiments of the present application provide an image processing method in which, for an original image and a reference image of the same size, an image processing device can replace the image of a key coordinate region in the original image with the image of a key coordinate region in the reference image to obtain a first target image. Because this key coordinate region at least partially overlaps with the recognition coordinate regions of at least n of the m image recognition models, and this recognition coordinate region is the region that the image recognition models need to focus on when performing image recognition, when the first target image is used as an adversarial sample to train the m image recognition models, the image recognition models can be trained adversarially on this key coordinate region, resulting in a better training effect.

[0097] The embodiment of the present application only illustrates the generation process of one adversarial sample as an example. It can be understood that the generation process of multiple adversarial samples can refer to the above introduction to the generation process of the one adversarial sample, and this application will not go into details.

[0098] FIG4 is a flowchart of an image processing method provided by an embodiment of the present application. The method can be applied to an image processing device, which can be the server 110 shown in FIG2 . Referring to FIG4 , the method includes:

[0099] Step 201: Acquire an original image and a reference image of the same size.

[0100] The image processing device can obtain the original image and reference image stored in advance, or can receive the original image and reference image sent by other devices (such as terminals). The original image and the reference image have the same size but different image contents. In addition, in order to facilitate subsequent operations, the arrangement of the image content in the original image should correspond to the arrangement of the image content in the reference image. For example, the original image and the reference image can both be facial images. In addition, the arrangement area and proportion of the facial features in the two facial images are basically the same.

[0101] It is understood that the original image acquired by the image processing device may be the image to be detected by the image recognition model when performing an image recognition task. The image processing device may identify the original image by calculating the similarity between the original image and a reference image. For example, if the image processing device determines that the calculated similarity is greater than a pre-set similarity threshold, the original image and the reference image may be determined to be the same image.

[0102] Step 202: For one of the m image recognition models, perform similarity determination operations on multiple unit coordinate regions in sequence.

[0103] In an embodiment of the present application, the image processing device pre-stores m image recognition models, where m is an integer greater than 0. After acquiring an original image and a reference image, the image processing device may sequentially perform a similarity determination operation on multiple unit coordinate regions on the original image and the reference image for one of the m image recognition models. The unit coordinate region has a size smaller than that of the reference image, and the multiple unit coordinate regions have the same size but different positions.

[0104] Optionally, the image processing device may determine multiple unit coordinate areas on the original image and the reference image in sequence by means of a sliding window. The area of ​​the unit coordinate area is the area of ​​the sliding window (also referred to as the kernel size). For example, the kernel size that the image processing device may use during the sliding window process may be 13x10, that is, the sliding window area may include 13x10 pixels. The step size used during the sliding window process may be 7x5. 7 is the moving step size of the sliding window in the pixel row direction, and 5 is the moving step size of the sliding window in the pixel column direction.

[0105] During the sliding window process, the image processing device can perform a similarity determination operation on each unit coordinate region each time the sliding window slides to the original image, thereby completing the similarity determination operation on multiple unit coordinate regions.

[0106] The m image recognition models involved in this application can all be face recognition models. At present, face recognition models are mainly divided into three categories: the InceptionResnet (a face recognition model) model based on FaceNet (an open source face recognition algorithm), the Iresnet (a face recognition model, referred to as IR) model based on insightface (an open source face recognition algorithm), and the MobileFaceNet (a face recognition model) model based on mobilenet (an open source face recognition algorithm). There are currently four types of public training data for face recognition models: the MS1M dataset, the VGGFACE2 dataset, the glint260k dataset, and the Webface260M dataset. The m image recognition models used in this application can be obtained by training the above three face models using different datasets.

[0107] Each image recognition model can be represented by "face recognition algorithm - network depth - training dataset name" or "model structure - training dataset name". For example, the m recognition models mentioned above may include: IR-50-vggface2, IR-50-MS1M, IR-101-glint360k, IR-101-webface260M, MobileFaceNet-MS1M, MobileFaceNet-glint360k, and InceptionResnetV1-vggface2.

[0108] Optionally, referring to FIG5 , when the image processing apparatus adopts an image recognition model to perform a similarity determination operation on one unit coordinate region among a plurality of unit coordinate regions, step 202 may include:

[0109] Step 2021: Replace the image of the unit coordinate area in the original image with the image of the unit coordinate area in the reference image to obtain an intermediate image corresponding to the unit coordinate area.

[0110] Only the image of that unit coordinate region in the intermediate image is replaced with the image of that unit coordinate region in the reference image. The images of all other unit coordinate regions remain unchanged. Therefore, for multiple intermediate images corresponding to multiple unit coordinate regions, the replaced images in each of these multiple intermediate images have different unit coordinate regions.

[0111] Step 2022: Using the image recognition model, determine the similarity between the intermediate image corresponding to the unit coordinate area and the reference image to obtain the similarity corresponding to the unit coordinate area.

[0112] After obtaining the intermediate image corresponding to the unit coordinate region, the image processing device can input the intermediate image and the reference image into an image recognition model. The image recognition model can extract features from the intermediate image and the reference image and output them for the image processing device to calculate similarity. The image processing device can then determine the calculated similarity as the similarity corresponding to the unit coordinate region.

[0113] The image processing device may use a similarity algorithm such as cosine distance and Euclidean distance to calculate the similarity between the intermediate image and the reference image.

[0114] For example, referring to Figure 6 , after intermediate image P1 and reference image P2 are input to image recognition model M1, model M1 extracts and outputs feature f1 of intermediate image P1 and feature f2 of reference image P2. The image processing device can then determine the similarity between intermediate image P1 and reference image P2 based on feature f1 and feature f2.

[0115] Step 203: Among the similarities corresponding to the multiple unit coordinate regions, the x unit coordinate regions with the greatest similarity are used as recognition coordinate regions of the image recognition model.

[0116] It can be understood that since the image of the unit coordinate area in the intermediate image corresponding to the unit coordinate area is identical to the image of the unit coordinate area in the reference image, the images of the unit coordinate area in the two images are completely similar. The greater the similarity between the intermediate image corresponding to the unit coordinate area and the reference image, the higher the proportion of similarity between the images in the unit coordinate area when the image recognition model calculates the similarity between the two images. In other words, the image of the unit coordinate area can significantly represent the characteristics of the image.

[0117] Based on this, the image processing device can use the x unit coordinate areas with the greatest similarity among the similarities corresponding to multiple unit coordinate areas as the recognition coordinate areas of the image recognition model (that is, the whole composed of the x unit coordinate areas as the recognition coordinate area of ​​the image recognition model). The recognition coordinate area is the area that the image recognition model needs to focus on identifying and detecting when recognizing an image. Wherein, x is an integer greater than or equal to 1, and x is less than the number of unit coordinate areas. Accordingly, for m image recognition models, the image processing device can determine m*x unit coordinate areas. For example, m=7, x=10.

[0118] Step 204: Determine at least one target unit coordinate area among the multiple unit coordinate areas.

[0119] In an embodiment of the present application, the recognition coordinate areas of the m image recognition models include a total of m*x unit coordinate areas. The m*x unit coordinate areas have the same unit coordinate area. The image processing device can determine at least two identical unit coordinate areas in the m*x unit coordinate areas as a target unit coordinate area, thereby determining at least one target unit coordinate area from the m*x unit coordinate areas. Each target unit coordinate area belongs to the recognition coordinate area of ​​at least two image recognition models in the m image recognition models. The at least one target unit coordinate area is also referred to as the overall sensitive coordinate area of ​​the m image recognition models.

[0120] It can be understood that the above-mentioned m image recognition models all have multiple unit coordinate areas, that is, the multiple unit coordinate areas possessed by the m image recognition models are the same. However, affected by factors such as the internal structure and training parameters of the m image recognition models, the recognition coordinate areas of the m image recognition models may not be the same. For example, a unit coordinate area Z1 among the multiple unit coordinate areas may belong to the recognition coordinate areas of model M1 and model M2 in the m image recognition models, but not to the recognition coordinate area of ​​model M3. Based on this, the image processing device can determine the unit coordinate area among the multiple unit coordinate areas, which belongs to the recognition coordinate areas of at least two image recognition models, as a target unit coordinate area.

[0121] Step 205 : Detect whether the area ratio of the image of at least one target unit coordinate region in the original image is less than a ratio threshold.

[0122] In the embodiment of the present application, if the image processing device detects that the area ratio of the image of at least one target unit coordinate region in the original image is greater than or equal to the ratio threshold, the image processing device may execute the following step 206. If the image processing device determines that the area ratio is less than the ratio threshold, the image processing device may execute the following step 207. The ratio threshold may be a preset fixed value, for example, the ratio threshold may be 10%.

[0123] Step 206: Reduce at least one target unit coordinate area to obtain a key coordinate area.

[0124] In step 205, if the image processing device detects that the area ratio of the image of at least one target unit coordinate region in the original image is greater than or equal to a ratio threshold, the image processing device may reduce the at least one target unit coordinate region to obtain a key coordinate region. The area ratio of the image of the key coordinate region in the original image is less than the ratio threshold. Optionally, the image processing device may use an image erosion algorithm to reduce the at least one target unit coordinate region. The degree of reduction of the at least one target unit coordinate region is the same.

[0125] It is understandable that if at least one target unit coordinate region whose area ratio is greater than or equal to the ratio threshold is directly determined as the key coordinate region, then when the image of the key coordinate region in the original image is replaced (also called a mask), the area ratio of the replacement region (i.e., the mask region) will be relatively large. As a result, the adversarial samples generated by masking are easily perceived by the naked eye. Based on this, after determining at least one target unit coordinate region, the image processing device can first detect whether the area ratio of at least one target unit coordinate region in the original image is less than the ratio threshold, and when the area ratio is greater than or equal to the ratio threshold, the target unit coordinate region is reduced so that the area ratio is less than the ratio threshold.

[0126] Step 207: Determine at least one target unit coordinate area as a key coordinate area.

[0127] In step 205 , if the image processing apparatus detects that the area ratio of the image of at least one target unit coordinate region in the original image is smaller than the ratio threshold, the at least one target unit coordinate region may be directly determined as a key coordinate region.

[0128] Step 208: Replace the image of the key coordinate region in the original image with the image of the key coordinate region in the reference image to obtain a first target image.

[0129] The first target image can be used to train any of the m image recognition models (e.g., perform adversarial training). Since the images in the key coordinate areas of the first target image and the reference image are identical, when the image recognition model recognizes the first target image and the reference image, the probability that the two images are the same is high in the recognition result, but the two images are essentially two different images. Based on this, the first target image can be used as an adversarial sample to perform adversarial training on any of the m image recognition models.

[0130] For example, referring to Figure 7 , the key coordinate region may include multiple target unit coordinate regions. The image processing apparatus may replace the images of the multiple target unit coordinate regions in the original image with the images of the multiple target unit coordinate regions on the reference image.

[0131] Step 209: Update the image of the key coordinate area of ​​the first target image to obtain a second target image.

[0132] It is understandable that since the key coordinate area accounts for a small proportion of the first target image and the key coordinate area does not include the detection coordinate area of ​​each image recognition model in the m image recognition models (which can be called the entire image), the image processing device calculates the similarity between the first target image and the reference image based on the features of the first target image extracted by the image recognition model (which are the features of the entire image) and the features of the reference image, which may still be low. Based on this, in order to achieve the effect of making the image used as the adversarial sample more similar to the reference image, after obtaining the first target image, the image processing device can further update the image of the key coordinate area of ​​the first target image to obtain the second target image.

[0133] The second target image is different from the reference image, and the similarity between the updated second target image and the reference image is greater than the similarity between the first target image and the reference image. In other words, the update can further reduce the difference between the updated second target image and the reference image, thereby achieving better training results.

[0134] It is understandable that if the second target image obtained after the update is identical to the reference image (i.e., completely similar), then the second target image cannot be used as an adversarial example for adversarial training of the image recognition model. Therefore, when updating the first target image, it is necessary to ensure that the second target image is completely similar to the reference image while gradually increasing the similarity between the two images.

[0135] Optionally, when processing the first target image, the image processing apparatus may update the loss values ​​of the m image recognition models. Accordingly, referring to FIG8 , step 209 may include:

[0136] Step 2091: Determine the loss values ​​of the m image recognition models respectively.

[0137] Among them, for one image recognition model among the m image recognition models, the loss value of the image recognition model can be determined based on the features of the first target image and the features of the reference image extracted by the image recognition model.

[0138] In an embodiment of the present application, when determining the loss value of an image recognition model, an image processing device may input a first target image and a reference image into the image recognition model. The image recognition model is capable of extracting features of the first target image and features of the reference image. The image processing device may then determine the loss value of the image recognition model based on the features of the two images.

[0139] Optionally, for an image recognition model, the loss value of the image recognition model is: the mean squared error (MSE) between the features of the first target image extracted by the image recognition model and the features of the reference image.

[0140] The mean square error MSE(y,y′) between the feature y of the first target image and the feature y′ of the reference image satisfies:

[0141] In formula (1), d is the dimension used by the image recognition model to extract image features. MSE(y,y′) is negatively correlated with the similarity between the first target image and the reference image. That is, the more similar the first target image is to the reference image, the smaller the MSE(y,y′) is.

[0142] Step 2092: Based on the sum of the loss values ​​of the m image recognition models, update the key coordinate area of ​​the first target image to obtain a second target image.

[0143] The sum of the loss values ​​of the m image recognition models is negatively correlated with the similarity between the first target image and the reference image. That is, when updating the first target image, the image processing device can update the image in a direction that reduces the sum of the loss values ​​of the m image recognition models, thereby increasing the similarity between the updated image and the reference image.

[0144] In an embodiment of the present application, the image processing device may use an image adversarial algorithm to update the first target image. Optionally, the image processing device may use a fast gradient sign method (FGSM) to update the first image processing device. Based on this, the image X2 obtained after the first target image X1 is updated may satisfy:

[0145] Here, ∈ represents the learning rate, and ∈ can be a pre-set fixed value used to control the degree of change in the key coordinate area within a certain range. For example, the value of ∈ can be 3 / 255. L1 represents the sum of the loss values ​​of the m image recognition models, and MASK represents the mask image. The pixel values ​​of the image in the key coordinate area in the mask image are all 1, and the pixel values ​​of the images in the other areas of the mask image except the key coordinate area are all 0. In addition, the size of the mask image is the same as that of the reference image.

[0146] In an embodiment of the present application, after determining the key coordinate area, the image processing device can initialize an initial image whose pixel values ​​are all 1 and whose size is the same as that of the reference image. Afterwards, the image processing device can set the pixel values ​​of the image in the key coordinate area in the initial image to 0 to obtain a mask image. Through the (1-MASK) in the above formula (2), the pixel values ​​of the image in the key coordinate area in the mask image can be set to 1, and the pixel values ​​of the images in other areas except the key coordinate area can be set to 0. Therefore, through the above formula (2), the key coordinate area in the first target image can be updated.

[0147] In the above formula (2), the gradient of the sum of the loss values ​​of the m image recognition models is calculated (i.e. ), and then the direction of the gradient change is calculated using the sign function. Adding a negative sign to the sign function can make the direction of change in the key coordinate area opposite to the direction of the gradient change. For example, when the gradient change direction is positive (i.e., the sum of the loss values ​​of multiple image recognition models increases), the image update in the key coordinate area is used to reduce the sum of the loss values ​​of multiple image recognition models.

[0148] Based on the above analysis, it can be seen that the image processing device updates the key coordinate area of ​​the first target image through the above formula (2), which can make the similarity calculated based on the features of the updated image and the features of the reference image increasingly higher.

[0149] Optionally, in order to make the image obtained by the image processing device updating the first target image more effective when used as an adversarial sample for adversarial training, the image processing device may iteratively update the key coordinate area of ​​the first target image to obtain a second target image. During the iterative update process, the image processing device may stop the iteration when the number of iterations reaches a preset target number, and determine the image obtained by the current iteration as the second target image. For example, the target number of iterations may be 150 times. Alternatively, the image processing device may decide whether to stop the iteration based on other parameters (such as the similarity between the iterative image and the reference image), which is not limited in this application.

[0150] Referring to Figure 9 , when the image processing device updates the key coordinate region of the first target image, it can first input the reference image P2 and the first target image P3 into m image recognition models for feature extraction. The image processing device can then determine the loss value of each image recognition model based on the features extracted by that model. The image processing device can then update the key coordinate region of the first target image P3 based on the sum of the loss values ​​of the m image recognition models to obtain an intermediate target image P4. The image processing device can then repeatedly use the intermediate target image obtained by updating the key coordinate region of the first target image as the first target image and update the first target image until the second target image is obtained.

[0151] In summary, the embodiments of the present application provide an image processing method. For an original image and a reference image of the same size, an image processing device can replace the image of the key coordinate area in the original image with the image of the key coordinate area in the reference image to obtain a first target image. Since the key coordinate area at least partially overlaps with the recognition coordinate area of ​​at least n of the m image recognition models, and the recognition coordinate area is the area that the image recognition model needs to focus on when performing image recognition, when the first target image is used as an adversarial sample to train the m image recognition models, the image recognition models can be trained adversarially on the key coordinate area, and the training effect is better.

[0152] It is understandable that the order of the steps of the image processing method provided in the embodiment of the present application can be appropriately adjusted, and the steps can also be deleted according to the circumstances. For example, the above steps 202 and 203 can be deleted according to the circumstances, that is, the image processing device can pre-store the recognition coordinate areas of m image recognition models. Alternatively, the above steps 205 and 206 can be deleted according to the circumstances, that is, the image processing device can directly determine at least one target unit coordinate area as a key coordinate area. Alternatively, the above step 209 can be deleted according to the circumstances, that is, the image processing device can directly output the first target image as an adversarial sample. Any technician familiar with the technical field of this technology can easily think of a method of change within the technical scope disclosed in this application, which should be covered within the scope of protection of this application, and therefore will not be repeated.

[0153] FIG10 is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application, which can execute the image processing method provided in the above method embodiment. As shown in FIG10 , the device includes:

[0154] The acquisition module 301 is used to acquire an original image and a reference image of the same size.

[0155] The first determination module 302 is used to determine a key coordinate area; wherein the key coordinate area at least partially overlaps with the recognition coordinate areas of at least n image recognition models among the m image recognition models; m and n are both integers and m≥n≥1, and the image recognition model is used to determine the similarity between the original image and the reference image based on the characteristics of the recognition coordinate area in the image.

[0156] The replacement module 303 is configured to replace the image in the key coordinate region in the original image with the image in the key coordinate region in the reference image to obtain a first target image.

[0157] In summary, an embodiment of the present application provides an image processing device that, for an original image and a reference image of the same size, can replace the image of the key coordinate area in the original image with the image of the key coordinate area in the reference image to obtain a first target image. Since the key coordinate area at least partially overlaps with the recognition coordinate area of ​​at least n of the m image recognition models, and the recognition coordinate area is the area that the image recognition model needs to focus on when performing image recognition, when the first target image is used as an adversarial sample to train the m image recognition models, the image recognition models can be trained adversarially on the key coordinate area, and the training effect is better.

[0158] Optionally, m>n>1.

[0159] Optionally, as shown in FIG11 , the image processing apparatus further includes:

[0160] The second determination module 304 is used to perform similarity determination operations on multiple unit coordinate areas in sequence for one image recognition model among the m image recognition models, where the size of the unit coordinate area is smaller than the size of the reference image, and the multiple unit coordinate areas have the same size but different positions; wherein the similarity determination operation on one unit coordinate area among the multiple unit coordinate areas includes: replacing the image of the unit coordinate area in the original image with the image of the unit coordinate area in the reference image to obtain an intermediate image corresponding to the unit coordinate area; and using the image recognition model, determining the similarity between the intermediate image corresponding to the unit coordinate area and the reference image to obtain the similarity corresponding to the unit coordinate area.

[0161] The selection module 305 is used to select the x unit coordinate areas with the greatest similarity among the similarities corresponding to the multiple unit coordinate areas as the recognition coordinate areas of the image recognition model, where x is an integer greater than or equal to 1 and is less than the number of unit coordinate areas.

[0162] Optionally, the area ratio of the image in the key coordinate region in the original image is less than a ratio threshold.

[0163] Optionally, the first determination module 302 is used to: determine at least one target unit coordinate area among multiple unit coordinate areas; the target unit coordinate area belongs to the recognition coordinate area of ​​at least two image recognition models; when the area ratio of the image of at least one target unit coordinate area in the original image is greater than or equal to a proportional threshold, reduce the at least one target unit coordinate area to obtain a key coordinate area; when the area ratio of the image of at least one target unit coordinate area in the original image is less than the proportional threshold, determine the at least one target unit coordinate area as a key coordinate area.

[0164] Optionally, as shown in FIG11 , the image processing apparatus further includes:

[0165] The updating module 306 is configured to update the image of the key coordinate region of the first target image to obtain a second target image, wherein the second target image is different from the reference image and the similarity between the second target image and the reference image is greater than the similarity between the first target image and the reference image.

[0166] Optionally, the updating module 306 is configured to:

[0167] Loss values ​​of the m image recognition models are determined respectively. For one image recognition model, the loss value of the image recognition model is determined based on features of the first target image and features of the reference image extracted by the image recognition model.

[0168] Based on the sum of the loss values ​​of the m image recognition models, the image of the key coordinate region of the first target image is updated to obtain a second target image. The sum of the loss values ​​of the m image recognition models is negatively correlated with the similarity between the first target image and the reference image.

[0169] Optionally, for an image recognition model, the loss value of the image recognition model is: the mean square error between the features of the first target image extracted by the image recognition model and the features of the reference image.

[0170] Optionally, the updating module 306 is configured to iteratively update the image of the key coordinate region of the first target image to obtain the second target image.

[0171] Optionally, the updating module 306 is configured to further implement the updating of the image of the key coordinate area of ​​the first target image by using the following formula:

[0172] Wherein, X1 represents the first target image, X2 represents the image obtained by updating the key coordinate area of ​​the first target image X1, ∈ represents the learning rate, L1 represents the sum of the loss values ​​of m image recognition models; MASK represents the mask image, the pixel values ​​of the image in the key coordinate area in the mask image are all 1, and the pixel values ​​of the images in other areas of the mask image except the key coordinate area are all 0.

[0173] In summary, an embodiment of the present application provides an image processing device that, for an original image and a reference image of the same size, can replace the image of the key coordinate area in the original image with the image of the key coordinate area in the reference image to obtain a first target image. Since the key coordinate area at least partially overlaps with the recognition coordinate area of ​​at least n of the m image recognition models, and the recognition coordinate area is the area that the image recognition model needs to focus on when performing image recognition, when the first target image is used as an adversarial sample to train the m image recognition models, the image recognition models can be trained adversarially on the key coordinate area, and the training effect is better.

[0174] It can be understood that the image processing device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0175] In addition, the image processing device and image processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation processes are detailed in the method embodiments, which will not be repeated here.

[0176] The present application also provides an image processing device, which may be the server 110 in the scenario shown in Figure 1. The image processing device may include a processor and a memory, wherein the memory stores instructions that are loaded and executed by the processor to implement the image processing method provided in the above method embodiment.

[0177] An embodiment of the present application further provides a non-transitory computer-readable storage medium, which stores instructions, and the instructions are loaded and executed by a processor to implement the image processing method provided by the above method embodiment.

[0178] An embodiment of the present application further provides a computer program product or a computer program, which includes computer instructions loaded and executed by a processor to implement the image processing method as described in the above aspects.

[0179] It is understood that the term "at least one" in this application means one or more, and "plurality" means two or more.

[0180] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on the quantity and execution order.

[0181] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0182] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Get the original image and reference image of the same size; Determine a key coordinate region; wherein the key coordinate region at least partially overlaps with the recognition coordinate regions of at least n image recognition models among the m image recognition models; m and n are both integers and m≥n≥1, and the image recognition model is used to determine the similarity between the original image and the reference image based on the features of the recognition coordinate region in the image; The image of the key coordinate area in the original image is replaced by the image of the key coordinate area in the reference image to obtain a first target image.

2. The method according to claim 1, characterized in that m>n>1.

3. The method according to claim 1 or 2, characterized in that: Before determining the key coordinate area, the method further includes: For one image recognition model among the m image recognition models: A similarity determination operation is sequentially performed on a plurality of unit coordinate regions, wherein the size of the unit coordinate region is smaller than the size of the reference image, and the plurality of unit coordinate regions have the same size and different positions; wherein the similarity determination operation on a unit coordinate region among the plurality of unit coordinate regions comprises: replacing an image of the unit coordinate region in the original image with an image of the unit coordinate region in the reference image, to obtain an intermediate image corresponding to the unit coordinate region; and determining, by using the image recognition model, a similarity between the intermediate image corresponding to the unit coordinate region and the reference image, to obtain a similarity corresponding to the unit coordinate region; Among the similarities corresponding to the multiple unit coordinate areas, the x unit coordinate areas with the largest similarities are used as the recognition coordinate areas of the image recognition model, where x is an integer greater than or equal to 1, and x is less than the number of the unit coordinate areas.

4. The method according to any one of claims 1 to 3, characterized in that: The area ratio of the image in the key coordinate region in the original image is less than a ratio threshold.

5. The method according to claim 3, characterized in that: m>n>1, the area ratio of the image of the key coordinate region in the original image is less than the ratio threshold; the determining of the key coordinate region includes: Determine at least one target unit coordinate area among the multiple unit coordinate areas; the target unit coordinate area belongs to the recognition coordinate areas of at least two image recognition models; When the area ratio of the image of the at least one target unit coordinate region in the original image is greater than or equal to the ratio threshold, shrinking the at least one target unit coordinate region to obtain the key coordinate region; When the area ratio of the image of the at least one target unit coordinate region in the original image is less than the ratio threshold, the at least one target unit coordinate region is determined as the key coordinate region.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Updating the image of the key coordinate area of ​​the first target image to obtain a second target image; The second target image is different from the reference image, and the similarity between the second target image and the reference image is greater than the similarity between the first target image and the reference image.

7. The method according to claim 6, characterized in that The updating of the image of the key coordinate area of ​​the first target image to obtain the second target image includes: Determine loss values ​​of the m image recognition models respectively, and for one of the image recognition models, the loss value of the image recognition model is determined based on features of the first target image and features of the reference image extracted by the image recognition model; Based on the sum of the loss values ​​of the m image recognition models, the image of the key coordinate area of ​​the first target image is updated to obtain a second target image; the sum of the loss values ​​of the m image recognition models is negatively correlated with the similarity between the first target image and the reference image.

8. The method according to claim 7, characterized in that For one of the image recognition models, the loss value of the image recognition model is: the mean square error between the features of the first target image extracted by the image recognition model and the features of the reference image.

9. The method according to any one of claims 6 to 8, characterized in that: Updating the image of the key coordinate area of ​​the first target image to obtain a second target image includes: The image of the key coordinate area of ​​the first target image is iteratively updated to obtain the second target image.

10. The method according to any one of claims 6 to 9, characterized in that: The updating of the image of the key coordinate area of ​​the first target image is further achieved by the following formula: Among them, X1 represents the first target image, X2 represents the image obtained by updating the key coordinate area of ​​the first target image X1, ∈ represents the learning rate, and L1 represents the sum of the loss values ​​of the m image recognition models; for one of the image recognition models, the loss value of the image recognition model is determined based on the features of the first target image extracted by the image recognition model and the features of the reference image; the sum of the loss values ​​of the m image recognition models is negatively correlated with the similarity between the first target image and the reference image; MASK represents a mask image, and the pixel values ​​of the image in the key coordinate area in the mask image are all 1, and the pixel values ​​of the images in other areas of the mask image except the key coordinate area are all 0.

11. An image processing device, characterized in that: The device comprises: An acquisition module, used for acquiring an original image and a reference image of the same size; A first determination module is used to determine a key coordinate region; wherein the key coordinate region at least partially overlaps with the recognition coordinate regions of at least n image recognition models among the m image recognition models; m≥n≥1, and the image recognition model is used to determine the similarity between the original image and the reference image based on the features of the recognition coordinate region in the image; A replacement module is used to replace the image of the key coordinate area in the original image with the image of the key coordinate area in the reference image to obtain a first target image.

12. The device according to claim 11, characterized in that The device also includes: A second determining module is configured to: for one image recognition model among the at least one image recognition model: The similarity determination operation is performed sequentially for a plurality of unit coordinate regions, the size of the unit coordinate region being The size of the unit coordinate area is smaller than that of the reference image, and the sizes of the multiple unit coordinate areas are the same and the positions are different; wherein the similarity determination operation for one unit coordinate area among the multiple unit coordinate areas includes: replacing the image of the unit coordinate area in the original image with the image of the unit coordinate area in the reference image to obtain an intermediate image corresponding to the unit coordinate area; using the image recognition model to determine the similarity between the intermediate image corresponding to the unit coordinate area and the reference image to obtain the similarity corresponding to the unit coordinate area; A selection module is used to select the x unit coordinate areas with the largest similarities among the similarities corresponding to the multiple unit coordinate areas as the recognition coordinate areas of the image recognition model, where x is an integer greater than or equal to 1 and x is less than the number of the unit coordinate areas.

13. The device according to claim 11 or 12, characterized in that m>n>1, the first determining module is used for: Determine at least one target unit coordinate area among the multiple unit coordinate areas; the target unit coordinate area belongs to the recognition coordinate areas of at least two image recognition models; When the area ratio of the image of the at least one target unit coordinate region in the original image is greater than or equal to a ratio threshold, shrinking the at least one target unit coordinate region to obtain the key coordinate region; When the area ratio of the image of the at least one target unit coordinate region in the original image is less than the ratio threshold, the at least one target unit coordinate region is determined as the key coordinate region.

14. An image processing device, characterized in that: The image processing device comprises a processor and a memory, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement the image processing method according to any one of claims 1 to 10.

15. A non-transitory computer-readable storage medium, characterized in that: The storage medium stores instructions, and the instructions are loaded and executed by the processor to implement the image processing method according to any one of claims 1 to 10.

Images