An image processing method, apparatus, electronic device, and storage medium

Abstract

The specification discloses an image processing method and device, electronic equipment and storage medium. The method comprises: generating an initial attention map corresponding to a target image based on an attention neural network for a target content entity, wherein the value of a target pixel parameter in the initial attention map reflects the probability of the pixel being extracted by the attention neural network. The pixel in the initial attention map that meets the preset value standard of the target pixel parameter is set as a target gray scale, and a processed attention map corresponding to the initial attention map is obtained. Based on the pixel of the target gray scale in the processed attention map, a content detection region is determined. The content detection region in the target image is detected for the target content entity.

Description

Technical Field

[0001] This document belongs to the field of artificial intelligence technology, and in particular relates to an image processing method, apparatus, electronic device and storage medium. Background Technology

[0002] With the development of artificial intelligence technology, machine-based image processing applications are becoming increasingly widespread. Content detection is one such common image processing technique. As the name suggests, content detection uses artificial intelligence models to detect content within images.

[0003] Currently, for a model to accurately detect content in an image, it needs to be trained using sample images with finely annotated content locations. However, model training requires a large number of sample images, resulting in a significant workload for annotation and making content detection applications a high-barrier endeavor.

[0004] Therefore, how to solve the problem of content detection models relying on sample images for training to identify content locations is the technical problem that this application aims to solve. Summary of the Invention

[0005] This specification provides an image processing method, apparatus, electronic device, and storage medium that enables a content detection model to determine the location of content in an image without relying on sample image training.

[0006] To achieve the above-mentioned objectives, the embodiments in this specification are implemented as follows:

[0007] Firstly, an image processing method is proposed, including:

[0008] Based on the attention neural network targeting the target content entity, an initial attention map corresponding to the target image is generated. The pixel in the initial attention map reflects the probability of the pixel being extracted by the attention neural network through the value of the target pixel parameter.

[0009] Pixels whose target pixel parameters in the initial attention map meet the preset value standard are set as target grayscale to obtain the processed attention map corresponding to the initial attention map.

[0010] Based on the target grayscale pixels in the processed attention map, the content detection region is determined;

[0011] Content detection of the target content entity is performed on the corresponding content detection region in the target image.

[0012] Secondly, an image processing apparatus is proposed, comprising:

[0013] The attention map generation module generates an initial attention map corresponding to the target image based on an attention neural network for the target content entity. The pixels in the initial attention map reflect the probability of the pixel being extracted by the attention neural network through the value of the target pixel parameter.

[0014] The attention map processing module sets the pixels whose target pixel parameters in the initial attention map reach a preset value standard as the target grayscale, thereby obtaining the processed attention map corresponding to the initial attention map.

[0015] The detection region determination module determines the content detection region based on the target grayscale pixels in the processed attention map;

[0016] The detection execution module performs content detection on the target content entity in the corresponding content detection region of the target image.

[0017] Thirdly, an electronic device is proposed, comprising: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the following operations:

[0018] Based on the attention neural network targeting the target content entity, an initial attention map corresponding to the target image is generated. The pixel in the initial attention map reflects the probability of the pixel being extracted by the attention neural network through the value of the target pixel parameter.

[0019] Pixels whose target pixel parameters in the initial attention map meet the preset value standard are set as target grayscale to obtain the processed attention map corresponding to the initial attention map.

[0020] Based on the target grayscale pixels in the processed attention map, the content detection region is determined;

[0021] Content detection of the target content entity is performed on the corresponding content detection region in the target image.

[0022] Fourthly, a computer-readable storage medium is proposed, which stores computer-executable instructions that, when executed by a processor, perform the following operations:

[0023] Based on the attention neural network targeting the target content entity, an initial attention map corresponding to the target image is generated. The pixel in the initial attention map reflects the probability of the pixel being extracted by the attention neural network through the value of the target pixel parameter.

[0024] Pixels whose target pixel parameters in the initial attention map meet the preset value standard are set as target grayscale to obtain the processed attention map corresponding to the initial attention map.

[0025] Based on the target grayscale pixels in the processed attention map, the content detection region is determined;

[0026] Content detection of the target content entity is performed on the corresponding content detection region in the target image.

[0027] The solution in this specification, when performing content detection on a target image for target content entities, uses an attention neural network for the target content entities to generate an initial attention map corresponding to the target image. In this initial attention map, the pixel values ​​under the target pixel parameters reflect the probability that the pixel will be feature-extracted by the attention neural network. Since pixels with a high probability of feature extraction by the attention neural network are more likely to belong to the target content entity, the next step is simply to set the pixels in the initial attention map whose target pixel parameters meet a preset value standard to the target grayscale, converting them into a processed attention map. Based on the pixels with the target grayscale in the processed attention map, the accurate content detection region can be determined. Finally, the content detection model only needs to be used to attempt to identify the target content entity in the corresponding content detection region of the target image to complete the content detection. The entire solution does not require using sample images with finely annotated content locations to train the content detection model, thus greatly improving the practicality of content detection applications. Attached Figure Description

[0028] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and their descriptions, serving to explain this specification and do not constitute an undue limitation thereof.

[0029] In the picture:

[0030] Figure 1 This is a schematic flowchart of the image processing method according to an embodiment of this specification.

[0031] Figure 2 This is a schematic diagram illustrating the first application of the image processing method in the embodiments of this specification.

[0032] Figure 3 This is a schematic diagram illustrating a second application of the image processing method described in this specification.

[0033] Figure 4 This is a schematic diagram illustrating the third application of the image processing method in the embodiments of this specification.

[0034] Figure 5This is a schematic diagram of the structure of the image processing apparatus according to an embodiment of this specification.

[0035] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this specification. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this document clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments in this document, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this document.

[0037] As mentioned earlier, to enable a model to accurately detect content in an image, it is necessary to train the model's ability to identify content locations using sample images with finely annotated content locations. However, model training requires a large number of sample images, resulting in a huge workload for annotation and making the application of content detection a high barrier to entry.

[0038] Therefore, this specification aims to propose a novel content detection scheme that can train a model to identify the location of content in an image without relying on sample images.

[0039] On the one hand, embodiments of this specification provide an image processing method, which can be executed by the apparatus described below. Wherein, Figure 1 This is a flowchart illustrating an image processing method, which includes the following steps:

[0040] S102, Based on the attention neural network for the target content entity, an initial attention map corresponding to the target image is generated. The pixel in the initial attention map reflects the probability of the pixel being extracted by the attention neural network through the value of the target pixel parameter.

[0041] In this embodiment, the attention neural network for the target content entity is pre-built. This step involves inputting the target image data into the attention neural network, which then determines the attention value of each pixel in the target image. Subsequently, the target pixel parameters are used to reflect the magnitude of the attention value to generate an initial attention map corresponding to the target image.

[0042] Here, we take the target pixel parameter as grayscale as an example. In this embodiment, grayscale can be used to represent the attention value. That is, the larger the grayscale, the larger the attention value; the smaller the grayscale, the smaller the attention value. Since the attention value of a pixel reflects the probability that the pixel will be used for feature extraction by the attention neural network, for an attention neural network targeting target content entities, pixels belonging to the target content entity area obviously have a higher probability of being extracted, and their corresponding grayscale values ​​are larger and closer to white; while pixels not belonging to the target content entity area have a lower probability of being extracted, and their corresponding grayscale values ​​are smaller and closer to black.

[0043] For example Figure 2 As shown, assuming the target image represents the moon, when the target image is imported into an attention neural network designed for the moon, pixels closer to white in the generated initial attention map are more easily extracted by the attention neural network. It can be seen that... Figure 2 The initial attention map in the image shows a general outline of the moon.

[0044] In practical applications, the attention neural network for the target content entity in this embodiment can come from existing models. For example, the text-to-image model of AI-Generated Content (AIGC) technology.

[0045] This example uses the open-source Stable Diffusion model. Currently, the Stable Diffusion model primarily offers the following two functionalities.

[0046] One function is to generate corresponding creative images based on the descriptive prompts in the text guidance information;

[0047] Another function is to modify an existing image based on the descriptive prompts in the text guidance information.

[0048] For the latter function, the Stable Diffusion model takes both text guidance information and an image as input. The basic principle of the Stable Diffusion model is to encode the text guidance information and the image separately, and then fuse and decode them based on a cross-modal attention mechanism between text and image. This allows the creation of content corresponding to the text guidance information based on the input image.

[0049] In this embodiment, it is only necessary to import the text guidance information describing the image of the target content entity and the target image into the text-to-image model. The attention neural network of the Stable Diffusion model can then generate the initial attention map of the target content entity corresponding to the target image based on the cross-modal cross attention between the text identifier of the target content entity in the text guidance information and the target image.

[0050] for example Figure 2 The target image shown can be input into the Stable Diffusion model along with the natural language phrase "a photo of Mooncelestialbody". The Stable Diffusion model's attention neural network is then used to generate an initial attention map based on the cross-modal cross-attention between the Moon text identifier "Moon" and the target image.

[0051] It should be noted that, Figure 2 Using grayscale to represent attention is merely to make it easier for the human eye to see the difference in the probability of pixels being extracted for features in the initial attention map. In practical applications, machines do not need to make distinctions with the human eye, so the target pixel parameters can also be brightness, color, etc., which are not specifically limited here.

[0052] S104, set the pixels in the initial attention map whose target pixel parameters meet the preset value standard as the target grayscale, and obtain the processed attention map corresponding to the initial attention map.

[0053] For ease of understanding, we will use a binary attention map after processing as an example. Regarding... Figure 2 The initial attention map shown can be used in this step to set pixels with a grayscale value (probability of being extracted as a feature) lower than a preset standard to the lowest grayscale value of 0, meaning pixels with a low probability of belonging to the "moon" are uniformly set to black; at the same time, pixels with a grayscale value reaching the preset standard are set to the highest grayscale value of 255, meaning pixels with a high probability of belonging to the "moon" are uniformly set to white, thus obtaining the desired result. Figure 2 The modified attention map is shown.

[0054] from Figure 2 As can be seen, the modified attention map can clearly present the outline of the content entity "moon".

[0055] Similarly, Figure 2 The binary image is used as an example of the post-processed attention map only to make it easier for the human eye to see the difference between pixels belonging to the "moon" and other pixels. In practical applications, the machine does not need to make this distinction with the human eye, so it is only necessary to set the target grayscale to pixels whose target pixel parameters meet the preset value standard.

[0056] S106, Determine the content detection region based on the target grayscale pixels in the processed attention map.

[0057] still Figure 2 For example, the processed attention map clearly shows the outline of the "moon". Therefore, this step only needs to determine the pixel area of ​​the target grayscale in the processed attention map as the content detection area.

[0058] Here, assuming the content detection region is a box, the minimum x-coordinate of the box is the minimum x-coordinate corresponding to the target grayscale pixel in the processed attention map, and the maximum x-coordinate of the box is the maximum x-coordinate corresponding to the target grayscale pixel in the processed attention map; similarly, the minimum y-coordinate of the box is the minimum y-coordinate corresponding to the target grayscale pixel in the processed attention map, and the maximum y-coordinate of the box is the maximum y-coordinate corresponding to the target grayscale pixel in the processed attention map.

[0059] Furthermore, in this embodiment, if there are at least two target grayscale pixel regions in the processed attention map, then each target grayscale pixel region can be used as a content detection region.

[0060] Furthermore, considering that attention neural networks have certain errors, resulting in some small areas of target grayscale pixels appearing in the processed attention map, these small areas of target grayscale pixels can be treated as noise and filtered out. That is, in this embodiment, the target grayscale pixel areas that do not meet the preset size standard in the processed attention map are removed, and the remaining target grayscale pixel areas in the processed attention map are determined as content detection areas.

[0061] S108, perform content detection on the corresponding content detection region in the target image for the target content entity.

[0062] still Figure 2 For example, after generating the processed attention map of the content entity "Moon", the "Moon" can be identified based on the content detection area selected in the processed attention map.

[0063] Based on the above, it can be seen that the method in this embodiment, when performing content detection on a target image for target content entities, uses an attention neural network for the target content entities to generate an initial attention map corresponding to the target image. In this initial attention map, the pixel's probability of being feature-extracted by the attention neural network is reflected by the value of its target pixel parameter. Since pixels with a high probability of feature extraction by the attention neural network are more likely to belong to the target content entity, it is only necessary to set the pixels in the initial attention map whose target pixel parameters meet a preset value standard to target grayscale to convert them into a processed attention map. Based on the pixels with target grayscale in the processed attention map, the accurate content detection region can be determined. Finally, the content detection model only needs to be used to attempt to identify the target content entity in the corresponding content detection region of the target image to complete the content detection. The entire scheme does not require using sample images with finely annotated content locations to train the content detection model, thus greatly improving the practicality of content detection applications.

[0064] Furthermore, the method of this embodiment can be applied to scenarios involving the review of images containing illegal content.

[0065] The following example illustrates the application scenarios for reviewing whether a target image contains illegal content.

[0066] This application scenario uses content detection technology to identify whether a target image contains target content entities that belong to the category of illegal content.

[0067] Here, assuming the target image shows a table with the target entity (contraband) placed on it, the process is as follows:

[0068] Phase 1: Generating the initial attention map of the target image based on the Stable Diffusion model.

[0069] This stage first generates the natural language translation "a photo of Illegal items on table" for the Stable Diffusion model based on the scene presented in the target image.

[0070] Next, the natural language phrase "a photo of Illegal items on table" and the target image are imported into the Stable Diffusion model, which then guides the Stable Diffusion model to create another image of "the target content entity placed on the table".

[0071] For this application scenario, we need to focus on the images created by the Stable Diffusion model. We only need to obtain the attention map of its attention neural network during the calculation process.

[0072] As mentioned earlier, the Stable Diffusion model extracts features based on cross-modal cross-attention between text labels and images. Therefore, the Stable Diffusion model will... Figure 3 As shown, the initial attention map corresponding to each text identifier in "a photo of Illegal items on table" is generated, as well as the initial attention map of some unique identifier bits generated by the Stable Diffusion model for recognizing natural language (this paper does not focus on the initial attention map of the identifier bits).

[0073] Based on the current capabilities of the Stable Diffusion model, the initial attention for different text identifiers presents an initial attention map related to their text meaning. Assuming... Figure 3 The initial attention map generated in the process visualizes the probability of a pixel being extracted by a feature. For example, the initial attention map for the text label "table" will show the outline of a table, while this stage only needs to extract the initial attention map for the text label "Illegalitems".

[0074] Phase Two: Create a modified attention map of the target image to determine the content detection region.

[0075] refer to Figure 4 As shown, this stage performs a binary image transformation on the initial attention map of the text identifier "Illegalitems". That is, the pixels in the initial attention map of "Illegalitems" that have reached a certain attention threshold are set to white, and the remaining pixels are set to white, so as to obtain a modified attention map that clearly presents the outline of "Illegalitems".

[0076] Then, the content detection region is constructed using the white pixels in the modified attention map according to "Illegalitems".

[0077] For example, a bounding box can be constructed based on the maximum and minimum coordinates of the white pixels in the modified attention map, and the area selected by this bounding box becomes the content detection region. Alternatively, the white pixel area in the modified attention map can be directly used as the content detection region, meaning the content detection region has the same shape as "Illegalitems".

[0078] Phase 3 involves identifying the target content entities within the corresponding content detection region of the target image.

[0079] After determining the content detection region in this stage, it can be mapped to the target image. Then, the content detection region is extracted from the target image and input into a classifier used to identify target content entities, thus obtaining the content detection result of whether the target image contains illegal items.

[0080] The above is an introduction to reviewing whether a target image contains inappropriate content. In this application scenario, after the content detection result for the target image indicates that it contains the target content entity, a matching violation processing operation can be performed on the target image.

[0081] For example, if the target image is published on an online platform, the violation handling actions include at least one of the following:

[0082] Delete the metadata of the target image in the network platform;

[0083] The network platform accounts that published the target image will be banned;

[0084] The network platform account that published the target image was downgraded in terms of access privileges.

[0085] For example, if the target image is carried in the target business request, the violation handling operations include:

[0086] The request for the target business was rejected.

[0087] On the other hand, corresponding to Figure 1 The method shown in this specification provides an image processing apparatus according to embodiments. Wherein, Figure 5 This is a schematic diagram of the structure of image processing, including:

[0088] The attention map generation module 510 generates an initial attention map corresponding to the target image based on an attention neural network for the target content entity. The pixels in the initial attention map reflect the probability of feature extraction by the attention neural network through the value of the target pixel parameter.

[0089] The attention map processing module 520 sets the target pixel parameters that meet the preset value standard in the initial attention map as the target grayscale, thereby obtaining the processed attention map corresponding to the initial attention map.

[0090] The detection region determination module 530 determines the content detection region based on the target grayscale pixels in the processed attention map.

[0091] The detection execution module 540 performs content detection on the target content entity in the corresponding content detection region of the target image.

[0092] The apparatus in this embodiment of the specification, when performing content detection on a target image for target content entities, uses an attention neural network for the target content entities to generate an initial attention map corresponding to the target image. In this initial attention map, the pixel values ​​under the target pixel parameters reflect the probability that the pixel will be feature-extracted by the attention neural network. Since pixels with a high probability of feature extraction by the attention neural network are more likely to belong to the target content entity, it is only necessary to set the pixels in the initial attention map whose target pixel parameters meet a preset value standard to the target grayscale to convert them into a processed attention map. Based on the pixels with the target grayscale in the processed attention map, the accurate content detection region can be determined. Finally, the content detection model only needs to be used to attempt to identify the target content entity in the corresponding content detection region of the target image to complete the content detection. The entire scheme does not require using sample images with finely annotated content locations to train the content detection model, thus greatly improving the practicality of content detection-related applications.

[0093] Optionally, the attention neural network comes from a text-to-image model; the attention map generation module 510 generates an initial attention map of the target image corresponding to the target content entity based on the attention neural network for the target content entity, including: importing text guidance information describing the image for generating the target content entity and the target image into the text-to-image model, so that the attention neural network generates an initial attention map of the target image corresponding to the target content entity based on the cross-modal cross attention between the text identifier corresponding to the target content entity in the text guidance information and the target image.

[0094] Optionally, the detection region determination module 530 determines the content detection region based on the target grayscale pixels in the processed attention map, including: determining the target grayscale pixel region in the processed attention map as the content detection region.

[0095] Optionally, if there are at least two target grayscale pixel regions in the processed attention map, the step of determining the target grayscale pixel regions in the processed attention map as content detection regions by the detection region determination module 530 includes: removing the target grayscale pixel regions in the processed attention map that do not meet the preset size standard, and determining the remaining target grayscale pixel regions in the processed attention map as content detection regions.

[0096] Optionally, the target content entity belongs to the category of illegal content; the apparatus in this embodiment further includes:

[0097] The violation handling module performs a matching violation handling operation on the target image if the content detection result corresponding to the target image indicates that the target content entity is contained.

[0098] Optionally, the target image is published on a network platform, and the violation handling operation includes at least one of the following:

[0099] Delete the metadata of the target image in the network platform;

[0100] The network platform accounts that published the target image will be banned;

[0101] The network platform account that published the target image was downgraded in terms of access privileges.

[0102] Optionally, the target image is carried in the target service request, and the violation handling operation includes:

[0103] The requested service request was rejected.

[0104] It should be understood that the image processing apparatus in the embodiments of this specification can be used as... Figure 1 The entity executing the method shown is therefore capable of implementing the corresponding steps and functions in the method, which will not be elaborated here.

[0105] Figure 6 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this specification. Please refer to it. Figure 6 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0106] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0107] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0108] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming the exception handling device of the aforementioned business system at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0109] An initial attention map corresponding to the target image is generated based on an attention neural network targeting the target content entity. The pixel in the initial attention map reflects the probability that the pixel will be used for feature extraction by the attention neural network through the value of the target pixel parameter.

[0110] Pixels whose target pixel parameters in the initial attention map meet the preset value standard are set as target grayscale to obtain the processed attention map corresponding to the initial attention map.

[0111] Based on the target grayscale pixels in the processed attention map, the content detection region is determined.

[0112] Content detection of the target content entity is performed on the corresponding content detection region in the target image.

[0113] When performing content detection on a target image, the electronic device in this embodiment uses an attention neural network for the target content entity to generate an initial attention map corresponding to the target image. In this initial attention map, the pixel's probability of being feature-extracted by the attention neural network is reflected by the value of its target pixel parameter. Since pixels with a high probability of feature extraction by the attention neural network are more likely to belong to the target content entity, the next step is to set the pixels in the initial attention map whose target pixel parameter reaches a preset value standard to a target grayscale to convert it into a processed attention map. Based on the pixels with target grayscale in the processed attention map, the accurate content detection region can be determined. Finally, the content detection model only needs to be used to attempt to identify the target content entity in the corresponding content detection region of the target image to complete the content detection. The entire scheme does not require training the content detection model with finely annotated sample images of content locations, thus greatly improving the practicality of content detection applications.

[0114] The above is as described in this instruction manual. Figure 1The methods of the illustrated embodiments can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in one or more embodiments of this specification. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in one or more embodiments of this specification can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0115] The electronic device can also perform Figure 1 The methods described herein will not be elaborated further.

[0116] Of course, in addition to software implementation, the electronic device described in this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0117] This specification also provides an embodiment of a computer-readable storage medium that stores one or more programs.

[0118] The program includes instructions that, when executed by a portable electronic device comprising multiple applications, enable the portable electronic device to perform... Figure 1 The method of the illustrated embodiment is specifically used to perform the following operations:

[0119] An initial attention map corresponding to the target image is generated based on an attention neural network targeting the target content entity. The pixel in the initial attention map reflects the probability that the pixel will be used for feature extraction by the attention neural network through the value of the target pixel parameter.

[0120] Pixels whose target pixel parameters in the initial attention map meet the preset value standard are set as target grayscale to obtain the processed attention map corresponding to the initial attention map.

[0121] Based on the target grayscale pixels in the processed attention map, the content detection region is determined.

[0122] Content detection of the target content entity is performed on the corresponding content detection region in the target image.

[0123] In summary, the above description is merely a preferred embodiment of this specification and is not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the scope of protection of one or more embodiments of this specification.

[0124] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0125] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0126] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0127] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. An image processing method, comprising: generating an initial attention map corresponding to a target image based on an attention neural network for a target content entity, wherein a pixel in the initial attention map reflects a probability of the pixel being extracted by the attention neural network for feature extraction through a value of a target pixel parameter; setting a pixel in the initial attention map, for which the target pixel parameter reaches a preset value standard, as a target gray scale to obtain a processed attention map corresponding to the initial attention map; determining a content detection region based on a pixel of the target gray scale in the processed attention map; and performing content detection for the target content entity on a corresponding content detection region in the target image.

2. The method of claim 1, the attention neural network is from a text-to-image model; the generating of the initial attention map corresponding to the target content entity of the target image based on the attention neural network for the target content entity comprises: introducing text guidance information used to describe a generated image of the target content entity and the target image into the text-to-image model to generate the initial attention map corresponding to the target content entity of the target image by the attention neural network according to cross-modal cross-attention between the target content entity and the target image corresponding to a text identifier of the target content entity in the text guidance information.

3. The method of claim 1, the determining of the content detection region based on the pixel of the target gray scale in the processed attention map comprises: determining a pixel region of the target gray scale in the processed attention map as the content detection region.

4. The method of claim 3, if the pixel region of the target gray scale in the processed attention map is not less than two, the determining of the content detection region from the pixel region of the target gray scale in the processed attention map comprises: removing a pixel region of the target gray scale in the processed attention map that does not reach a preset size standard, and determining a remaining pixel region of the target gray scale in the processed attention map as the content detection region.

5. The method of any one of claims 1 to 4, the target content entity belongs to a violation content category; the method further comprises: if a content detection result corresponding to the target image indicates that the target content entity is contained, performing a matching violation processing operation on the target image.

6. The method of claim 5, the target image is published on a network platform, and the violation processing operation comprises at least one of: deleting metadata of the target image in the network platform; banning a network platform account that publishes the target image; degrading the authority of the network platform account that publishes the target image.

7. The method of claim 6, the target image is carried in a target service request, and the violation processing operation comprises: rejecting the target service request.

8. An image processing apparatus, comprising: An attention map generation module generates an initial attention map corresponding to the target image based on an attention neural network for the target content entity, wherein a pixel in the initial attention map reflects a probability of being extracted by the attention neural network for a feature through a value size under a target pixel parameter; An attention map processing module sets a pixel in the initial attention map, in which a target pixel parameter reaches a preset value standard, as a target gray scale to obtain a processed attention map corresponding to the initial attention map; A detection region determination module determines a content detection region based on the pixel of the target gray scale in the processed attention map; A detection execution module performs content detection for the target content entity on the content detection region corresponding to the target image.

9. An electronic device comprising: A processor; and a memory arranged to store computer-executable instructions that, when executed, cause the processor to perform the following operations: generate an initial attention map corresponding to the target image based on an attention neural network for the target content entity, wherein a pixel in the initial attention map reflects a probability of being extracted by the attention neural network for a feature through a value size under a target pixel parameter; set a pixel in the initial attention map, in which a target pixel parameter reaches a preset value standard, as a target gray scale to obtain a processed attention map corresponding to the initial attention map; determine a content detection region based on the pixel of the target gray scale in the processed attention map; perform content detection for the target content entity on the content detection region corresponding to the target image.

10. A computer-readable storage medium for storing computer-executable instructions that, when executed by a processor, implement the following operations: generating, based on an attention neural network for the target content entity, an initial attention map corresponding to the target image, wherein a pixel in the initial attention map reflects a probability of being extracted by the attention neural network for a feature through a value size under a target pixel parameter; set a pixel in the initial attention map, in which a target pixel parameter reaches a preset value standard, as a target gray scale to obtain a processed attention map corresponding to the initial attention map; determine a content detection region based on the pixel of the target gray scale in the processed attention map; perform content detection for the target content entity on the content detection region corresponding to the target image.

Images