Illegal image recognition method, device, electronic device and program product

Abstract

The present application provides a method, device, equipment and program product for identifying illegal images. The method includes: performing illegal identification on a target image based on at least two illegal identification models to obtain corresponding at least two initial illegal identification results; wherein the at least two illegal identification models are provided with a first enhanced feature extraction module, the first enhanced feature extraction module is used to upsample the input data, and perform feature extraction on the upsampled results based on a spatial attention mechanism, and the feature extraction results are used for illegal identification calculations. Based on the at least two initial illegal identification results, a target illegal identification result corresponding to the target image is determined. The present application is used to solve the problems of low efficiency and poor accuracy in manual detection of illegal images.

Description

Technical Field

[0001] The present application relates to the field of risk control, and in particular to a method, device, equipment and program product for identifying illegal images. Background Art

[0002] Illegal images are those that contain information that is prohibited in business scenarios.

[0003] Currently, illegal images mainly rely on manual detection, but due to the huge number of images on the Internet, manual detection lacks practicality due to its low efficiency and the problem of misjudgment. Summary of the invention

[0004] The purpose of this application is to provide a method, device, equipment and program product for identifying illegal images, which can solve the problems of low efficiency and poor accuracy in manual detection of illegal images.

[0005] In order to achieve the above purpose, the embodiment of the present application is implemented as follows: In a first aspect, a method for identifying illegal images is provided, comprising: Based on at least two violation recognition models, the target image is subjected to violation recognition to obtain at least two corresponding initial violation recognition results; wherein the at least two violation recognition models are provided with a first enhanced feature extraction module, the first enhanced feature extraction module is used to upsample the input data, and perform feature extraction on the upsampled result based on a spatial attention mechanism, and the feature extraction result is used for violation recognition calculation; Based on the at least two initial violation recognition results, a target violation recognition result corresponding to the target image is determined.

[0006] In a second aspect, a device for determining illegal web pages is provided, comprising: A computing module, which performs violation recognition on a target image based on at least two violation recognition models to obtain corresponding at least two initial violation recognition results; wherein the at least two violation recognition models are provided with a first enhanced feature extraction module, the first enhanced feature extraction module is used to upsample input data, and perform feature extraction on the upsampled result based on a spatial attention mechanism, and the feature extraction result is used for violation recognition calculation; A decision module is used to determine a target violation recognition result corresponding to the target image based on the at least two initial violation recognition results.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor; and a memory configured to store computer-executable instructions, wherein the computer-executable instructions, when executed, cause the processor to execute the method described in the first aspect.

[0008] According to a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions implement the method described in the first aspect when executed by a processor.

[0009] In order to solve the problems of low efficiency and poor accuracy in manual detection of illegal images, the embodiments of the present application propose a multi-model violation recognition scheme based on a special attention mechanism. That is, at least two violation recognition models are prepared, and a first enhanced feature extraction module is deployed in each violation recognition model to upsample the input data, and feature extraction is performed on the upsampled results based on the spatial attention mechanism, and the feature extraction results are used for violation recognition calculations. Among them, upsampling and spatial attention can make the model focus on areas with low resolution and small targets in the target image to improve the problem of the model's easy loss of information. Then, by weighing the initial violation recognition results given by different violation recognition models for the target image, a high-accuracy target violation recognition result can be determined. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0011] Figure 1 This is a schematic diagram of the first flow chart of the illegal image recognition method according to an embodiment of the present application.

[0012] Figure 2 A schematic diagram for extracting information from illegal graphics.

[0013] Figure 3 This is a schematic diagram of the structure of the Resnet-50 model of an embodiment of the present application.

[0014] Figure 4 This is a schematic diagram of the structure of the first enhanced feature extraction module in the Resnet-50 model of an embodiment of the present application.

[0015] Figure 5 Schematic diagram of the structure of the YOLOV5 model of the embodiment of the present application.

[0016] Figure 6 This is a schematic diagram of the structure of the first enhanced feature extraction module in the YOLOV5 model of an embodiment of the present application.

[0017] Figure 7 This is a schematic diagram of the structure of the second enhanced feature extraction module in the YOLOV5 model of an embodiment of the present application.

[0018] Figure 8 This is a second flow chart of the illegal image recognition method according to an embodiment of the present application.

[0019] Fig. 9 Schematic diagram of the structure of the illegal image recognition device according to an embodiment of the present application.

[0020] Fig.10 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In response to the problems of low efficiency and poor accuracy in manual detection of illegal images, this application proposes an illegal image recognition solution executed by a machine based on artificial intelligence technology, which can realize compliance detection of image content with a multi-model with an improved attention mechanism.

[0022] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0023] An embodiment of the present application provides a method for identifying illegal images. Figure 1 : is a flowchart of the illegal image recognition method, including the following steps: S102, performing violation identification on the target image based on at least two violation identification models to obtain corresponding at least two initial violation identification results; wherein, at least two violation identification models are provided with a first enhanced feature extraction module, the first enhanced feature extraction module is used to upsample the input data, and perform feature extraction on the upsampled result based on the spatial attention mechanism, and the feature extraction result is used for violation identification calculation.

[0024] The at least two violation recognition models described in this embodiment can be used to identify the same type of violation images or different types of violation images. In actual scenarios, some violation images use specific methods to conceal violation information. Considering that a single model may not be able to effectively identify due to limitations, a multi-model with an improved attention mechanism is used to achieve collaborative compliance detection, so as to comprehensively utilize the advantages of different models to improve recognition accuracy.

[0025] The improved attention mechanism described here is implemented by the first enhanced feature extraction module. The first enhanced feature extraction module can supplement the missing pixel information for the low-resolution image area through upsampling operation, making it easier for the violation recognition model to recognize and extract features in the image; in addition, feature extraction of the upsampling results based on spatial attention can focus on specific areas of small targets in the image in a targeted manner, thereby improving the accuracy of compliance detection.

[0026] for example Figure 2 As shown in the figure, when the illegal content and regular content in the image are separated and mixed together to a certain extent, the content extracted by ORC technology may not be recognized by the conventional illegal recognition model. However, after the first enhanced feature extraction module is introduced, feature extraction is performed with the help of the first enhanced feature extraction module. The attention weight can be calculated according to the correlation between the small target content and the small target content in the image. The higher the correlation, the higher the attention weight. Therefore, the whole block of content with a higher attention weight is encoded to obtain the feature extraction result of the complete illegal content.

[0027] It should be noted that how the first enhanced feature extraction module of this embodiment is set in the violation identification model depends on the specific structure of the violation identification model, which is not limited in this article.

[0028] The following describes the configuration of the first enhanced feature extraction module of this embodiment in combination with two implementation methods.

[0029] Implementation method 1 In this embodiment, one of the violation identification models is a Resnet-50 model. ResNet50 is a typical image recognition task model, which consists of five stages (stage0 to stage4). Each stage gradually extracts image features through convolution and pooling operations, and finally classifies through a fully connected layer. Correspondingly, the first enhanced feature extraction module is set at the output end of at least part of the stage modules of the Resnet-50 model. For example Figure 3 As shown, there are two first enhanced feature extraction modules in the Resnet-50 model, one of which is set at the output end of the stage0 module of the Resnet-50 model, and the other is set at the output end of the stage4 module in the Resnet-50 model.

[0030] Specifically, refer to Figure 4 As shown, the first enhanced feature extraction module set in the Resnet-50 model includes: a first branch, a second branch, a third branch, a first dot product unit and a second dot product unit.

[0031] The first branch is used to: perform a first upsampling operation on the input data to obtain first upsampling data; perform a first convolution operation on the first upsampling data to obtain first convolution data; perform a second convolution operation on the first convolution data to obtain second convolution data ; The second branch is used to: perform a second upsampling operation on the input data to obtain second upsampling data; perform a third convolution operation on the second upsampling data to obtain third convolution data; perform a fourth convolution operation on the third convolution data to obtain fourth convolution data ; The third branch is used to: perform a fifth convolution operation on the input data to obtain fifth convolution data; The first dot product unit is used to: perform the second convolution on the data based on the spatial attention mechanism And the fourth convolution data Perform the first dot product operation to obtain the first dot product data containing spatial attention weight information ; The second dot product unit is used to: perform a second dot product operation on the first dot product data and the fifth convolution data based on the spatial attention mechanism to obtain the second dot product data containing the spatial attention weight information , the second dot product data As the feature extraction result of the first enhanced feature extraction module.

[0032] Among them, the first convolution operation and the third convolution operation use a 3×3 convolution kernel, and the second convolution operation, the fourth convolution operation, and the fifth convolution operation use a 1×1 convolution kernel.

[0033] The expression formula involved is as follows:

[0034]

[0035]

[0036]

[0037] in, , For input data, is the image size parameter.

[0038] It should be understood that the upsampling operation in the first enhanced feature extraction module is conducive to retaining more detail information for low-resolution, blurred images; the two convolution information is conducive to extracting feature representations at a higher semantic level; the spatial attention weight information attention map calculated in the feature fusion extraction module fully considers the relationship between each position in the image, reduces the importance of parts that are easily misjudged in the image, and the feature map with spatial weight information obtained after calculation by the self-attention module is used for subsequent classification operations to improve the recognition accuracy.

[0039] Implementation method 2 In this embodiment, one of the violation identification models is the YOLOV5 model. YOLOv5 is a target detection algorithm model, and its network structure mainly includes three parts: the backbone network (Backbone), the neck network (Neck) and the head network (Head). Among them, the backbone network and the neck network are responsible for feature extraction and fusion, and the head network is responsible for prediction, that is, the violation identification calculation described in this article. Since this embodiment does not involve improvements to the head network, the backbone network and the neck network are introduced below: 1. Backbone The backbone network is the feature extraction core of the YOLOv5 model, which is responsible for extracting useful features from the input image. In YOLOv5, the backbone network adopts the CSPNet (Cross Stage Partial Network) structure, which effectively improves the network's feature extraction capability and computational efficiency through cross-stage partial connections.

[0040] The CSPNet structure mainly includes: 1) CSP (Cross Stage Partial Networks) module: It mainly processes the input data in two parts, one part is subjected to normal convolution operation, and the other part is directly spliced ​​with the result of the convolution operation. This design not only retains the diversity of feature information, but also reduces the complexity of the model by reducing redundant calculations. In the CSP structure, different parts of the feature map are processed at different network stages and fused at the final stage, thus realizing cross-stage feature utilization and enhancement.

[0041] 2) CBS (Conv+BatchNorm+SiLU) module: Conv represents convolutional layer (Conv), BatchNorm represents batch normalization layer (BatchNorm), and SiLU represents activation function.

[0042] Conv: Responsible for extracting features from the image. The convolution layer performs local perception and feature extraction on the input feature map through convolution operations to generate a new feature map.

[0043] BatchNorm: used to normalize the output of the convolutional layer. The batch normalization layer can accelerate the convergence of the model, improve the stability of the model, and help prevent overfitting.

[0044] SiLU: Used to increase the nonlinear characteristics of the network. The SiLU function (also known as the Swish function or Sigmoid-weighted Linear Unit) has a linear growth characteristic when the input is positive and tends to zero when the input is negative, which helps the model capture complex feature relationships.

[0045] 3) SPPF (Spatial Pyramid Pooling Fast) module: It is used to divide the input feature map into sub-regions of different sizes through pooling operations of different scales, and pool each sub-region to capture contextual information of different scales. This helps the model better handle objects of different sizes and improves the accuracy of detection.

[0046] 2. Neck network The neck network is located between the backbone network and the head network. It is responsible for further fusing and enhancing features to improve detection performance. In YOLOv5, the neck network adopts a hybrid structure of FPN (Feature Pyramid Networks) and PAN (Path Aggregation Network).

[0047] 1) FPN structure: FPN passes high-level semantic features to low-level layers through a top-down path, enhancing the semantic information of low-level feature maps. This structure helps the model detect small targets more accurately. The FPN structure includes: CBS module and upsampling module. The role of the CBS module is described above; the upsampling module is mainly used to restore low-resolution feature maps to high resolution for better feature fusion and target detection. Figure 5 As shown, this embodiment uses the Deconv module as the upsampling module. Deconv is a deconvolution operation. Compared with upsampling, deconvolution can more accurately reconstruct the original image or feature map and reduce information loss to a certain extent. The deconvolution parameters are: kernel=3, step=2, and padding=1. In addition, other modules in the neck network can also use deconvolution to perform upsampling operations.

[0048] 2) PAN structure: PAN passes low-level graphic features to high-level ones through a bottom-up path, achieving bidirectional fusion of features. This structure further enhances the network's feature extraction capabilities and helps improve the detection accuracy of the model. The PAN structure includes CSP module, CBS module, and Concat module. The functions of the CSP module and CBS module are described above; the Concat module is used to perform connection operations (also called splicing operations). In neural networks, Concat is usually used to fuse feature maps of different layers or scales, thereby enhancing the model's feature extraction capabilities.

[0049] In summary, the backbone network and neck network of the YOLOv5 model work together to achieve efficient feature extraction and fusion of the input image. By optimizing the network structure and adopting advanced components, YOLOv5 has achieved outstanding performance in target detection tasks.

[0050] refer to Figure 5 As shown, the first enhanced feature extraction module of this embodiment replaces the CSP module of the backbone network of the YOLOV5 model.

[0051] Specifically, refer to Figure 6 As shown, the first enhanced feature extraction module set in the YOLOV5 model backbone network includes: a fourth branch, a fifth branch and a first connection unit.

[0052] The fourth branch is used to perform the first CBS operation on the input data (the CBS operation includes convolution calculation, batch normalization calculation and activation function calculation) to obtain the first CBS data ; Perform multiple first composite CBS operations on the first CBS data to obtain multiple first composite CBS data ; Perform a first connection operation on the first CBS data and the first composite CBS data to obtain the first connection data ; The first composite CBS operation includes at least two CBS operations performed consecutively.

[0053] The fifth branch is used to: perform a third upsampling operation on the input data to obtain third upsampling data; perform a second CBS operation on the third upsampling data to obtain second CBS data; perform a third CBS operation on the second CBS data to obtain third CBS data; perform a first sigmod function calculation operation on the third CBS data to obtain first sigmod function calculation data ; Based on the spatial attention mechanism, the third dot product operation is performed on the first sigmoid function calculation number to obtain the third dot product data containing the spatial attention weight information ; The first connection unit is used to: based on the spatial attention mechanism, And the third dot product data A second point connection operation is performed to obtain second connection data containing spatial attention weight information, and the second connection data is used as the feature extraction result of the first enhanced feature extraction module.

[0054] Among them, the first CBS operation and the third CBS operation use a 3×3 convolution kernel; the second CBS operation uses a 1×1 convolution kernel; the first composite CBS operation includes executing two CBS operations in succession, the first CBS operation uses a 1×1 convolution kernel, and the second CBS operation uses a 3×3 convolution kernel.

[0055] The expression formula involved is as follows: ))

[0056]

[0057]

[0058] Sigmod

[0059]

[0060] Sigmod

[0061]

[0062] Among them, n represents the convolution kernel size, Represents input data.

[0063] It should be understood that the first enhanced feature extraction module can supplement the missing pixel information for the low-resolution image area through upsampling operation, making it easier for the model to recognize and extract features in the image; the two convolution operations calculate the weight information of different positions in the image, so that the network pays more attention to the image information of sensitive parts, and obtains the image with importance information. ,The backbone of the improved spatial attention module can focus on specific areas in the image containing small objects, improving the image quality and the detection accuracy.

[0064] Further references Figure 5 As shown, in addition to setting the first enhanced feature extraction module, this embodiment can also replace the CSP module in Neck with a second enhanced feature extraction module, where the second enhanced feature extraction module is used to upsample the input data and perform feature extraction on the upsampled results based on the channel attention mechanism.

[0065] in, Figure 7 The structure of the second enhanced feature extraction module is illustrated, and the second enhanced feature extraction module includes: a sixth branch, a seventh branch, and a second connection unit. The sixth branch is used to: perform a fourth CBS operation on the input data to obtain fourth CBS data; perform multiple second composite CBS operations on the fourth CBS data to obtain multiple second composite CBS data; the second composite CBS operation includes at least two CBS operations executed continuously.

[0066] The seventh branch is used to: perform a fourth upsampling operation on the input data to obtain fourth upsampling data; perform a global averaging operation and a global maximum pooling operation on the fourth upsampling data to obtain global average data And global maximum pooling data ; For global average data And global maximum pooling data Perform a sum operation to obtain sum data; perform a multi-channel full connection operation and a second sigmod function calculation operation on the sum data to obtain full connection data and second sigmod function calculation data ; Based on the channel attention mechanism, a fourth dot product operation is performed on the fully connected data and the second sigmoid function calculation data to obtain the fourth dot product data containing the channel attention weight information.

[0067] The second connection unit is used to perform a third connection operation on the plurality of second composite CBS data and the fourth dot product data to obtain third connection data, and the third connection data is used as a feature extraction result of the second enhanced feature extraction module.

[0068] The expression formula involved is as follows: UpSample(X) UpSample(X)

[0069] It should be understood that the second enhanced feature extraction module first performs an upsampling operation to enrich the feature information of the low-resolution feature map through interpolation and other methods, thereby increasing the spatial information content of the feature map, helping to retain more detailed features and local information, and significantly optimizing the detection of small targets; and then obtains the channel weight information through two pooling operations and the fusion of their corresponding results. , which can help the model automatically filter and focus on the most relevant feature channels during the learning process. The final feature map F with channel weights has a strong representation ability for the image, thereby improving the efficiency and generalization ability of the model.

[0070] In addition, after the above improvements to the YOLOV5 model, the YOLOV5 model can use the normalized Wasserstein distance (NWD) loss function to complete the training, replacing the original complete intersection over union (CIOU) loss function. The NWD loss function introduces two additional parameters, the center point distance weight of the bounding box and the width and height distance weight of the bounding box, on the basis of the CIOU loss function. By adjusting the center point distance weight and the width and height distance weight of the bounding box, it can show higher sensitivity when detecting small objects, so as to improve the accuracy of violation recognition.

[0071] Specifically, the loss function of NWD is: ; in, a distance metric representing the loss function; Represents the bounding box Modeled Gaussian distribution; Represents the bounding box Modeled Gaussian distribution; represents the center point distance weight, represents the width-height distance weight, .

[0072] S104: Determine a target violation recognition result corresponding to the target image based on at least two initial violation recognition results.

[0073] It should be understood that the target violation identification result in this embodiment is obtained after weighing all the initial violation identification results. Since the weighing method is not unique, it is not specifically limited here.

[0074] As an example, if the initial violation recognition results indicating violations among at least two initial violation recognition results reach a preset proportion, a first target violation recognition result corresponding to the target image is determined, otherwise a second target violation recognition result corresponding to the target image is determined; the first target violation recognition result indicates that the target image is in violation, and the second target violation recognition result indicates that the target image is not in violation. Figure 8 As shown, assuming that the violation recognition models of this embodiment include Resnet-50 model 1, Resnet-50 model 2, YOLOV5 model 1 and YOLOV5 model 2, as long as any two of the models recognize violations (reaching 50%), the final result can be determined as a violation image.

[0075] Alternatively, in order to enhance the strength of violation identification, as long as all initial violation identification results include an initial violation identification result indicating a violation, the first target violation identification result is determined; in other cases, the second target violation identification result is determined.

[0076] In addition, corresponding to Figure 1 In addition to the method shown, another embodiment of this embodiment further provides a device for identifying illegal images. Fig. 9 : is a schematic diagram of the structure of the illegal webpage judging device 900, comprising: The counterfeit webpage determination module 610 is used to determine a potential counterfeit webpage corresponding to the target webpage.

[0077] The operation module 910 performs violation identification on the target image based on at least two violation identification models to obtain at least two corresponding initial violation identification results; wherein the at least two violation identification models are provided with a first enhanced feature extraction module, the first enhanced feature extraction module is used to upsample the input data, and extract features from the upsampled results based on the spatial attention mechanism, and the feature extraction results are used for violation identification calculation. The decision module 920 is configured to determine a target violation recognition result corresponding to the target image based on the at least two initial violation recognition results.

[0078] The device of this embodiment prepares at least two violation recognition models, and deploys a first enhanced feature extraction module in each violation recognition model to upsample the input data, and extract features from the upsampled results based on the spatial attention mechanism, and the feature extraction results are used for violation recognition calculations. Among them, upsampling and spatial attention can make the model focus on the areas of low resolution and small targets in the target image to improve the problem of the model's easy loss of information. Then, by weighing the initial violation recognition results given by different violation recognition models for the target image, a high-accuracy target violation recognition result can be determined.

[0079] The at least two violation identification models include a Resnet-50 model, and the first enhanced feature extraction module is arranged at an output end of at least part of the stage modules of the Resnet-50 model.

[0080] Optionally, the first enhanced feature extraction module in the Resnet-50 model includes a first branch, a second branch, a third branch, a first dot product unit and a second dot product unit; the first branch is used to: perform a first upsampling operation on the input data to obtain first upsampling data; perform a first convolution operation on the first upsampling data to obtain first convolution data; perform a second convolution operation on the first convolution data to obtain second convolution data; The second branch is used to: perform a second upsampling operation on the input data to obtain second upsampling data; perform a third convolution operation on the second upsampling data to obtain third convolution data; perform a fourth convolution operation on the third convolution data to obtain fourth convolution data; the third branch is used to: perform a fifth convolution operation on the input data to obtain fifth convolution data; the first dot product unit is used to: perform a first dot product operation on the second convolution data and the fourth convolution data based on the spatial attention mechanism to obtain first dot product data containing spatial attention weight information; the second dot product unit is used to: perform a second dot product operation on the first dot product data and the fifth convolution data based on the spatial attention mechanism to obtain second dot product data containing spatial attention weight information, and the second dot product data is used as the feature extraction result of the first enhanced feature extraction module.

[0081] Optionally, the first convolution operation and the third convolution operation use a 3×3 convolution kernel, and the second convolution operation and the fourth convolution operation use a 1×1 convolution kernel.

[0082] Optionally, there are two first enhanced feature extraction modules in the Resnet-50 model, one of which is set at the output end of the stage0 module in the Resnet-50 model, and the other is set at the output end of the stage4 module in the Resnet-50 model.

[0083] Optionally, the at least two violation identification models include a YOLOV5 model, and the first enhanced feature extraction module serves as a CSP module of a backbone network of the YOLOV5 model.

[0084] Optionally, the first enhanced feature extraction module arranged in the backbone network includes: a fourth branch, a fifth branch and a first connection unit; the fourth branch is used to: perform a first CBS operation on the input data to obtain first CBS data; perform multiple first composite CBS operations on the first CBS data to obtain multiple first composite CBS data; perform a first connection operation on the first CBS data and the first composite CBS data to obtain first connection data; the first composite CBS operation includes at least two CBS operations performed continuously; the fifth branch is used to: perform a third upsampling operation on the input data to obtain third upsampling data; perform a second CBS operation on the third upsampling data operation to obtain second CBS data; perform a third CBS operation on the second CBS data to obtain third CBS data; perform a first sigmod function calculation operation on the third CBS data to obtain first sigmod function calculation data; perform a third dot product operation on the first sigmod function calculation data based on the spatial attention mechanism to obtain third dot product data containing spatial attention weight information; the first connection unit is used to: perform a second dot connection operation on the first connection data and the third dot product data based on the spatial attention mechanism to obtain second connection data containing spatial attention weight information, and the second connection data is used as the feature extraction result of the first enhanced feature extraction module.

[0085] Optionally, the first CBS operation and the third CBS operation use a 3×3 convolution kernel; the second CBS operation uses a 1×1 convolution kernel; the first composite CBS operation includes executing two CBS operations in succession, the first CBS operation uses a 1×1 convolution kernel, and the second CBS operation uses a 3×3 convolution kernel.

[0086] Optionally, the YOLOV5 model also includes a second enhanced feature extraction module, which is used to upsample the input data and perform feature extraction on the upsampling result based on a channel attention mechanism.

[0087] Optionally, the second enhanced feature extraction module serves as a CSP module of the neck network in the YOLOV5 model.

[0088] Optionally, the second enhanced feature extraction module includes: a sixth branch, a seventh branch, and a second connection unit; The sixth branch is used to: perform a fourth CBS operation on the input data to obtain fourth CBS data; perform multiple second composite CBS operations on the fourth CBS data to obtain multiple second composite CBS data; the second composite CBS operation includes at least two CBS operations performed continuously; the seventh branch is used to: perform a fourth upsampling operation on the input data to obtain fourth upsampling data; perform global averaging operation and global maximum pooling operation on the fourth upsampling data to obtain global averaging data and global maximum pooling data; perform a summing operation on the global averaging data and the global maximum pooling data to obtain summed data; perform a multi-channel full connection operation and a second sigmod function calculation operation on the summed data to obtain fully connected data and second sigmod function calculation data; perform a fourth dot product operation on the fully connected data and the second sigmod function calculation data based on the channel attention mechanism to obtain fourth dot product data containing channel attention weight information; the second connection unit is used to: perform a third connection operation on the multiple second composite CBS data and the fourth dot product data to obtain third connection data, and the third connection data is used as the feature extraction result of the second enhanced feature extraction module.

[0089] Optionally, the fourth CBS operation adopts a 1×1 convolution kernel; the second composite CBS operation includes executing two CBS operations in succession, the first CBS operation adopts a 1×1 convolution kernel, and the second CBS operation adopts a 3×3 convolution kernel.

[0090] Optionally, the YOLOV5 model uses a normalized Wasserstein distance loss function to complete training, and function parameters of the loss function include a center point distance weight of the bounding box and a width and height distance weight of the bounding box; and / or, the YOLOV5 model uses a deconvolution method to perform upsampling operations.

[0091] Optionally, the loss function is: ; in, a distance metric representing the loss function; Represents the bounding box Modeled Gaussian distribution; Represents the bounding box Modeled Gaussian distribution; represents the center point distance weight, represents the width-height distance weight, .

[0092] Optionally, the decision module 920 determines a target violation identification result corresponding to the target image based on the at least two initial violation identification results, including: if the initial violation identification results indicating a violation among the at least two initial violation identification results reach a preset proportion, then determining a first target violation identification result corresponding to the target image; otherwise determining a second target violation identification result corresponding to the target image; the first target violation identification result indicates that the target image is in violation, and the second target violation identification result indicates that the target image is not in violation.

[0093] It should be noted that the illegal image recognition device of this embodiment can be used as Figure 1 The execution subject of the method shown can thus realize Figure 1 The steps and functions in the method shown.

[0094] Fig.10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.10 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0095] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0096] The memory is used to store the computer program. Specifically, the computer program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides the computer program to the processor.

[0097] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming the above-mentioned Fig. 9Correspondingly, the processor executes the program stored in the memory and is specifically used to perform the following operations: Violation recognition is performed on a target image based on at least two violation recognition models to obtain corresponding at least two initial violation recognition results; wherein the at least two violation recognition models are provided with a first enhanced feature extraction module, the first enhanced feature extraction module is used to upsample the input data, and perform feature extraction on the upsampled result based on a spatial attention mechanism, and the feature extraction result is used for violation recognition calculation.

[0098] Based on the at least two initial violation recognition results, a target violation recognition result corresponding to the target image is determined.

[0099] The above is as in this manual Figure 1 The illegal webpage judgment method disclosed in the illustrated embodiment can be applied to a processor and implemented by the processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module can be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0100] Of course, in addition to software implementation, the electronic device of this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0101] In addition, the present application also provides a computer program product, which includes a computer-readable storage medium, the computer-readable storage medium storing one or more computer programs, and the one or more computer programs including instructions. When the above instructions are executed by a portable electronic device including multiple application programs, the portable electronic device can execute Figure 1 The steps in the method shown include: Violation recognition is performed on a target image based on at least two violation recognition models to obtain corresponding at least two initial violation recognition results; wherein the at least two violation recognition models are provided with a first enhanced feature extraction module, the first enhanced feature extraction module is used to upsample the input data, and perform feature extraction on the upsampled result based on a spatial attention mechanism, and the feature extraction result is used for violation recognition calculation.

[0102] Based on the at least two initial violation recognition results, a target violation recognition result corresponding to the target image is determined.

[0103] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0105] The above are only embodiments of this specification and are not intended to limit this specification. For those skilled in the art, this specification may be subject to various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the scope of the claims of this specification. In addition, all other embodiments obtained by ordinary technicians in this field without creative work shall fall within the scope of protection of this document.

Claims

1. A method for identifying illegal images, characterized in that: include: Based on at least two violation recognition models, the target image is subjected to violation recognition to obtain at least two corresponding initial violation recognition results; wherein the at least two violation recognition models are provided with a first enhanced feature extraction module, the first enhanced feature extraction module is used to upsample the input data, and perform feature extraction on the upsampled result based on a spatial attention mechanism, and the feature extraction result is used for violation recognition calculation; Based on the at least two initial violation recognition results, a target violation recognition result corresponding to the target image is determined.

2. The method according to claim 1, characterized in that The at least two violation identification models include a Resnet-50 model, and the first enhanced feature extraction module is arranged at an output end of at least part of the stage modules of the Resnet-50 model.

3. The method according to claim 2, characterized in that The first enhanced feature extraction module in the Resnet-50 model includes a first branch, a second branch, a third branch, a first dot product unit and a second dot product unit; wherein: The first branch is used to: perform a first upsampling operation on the input data to obtain first upsampling data; perform a first convolution operation on the first upsampling data to obtain first convolution data; perform a second convolution operation on the first convolution data to obtain second convolution data; The second branch is used to: perform a second upsampling operation on the input data to obtain second upsampling data; perform a third convolution operation on the second upsampling data to obtain third convolution data; perform a fourth convolution operation on the third convolution data to obtain fourth convolution data; The third branch is used to: perform a fifth convolution operation on the input data to obtain fifth convolution data; The first dot product unit is used to: perform a first dot product operation on the second convolution data and the fourth convolution data based on a spatial attention mechanism to obtain first dot product data containing spatial attention weight information; The second dot product unit is used to perform a second dot product operation on the first dot product data and the fifth convolution data based on the spatial attention mechanism to obtain second dot product data containing spatial attention weight information, and the second dot product data is used as the feature extraction result of the first enhanced feature extraction module.

4. The method according to claim 3, characterized in that The first convolution operation and the third convolution operation use a 3×3 convolution kernel, and the second convolution operation, the fourth convolution operation, and the fifth convolution operation use a 1×1 convolution kernel.

5. The method according to claim 2, characterized in that: There are two first enhanced feature extraction modules in the Resnet-50 model, one of which is set at the output end of the stage0 module in the Resnet-50 model, and the other is set at the output end of the stage4 module in the Resnet-50 model.

6. The method according to claim 1, characterized in that The at least two violation identification models include a YOLOV5 model, and the first enhanced feature extraction module serves as a CSP module of a backbone network in the YOLOV5 model.

7. The method according to claim 6, characterized in that The first enhanced feature extraction module arranged in the backbone network includes: a fourth branch, a fifth branch and a first connection unit; The fourth branch is used to: perform a first CBS operation on the input data to obtain first CBS data; perform multiple first composite CBS operations on the first CBS data to obtain multiple first composite CBS data; perform a first connection operation on the first CBS data and multiple first composite CBS data to obtain first connection data; the first composite CBS operation includes at least two CBS operations performed continuously; The fifth branch is used to: perform a third upsampling operation on the input data to obtain third upsampling data; perform a second CBS operation on the third upsampling data to obtain second CBS data; perform a third CBS operation on the second CBS data to obtain third CBS data; perform a first sigmod function calculation operation on the third CBS data to obtain first sigmod function calculation data; perform a third dot product operation on the first sigmod function calculation data based on the spatial attention mechanism to obtain third dot product data containing spatial attention weight information; The first connection unit is used to: perform a second point connection operation on the first connection data and the third dot product data based on the spatial attention mechanism to obtain second connection data containing spatial attention weight information, and the second connection data is used as the feature extraction result of the first enhanced feature extraction module.

8. The method according to claim 7, characterized in that The first CBS operation and the third CBS operation use a 3×3 convolution kernel; the second CBS operation uses a 1×1 convolution kernel; the first composite CBS operation includes executing two CBS operations in succession, the first CBS operation uses a 1×1 convolution kernel, and the second CBS operation uses a 3×3 convolution kernel.

9. The method according to claim 6, characterized in that The YOLOV5 model also includes a second enhanced feature extraction module, which is used to upsample the input data and perform feature extraction on the upsampled results based on a channel attention mechanism.

10. The method according to claim 9, characterized in that The second enhanced feature extraction module serves as a CSP module of the neck network in the YOLOV5 model.

11. The method according to claim 10, characterized in that The second enhanced feature extraction module includes: a sixth branch, a seventh branch, and a second connection unit; The sixth branch is used to: perform a fourth CBS operation on the input data to obtain fourth CBS data; perform a plurality of second composite CBS operations on the fourth CBS data to obtain a plurality of second composite CBS data; the second composite CBS operation includes at least two CBS operations performed continuously; The seventh branch is used to: perform a fourth upsampling operation on the input data to obtain fourth upsampling data; perform a global averaging operation and a global maximum pooling operation on the fourth upsampling data to obtain global averaged data and global maximum pooled data; perform a summing operation on the global averaged data and the global maximum pooled data to obtain summed data; perform a multi-channel full connection operation and a second sigmoid function calculation operation on the summed data to obtain fully connected data and second sigmoid function calculation data; perform a fourth dot product operation on the fully connected data and the second sigmoid function calculation data based on a channel attention mechanism to obtain fourth dot product data containing channel attention weight information; The second connection unit is used to perform a third connection operation on the multiple second composite CBS data and the fourth dot product data to obtain third connection data, and the third connection data is used as the feature extraction result of the second enhanced feature extraction module.

12. The method according to claim 11, characterized in that The fourth CBS operation uses a 1×1 convolution kernel; the second composite CBS operation includes executing two CBS operations in succession, the first CBS operation uses a 1×1 convolution kernel, and the second CBS operation uses a 3×3 convolution kernel.

13. The method according to claim 11, characterized in that The YOLOV5 model uses a normalized Wasserstein distance loss function to complete training, and the function parameters of the loss function include a center point distance weight of a bounding box and a width and height distance weight of a bounding box; and / or, the YOLOV5 model uses a deconvolution method to perform an upsampling operation.

14. The method according to claim 13, characterized in that The loss function is: ; in, a distance metric representing the loss function; Represents the bounding box Modeled Gaussian distribution; Represents the bounding box Modeled Gaussian distribution; represents the center point distance weight, represents the width-height distance weight, .

15. The method according to any one of claims 1 to 14, characterized in that: Determining a target violation recognition result corresponding to the target image based on the at least two initial violation recognition results includes: If the initial violation recognition results indicating violations among the at least two initial violation recognition results reach a preset proportion, a first target violation recognition result corresponding to the target image is determined, otherwise a second target violation recognition result corresponding to the target image is determined; the first target violation recognition result indicates that the target image has violated a violation, and the second target violation recognition result indicates that the target image has not violated a violation.

16. A device for identifying illegal images, characterized in that: include: A computing module, which performs violation recognition on a target image based on at least two violation recognition models to obtain corresponding at least two initial violation recognition results; wherein the at least two violation recognition models are provided with a first enhanced feature extraction module, the first enhanced feature extraction module is used to upsample input data, and perform feature extraction on the upsampled result based on a spatial attention mechanism, and the feature extraction result is used for violation recognition calculation; A decision module is used to determine a target violation recognition result corresponding to the target image based on the at least two initial violation recognition results.

17. An electronic device comprising: processor; and a memory arranged to store computer executable instructions, wherein the executable instructions, when executed, cause the processor to perform the method according to any one of claims 1 to 15.

18. A computer program product, comprising a computer-readable storage medium storing a computer program, characterized in that: The computer program is operable to cause a computer to perform the method according to any one of claims 1 to 15.

Images