Picture content safety monitoring system based on attention mechanism

Abstract

The invention discloses a picture content safety monitoring system based on an attention mechanism, and belongs to the technical field of data processing. Comprising a picture collection unit, the picture collection unit is connected to a safety monitoring system, the safety monitoring system comprises a picture classification unit, an image content recognition unit and a safety judgment unit, after the picture classification unit obtains a picture, the picture content is divided into N areas, all the areas of the picture are obtained, the areas are recognized, and the safety judgment unit judges the safety of the picture; an identification picture group is obtained, the identification picture group is composed of N area pictures, and each area picture has a classification area; the image content identification unit comprises a task distribution unit and a plurality of feature identification units; an attention mechanism unit is arranged in each feature recognition unit; according to the picture content safety monitoring system based on the attention mechanism, the picture content can be quickly subjected to safety judgment, and the safety judgment precision is high.

Description

Technical Field

[0001] The present invention specifically relates to an image content security monitoring system based on an attention mechanism, and belongs to the technical field of data processing. Background Art

[0002] As information exchange becomes more and more frequent, a large number of pictures and texts are transmitted to each other through the network. Each public network platform needs to receive and display a large number of pictures and texts every day. For example, email or communication software can be used to send and receive any pictures and texts. In order to make users comply with the picture upload requirements and avoid sensitive data from being displayed on the front end of the public network platform, pictures and texts need to be reviewed. In addition, for users with confidentiality requirements, when data is exchanged, it is necessary to monitor the sent content to avoid information leakage. In the prior art, for a system for monitoring data interaction content, such as Chinese patent publication number: CN115567284A, a text and picture based system is disclosed. A content security filtering method and system, the method comprising: obtaining data to be transmitted; performing keyword verification on text information to be verified to obtain text security detection results; performing image recognition, extracting image content, performing picture content verification, and obtaining picture security monitoring results; obtaining the user authority for receiving data, shielding the text information to be verified and the picture information to be verified, and sending them; the system can monitor and shield sensitive data, but the filtering method can ensure the accuracy of text content data extraction in the picture, but cannot achieve efficient and accurate recognition of non-text content in the picture content, and is prone to problems such as inability to distinguish the picture content or low recognition accuracy. Summary of the invention

[0003] To solve the above problems, the present invention proposes an image content security monitoring system based on an attention mechanism, which can quickly make security judgments on image content with high security judgment accuracy.

[0004] The image content security monitoring system based on the attention mechanism of the present invention includes an image acquisition unit, which is connected to the security monitoring system. The security monitoring system is implanted with a sending end or a receiving end of data interaction. The image acquisition unit acquires images according to the application object of the image content security monitoring system. For example, the image is first sent to the security monitoring system before being sent, or after the public network platform receives the image, it first performs security monitoring through the security monitoring system. Only after the monitoring passes can the image content be displayed on the public network platform; the security monitoring system includes: A picture classification unit, after acquiring the picture, the picture classification unit divides the picture content into N regions, obtains all the regions of the picture, and identifies the regions to obtain an identification picture group, wherein the identification picture group consists of N region pictures, and each region picture has a classification region; The picture classification unit only performs large classifications. For example, a picture containing people, animals, plants, tools and other contents is sent to the picture classification unit. After the picture classification unit obtains the picture, it generates regional pictures with separate features in sequence, such as people region pictures, animal region pictures, plant region pictures and tool region pictures. The picture classification model can use the existing trained software model of Identify Everything or a self-trained picture classification model according to application requirements, such as an image classification model formed by a trained ResNet50 network structure. The picture classification unit only obtains regional pictures and identifies the features of the regional pictures. For example, after the picture classification unit classifies and generates a person region picture, it does not perform feature recognition on the person region picture, such as not determining the features of the person's facial expression, person's gesture, person's clothing and the like. This greatly improves the classification efficiency of the picture classification unit, and can quickly generate multiple regional pictures of set sizes corresponding to the input picture to form a picture group. An image content recognition unit, the image content recognition unit includes a task dispatching unit and a plurality of feature recognition units; each of the feature recognition units has an attention mechanism unit built in, when the task dispatching unit obtains the recognition picture group, and according to the classification of the picture group, N area pictures are sent to the corresponding feature recognition unit; after the feature recognition unit obtains the area picture, it performs operation through the attention mechanism unit, and then recognizes the area picture, and outputs the area picture content, the picture content includes text content and feature data; the image content recognition unit obtains all the area picture contents, and maps the picture contents to the positional relationship of the original picture to obtain a matrix data group; After the task dispatching unit obtains the recognition picture group, it is sent to the corresponding feature recognition unit respectively; for example, the person area picture is sent to the person feature recognition unit, the animal area picture is sent to the animal feature recognition unit; the plant area picture is sent to the plant feature recognition unit; the tool area picture is sent to the tool feature recognition unit; after the feature recognition unit obtains the area picture, such as the person feature recognition unit obtains the person area picture, then, the attention mechanism unit is introduced, and the face, hands, private parts of the body and clothing of the person area picture are taken as key areas through the multi-head attention mechanism unit; the person feature recognition unit recognizes the key areas, which can improve the recognition efficiency and recognition accuracy, and after recognition, the text content and / or feature data are obtained; for example, the tool area picture is sent to the tool feature recognition unit, and the head, middle section and tail of the tool area picture are taken as key areas through the multi-head attention mechanism unit; the feature data of quickly determining whether the tool is a controlled knife, and the feature data of determining whether the tool is violent and bloody; The security determination unit performs a security determination on the matrix data group after acquiring the matrix data group. When sensitive data appears, a risk instruction is issued; otherwise, the picture content is deemed safe by default; the security determination unit determines whether sensitive data exists based on the matrix data group; if so, a risk instruction is issued, and the picture can only be sent or uploaded to the platform after manual confirmation that there is no risk.

[0005] Further, the security determination unit includes a sensitive data determination unit and a unit block determination unit; the text content is input into the sensitive data determination unit; the feature data group is input into the unit block determination unit, and the unit block determination unit includes: A picture feature conversion unit, the picture feature conversion unit is connected to the key-value dictionary library, the picture feature conversion unit first obtains all feature data of the input picture and the position relationship of the feature data, then uses the feature data as an index, compares the key-value dictionary library, matches each feature data with a key value, and outputs a key-value data group; each data unit in the key-value data group includes key-value data and position assignment; when a feature is not recognized, the monitoring operation ignores the feature and registers the feature as a candidate; The safety determination module includes a quick determination module and a deep inspection determination module. The deep inspection determination module includes a feature distribution monitoring unit, a position monitoring unit and an associated monitoring unit. The working process of the quick determination module is as follows: After the quick determination module obtains the key value data group, each key value data of the key value data group is traversed through the sensitive dictionary library. When a certain key value data is retrieved, the quick determination module issues a risk instruction; When the quick determination module does not issue a risk instruction, it directly issues a safety instruction or enters the deep inspection determination module. When the deep inspection determination module is triggered, it enters at least one of the feature distribution monitoring unit, the location monitoring unit, and the associated monitoring unit. When the deep inspection determination module detects a risk feature, it issues a risk instruction. When the deep inspection determination module does not retrieve a risk feature, it outputs a safety instruction. The image content security review is completed. The feature distribution monitoring unit is connected to the feature weight dictionary library and pre-sets a first weight threshold. The feature distribution monitoring unit obtains a key value data group, and traverses the feature weight dictionary library for each key value of the key value data group to obtain a weight data group; then, the number of weight data groups exceeding the first weight threshold is counted; when the number reaches a set value, a risk instruction is issued, otherwise an output safety instruction is output; When the position monitoring unit is working, the data unit is input into the position monitoring unit, and the position monitoring unit forms a combination of two key values ​​that are adjacent in row or vertically according to the position assignment, and determines whether the combination is hit by the data in the associated database by comparing with the associated database. If it is hit, a risk instruction is issued, otherwise a safety instruction is output; The association monitoring unit is connected to the association database and pre-sets a second weight threshold, where the second weight threshold is less than the first weight threshold; the association monitoring unit obtains key-value data exceeding the weight threshold in the key-value data group, and determines whether all combinations of key-value data in the key-value data group are consistent with those in the random combination database by traversing the random combination database, and if so, issues a risk instruction, otherwise outputs a safety instruction; the association database is a true subset of the random combination database.

[0006] Furthermore, the picture classification unit includes an image classification model composed of a trained ResNet50 network structure; the picture classification unit completes the picture content classification, and intercepts each classification object to form an identification picture group; the feature recognition unit is composed of a YOLOV3 model or a YOLOV5 model; the attention mechanism unit is composed of one or a combination of two of a CAM module and a SAM module.

[0007] The CAM of the attention mechanism unit is a channel attention module, which uses the relationship between channels to generate a channel attention map. In the forward propagation process, CAM assigns different weights to the feature channels transmitted in the neural network, so that the feature channels with larger weights will frequently update their parameters. Important feature channels usually have a larger proportion than other channels, and the parts that the network focuses on can be more clearly reflected in the final output image. When processing with CAM, the input feature map is first subjected to spatial global average pooling and maximum pooling operations to obtain two one-dimensional feature vectors that aggregate the spatial information of the feature map, and then two fully connected networks are used to filter the channel information contained therein. Finally, the two filtered features are added together and the channel attention feature map is obtained through the Sigmoid function. ; The SAM of the attention mechanism unit is a spatial attention module, which mainly focuses on the location information of the features and generates a spatial attention feature map using the spatial relationship between the features. On the two-dimensional plane, the spatial attention applies different weights to each pixel of all channel features of the input feature map, and these weights represent the importance of a certain spatial location information. During SAM processing, two pooling operations are used to aggregate the channel information of the feature map to generate two two-dimensional maps, which are then added based on the channel, and then reduced to a single channel through a convolutional layer. Finally, the sigmoid function is used to generate the spatial attention map. CAM improves the model's attention to important features by weighting the channel relationship of the feature map, while SAM obtains more effective positional relationship information by modeling the spatial relationship of the features. When used in combination, the two can complement and promote each other. CAM and SAM are combined to form a composite attention mechanism unit. When used, the CAM operation is completed first, and then the SAM operation is performed. Specifically, the original feature map F is input into the attention module, and the size of the feature map F is H*W*C, where H is the height of the original feature map F, W is the width of the original feature map F, and C is the number of channels of the original feature map F. CAM performs global average pooling and maximum pooling operations on it respectively, retains the different spatial information of the feature map F, and generates two 1*1*C different spatial channel descriptions, recorded as and .in, Contains the global background information of the target object, The salient features of the target object are highlighted; let the convolution set input into the attention structure be ,and ;in, For the The parameters of the convolution kernel; among them, (1); (2); In the above formula (1) and formula (2), For the The spatial information of the parameters of the convolution kernel, For high information, is the width information; () is a function that finds the parameter (set) of a function; For the The parameters of the convolution kernel; Will and The two channels containing different spatial features of the target are described through a two-layer shared network, which consists of two layers of fully connected networks FC1 and FC2 with C / r and C neurons respectively, and the activation function uses ReLU; the outputs of the two branches are: (3); (4); The two features obtained by formula (3) and formula (4) are added together and then activated by the Sigmoid function to obtain the output of the attention module, as follows: (5); Then, the feature map is weighted by element-by-element multiplication to obtain the channel attention map after filtering the features. ; (6); CAM output channel attention diagram , and the channel attention map Sent to SAM; the SAM operation output ; Specifically: (7); in, Generate a spatial attention map using the Sigmoid function; is the Sigmoid function, Indicates a convolution operation with a convolution kernel size of 3*3; It is a channel feature map of global average pooling; is a channel feature map of the maximum pooling operation, Depend on Output and The output is obtained by adding channels; the SAM operation outputs the spatial attention map After that, in the channel attention map Finally, the attention mechanism unit outputs the final attention feature map ; (8).

[0008] The attention mechanism unit can be seamlessly embedded in the network of the feature recognition unit, such as implanting the attention mechanism unit into the backbone network of YOLOV3. The backbone network of YOLOV3 is composed of multiple Darknet-53 residual structures superimposed on each other. It directly adds the underlying features and high-level features through skip connections to solve the learning saturation problem of YOLOV3. The Darknet-53 network structure is mainly composed of DBL and ResBlock. DBL is composed of convolutional layer, BN layer and LeakyReLU activation function. When the attention mechanism unit is implanted into the feature recognition unit, the attention mechanism unit is directly implanted into the ResBlock of the Darknet-53 network structure, specifically: the attention mechanism unit is implanted into the DBL of ResBlock, so that each ResBlock is composed of DBL and multiple attention mechanism units; so that the original feature map can be feature-weighted in the deeper network layer of YOLOV3; so that the key areas of the image content can be efficiently and accurately identified.

[0009] Furthermore, when the feature recognition unit recognizes text content, the feature recognition unit is an OCR recognition unit; when the feature recognition unit recognizes non-text content, the feature recognition unit is a feature extraction unit; the area image of the non-text content is first operated by the attention mechanism unit, and then the feature extraction unit performs feature recognition to obtain all feature unit blocks, and the feature recognition unit outputs all feature unit blocks or feature unit blocks that cannot be recognized by the OCR recognition unit as a feature data group; the image feature conversion unit first obtains the feature data group of the input image, then recognizes the features to obtain the recognized characters, and then uses the recognized characters as indexes to compare the key value dictionary library, matches the feature data to obtain the key values ​​corresponding to each feature data, and outputs the key value data group.

[0010] Furthermore, the sensitive data determination unit is connected to a sensitive character dictionary library, and the sensitive character dictionary library contains sensitive characters, phrases and sentences.

[0011] Further, the unit block determination unit includes a safety monitoring mode selection unit, and the safety monitoring mode selection unit includes: A first-level safety monitoring mode, wherein only a quick determination module is executed; Secondary safety monitoring mode, the secondary safety monitoring mode first executes the primary safety monitoring mode, and when the primary safety monitoring mode does not detect a risk instruction, it enters the characteristic distribution monitoring unit for monitoring; when the characteristic distribution monitoring unit does not detect a risk instruction, it enters the position monitoring unit for safety monitoring; The third-level safety monitoring mode first executes the second-level safety monitoring mode, and then enters the associated monitoring unit when the second-level safety monitoring mode does not detect any risk instructions.

[0012] Furthermore, the registration candidate process is as follows: when a feature is not recognized, the unrecognized feature data is output, and the recognition character, key value and weight value of the recognition character are manually set for the feature data, and automatically updated to the feature weight dictionary library and the random combination database.

[0013] Compared with the prior art, the image content security monitoring system based on the attention mechanism of the present invention can quickly classify the image content and send the classified images to the corresponding feature recognition unit. The feature recognition unit introduces the attention mechanism, which can quickly detect key areas, improve detection efficiency and detection accuracy, and obtain detection features. Finally, based on the detection feature calculation, it is determined whether the image content contains sensitive information, and the image content can be quickly judged for security with high accuracy. When making a security judgment, the quick judgment module is used to judge whether each independent feature contains sensitive data, and the judgment speed is fast. For important information or occasions with low timeliness requirements, the deep inspection and judgment module can be used to make a comprehensive judgment on whether the image content contains sensitive information. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the overall structure of the picture content security monitoring system of the present invention.

[0015] Figure 2 It is a schematic diagram of the overall structure of the safety determination module of the present invention.

[0016] Figure 3 Schematic diagram of the working process of the image feature conversion unit of the present invention.

[0017] Figure 4 Schematic diagram of the working process of the safety determination module of the present invention.

[0018] Figure 5 Schematic diagram of the workflow of the deep inspection and determination module of the present invention.

[0019] Figure 6 It is a schematic diagram of the working process of the safety monitoring mode selection unit of the present invention. DETAILED DESCRIPTION

[0020] Embodiment 1: like Figures 1 to 6 The picture content security monitoring system based on the attention mechanism shown in the figure includes a picture acquisition unit, which is connected to the security monitoring system. The security monitoring system is embedded in the sending end or receiving end of data interaction. The picture acquisition unit acquires pictures according to the application object of the picture content security monitoring system. For example, the picture is sent to the security monitoring system before being sent, or after the public network platform receives the picture, it is firstly monitored by the security monitoring system. Only after the monitoring is passed can the picture content be displayed on the public network platform; the security monitoring system includes: A picture classification unit, after acquiring the picture, the picture classification unit divides the picture content into N regions, obtains all the regions of the picture, and identifies the regions to obtain an identification picture group, wherein the identification picture group consists of N region pictures, and each region picture has a classification region; The picture classification unit only performs large classifications. For example, a picture containing people, animals, plants, tools and other contents is sent to the picture classification unit. After the picture classification unit obtains the picture, it generates regional pictures with separate features in sequence, such as people region pictures, animal region pictures, plant region pictures and tool region pictures. The picture classification model can use the existing trained software model of Identify Everything or a self-trained picture classification model according to application requirements, such as an image classification model formed by a trained ResNet50 network structure. The picture classification unit only obtains regional pictures and identifies the features of the regional pictures. For example, after the picture classification unit classifies and generates a person region picture, it does not perform feature recognition on the person region picture, such as not determining the features of the person's facial expression, person's gesture, person's clothing and the like. This greatly improves the classification efficiency of the picture classification unit, and can quickly generate multiple regional pictures of set sizes corresponding to the input picture to form a picture group. An image content recognition unit, the image content recognition unit includes a task dispatching unit and a plurality of feature recognition units; each of the feature recognition units has an attention mechanism unit built in, when the task dispatching unit obtains the recognition picture group, and according to the classification of the picture group, N area pictures are sent to the corresponding feature recognition unit; after the feature recognition unit obtains the area picture, it performs operation through the attention mechanism unit, and then recognizes the area picture, and outputs the area picture content, the picture content includes text content and feature data; the image content recognition unit obtains all the area picture contents, and maps the picture contents to the positional relationship of the original picture to obtain a matrix data group; After the task dispatching unit obtains the recognition picture group, it is sent to the corresponding feature recognition unit respectively; for example, the person area picture is sent to the person feature recognition unit, the animal area picture is sent to the animal feature recognition unit; the plant area picture is sent to the plant feature recognition unit; the tool area picture is sent to the tool feature recognition unit; after the feature recognition unit obtains the area picture, such as the person feature recognition unit obtains the person area picture, then, the attention mechanism unit is introduced, and the face, hands, private parts of the body and clothing of the person area picture are taken as key areas through the multi-head attention mechanism unit; the person feature recognition unit recognizes the key areas, which can improve the recognition efficiency and recognition accuracy, and after recognition, the text content and / or feature data are obtained; for example, the tool area picture is sent to the tool feature recognition unit, and the head, middle section and tail of the tool area picture are taken as key areas through the multi-head attention mechanism unit; the feature data of quickly determining whether the tool is a controlled knife, and the feature data of determining whether the tool is violent and bloody; The security determination unit performs a security determination on the matrix data group after acquiring the matrix data group. When sensitive data appears, a risk instruction is issued; otherwise, the picture content is deemed safe by default; the security determination unit determines whether sensitive data exists based on the matrix data group; if so, a risk instruction is issued, and the picture can only be sent or uploaded to the platform after manual confirmation that there is no risk.

[0021] The security determination unit includes a sensitive data determination unit and a unit block determination unit; the text content is input into the sensitive data determination unit; the feature data group is input into the unit block determination unit, and the unit block determination unit includes: A picture feature conversion unit, the picture feature conversion unit is connected to the key-value dictionary library, the picture feature conversion unit first obtains all feature data of the input picture and the position relationship of the feature data, then uses the feature data as an index, compares the key-value dictionary library, matches each feature data with a key value, and outputs a key-value data group; each data unit in the key-value data group includes key-value data and position assignment; when a feature is not recognized, the monitoring operation ignores the feature and registers the feature as a candidate; The safety determination module includes a quick determination module and a deep inspection determination module. The deep inspection determination module includes a feature distribution monitoring unit, a position monitoring unit and an associated monitoring unit. The working process of the quick determination module is as follows: After the quick determination module obtains the key value data group, each key value data of the key value data group is traversed through the sensitive dictionary library. When a certain key value data is retrieved, the quick determination module issues a risk instruction; When the quick determination module does not issue a risk instruction, it directly issues a safety instruction or enters the deep inspection determination module. When the deep inspection determination module is triggered, it enters at least one of the feature distribution monitoring unit, the location monitoring unit, and the associated monitoring unit. When the deep inspection determination module detects a risk feature, it issues a risk instruction. When the deep inspection determination module does not retrieve a risk feature, it outputs a safety instruction. The image content security review is completed. The feature distribution monitoring unit is connected to the feature weight dictionary library and pre-sets a first weight threshold. The feature distribution monitoring unit obtains a key value data group, and traverses the feature weight dictionary library for each key value of the key value data group to obtain a weight data group; then, the number of weight data groups exceeding the first weight threshold is counted; when the number reaches a set value, a risk instruction is issued, otherwise an output safety instruction is output; When the position monitoring unit is working, the data unit is input into the position monitoring unit, and the position monitoring unit forms a combination of two key values ​​that are adjacent in row or vertically according to the position assignment, and determines whether the combination is hit by the data in the associated database by comparing with the associated database. If it is hit, a risk instruction is issued, otherwise a safety instruction is output; The association monitoring unit is connected to the association database and pre-sets a second weight threshold, where the second weight threshold is less than the first weight threshold; the association monitoring unit obtains key-value data exceeding the weight threshold in the key-value data group, and determines whether all combinations of key-value data in the key-value data group are consistent with those in the random combination database by traversing the random combination database, and if so, issues a risk instruction, otherwise outputs a safety instruction; the association database is a true subset of the random combination database.

[0022] The picture classification unit includes an image classification model composed of a trained ResNet50 network structure; the picture classification unit completes the picture content classification, and intercepts each classification object to form an identification picture group; the feature recognition unit is composed of a YOLOV3 model or a YOLOV5 model; the attention mechanism unit is composed of one or a combination of two of the CAM module and the SAM module.

[0023] The CAM of the attention mechanism unit is a channel attention module, which uses the relationship between channels to generate a channel attention map. In the forward propagation process, CAM assigns different weights to the feature channels transmitted in the neural network, so that the feature channels with larger weights will frequently update their parameters. Important feature channels usually have a larger proportion than other channels, and the parts that the network focuses on can be more clearly reflected in the final output image. When processing with CAM, the input feature map is first subjected to spatial global average pooling and maximum pooling operations to obtain two one-dimensional feature vectors that aggregate the spatial information of the feature map, and then two fully connected networks are used to filter the channel information contained therein. Finally, the two filtered features are added together and the channel attention feature map is obtained through the Sigmoid function. ; The SAM of the attention mechanism unit is a spatial attention module, which mainly focuses on the location information of the features and generates a spatial attention feature map using the spatial relationship between the features. On the two-dimensional plane, the spatial attention applies different weights to each pixel of all channel features of the input feature map, and these weights represent the importance of a certain spatial location information. During SAM processing, two pooling operations are used to aggregate the channel information of the feature map to generate two two-dimensional maps, which are then added based on the channel, and then reduced to a single channel through a convolutional layer. Finally, the sigmoid function is used to generate the spatial attention map. CAM improves the model's attention to important features by weighting the channel relationship of the feature map, while SAM obtains more effective positional relationship information by modeling the spatial relationship of the features. When used in combination, the two can complement and promote each other. CAM and SAM are combined to form a composite attention mechanism unit. When used, the CAM operation is completed first, and then the SAM operation is performed. Specifically, the original feature map F is input into the attention module, and the size of the feature map F is H*W*C, where H is the height of the original feature map F, W is the width of the original feature map F, and C is the number of channels of the original feature map F. CAM performs global average pooling and maximum pooling operations on it respectively, retains the different spatial information of the feature map F, and generates two 1*1*C different spatial channel descriptions, recorded as and .in, Contains the global background information of the target object, The salient features of the target object are highlighted; let the convolution set input into the attention structure be ,and ;in, For the The parameters of the convolution kernel; among them, (1); (2); In the above formula (1) and formula (2), For the The spatial information of the parameters of the convolution kernel, For high information, is the width information; () is a function that finds the parameter (set) of a function; For the The parameters of the convolution kernel; Will and The two channels containing different spatial features of the target are described through a two-layer shared network, which consists of two layers of fully connected networks FC1 and FC2 with C / r and C neurons respectively, and the activation function uses ReLU; the outputs of the two branches are: (3); (4); The two features obtained by formula (3) and formula (4) are added together and then activated by the Sigmoid function to obtain the output of the attention module, as follows: (5); Then, the feature map is weighted by element-by-element multiplication to obtain the channel attention map after filtering the features. ; (6); CAM output channel attention diagram , and the channel attention map Sent to SAM; the SAM operation output ; Specifically: (7); in, Generate a spatial attention map using the Sigmoid function; is the Sigmoid function, Indicates a convolution operation with a convolution kernel size of 3*3; It is a channel feature map of global average pooling; is a channel feature map of the maximum pooling operation, Depend on Output and The output is obtained by adding channels; the SAM operation outputs the spatial attention map After that, in the channel attention map Finally, the attention mechanism unit outputs the final attention feature map ; (8).

[0024] The attention mechanism unit can be seamlessly embedded in the network of the feature recognition unit, such as implanting the attention mechanism unit into the backbone network of YOLOV3. The backbone network of YOLOV3 is composed of multiple Darknet-53 residual structures superimposed on each other. It directly adds the underlying features and high-level features through skip connections to solve the learning saturation problem of YOLOV3. The Darknet-53 network structure is mainly composed of DBL and ResBlock. DBL is composed of convolutional layer, BN layer and LeakyReLU activation function. When the attention mechanism unit is implanted into the feature recognition unit, the attention mechanism unit is directly implanted into the ResBlock of the Darknet-53 network structure, specifically: the attention mechanism unit is implanted into the DBL of ResBlock, so that each ResBlock is composed of DBL and multiple attention mechanism units; so that the original feature map can be feature-weighted in the deeper network layer of YOLOV3; so that the key areas of the image content can be efficiently and accurately identified.

[0025] When the feature recognition unit recognizes text content, the feature recognition unit is an OCR recognition unit; when the feature recognition unit recognizes non-text content, the feature recognition unit is a feature extraction unit; the area image of the non-text content is first operated by the attention mechanism unit, and then the feature extraction unit performs feature recognition to obtain all feature unit blocks, and the feature recognition unit outputs all feature unit blocks or feature unit blocks that cannot be recognized by the OCR recognition unit as a feature data group; the image feature conversion unit first obtains the feature data group of the input image, then recognizes the features to obtain the recognized characters, and then uses the recognized characters as indexes to compare the key value dictionary library, matches the feature data to obtain the key value corresponding to each feature data, and outputs the key value data group.

[0026] The sensitive data determination unit is connected to a sensitive character dictionary library, and the sensitive character dictionary library has sensitive characters, phrases and sentences built in it.

[0027] The unit block determination unit includes a safety monitoring mode selection unit, and the safety monitoring mode selection unit includes: A first-level safety monitoring mode, wherein only a quick determination module is executed; Secondary safety monitoring mode, the secondary safety monitoring mode first executes the primary safety monitoring mode, and when the primary safety monitoring mode does not detect a risk instruction, it enters the characteristic distribution monitoring unit for monitoring; when the characteristic distribution monitoring unit does not detect a risk instruction, it enters the position monitoring unit for safety monitoring; The third-level safety monitoring mode first executes the second-level safety monitoring mode, and then enters the associated monitoring unit when the second-level safety monitoring mode does not detect any risk instructions.

[0028] The registration candidate process is as follows: when a feature is not recognized, the unrecognized feature data is output, and the recognition character, key value and weight value of the recognition character are manually set for the feature data, and automatically updated to the feature weight dictionary library and the random combination database.

[0029] The above embodiments are only preferred implementations of the present invention, so any equivalent changes or modifications made according to the structures, features and principles described in the scope of application of the present invention are included in the scope of application of the present invention.

Claims

1. A picture content security monitoring system based on attention mechanism, characterized by: It includes a picture acquisition unit, which is connected to a security monitoring system. The security monitoring system includes: A picture classification unit, after acquiring the picture, the picture classification unit divides the picture content into N regions, obtains all the regions of the picture, and identifies the regions to obtain an identification picture group, wherein the identification picture group consists of N region pictures, and each region picture has a classification region; An image content recognition unit, the image content recognition unit includes a task dispatching unit and a plurality of feature recognition units; each of the feature recognition units has an attention mechanism unit built in, when the task dispatching unit obtains the recognition picture group, and according to the classification of the picture group, N area pictures are sent to the corresponding feature recognition unit; after the feature recognition unit obtains the area picture, it performs operation through the attention mechanism unit, and then recognizes the area picture, and outputs the area picture content, the picture content includes text content and feature data; the image content recognition unit obtains all the area picture contents, and maps the picture contents to the positional relationship of the original picture to obtain a matrix data group; The security determination unit performs security determination on the matrix data group after acquiring the matrix data group, and issues a risk instruction when sensitive data appears; otherwise, the image content is deemed safe by default.

2. The image content security monitoring system based on the attention mechanism according to claim 1, characterized in that: The security determination unit includes a sensitive data determination unit and a unit block determination unit; The text content is input into a sensitive data determination unit; The feature data group is input to a unit block determination unit, and the unit block determination unit includes: A picture feature conversion unit, the picture feature conversion unit is connected to the key-value dictionary library, the picture feature conversion unit first obtains all feature data of the input picture and the position relationship of the feature data, then uses the feature data as an index, compares the key-value dictionary library, matches each feature data with a key value, and outputs a key-value data group; each data unit in the key-value data group includes key-value data and position assignment; when a feature is not recognized, the monitoring operation ignores the feature and registers the feature as a candidate; The safety determination module includes a quick determination module and a deep inspection determination module. The deep inspection determination module includes a feature distribution monitoring unit, a position monitoring unit and an associated monitoring unit. The working process of the quick determination module is as follows: After the quick determination module obtains the key value data group, each key value data of the key value data group is traversed through the sensitive dictionary library. When a certain key value data is retrieved, the quick determination module issues a risk instruction; When the quick determination module does not issue a risk instruction, it directly issues a safety instruction or enters the deep inspection determination module. When the deep inspection determination module is triggered, it enters at least one of the feature distribution monitoring unit, the location monitoring unit, and the associated monitoring unit. When the deep inspection determination module detects a risk feature, it issues a risk instruction. When the deep inspection determination module does not retrieve a risk feature, it outputs a safety instruction. The image content security review is completed. The feature distribution monitoring unit is connected to the feature weight dictionary library and pre-sets a first weight threshold. The feature distribution monitoring unit obtains a key value data group, and traverses the feature weight dictionary library for each key value of the key value data group to obtain a weight data group; then, the number of weight data groups exceeding the first weight threshold is counted; when the number reaches a set value, a risk instruction is issued, otherwise an output safety instruction is output; When the position monitoring unit is working, the data unit is input into the position monitoring unit, and the position monitoring unit forms a combination of two key values ​​that are adjacent in row or vertically according to the position assignment, and determines whether the combination is hit by the data in the associated database by comparing with the associated database. If it is hit, a risk instruction is issued, otherwise a safety instruction is output; The association monitoring unit is connected to the association database and pre-sets a second weight threshold, where the second weight threshold is less than the first weight threshold; the association monitoring unit obtains key-value data exceeding the weight threshold in the key-value data group, and determines whether all combinations of key-value data in the key-value data group are consistent with those in the random combination database by traversing the random combination database, and if so, issues a risk instruction, otherwise outputs a safety instruction; the association database is a true subset of the random combination database.

3. The image content security monitoring system based on the attention mechanism according to claim 1, characterized in that: The picture classification unit includes an image classification model composed of a trained ResNet50 network structure; the picture classification unit completes the picture content classification, and intercepts each classification object to form an identification picture group; the feature recognition unit is composed of a YOLOV3 model or a YOLOV5 model; the attention mechanism unit is composed of one or a combination of two of the CAM module and the SAM module.

4. The image content security monitoring system based on the attention mechanism according to claim 1, characterized in that: When the feature recognition unit recognizes text content, the feature recognition unit is an OCR recognition unit; when the feature recognition unit recognizes non-text content, the feature recognition unit is a feature extraction unit; the area image of the non-text content is first operated by the attention mechanism unit, and then the feature extraction unit performs feature recognition to obtain all feature unit blocks, and the feature recognition unit outputs all feature unit blocks or feature unit blocks that cannot be recognized by the OCR recognition unit as a feature data group; the image feature conversion unit first obtains the feature data group of the input image, then recognizes the features to obtain the recognized characters, and then uses the recognized characters as indexes to compare the key value dictionary library, matches the feature data to obtain the key value corresponding to each feature data, and outputs the key value data group.

5. The image content security monitoring system based on the attention mechanism according to claim 1, characterized in that: The sensitive data determination unit is connected to a sensitive character dictionary library, and the sensitive character dictionary library has sensitive characters, phrases and sentences built in it.

6. The image content security monitoring system based on the attention mechanism according to claim 1, characterized in that: The unit block determination unit includes a safety monitoring mode selection unit, and the safety monitoring mode selection unit includes: A first-level safety monitoring mode, wherein only a quick determination module is executed; Secondary safety monitoring mode, the secondary safety monitoring mode first executes the primary safety monitoring mode, and when the primary safety monitoring mode does not detect a risk instruction, it enters the characteristic distribution monitoring unit for monitoring; when the characteristic distribution monitoring unit does not detect a risk instruction, it enters the position monitoring unit for safety monitoring; The third-level safety monitoring mode first executes the second-level safety monitoring mode, and then enters the associated monitoring unit when the second-level safety monitoring mode does not detect any risk instructions.

7. The image content security monitoring system based on the attention mechanism according to claim 1, characterized in that: The registration candidate process is as follows: when a feature is not recognized, the unrecognized feature data is output, and the recognition character, key value and weight value of the recognition character are manually set for the feature data, and automatically updated to the feature weight dictionary library and the random combination database.

Images