Abnormal image identification method and device, computer device and storage medium

Abstract

The application relates to an abnormal image recognition method and device and a computer device. The method comprises the following steps: extracting a face region in a to-be-detected image, performing abnormal face recognition on the face region to obtain an abnormal face possibility; extracting a background region in the to-be-detected image, performing abnormal scene recognition on the background region to obtain an abnormal scene possibility; performing abnormal object detection on an object region in the to-be-detected image to obtain an abnormal object possibility; performing attention weight calculation based on the face region, the background region and the object region to obtain the attention weights corresponding to the face region, the background region and the object region respectively; performing weighted calculation based on the attention weights, the abnormal face possibility, the abnormal scene possibility and the abnormal object possibility to obtain a target abnormal possibility corresponding to the to-be-detected image; and determining that the to-be-detected image is an abnormal image when the target abnormal possibility meets a preset abnormal possibility threshold. The method can improve the abnormal image recognition accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to an abnormal image identification method and device, a computer device, a storage medium, and a computer program product. BACKGROUND

[0002] With the development of computer technology, Internet product transfer resource acquisition technology has emerged, such as loan applications of financial companies. Internet product transfer resource acquisition refers to a way in which a user sends an acquisition request through an Internet product and obtains a transfer resource after risk assessment by a server corresponding to the Internet product. At present, common risk assessment methods include credit assessment of users or abnormal image identification of user images, such as identity verification through face recognition to assess whether there is a risk. However, the traditional abnormal image identification method cannot accurately identify whether a user image is an abnormal image, and has the problem of low identification accuracy. SUMMARY

[0003] Therefore, it is necessary to provide a risk identification method, device, computer device, computer readable storage medium, and computer program product capable of improving the identification accuracy of abnormal images to solve the above technical problems.

[0004] In a first aspect, the present application provides an abnormal image identification method. The method comprises:

[0005] obtaining a to-be-detected image, extracting a face region in the to-be-detected image, performing abnormal face identification on the face region, and obtaining an abnormal face possibility corresponding to the face region;

[0006] extracting a background region in the to-be-detected image, performing abnormal scene identification on the background region, and obtaining an abnormal scene possibility corresponding to the background region;

[0007] performing abnormal object detection on an object region in the to-be-detected image, and obtaining an abnormal object possibility corresponding to the to-be-detected image;

[0008] performing attention weight calculation based on the face region, the background region, and the object region to obtain attention weights corresponding to the face region, the background region, and the object region respectively, and performing weighted calculation based on the attention weights, the abnormal face possibility, the abnormal scene possibility, and the abnormal object possibility to obtain a target abnormal possibility corresponding to the to-be-detected image;

[0009] When the target abnormal possibility meets a preset abnormal possibility threshold, the to-be-detected image is determined to be an abnormal image.

[0010] In a second aspect, the present application further provides an abnormal image identification device. The device comprises:

[0011] The face recognition module is configured to acquire a to-be-detected image, extract a face region in the to-be-detected image, perform abnormal face recognition on the face region, and obtain an abnormal face possibility corresponding to the face region.

[0012] The scene recognition module is configured to extract a background region in the to-be-detected image, perform abnormal scene recognition on the background region, and obtain an abnormal scene possibility corresponding to the background region.

[0013] The object recognition module is configured to perform abnormal object detection on an object region in the to-be-detected image, and obtain an abnormal object possibility corresponding to the to-be-detected image.

[0014] The calculation module is configured to perform attention weight calculation based on the face region, the background region, and the object region, obtain attention weights corresponding to the face region, the background region, and the object region respectively, perform weighted calculation based on the attention weights, the abnormal face possibility, the abnormal scene possibility, and the abnormal object possibility, and obtain a target abnormal possibility corresponding to the to-be-detected image.

[0015] The judgment module is configured to determine that the to-be-detected image is an abnormal image when the target abnormal possibility meets a preset abnormal possibility threshold.

[0016] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0017] The face recognition module is configured to acquire a to-be-detected image, extract a face region in the to-be-detected image, perform abnormal face recognition on the face region, and obtain an abnormal face possibility corresponding to the face region.

[0018] The scene recognition module is configured to extract a background region in the to-be-detected image, perform abnormal scene recognition on the background region, and obtain an abnormal scene possibility corresponding to the background region.

[0019] The object recognition module is configured to perform abnormal object detection on an object region in the to-be-detected image, and obtain an abnormal object possibility corresponding to the to-be-detected image.

[0020] The calculation module is configured to perform attention weight calculation based on the face region, the background region, and the object region, obtain attention weights corresponding to the face region, the background region, and the object region respectively, perform weighted calculation based on the attention weights, the abnormal face possibility, the abnormal scene possibility, and the abnormal object possibility, and obtain a target abnormal possibility corresponding to the to-be-detected image.

[0021] The judgment module is configured to determine that the to-be-detected image is an abnormal image when the target abnormal possibility meets a preset abnormal possibility threshold.

[0022] In a fourth aspect, the present application also provides a computer readable storage medium. The computer readable storage medium has a computer program stored thereon, and the computer program, when executed by a processor, implements the following steps:

[0023] An image to be detected is acquired, a face region in the image to be detected is extracted, the face region is subjected to abnormal face recognition, and an abnormal face possibility corresponding to the face region is obtained;

[0024] A background region in the image to be detected is extracted, the background region is subjected to abnormal scene recognition, and an abnormal scene possibility corresponding to the background region is obtained;

[0025] An object region in the image to be detected is subjected to abnormal object detection, and an abnormal object possibility corresponding to the image to be detected is obtained;

[0026] Attention weight calculation is performed based on the face region, the background region and the object region, to obtain attention weights corresponding to the face region, the background region and the object region respectively, and weighted calculation is performed based on the attention weights, the abnormal face possibility, the abnormal scene possibility and the abnormal object possibility, to obtain a target abnormal possibility corresponding to the image to be detected;

[0027] When the target abnormal possibility meets a preset abnormal possibility threshold, the image to be detected is determined as an abnormal image.

[0028] In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program, and the computer program, when executed by a processor, implements the following steps:

[0029] An image to be detected is acquired, a face region in the image to be detected is extracted, the face region is subjected to abnormal face recognition, and an abnormal face possibility corresponding to the face region is obtained;

[0030] A background region in the image to be detected is extracted, the background region is subjected to abnormal scene recognition, and an abnormal scene possibility corresponding to the background region is obtained;

[0031] An object region in the image to be detected is subjected to abnormal object detection, and an abnormal object possibility corresponding to the image to be detected is obtained;

[0032] Attention weight calculation is performed based on the face region, the background region and the object region, to obtain attention weights corresponding to the face region, the background region and the object region respectively, and weighted calculation is performed based on the attention weights, the abnormal face possibility, the abnormal scene possibility and the abnormal object possibility, to obtain a target abnormal possibility corresponding to the image to be detected;

[0033] When the target abnormal possibility meets a preset abnormal possibility threshold, the image to be detected is determined as an abnormal image.

[0034] The abnormal image recognition method, device, computer device, storage medium and computer program product, calculate the abnormal face possibility by extracting the face region in the to-be-detected image, calculate the abnormal scene possibility by extracting the background region in the to-be-detected image, then perform abnormal object detection on the object region in the to-be-detected image to obtain the abnormal object possibility. Then, the attention weights corresponding to the face region, the background region and the object region are calculated, the target abnormal possibility corresponding to the to-be-detected image is obtained by weighted calculation of the attention weights, the abnormal face possibility, the abnormal scene possibility and the abnormal object possibility, and the target abnormal possibility is judged with the abnormal possibility threshold to determine whether the to-be-detected image is an abnormal image. The target abnormal possibility is more accurate by joint judgment of the face region, the background region and the object region in the to-be-detected image, and the abnormal image detection of the to-be-detected image is more accurate, thereby improving the recognition accuracy of the abnormal image. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 An application environment diagram of an abnormal image recognition method in an embodiment;

[0036] Figure 2 A flowchart of an abnormal image recognition method in an embodiment;

[0037] Figure 3 A face region diagram in an embodiment;

[0038] Figure 4 A background region diagram in an embodiment;

[0039] Figure 5 A target object diagram in an embodiment;

[0040] Figure 6 A flowchart of an abnormal image detection in an embodiment;

[0041] Figure 7 A structure block diagram of an abnormal image recognition device in an embodiment;

[0042] Figure 8 An internal structure diagram of a computer device in an embodiment;

[0043] Figure 9 An internal structure diagram of a computer device in another embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical scheme and advantages of the present application clearer, further detailed description will be made to the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0045] The abnormal image recognition method provided by the embodiments of the present application can be applied to the application environment as shown in Figure 1 The terminal 102 communicates with the server 104 through a network. The data storage system can store data required to be processed by the server 104. The data storage system can be integrated on the server 104, or placed on a cloud or other network server. The server 104 obtains the to-be-detected image uploaded by the terminal 102, extracts the face region in the to-be-detected image, performs abnormal face recognition on the face region, and obtains the abnormal face possibility corresponding to the face region. The server 104 extracts the background region in the to-be-detected image, performs abnormal scene recognition on the background region, and obtains the abnormal scene possibility corresponding to the background region. The server 104 performs abnormal object detection on the object region in the to-be-detected image, and obtains the abnormal object possibility corresponding to the to-be-detected image. The server 104 performs attention weight calculation based on the face region, the background region and the object region, obtains the attention weight corresponding to the face region, the background region and the object region respectively, and performs weighted calculation based on the attention weight, the abnormal face possibility, the abnormal scene possibility and the abnormal object possibility, to obtain the target abnormal possibility. When the target abnormal possibility meets a preset abnormal possibility threshold, the server 104 determines that the to-be-detected image is an abnormal image. The terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle-mounted device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0046] In one embodiment, as shown in Figure 2 An abnormal image recognition method is provided. Taking the server in Figure 1 as an example, the method includes the following steps:

[0047] In step 202, a to-be-detected image is obtained, a face region in the to-be-detected image is extracted, and abnormal face recognition is performed on the face region to obtain an abnormal face possibility corresponding to the face region.

[0048] The to-be-detected image refers to an image to be detected for abnormal image detection. The face region refers to the region of the face image in the to-be-detected image. The abnormal face possibility refers to the determination degree of the face in the to-be-detected image as an abnormal face. The abnormal face refers to a pre-stored face marked as abnormal.

[0049] Specifically, the server acquires a to-be-detected image uploaded by the user terminal, and the to-be-detected image includes a user face image corresponding to the user terminal. The server can input the to-be-detected image into a pre-set face detection model for face recognition, and output a face image in the to-be-detected image. Then, the output face image is recognized, and by acquiring a plurality of pre-stored abnormal face images, the face image in the to-be-detected image is compared with the plurality of pre-stored abnormal face images respectively, to obtain an abnormal face possibility of the face image in the to-be-detected image belonging to an abnormal face.

[0050] When the server detects that there is no face image in the to-be-detected image through the face detection model, it is determined that the to-be-detected image is an invalid image, and a face image acquisition request is generated, which is returned to the user terminal to reacquire the to-be-detected image including the user face image.

[0051] Step 204, extracting a background region in the to-be-detected image, performing abnormal scene recognition on the background region, and obtaining an abnormal scene possibility corresponding to the background region.

[0052] The background region refers to an image of a non-portrait region in the to-be-detected image. The abnormal scene possibility refers to a determination degree of the background region in the to-be-detected image being an abnormal scene region. The abnormal scene refers to a pre-stored background marked as abnormal.

[0053] Specifically, the server performs human body detection on the to-be-detected image, determines a background region in the to-be-detected image according to the human body detection result, extracts the background region in the to-be-detected image, and obtains a background image. The server can acquire a plurality of pre-stored abnormal scene images, compare the background image in the to-be-detected image with the plurality of abnormal scenes respectively, and obtain an abnormal scene possibility of the background image in the to-be-detected image belonging to an abnormal scene image.

[0054] Step 206, performing abnormal object detection on an object region in the to-be-detected image, and obtaining an abnormal object possibility corresponding to the to-be-detected image.

[0055] The object region refers to an object image in the to-be-detected image. The abnormal object possibility refers to a determination degree of the object in the to-be-detected image being an abnormal object. The abnormal object refers to a pre-marked abnormal object.

[0056] Specifically, the server can use a pre-set object detection algorithm to perform object detection on the to-be-detected image, to obtain each object in the to-be-detected image. Then, the server determines whether there is an abnormal object in the to-be-detected image according to a pre-set abnormal object, and the abnormal object can be a dangerous object, to obtain an abnormal object possibility corresponding to the to-be-detected image.

[0057] In step 208, attention weight calculation is performed based on the face region, the background region, and the object region to obtain attention weights corresponding to the face region, the background region, and the object region respectively. Weighted calculation is performed based on the attention weights, the abnormal face possibility, the abnormal scene possibility, and the abnormal object possibility to obtain a target abnormality possibility.

[0058] The attention weight refers to the importance of the face region, the background region, and the object region. The target abnormality possibility refers to the determination degree of the to-be-detected image as an abnormal image.

[0059] Specifically, the server obtains feature data corresponding to the face region, the background region, and the object region in the to-be-detected image respectively, and performs attention weight calculation using the feature data corresponding to the face region, the background region, and the object region respectively to obtain attention weights corresponding to the face region, the background region, and the object region respectively. Then, the server performs weighted calculation on the abnormal face possibility, the abnormal scene possibility, and the abnormal object possibility according to the attention weights corresponding to the face region, the background region, and the object region respectively to obtain the target abnormality possibility.

[0060] In step 210, when the target abnormality possibility meets a preset abnormality possibility threshold, it is determined that the to-be-detected image is an abnormal image.

[0061] The preset abnormality possibility threshold is a threshold corresponding to the target abnormality possibility, which is used to determine whether the to-be-detected image is an abnormal image. The abnormal image refers to an image in which the face region, the background region, and the object region are abnormal.

[0062] Specifically, the server obtains a pre-stored abnormality possibility threshold, compares the target abnormality possibility corresponding to the to-be-detected image with the abnormality possibility threshold, and determines that the to-be-detected image is an abnormal image when the target abnormality possibility meets the abnormality possibility threshold.

[0063] In the above abnormal image recognition method, the abnormal face possibility is calculated by extracting the face region in the to-be-detected image, the abnormal scene possibility is calculated by extracting the background region in the to-be-detected image, and then the abnormal object possibility is obtained by performing abnormal object detection on the object region in the to-be-detected image. Then, the attention weights corresponding to the face region, the background region, and the object region are calculated, the target abnormality possibility corresponding to the to-be-detected image is obtained by performing weighted calculation based on the attention weights, the abnormal face possibility, the abnormal scene possibility, and the abnormal object possibility, and the target abnormality possibility is compared with the abnormality possibility threshold to determine whether the to-be-detected image is an abnormal image. The target abnormality possibility is calculated more accurately by joint determination of the face region, the background region, and the object region in the to-be-detected image, and the abnormal image detection of the to-be-detected image is more accurate, thereby improving the recognition accuracy of the abnormal image.

[0064] In one embodiment, step 202, the face region in the image to be detected is extracted, the face region is subjected to abnormal face recognition, and the abnormal face possibility corresponding to the face region is obtained, comprising:

[0065] The face region is subjected to face feature extraction, and face feature information is obtained;

[0066] The abnormal face feature information corresponding to each abnormal face image in the preset abnormal face image library is obtained;

[0067] Based on the face feature information and the abnormal face feature information corresponding to each abnormal face image, a similarity calculation is performed, and a face similarity result corresponding to each abnormal face image is obtained;

[0068] Based on the face similarity result corresponding to each abnormal face image, the abnormal face possibility corresponding to the face region is determined.

[0069] The face feature information refers to the feature vector in the face region. The preset abnormal face image library refers to a pre-set face image library for storing marked abnormal face images. The abnormal face feature information refers to the feature vector corresponding to the marked abnormal face image. The face similarity result refers to the similarity degree of the face region in the image to be detected and the abnormal face in the abnormal face image library.

[0070] Specifically, the server uses a face detection algorithm to perform face detection on the image to be detected, outputs a face frame in the image to be detected, determines a face region in the image to be detected according to the face frame, and extracts a face image in the face region. The server obtains a pre-set feature extraction network, inputs the face image into the feature extraction network, intercepts the output vector before the full connection layer as the feature output, and obtains the face feature information, i.e., the feature vector corresponding to the face image.

[0071] The feature extraction network can be a convolutional neural network, and the vector dimension can be (1, 512). The convolutional neural network encodes and reduces the dimension of the face feature. For example, a face image may contain features such as human features and contours, skin color, etc. Assuming that the resolution of the face image is 256*256, the face feature is encoded into a (1, 512) feature vector after convolution operation.

[0072] The server acquires various abnormal face images from a pre-defined abnormal face image database and extracts the abnormal face feature information corresponding to each image, i.e., the feature vector corresponding to each abnormal face image. The server calculates the similarity between the facial feature information of the face region in the image to be detected and the feature information of each abnormal face. The server can use a cosine similarity algorithm to calculate the similarity between the facial feature information and the abnormal face feature information, obtaining the face similarity result corresponding to each abnormal face image. Then, the server selects the face similarity result with the highest value as the probability of an abnormal face corresponding to the face region.

[0073] In one specific embodiment, such as Figure 3 As shown, a schematic diagram of a face region is provided. The server uses RetinaFace (a face detection network) to perform face detection on the image to be detected, and outputs the image to be detected with face bounding boxes. The face image within the face bounding box is extracted, and the face image is input into MobileNet (a lightweight neural network) for feature extraction, outputting the face vector features corresponding to the face image, with a vector dimension of (1, 512). The server obtains abnormal face images from a preset abnormal face image library. The abnormal face image library can be a blacklist face image database, and each stored abnormal face image can be a face image of a blacklisted user.

[0074] The server extracts feature vectors corresponding to each abnormal face image, with a vector dimension of (1, 512). It then uses cosine similarity to calculate the face similarity between the feature vector of the face region in the image to be detected and the feature vectors of each abnormal face image. The face similarity result with the highest value is taken as the abnormal face confidence score, which can be represented by `face_c`, where `face_c ∈ (0, 1)`. This indicates that the feature vectors of two faces are calculated using cosine similarity. The closer the calculated distance between the two vectors, the more similar the two faces are. The value range is (0, 1), meaning the more similar the faces are, the closer the value is to 1, and vice versa. The server can set the number of abnormal face images to 100, and the expression for the probability of an abnormal face is shown in Formula 1.

[0075] F_score = face_c * 100 (Formula 1)

[0076] F_score represents the probability of an abnormal face.

[0077] In this embodiment, facial feature information is extracted from the image to be detected, and the similarity between the facial image in the image to be detected and the abnormal facial image is calculated based on the facial feature information to obtain the probability of an abnormal face. This allows for subsequent calculation of the probability of an abnormal face in the target image, increasing the recognition angle of abnormal face recognition, thereby realizing multi-angle detection of abnormal images and improving the accuracy of abnormal image recognition.

[0078] In one embodiment, step 204, the background region is subjected to abnormal scene recognition to obtain the abnormal scene possibility corresponding to the background region, including:

[0079] The scene feature information is extracted from the background region;

[0080] The abnormal scene feature information corresponding to each abnormal scene image in the preset abnormal scene image library is obtained;

[0081] The similarity between the scene feature information and the abnormal scene feature information corresponding to each abnormal scene image is calculated based on the scene feature information and the abnormal scene feature information corresponding to each abnormal scene image to obtain the scene similarity result corresponding to each abnormal scene image;

[0082] The abnormal scene possibility corresponding to the background region is determined based on the scene similarity result corresponding to each abnormal scene image.

[0083] The scene feature information refers to the feature vector corresponding to the background region in the to-be-detected image. The preset abnormal scene image library refers to a pre-set scene image library for storing background images marked as abnormal. The scene similarity refers to the similarity between the background region of the to-be-detected image and the abnormal scene image in the abnormal scene image library.

[0084] Specifically, the server performs image semantic segmentation on the to-be-detected image, such as u-net (an image segmentation algorithm), sets the non-background pixels to 0, extracts the region with non-zero pixel values from the server to obtain the background image corresponding to the background region in the to-be-detected image, inputs the background image into the pre-set feature extraction network for scene feature extraction, and extracts the output vector before the full connection layer as the feature output to obtain the scene feature information, i.e., the feature vector corresponding to the background image, and the vector dimension can be (1, 512).

[0085] The server obtains each abnormal scene image in the preset abnormal scene image library and extracts the abnormal scene feature information corresponding to each abnormal scene image, i.e., the feature vector corresponding to each abnormal scene image. The server calculates the similarity between the scene feature information of the background region in the to-be-detected image and each abnormal scene feature information, and the server can use the cosine similarity algorithm to calculate the similarity between the scene feature information and the abnormal scene feature information to obtain the scene similarity result corresponding to each abnormal scene image. Then the server selects the maximum scene similarity result as the abnormal scene possibility corresponding to the background region.

[0086] In one specific embodiment, as Figure 4As shown, a background area schematic diagram is provided; the server uses a u-net (semantic segmentation neural network) for image semantic segmentation to obtain a background image corresponding to the to-be-detected image. The background image is input into a moblienet (lightweight neural network) for feature extraction, and a scene vector feature corresponding to the background image is output, and the vector dimension is (1, 512). The server obtains an abnormal scene image in a preset abnormal scene image library. The abnormal face image library can be a blacklist scene image database, and each abnormal scene image stored can be a place background image of performing illegal business.

[0087] The server extracts the feature vector corresponding to each abnormal place image, and the vector dimension is (1, 512), and uses cosine similarity to calculate the scene similarity result of the feature vector of the background image in the to-be-detected image and the feature vector of each abnormal scene image. The face similarity result with the largest value is taken as the abnormal scene confidence, which can be represented by background_c, and background_c∈(0,1), which means that the feature vectors of the two scene images are calculated by cosine similarity. If the result of the two vectors is closer, the two scenes are more similar, and the value range is (0, 1), that is, the more similar the scene, the closer the value to 1, and vice versa. The value is closer to 0. The server can set the abnormal scene image to 100, and the expression of the abnormal face possibility is shown in formula 2:

[0088] B_score=background_c*100 Formula 2

[0089] Wherein, B_score represents the abnormal scene possibility.

[0090] In one specific embodiment, the server can obtain a pre-set important scene object label. The important scene object can be an important object in the pre-stored plurality of abnormal scene images. The important object is a scene object in which the occurrence rate of the plurality of pre-set objects in each abnormal scene image reaches a preset threshold. The important object can be an entity or a non-entity, such as a fixed telephone, illegal propaganda slogans, illegal company icons, etc. The important object can be one or more.

[0091] The server detects the background image in the to-be-detected image, judges whether the important scene object exists in the background image of the to-be-detected image according to the important scene object label, detects that the important scene object exists in the background image of the to-be-detected image, and determines the region where the important scene object in the background image is located. Then the server cuts the region where the important scene object in the background image is located according to a preset size to obtain a local background image of the region where the important scene object in the background image is located. The local background image is input into a feature extraction network to obtain a feature vector corresponding to the local background image, and the vector dimension is (1, 512).

[0092] The server obtains an abnormal local background image of a corresponding position in the plurality of pre-stored abnormal scene images according to a preset size, and extracts a feature vector corresponding to the abnormal local background image in each abnormal scene image, with a vector dimension of (1, 512). The similarity of the local background image in the to-be-detected image to each abnormal local background image is calculated using a cosine similarity, to obtain a similarity result corresponding to each abnormal local background image, and the abnormal scene possibility corresponding to the to-be-detected image is obtained according to the similarity result corresponding to each abnormal local background image.

[0093] In this embodiment, the scene feature information of the to-be-detected image is extracted, and the similarity of the background image in the to-be-detected image to the abnormal scene image is calculated according to the scene feature information, to obtain the abnormal scene possibility. This enables the subsequent use of the scene face possibility to calculate the target abnormal possibility, increases the recognition angle of abnormal scene recognition, thereby realizing multi-angle detection of abnormal images and improving the recognition accuracy of abnormal images.

[0094] In one embodiment, step 206, abnormal object detection is performed on the object region in the to-be-detected image to obtain an abnormal object possibility corresponding to the to-be-detected image, including:

[0095] Target object detection is performed on the object region in the to-be-detected image to obtain target object information;

[0096] The target object information and the preset abnormal object information are used for feature vector conversion to obtain an abnormal object feature vector;

[0097] An abnormal weight vector corresponding to the preset abnormal object information is obtained, and the abnormal weight vector and the abnormal object feature vector are used for merging calculation to obtain an abnormal object possibility.

[0098] The target object refers to all objects existing in the to-be-detected image. The target object information refers to the information of all objects existing in the to-be-detected image, including the category and attribute of the objects in the to-be-detected image. The feature vector conversion refers to converting the existing objects in the to-be-detected image into a vector form. The preset abnormal object information refers to the information of various objects including abnormal objects, including the category of each pre-set object. The abnormal object feature vector refers to the feature vector of each object in the to-be-detected image. The abnormal weight vector refers to the weight vector corresponding to each object pre-set.

[0099] Specifically, the server uses a preset target detection algorithm to perform object detection on the to-be-detected image, such as dangerous objects such as knives, guns, contraband, and a large number of fixed phones. The target detection algorithm can be yolov5 (target detection algorithm), to obtain the category of each object contained in the to-be-detected image. For example, the target detection algorithm can be yolov5 (target detection algorithm), to obtain the category of each object contained in the to-be-detected image. Figure 5As shown, a target object sketch is provided. The server obtains the categories of each preset object in the preset abnormal object information, performs feature vector conversion according to the object categories in the to-be-detected image and the categories of each preset object, and obtains an abnormal object feature vector. The vector dimension corresponding to the abnormal object feature vector can be represented as (n, N), where n represents the number of object categories identified in the to-be-detected image, and N represents the number of categories of preset objects. Then the server obtains an abnormal weight vector corresponding to the preset abnormal object information, and performs merging calculation based on the abnormal weight vector and the abnormal object feature vector to obtain an abnormal object possibility. The expression of the merging calculation is shown in formula 3:

[0100]

[0101] wherein O_score represents the abnormal object possibility; n i represents the vector of each object in the to-be-detected image; and m represents the abnormal weight vector.

[0102] For example, 5 kinds of object categories are pre-set: phone, knife, fruit, clock, and display screen, and the weights corresponding to the 5 objects are pre-set, as shown in Table 1. The dimension of the abnormal weight vector is (5, 1), and the corresponding abnormal weight vector m is (10, 20, 0, 0, 5). The server performs target detection on the to-be-detected image, detects that there are 3 kinds of object categories in the to-be-detected image, namely knife, phone, and clock, i.e. n = 3, and the vector dimension corresponding to the abnormal object feature vector in the to-be-detected image is (3, 5). The vectors corresponding to each object are as follows: the vector of the knife n1 is (0, 1, 0, 0, 0), the vector of the phone n2 is (1, 0, 0, 0, 0), and the vector of the clock n3 is (0, 0, 0, 1, 0).

[0103] The abnormal object possibility is

[0104] Table 1

[0105] Pre-set item Telephone Knife Fruit Clock Display screen Weight 10 20 0 0 5

[0106] In this embodiment, the abnormal object possibility is calculated by using the abnormal weight vector and the abnormal object feature vector, so that the subsequent target abnormal possibility calculation is performed using the abnormal object possibility, the recognition angle of the abnormal scene recognition is increased, and the multi-angle detection of the abnormal image is realized, thereby improving the recognition accuracy of the abnormal image.

[0107] In one embodiment, in step 208, attention weight calculation is performed based on the face region, the background region, and the object region to obtain the attention weights corresponding to the face region, the background region, and the object region, respectively, including:

[0108] obtain face feature information corresponding to the face region, scene feature information corresponding to the background region, and an abnormal object feature vector corresponding to the object region;

[0109] input the face feature information, the scene feature information, and the abnormal object feature vector into a preset attention network to perform pooling calculation and full connection layer calculation, to obtain attention weights corresponding to the face region, the background region, and the object region respectively.

[0110] Specifically, the server can input the abnormal object feature vector into a convolution network to perform dimension conversion and complete dimension unification according to the dimensions corresponding to the face feature information and the scene feature information. Then, the server inputs the face feature information, the scene feature information, and the abnormal object feature vector after dimension unification into a preset attention network to perform pooling calculation and full connection layer calculation, to obtain attention weights corresponding to the face region, the background region, and the object region respectively.

[0111] In one embodiment, the server inputs the face feature information, the scene feature information, and the abnormal object feature vector after dimension unification into an SE attention module (channel attention module), to obtain attention weights corresponding to the face region, the object region, and the background region respectively: α, β, and γ.

[0112] According to the attention weights corresponding to the face region, the object region, and the background region respectively, the face possibility, the abnormal scene possibility, and the abnormal object possibility, weighted calculation is performed, and the calculation formula is shown in formula 4.

[0113] S = α * F_score + β * O_score + γ * B_score Formula 4

[0114] Wherein, S represents the target abnormal possibility.

[0115] In this embodiment, the face feature information, the scene feature information, and the abnormal object feature vector corresponding to the object region are dimension unified, and then the face feature information, the scene feature information, and the abnormal object feature vector corresponding to the object region after dimension unification are used for weight calculation, to obtain more accurate attention weights corresponding to the face region, the background region, and the object region respectively, so that the abnormal image detection of the to-be-detected image is more accurate.

[0116] In one embodiment, the abnormal image recognition method further comprises:

[0117] input the to-be-detected image into the abnormal image recognition model, extract a face region in the to-be-detected image through the abnormal image recognition model, perform abnormal face recognition on the face region, and obtain an abnormal face possibility corresponding to the face region;

[0118] The background region in the to-be-detected image is extracted through the abnormal image recognition model, an abnormal scene of the background region is recognized, and an abnormal scene possibility corresponding to the background region is obtained.

[0119] The abnormal object possibility corresponding to the to-be-detected image is obtained by performing abnormal object detection on the to-be-detected image through the abnormal image recognition model.

[0120] The attention weight corresponding to the face region, the background region and the object region is obtained by performing attention weight calculation on the abnormal face possibility, the abnormal scene possibility and the abnormal object possibility through the abnormal image recognition model. The target abnormal possibility is obtained by performing weighted calculation based on the attention weight, the abnormal face possibility, the abnormal scene possibility and the abnormal object possibility.

[0121] Specifically, as shown in Figure 6 An abnormal image detection flowchart is provided. The server obtains an abnormal image recognition model, including a feature extraction module, a target detection module, a similarity calculation module, a multi-attention module and a discrimination module. The server obtains a to-be-detected image uploaded by a user terminal, and inputs the to-be-detected image into the abnormal image recognition model for abnormal image recognition. The feature extraction module in the abnormal image recognition model extracts a face region in the to-be-detected image, performs abnormal face recognition on the face region through the similarity calculation module, and performs similarity calculation on the pre-stored abnormal face image obtained from the abnormal face image library to obtain a risk face feature vector result corresponding to the face region, i.e., an abnormal face possibility.

[0122] The feature extraction module extracts a background region in the to-be-detected image, performs abnormal scene recognition on the background region through the similarity calculation module, and performs similarity calculation on the pre-stored abnormal scene image obtained from the abnormal scene image library to obtain a background feature vector result corresponding to the background region, i.e., an abnormal scene possibility.

[0123] The target detection module in the abnormal image recognition model performs abnormal object detection on the to-be-detected image to obtain a target detection vector result corresponding to the to-be-detected image, i.e., an abnormal object possibility.

[0124] Then the abnormal image recognition model inputs the abnormal face possibility, the abnormal scene possibility and the abnormal object possibility into the multi-attention module for weight calculation to obtain the attention weight corresponding to the face region, the background region and the object region, and performs weighted calculation on the abnormal face possibility, the abnormal scene possibility, the abnormal object possibility and the attention weight to obtain the target abnormal possibility.

[0125] The determination module in the abnormal image recognition model determines the target abnormality possibility to determine the abnormal image, and outputs a determination result corresponding to the to-be-detected image. The server determines whether the to-be-detected image is an abnormal image according to the detection result. If the to-be-detected image is an abnormal image, the abnormal image is stored in the abnormal image database. The abnormal image database can be a high-risk fraud scene library.

[0126] In this embodiment, the abnormal image recognition model is constructed, the to-be-detected image uploaded by the user is recognized by the abnormal image recognition model, and the face, background and target object are jointly recognized, so that the accuracy of the abnormal image recognition is improved, and the diversity of the abnormal image samples is increased.

[0127] It should be understood that, although each step in the flowchart involved in each of the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0128] Based on the same inventive concept, the embodiments of the present application also provide an abnormal image recognition device for implementing the above-mentioned abnormal image recognition method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more abnormal image recognition device embodiments provided below can refer to the limitations of the abnormal image recognition method in the above text, which will not be repeated here.

[0129] In one embodiment, as shown in Figure 7 An abnormal image recognition device 700 is provided, comprising a face recognition module 702, a scene recognition module 704, an object recognition module 706, a calculation module 708 and a judgment module 710, wherein:

[0130] The face recognition module 702 is configured to obtain a to-be-detected image, extract a face region in the to-be-detected image, and perform abnormal face recognition on the face region to obtain an abnormal face possibility corresponding to the face region.

[0131] The scene recognition module 704 is configured to extract a background region in the to-be-detected image, perform abnormal scene recognition on the background region, and obtain an abnormal scene possibility corresponding to the background region.

[0132] the object recognition module 706 is configured to perform abnormal object detection on the object region in the to-be-detected image to obtain an abnormal object possibility corresponding to the to-be-detected image;

[0133] The computing module 708 is configured to perform attention weight calculation based on the face region, the background region, and the object region to obtain attention weights corresponding to the face region, the background region, and the object region respectively, and perform weighted calculation based on the attention weights, the abnormal face possibility, the abnormal scene possibility, and the abnormal object possibility to obtain a target abnormal possibility corresponding to the to-be-detected image.

[0134] The judging module 710 is configured to determine that the to-be-detected image is an abnormal image when the target abnormal possibility meets a preset abnormal possibility threshold.

[0135] In one embodiment, the face recognition module 702 includes:

[0136] The face extraction unit is configured to perform face feature extraction on the face region to obtain face feature information, obtain abnormal face feature information corresponding to each abnormal face image in a preset abnormal face image library, perform similarity calculation based on the face feature information and the abnormal face feature information corresponding to each abnormal face image to obtain a face similarity result corresponding to each abnormal face image, and determine an abnormal face possibility corresponding to the face region based on the face similarity result corresponding to each abnormal face image.

[0137] In one embodiment, the scene recognition module includes:

[0138] The scene extraction unit is configured to perform scene feature extraction on the background region to obtain scene feature information, obtain abnormal scene feature information corresponding to each abnormal scene image in a preset abnormal scene image library, perform similarity calculation based on the scene feature information and the abnormal scene feature information corresponding to each abnormal scene image to obtain a scene similarity result corresponding to each abnormal scene image, and determine an abnormal scene possibility corresponding to the background region based on the scene similarity result corresponding to each abnormal scene image.

[0139] In one embodiment, the object recognition module includes:

[0140] The object extraction unit is configured to perform target object detection on the object region in the to-be-detected image to obtain target object information, perform representation vector conversion based on the target object information and preset abnormal object information to obtain an abnormal object representation vector, obtain an abnormal weight vector corresponding to the preset abnormal object information, and perform merging calculation based on the abnormal weight vector and the abnormal object representation vector to obtain an abnormal object possibility.

[0141] In one embodiment, the computing module includes:

[0142] The weight calculation unit is configured to obtain face feature information corresponding to the face region, scene feature information corresponding to the background region, and an abnormal object representation vector corresponding to the object region; and input the face feature information, the scene feature information, and the abnormal object representation vector into a preset attention network to perform pooling calculation and full connection layer calculation, so as to obtain attention weights corresponding to the face region, the background region, and the object region respectively.

[0143] In an embodiment, the abnormal image recognition device 700 further includes:

[0144] The abnormal image recognition model unit is configured to input the to-be-detected image into the abnormal image recognition model, extract a face region in the to-be-detected image through the abnormal image recognition model, perform abnormal face recognition on the face region, and obtain an abnormal face possibility corresponding to the face region; extract a background region in the to-be-detected image through the abnormal image recognition model, perform abnormal scene recognition on the background region, and obtain an abnormal scene possibility corresponding to the background region; perform abnormal object detection on the to-be-detected image through the abnormal image recognition model, and obtain an abnormal object possibility corresponding to the to-be-detected image; perform attention weight calculation based on the abnormal face possibility, the abnormal scene possibility, and the abnormal object possibility through the abnormal image recognition model, and obtain attention weights corresponding to the face region, the background region, and the object region respectively; and perform weighted calculation based on the attention weights, the abnormal face possibility, the abnormal scene possibility, and the abnormal object possibility, and obtain a target abnormal possibility corresponding to the to-be-detected image.

[0145] Each of the above-described abnormal image recognition devices can be realized by software, hardware, or a combination thereof. Each of the above-described modules can be embedded in or independent of a processor in a computer device in a hardware form, or stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform operations corresponding to each of the above-described modules.

[0146] In an embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in Figure 8As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control ability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store abnormal face images, abnormal scene images and the like. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement an abnormal image recognition method.

[0147] In one embodiment, a computer device is provided, which can be a terminal, and its internal structure diagram can be as shown in the figure. Figure 9 As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control ability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store abnormal face images, abnormal scene images and the like. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement an abnormal image recognition method.

[0148] Those skilled in the art can understand that, Figures 8-9The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0149] In one embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method embodiments when executing the computer program. In one embodiment, the processor also implements the following steps when executing the computer program:

[0150] In one embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program implements the steps in the above method embodiments when executed by a processor.

[0151] In one embodiment, a computer program product is provided, including a computer program, and the computer program implements the steps in the above method embodiments when executed by a processor.

[0152] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0154] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0155] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be noted that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. An abnormal image recognition method characterized by comprising: The method comprises: acquiring a to-be-detected image, extracting a face region in the to-be-detected image, performing abnormal face recognition on the face region to obtain an abnormal face possibility corresponding to the face region; extracting a background region in the to-be-detected image, performing abnormal scene recognition on the background region to obtain an abnormal scene possibility corresponding to the background region; performing abnormal object detection on an object region in the to-be-detected image to obtain an abnormal object possibility corresponding to the to-be-detected image; performing attention weight calculation based on the face region, the background region and the object region to obtain attention weights corresponding to the face region, the background region and the object region respectively, performing weighted calculation based on the attention weights, the abnormal face possibility, the abnormal scene possibility and the abnormal object possibility to obtain a target abnormal possibility corresponding to the to-be-detected image; when the target abnormal possibility meets a preset abnormal possibility threshold, determining that the to-be-detected image is an abnormal image.

2. The method of claim 1, wherein, The extraction of the face region in the to-be-detected image, the abnormal face recognition on the face region, and the obtaining of the abnormal face possibility corresponding to the face region comprise: performing face feature extraction on the face region to obtain face feature information; obtaining abnormal face feature information corresponding to each abnormal face image in a preset abnormal face image library; performing similarity calculation based on the face feature information and the abnormal face feature information corresponding to each abnormal face image to obtain a face similarity result corresponding to each abnormal face image; determining the abnormal face possibility corresponding to the face region based on the face similarity result corresponding to each abnormal face image.

3. The method of claim 1, wherein, The abnormal scene recognition on the background region and the obtaining of the abnormal scene possibility corresponding to the background region comprise: performing scene feature extraction on the background region to obtain scene feature information; obtaining abnormal scene feature information corresponding to each abnormal scene image in a preset abnormal scene image library; performing similarity calculation based on the scene feature information and the abnormal scene feature information corresponding to each abnormal scene image to obtain a scene similarity result corresponding to each abnormal scene image; determining the abnormal scene possibility corresponding to the background region based on the scene similarity result corresponding to each abnormal scene image.

4. The method of claim 1, wherein, The abnormal object detection on the object region in the to-be-detected image and the obtaining of the abnormal object possibility corresponding to the to-be-detected image comprise: performing target object detection on the object region in the to-be-detected image to obtain target object information; performing representation vector conversion based on the target object information and preset abnormal object information to obtain an abnormal object representation vector; obtaining an abnormal weight vector corresponding to the preset abnormal object information, and performing merging calculation based on the abnormal weight vector and the abnormal object representation vector to obtain the abnormal object possibility.

5. The method of claim 1, wherein the attention weight calculation based on the face region, the background region and the object region to obtain the attention weight corresponding to the face region, the background region and the object region respectively comprises: obtaining face feature information corresponding to the face region, scene feature information corresponding to the background region and abnormal object feature vector corresponding to the object region; inputting the face feature information, the scene feature information and the abnormal object feature vector into a preset attention network to perform pooling calculation and full connection layer calculation, and obtaining the attention weight corresponding to the face region, the background region and the object region respectively.

6. The method of claim 1, further comprising: inputting the to-be-detected image into an abnormal image recognition model, extracting a face region in the to-be-detected image through the abnormal image recognition model, performing abnormal face recognition on the face region, and obtaining an abnormal face possibility corresponding to the face region; extracting a background region in the to-be-detected image through the abnormal image recognition model, performing abnormal scene recognition on the background region, and obtaining an abnormal scene possibility corresponding to the background region; performing abnormal object detection on the to-be-detected image through the abnormal image recognition model, and obtaining an abnormal object possibility corresponding to the to-be-detected image; performing attention weight calculation based on the abnormal face possibility, the abnormal scene possibility and the abnormal object possibility through the abnormal image recognition model, obtaining the attention weight corresponding to the face region, the background region and the object region respectively, and performing weighted calculation based on the attention weight, the abnormal face possibility, the abnormal scene possibility and the abnormal object possibility to obtain a target abnormal possibility corresponding to the to-be-detected image.

7. An abnormal image recognition device characterized by comprising: The device comprises: a face recognition module configured to obtain a to-be-detected image, extract a face region in the to-be-detected image, perform abnormal face recognition on the face region, and obtain an abnormal face possibility corresponding to the face region; a scene recognition module configured to extract a background region in the to-be-detected image, perform abnormal scene recognition on the background region, and obtain an abnormal scene possibility corresponding to the background region; an object recognition module configured to perform abnormal object detection on an object region in the to-be-detected image, and obtain an abnormal object possibility corresponding to the to-be-detected image; a calculation module configured to perform attention weight calculation based on the face region, the background region and the object region to obtain the attention weight corresponding to the face region, the background region and the object region respectively, and perform weighted calculation based on the attention weight, the abnormal face possibility, the abnormal scene possibility and the abnormal object possibility to obtain a target abnormal possibility corresponding to the to-be-detected image; a judgment module configured to determine that the to-be-detected image is an abnormal image when the target abnormal possibility meets a preset abnormal possibility threshold.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

Images