Image filtering method, device and electronic equipment

By compressing and pruning images using a convolutional neural network model, and combining multiple feature extraction methods, the issues of memory usage and computational efficiency when deploying convolutional neural networks on local user clients are resolved, thus achieving efficient risk image recognition.

CN118015343BActive Publication Date: 2024-11-08CHINA MOBILE INFORMATION SYST INTEGRATION CO LTD +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410078354.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-18
Publication Date
2024-11-08
Estimated Expiration
2044-01-18

AI Technical Summary

Technical Problem

In existing technologies, convolutional neural networks require a large amount of memory and have low computational efficiency when identifying risky images, making them unsuitable for successful deployment on user local clients.

Method used

A compressed convolutional neural network model is adopted. By discarding convolutional layer neurons and pruning fully connected layers, sparsification is performed to reduce model parameters. Combined with preset image feature extraction rules, the features of the target image are extracted to determine whether there are preset risks.

Benefits of technology

While ensuring accuracy, the model parameters were reduced, memory usage was decreased, computational efficiency was improved, and the accuracy of risk identification was enhanced through multiple feature extraction methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118015343B_ABST
    Figure CN118015343B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention discloses an image filtering method, device and electronic device, the method comprising: obtaining a target image to be detected for risk, inputting it into a pre-trained compressed convolutional neural network model, compressing and flattening the target image through the compressed convolutional neural network model, extracting features of the target image in a manner of converting the target image into one or more preset dimensional vectors, obtaining a first image feature, extracting a second image feature of the target image through a preset image feature extraction rule, and determining an image feature corresponding to the target image based on the second image feature and the first image feature, and judging whether the target image is an image with a preset risk based on the similarity between the image feature corresponding to the target image and the risk image feature corresponding to the preset risk image. Through this method, network parameters can be reduced, the memory capacity occupied by the client can be reduced, and the model operation efficiency can be improved, thereby facilitating the deployment of the model to the client.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to an image filtering method, device and electronic equipment. Background Art

[0002] With the development of Internet technology and people's emphasis on personal privacy protection, it is often necessary to filter and identify risk images. Taking online communication as an example, online communication has become an important way of communication and exchange in people's work and life. In order to maintain people's normal life and maintain social stability, it is necessary to identify and process risk images in online communication.

[0003] The current method for identifying and processing risky images in online communication is usually to detect Internet images through a pre-trained image classifier, determine the candidate image area containing the target attribute, and then input the candidate image area into a convolutional neural network to calculate the convolutional neural network features. Finally, the image classifier is used to classify whether the image contains the target attribute based on the convolutional neural network features to achieve image filtering.

[0004] However, in the current methods of identifying and processing risk images, the features of risk images need to be extracted through convolutional neural networks, which requires the convolutional neural network to be deployed on the user's local client. The huge number of parameters of the convolutional neural network requires a large amount of memory during deployment, and it consumes a huge amount of computer resources when performing image recognition, which ultimately leads to low computing efficiency and cannot be successfully deployed on the user's local client. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide an image filtering method, device and electronic device to solve the problem that the convolutional neural network in the prior art needs to occupy a large amount of memory and has low computing efficiency when performing risk image recognition.

[0006] To solve the above technical problems, the embodiments of the present invention are implemented as follows:

[0007] In a first aspect, an embodiment of the present invention provides an image filtering method, the method comprising: obtaining a target image to be detected for risk;

[0008] The target image is input into a pre-trained compressed convolutional neural network model, and the target image is compressed and flattened by the compressed convolutional neural network model, and the target image is converted into one or more preset dimension vectors to extract features of the target image, so as to obtain a first image feature of the target image, wherein the preset dimension is smaller than the dimension of the vector corresponding to the target image, and the compressed convolutional neural network model is a model constructed by a convolutional neural network after the convolutional layer is sparsely processed by discarding neurons in the convolutional layer and the fully connected layer is pruned based on a preset compression threshold;

[0009] Extracting a second image feature corresponding to the target image by using a preset image feature extraction rule, and determining an image feature corresponding to the target image based on the extracted second image feature and the first image feature, wherein the features extracted by using the image feature extraction rule include features not included in the features extracted by using the compressed convolutional neural network model;

[0010] Based on the similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image, it is determined whether the target image is an image with a preset risk.

[0011] Optionally, the inputting the target image into a pre-trained compressed convolutional neural network model, compressing and flattening the target image through the compressed convolutional neural network model, and extracting features of the target image in a manner of converting the target image into one or more preset dimensional vectors to obtain a first image feature of the target image, including:

[0012] Preprocessing the target image, and inputting the preprocessed target image into the convolution layer of the compressed convolutional neural network model for convolution processing to obtain a first preset dimension vector corresponding to the target image;

[0013] Inputting the first preset dimensional vector into the pooling layer of the compressed convolutional neural network model for maximum pooling processing to obtain a second preset dimensional vector corresponding to the target image, and inputting the second preset dimensional vector into a preset activation function to obtain a third preset dimensional vector corresponding to the target image, wherein the preset activation function is used to discard neurons of the convolution layer by controlling the function value of the negative domain;

[0014] In the fully connected layer of the compressed convolutional neural network model, the third preset dimensional vector is compressed and flattened based on pooling processing parameters of at least two preset sizes to obtain the preset dimensional vector corresponding to the target image, and the first image feature of the target image is determined based on the preset dimensional vector.

[0015] Optionally, the method of discarding neurons in the convolutional layer includes:

[0016] A method of randomly discarding neurons in the convolutional layer of the compressed convolutional neural network model according to a preset probability value.

[0017] Optionally, extracting the second image feature corresponding to the target image by using a preset image feature extraction rule includes:

[0018] Determine the image unit corresponding to each pixel point based on adjacent pixels of a plurality of pixels specified in the target image by using a preset image feature extraction rule;

[0019] determining an image block based on a plurality of image units;

[0020] Taking one image unit as a step length, scanning the target image based on each image block to obtain a feature vector corresponding to each image block;

[0021] The second image feature of the target image is determined according to the feature vector corresponding to each image block.

[0022] Optionally, the method for acquiring the risk image feature corresponding to the preset risk image includes:

[0023] Extract feature information according to the risk image set by the user to obtain risk image features corresponding to the preset risk image; or

[0024] According to the open source risk image feature information library, the risk image features corresponding to the preset risk images are obtained.

[0025] Optionally, the method for determining the similarity between the image feature corresponding to the target image and the risk image feature corresponding to the preset risk image includes:

[0026] Calculating the cosine similarity between the image feature corresponding to the target image and the risk image feature corresponding to a preset risk image; or,

[0027] The Pearson correlation coefficient between the image feature corresponding to the target image and the risk image feature corresponding to the preset risk image is calculated.

[0028] In a second aspect, an embodiment of the present invention provides a model training method, the method comprising:

[0029] Acquire image samples with label information, where the label information is used to mark risk images and normal images;

[0030] Preprocessing the image sample, and inputting the preprocessed image sample into the convolution layer of the compression convolution neural network model for convolution processing, to obtain a first preset dimension vector corresponding to the image sample, the compression convolution neural network model is used to compress and flatten the image sample, to extract features of the image sample by converting the image sample into one or more preset dimension vectors, to obtain a first image feature of the image sample, wherein the preset dimension is smaller than the dimension of the vector corresponding to the image sample;

[0031] Inputting the first preset dimensional vector into the pooling layer of the compressed convolutional neural network model for maximum pooling processing to obtain a second preset dimensional vector corresponding to the image sample, and inputting the second preset dimensional vector into a preset activation function to obtain a third preset dimensional vector corresponding to the image sample, wherein the preset activation function is used to discard neurons of the convolution layer by controlling the function value of the negative domain;

[0032] In the fully connected layer of the compressed convolutional neural network model, based on at least two pooling processing parameters of preset sizes, the third preset dimensional vector is compressed and flattened to obtain a preset dimensional vector corresponding to the image sample, and the first image feature of the image sample is determined according to the preset dimensional vector corresponding to the image sample;

[0033] Based on the image samples and a preset loss function, the compressed convolutional neural network model is trained to obtain a trained compressed convolutional neural network model.

[0034] In a third aspect, an embodiment of the present invention provides an image filtering device, the device comprising:

[0035] A target image acquisition module is used to acquire a target image to be detected for risk;

[0036] A first image feature extraction module, used for inputting the target image into a pre-trained compressed convolutional neural network model, compressing and flattening the target image through the compressed convolutional neural network model, and extracting features of the target image by converting the target image into one or more preset dimension vectors to obtain a first image feature of the target image, wherein the preset dimension is smaller than the dimension of the vector corresponding to the target image, and the compressed convolutional neural network model is a model constructed by a convolutional neural network after performing sparse processing on the convolutional layer by discarding neurons in the convolutional layer and pruning the fully connected layer based on a preset compression threshold;

[0037] An image feature determination module, configured to extract a second image feature corresponding to the target image by using a preset image feature extraction rule, and determine the image feature corresponding to the target image based on the extracted second image feature and the first image feature, wherein the features extracted by using the image feature extraction rule include features not included in the features extracted by using the compressed convolutional neural network model;

[0038] The recognition module is used to determine whether the target image is an image with preset risks based on the similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image.

[0039] Optionally, the first image feature extraction module includes:

[0040] A preprocessing unit, used to preprocess the target image, and input the preprocessed target image into the convolution layer of the compressed convolutional neural network model for convolution processing, so as to obtain a first preset dimension vector corresponding to the target image;

[0041] A neuron processing unit, used to input the first preset dimensional vector into the pooling layer of the compressed convolutional neural network model for maximum pooling processing to obtain a second preset dimensional vector corresponding to the target image, and input the second preset dimensional vector into a preset activation function to obtain a third preset dimensional vector corresponding to the target image, wherein the preset activation function is used to discard neurons of the convolution layer by controlling the function value of the negative domain;

[0042] A compression and flattening processing unit is used to perform compression and flattening processing on the third preset dimensional vector in the fully connected layer of the compressed convolutional neural network model based on at least two pooling processing parameters of preset sizes to obtain the preset dimensional vector corresponding to the target image, and determine the first image feature of the target image according to the preset dimensional vector.

[0043] Optionally, the first image feature extraction module discards neurons in the convolutional layer by:

[0044] A method of randomly discarding neurons in the convolutional layer of the compressed convolutional neural network model according to a preset probability value.

[0045] Optionally, the image feature determination module includes:

[0046] An image unit determination unit, configured to determine an image unit corresponding to each pixel point based on adjacent pixels of a plurality of pixels specified in the target image according to a preset image feature extraction rule;

[0047] An image block determining unit, configured to determine an image block based on a plurality of image units;

[0048] An image block feature vector determination unit, configured to scan the target image based on each image block with one image unit as a step length, to obtain a feature vector corresponding to each image block;

[0049] The second image feature determination unit is used to determine the second image feature of the target image according to the feature vector corresponding to each image block.

[0050] Optionally, the method for acquiring the risk image features corresponding to the risk image preset in the recognition module includes:

[0051] Feature information is extracted based on the risk image set by the user to obtain risk image features corresponding to the preset risk image; or, risk image features corresponding to the preset risk image are obtained based on an open source risk image feature information library.

[0052] Optionally, the method for determining the similarity between the image feature corresponding to the target image and the risk image feature corresponding to the preset risk image in the recognition module includes:

[0053] Calculate the cosine similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image; or calculate the Pearson correlation coefficient between the image features corresponding to the target image and the risk image features corresponding to the preset risk image.

[0054] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the image filtering method and the steps of the model training method provided in the above-mentioned embodiment are implemented.

[0055] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image filtering method and the steps of the model training method provided in the above embodiments are implemented.

[0056] It can be seen from the technical solution provided by the above embodiments of the present invention that the embodiments of the present invention obtain a target image to be detected for risk, input the target image into a pre-trained compressed convolutional neural network model, compress and flatten the target image through the compressed convolutional neural network model, and extract features of the target image by converting the target image into one or more preset dimensional vectors to obtain a first image feature of the target image, extract a second image feature corresponding to the target image through a preset image feature extraction rule, and determine the image feature corresponding to the target image based on the extracted second image feature and the first image feature, and judge whether the target image is an image with preset risks based on the similarity between the image feature corresponding to the target image and the risk image feature corresponding to the preset risk image. In this way, by using the compressed convolutional neural network model to compress and flatten the target image, specifically by discarding the convolutional layer neurons to perform sparse processing on the convolutional layer and pruning the fully connected layer, the redundant parameters of the convolutional layer in the convolutional neural network and the redundant nodes in the fully connected layer that have little effect on the output of the neural network are eliminated while ensuring the accuracy of model recognition, thereby greatly reducing the parameters of the convolutional neural network, reducing the memory capacity of the compressed convolutional neural network model deployed to the user's local client, and improving the model's operating efficiency, which is conducive to successfully deploying the model to the client. In addition, in the embodiments of this specification, two image feature extraction methods are used to extract the first image feature and the second image feature respectively. By combining the two image feature extraction methods, the acquired image features can more comprehensively reflect the target image, which is conducive to improving the accuracy of risk identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0058] Figure 1 A schematic diagram of a flow chart of an image filtering method of the present invention;

[0059] Figure 2 A schematic diagram of another image filtering method of the present invention;

[0060] Figure 3 A schematic diagram of a flow chart of a model training method of the present invention;

[0061] Figure 4 It is a structural schematic diagram of an image filtering device of the present invention;

[0062] Figure 5The figure is a schematic structural diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0063] The embodiments of the present invention provide an image filtering method, an image filtering device and an electronic device.

[0064] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0065] Embodiment 1

[0066] like Figure 1 As shown, an embodiment of the present invention provides an image filtering method, and the execution subject of the method may be a terminal device. The method may specifically include the following steps:

[0067] In S102, a target image to be detected for risk is obtained.

[0068] In the embodiment of the present invention, the target image may be a risk image with a preset risk, or may be a normal image. The target image may come from a network communication process, such as a chat record, or may come from other output carriers containing images.

[0069] In S104, the target image is input into a pre-trained compressed convolutional neural network model, and the target image is compressed and flattened by the compressed convolutional neural network model, and features of the target image are extracted by converting the target image into one or more preset dimensional vectors to obtain a first image feature of the target image.

[0070] Among them, the preset dimension is smaller than the dimension of the vector corresponding to the target image. The compressed convolutional neural network model is a model constructed by the convolutional neural network after sparse processing of the convolutional layer by discarding the convolutional layer neurons and pruning the fully connected layer based on the preset compression threshold.

[0071] In implementation, the pre-trained compressed convolutional neural network model can be first deployed to the user's local client, and then the target image to be detected for risk is obtained and input into the compressed convolutional neural network model.

[0072] The compressed convolutional neural network model in the embodiment of this specification is used to compress and flatten the target image, thereby reducing the dimension of the target image and extracting features to obtain the first image feature. By performing sparse processing on the convolutional layer in the compressed convolutional neural network model and pruning the fully connected layer, some neurons in the convolutional layer and redundant nodes in the fully connected layer that contribute less to the output result are removed, thereby reducing redundant parameters and thus reducing model parameters.

[0073] In implementation, the method of removing some neurons from the convolution layer to achieve sparse processing can be to discard some convolution layer neurons through an activation function (or piecewise function), or to directly discard some convolution layer neurons according to specific application scenarios, or to discard some convolution layer neurons according to a preset first weight threshold.

[0074] The fully connected layer can be pruned according to a preset second weight threshold, for example, by deleting nodes whose weight is less than the preset second weight threshold. Nodes whose weight is less than the preset second weight threshold are determined as nodes that contribute less to the output result of the neural network, and are deleted. This can greatly reduce model parameters and improve model operation efficiency while ensuring the recognition accuracy of the model.

[0075] In addition, the compressed convolutional neural network in the embodiments of this specification can be pre-trained on the server side and then applied to the client side, or it can be pre-trained on the client side and then applied to the image filtering processing on the client side.

[0076] In S106, the second image features corresponding to the target image are extracted by a preset image feature extraction rule, and the image features corresponding to the target image are determined based on the extracted second image features and the first image features, and the features extracted by the image feature extraction rule include features that are not included in the features extracted by the compressed convolutional neural network model.

[0077] In the embodiment of the present specification, the first image feature is extracted by a compressed convolutional neural network, and the second image feature is extracted by a preset image feature extraction rule. The features extracted by the image extraction rule include features that are not included in the features extracted by the compressed convolutional neural network model. By combining the two image feature extraction methods, the acquired image features can more comprehensively reflect the characteristics of the target image, thereby improving the accuracy and efficiency of risk identification.

[0078] In implementation, the preset image feature extraction rules can be used to supplement the extraction of image features that cannot be extracted or that are not extracted well based on the compressed convolutional neural network model, such as image contours, image colors, image-perceived hash values, etc. Since the compressed convolutional neural network model is a model obtained by sparse processing and pruning of the convolutional neural network, if the image contour and image color are extracted based on the compressed convolutional neural network model, the accuracy of the extracted image features may be affected due to incomplete extraction information. Therefore, extracting the second image feature through the preset image feature extraction rules can improve the accuracy and comprehensiveness of image feature extraction.

[0079] In implementation, the method of extracting the second image feature corresponding to the target image through the preset image feature extraction rule can act on the target image according to the preset image feature extraction rule to directly obtain the second image feature, or the target image can be partitioned and the feature vector of each partition can be extracted according to the preset image feature extraction rule, and finally the second image feature can be determined based on the feature vectors of all partitions.

[0080] In S108, based on the similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image, it is determined whether the target image is an image with preset risks.

[0081] The risk image features corresponding to the preset risk image are used as reference image features, and the image features corresponding to the target image are compared with the reference image features for similarity. If the similarity between the two is greater than the preset similarity threshold, the target image is very similar to the risk image, and the target image is determined to be an image with a preset risk. Otherwise, the target image is not similar to the risk image, and the target image is determined to be an image without a preset risk.

[0082] In implementation, the method for determining the similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image may use Euclidean distance to calculate the similarity between the two, or may use cosine similarity to determine the similarity between the two, which is not limited in the embodiments of the present invention.

[0083] An embodiment of the present invention provides an image filtering method, which obtains a target image to be detected for risk, inputs the target image into a pre-trained compressed convolutional neural network model, compresses and flattens the target image through the compressed convolutional neural network model, and extracts features of the target image by converting the target image into one or more preset dimensional vectors to obtain a first image feature of the target image, extracts a second image feature corresponding to the target image through a preset image feature extraction rule, and determines image features corresponding to the target image based on the extracted second image features and the first image features, and determines whether the target image is an image with preset risks based on the similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image. In this way, by using the compressed convolutional neural network model to compress and flatten the target image, specifically by discarding the convolutional layer neurons to perform sparse processing on the convolutional layer and pruning the fully connected layer, the redundant parameters of the convolutional layer in the convolutional neural network and the redundant nodes in the fully connected layer that have little effect on the output of the neural network are eliminated while ensuring the accuracy of model recognition, thereby greatly reducing the parameters of the convolutional neural network, reducing the memory capacity of the compressed convolutional neural network model deployed to the user's local client, and improving the model's operating efficiency, which is conducive to successfully deploying the model to the client. In addition, in the embodiments of this specification, two image feature extraction methods are used to extract the first image feature and the second image feature respectively. By combining the two image feature extraction methods, the acquired image features can more comprehensively reflect the target image, which is conducive to improving the accuracy of risk identification.

[0084] Embodiment 2

[0085] like Figure 2 As shown, an embodiment of the present invention provides another image filtering method, and the execution subject of the method may be a terminal device. The method may specifically include the following steps:

[0086] In S202, a target image to be detected for risk is obtained.

[0087] The processing method of step S202 can refer to the processing of step S102 in embodiment 1, and will not be repeated here.

[0088] In S2042, the target image is preprocessed, and the preprocessed target image is input into the convolution layer of the compressed convolutional neural network model for convolution processing to obtain a first preset dimensional vector corresponding to the target image.

[0089] In implementation, the target image may be preprocessed by preprocessing the image size of the target image to obtain a preprocessed target image that matches the compressed convolutional neural network. For example, the original target image may be processed by image reduction or image enlargement to obtain a preprocessed target image with a size of 128*128*3.

[0090] Taking the conversion of a three-dimensional target image into multiple two-dimensional vectors as an example, after the preprocessed target image is input into the convolution layer of the compressed convolutional neural network model, the step size and padding content of the sliding window are set and convolution processing is performed to output multiple first two-dimensional vectors.

[0091] In S2044, the first preset dimensional vector is input into the pooling layer of the compressed convolutional neural network model for maximum pooling processing to obtain a second preset dimensional vector corresponding to the target image, and the second preset dimensional vector is input into a preset activation function to obtain a third preset dimensional vector corresponding to the target image. The preset activation function is used to discard neurons in the convolution layer by controlling the function value of the negative domain.

[0092] Continuing with the example of converting a three-dimensional target image into multiple two-dimensional vectors, after the convolution layer outputs multiple first two-dimensional vectors, the pooling layer selects the maximum value to obtain the second two-dimensional vector. The output result is expressed by the following formula: Among them, Z l is the maximum value of multiple first two-dimensional vectors (i.e., the second two-dimensional vector), are multiple first two-dimensional vectors.

[0093] The second two-dimensional vector is then input into the following activation function (piecewise function): The third two-dimensional vector can be expressed as follows: l =f(Z l ).

[0094] In the above piecewise function, the function value on the negative interval approaches zero infinitely. Therefore, if the input two-dimensional vector Z l If the value of is negative, the output function value approaches zero infinitely. In the embodiment of the present invention, the function value of the negative domain is controlled by the activation function to make it infinitely close to 0 to discard the neurons of the convolution layer, thereby realizing the sparse processing of the convolution layer. Discarding the convolution layer neurons in this way can effectively improve the efficiency and accuracy of the convolution layer sparse processing, avoid removing effective neurons, and thus improve the accuracy of image filtering.

[0095] In an embodiment of the present invention, the method of discarding neurons in the convolutional layer may be: randomly discarding neurons in the convolutional layer in the compressed convolutional neural network model according to a preset probability value. Compared with discarding convolutional neurons according to weights, the method of randomly discarding neurons in the convolutional layer according to a preset probability value can ensure that the number of neurons discarded each time is the same, and the random discard method can further avoid discarding effective neurons, thereby improving the accuracy of sparse processing of the convolutional layer, which is beneficial to the accuracy of image filtering.

[0096] In S2046, in the fully connected layer of the compressed convolutional neural network model, the third preset dimensional vector is compressed and flattened based on pooling processing parameters of at least two preset sizes to obtain the preset dimensional vector corresponding to the target image, and the first image feature of the target image is determined based on the preset dimensional vector.

[0097] In the implementation, taking the selection of three preset sizes of pooling processing parameters to compress and flatten the third preset dimensional vector as an example, first, the third two-dimensional vector is pooled with a sliding window of 4x4, and a set of maximum values ​​in the third two-dimensional vector under this sliding window are extracted and saved as the first set of one-dimensional vectors; then, the third two-dimensional vector is pooled with a sliding window of 2x2, and a set of maximum values ​​in the third two-dimensional vector under this sliding window are extracted and saved as the second one-dimensional vector; secondly, the maximum value of all data in the third two-dimensional vector is taken, and the third set of one-dimensional vectors is output; finally, the first set of one-dimensional vectors, the second set of one-dimensional vectors and the third set of one-dimensional vectors are concatenated to obtain the final one-dimensional feature vector (i.e., the first image feature of the target image).

[0098] According to the above embodiment, it is assumed that the original target image is a two-dimensional vector of size 224x224 after preprocessing, and the output result after five layers of convolution is a three-dimensional vector of 13x13x256 (i.e., 256 two-dimensional feature vectors of 13x13). Using the method of step S2046 in the embodiment of this specification, the three-dimensional vector is pooled in three sizes of 1x1, 2x2, and 4x4, and then the maximum value is taken respectively, and the output result is 256 one-dimensional vectors of (16+4+1)x 1, and the number of eigenvalues ​​is 5376. If the method of directly stretching the two-dimensional vector to a one-dimensional vector is adopted, for the three-dimensional vector with an output result of 13x13x256, the output is 256 one-dimensional vectors of (13x13)x 1, and the number of eigenvalues ​​is 43264. Therefore, the processing method of the compressed fully connected layer in the embodiment of this specification can greatly reduce the number of parameters, which is conducive to further reducing the memory capacity of the compressed convolutional neural network model deployed to the user's local client and improving the model operation efficiency.

[0099] In S206, the second image features corresponding to the target image are extracted by a preset image feature extraction rule, and the image features corresponding to the target image are determined based on the extracted second image features and the first image features, and the features extracted by the image feature extraction rule include features that are not included in the features extracted by the compressed convolutional neural network model.

[0100] In the embodiment of the present invention, there are various ways to extract the second image feature corresponding to the target image through the preset image feature extraction rule. An optional way is provided below:

[0101] In step A1, an image unit corresponding to each pixel point is determined based on adjacent pixels of a plurality of pixels specified in the target image by using a preset image feature extraction rule;

[0102] In step A2, an image block is determined based on a plurality of image units;

[0103] In step A3, the target image is scanned based on each image block with one image unit as the step length to obtain a feature vector corresponding to each image block;

[0104] In step A4, the second image feature of the target image is determined according to the feature vector corresponding to each image block.

[0105] According to the above steps A1-A4, take hog feature extraction as an example: first, divide the target image into several pixel units (i.e., image units), divide the gradient direction into 9 bins evenly, perform histogram statistics on the gradient directions of all pixels in each unit in each direction bin, and obtain a 9-dimensional feature vector. Each 4 adjacent units constitute an image block. Connect the feature vectors in an image block to obtain a 36-dimensional feature vector. Use the image block to scan the target image, and the scanning step is one unit. Finally, connect the features of all image blocks in series to obtain the second feature vector.

[0106] For other processing in step S206, reference may be made to step S106 in the first embodiment, which will not be described in detail here.

[0107] In S208, based on the similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image, it is determined whether the target image is an image with preset risks.

[0108] In the embodiment of the present invention, the method for obtaining the risk image features corresponding to the preset risk image can be to extract feature information according to the risk image set by the user to obtain the risk image features corresponding to the preset risk image; or to obtain the risk image features corresponding to the preset risk image according to the open source risk image feature information library. The former can be used in application scenarios where the user has specific requirements, and the latter can be used in application scenarios where the user does not have specific requirements.

[0109] In the embodiment of the present invention, the method for determining the similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image may be to calculate the cosine similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image. Assume that the cosine similarity expression of the two feature vectors x and y is: The closer the cosine similarity between two vectors is to 1, the more similar the two feature vectors are.

[0110] The method for determining the similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image can also be to calculate the Pearson correlation coefficient (i.e., Pearson correlation coefficient) between the image features corresponding to the target image and the risk image features corresponding to the preset risk image. Assume that the Pearson correlation coefficient expression of the two feature vectors X and Y is: The closer the absolute value of the Pearson correlation coefficient is to 1, the more similar the two eigenvectors are.

[0111] An embodiment of the present invention provides an image filtering method, which obtains a target image to be detected for risk, inputs the target image into a pre-trained compressed convolutional neural network model, compresses and flattens the target image through the compressed convolutional neural network model, and extracts features of the target image by converting the target image into one or more preset dimensional vectors to obtain a first image feature of the target image, extracts a second image feature corresponding to the target image through a preset image feature extraction rule, and determines image features corresponding to the target image based on the extracted second image features and the first image features, and determines whether the target image is an image with preset risks based on the similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image. In this way, by using the compressed convolutional neural network model to compress and flatten the target image, specifically by discarding the convolutional layer neurons to perform sparse processing on the convolutional layer and pruning the fully connected layer, the redundant parameters of the convolutional layer in the convolutional neural network and the redundant nodes in the fully connected layer that have little effect on the output of the neural network are eliminated while ensuring the accuracy of model recognition, thereby greatly reducing the parameters of the convolutional neural network, reducing the memory capacity of the compressed convolutional neural network model deployed to the user's local client, and improving the model's operating efficiency, which is conducive to successfully deploying the model to the client. In addition, in the embodiments of this specification, two image feature extraction methods are used to extract the first image feature and the second image feature respectively. By combining the two image feature extraction methods, the acquired image features can more comprehensively reflect the target image, which is conducive to improving the accuracy of risk identification.

[0112] Embodiment 3

[0113] like Figure 3 As shown, an embodiment of the present invention provides a model training method, the execution subject of the method can be a server, and the server can be an independent server or a server cluster composed of multiple servers. The method can specifically include the following steps:

[0114] In S302, image samples with label information are obtained, and the label information is used to mark risk images and normal images.

[0115] It should be noted that the model training method in the embodiment of the present invention is used to train a compressed convolutional neural network model, which is used to compress and flatten the target image to be detected for risk, and extract the features of the target image by converting the target image into one or more preset dimensional vectors to obtain the first image feature of the target image. Moreover, the compressed convolutional neural network model is a model constructed by a convolutional neural network that is based on a preset compression threshold, performs sparse processing on the convolutional layer by discarding the neurons in the convolutional layer, and performs pruning on the fully connected layer.

[0116] In S304, the image samples are preprocessed and the preprocessed image samples are input into the convolution layer of the compression convolutional neural network model for convolution processing to obtain a first preset dimension vector corresponding to the image sample. The compression convolutional neural network model is used to compress and flatten the image samples to extract features of the image samples by converting them into one or more preset dimension vectors to obtain a first image feature of the image sample, where the preset dimension is smaller than the dimension of the vector corresponding to the image sample.

[0117] In S306, the first preset dimensional vector is input into the pooling layer of the compressed convolutional neural network model for maximum pooling processing to obtain a second preset dimensional vector corresponding to the image sample, and the second preset dimensional vector is input into a preset activation function to obtain a third preset dimensional vector corresponding to the image sample. The preset activation function is used to discard neurons in the convolution layer by controlling the function value of the negative domain.

[0118] In S308, in the fully connected layer of the compressed convolutional neural network model, the third preset dimensional vector is compressed and flattened based on pooling processing parameters of at least two preset sizes to obtain the preset dimensional vector corresponding to the image sample, and the first image feature of the image sample is determined based on the preset dimensional vector corresponding to the image sample.

[0119] The processing method of the above steps S304-S308 can refer to steps S2042-S2046 in the second embodiment, which will not be repeated here.

[0120] In S310, based on the image samples and the preset loss function, the compressed convolutional neural network model is trained to obtain a trained compressed convolutional neural network model.

[0121] Embodiment 4

[0122] The above is an image filtering method provided by an embodiment of the present invention. Based on the same idea, an embodiment of the present invention also provides an image filtering device, such as Figure 4 shown.

[0123] The image filtering device comprises: a target image acquisition module 410, a first image feature extraction module 420, an image feature determination module 430 and a recognition module 440, wherein:

[0124] The target image acquisition module 410 is used to acquire the target image to be detected for risk;

[0125] A first image feature extraction module 420 is used to input a target image into a pre-trained compressed convolutional neural network model, compress and flatten the target image through the compressed convolutional neural network model, and extract features of the target image by converting the target image into one or more preset dimension vectors to obtain a first image feature of the target image, wherein the preset dimension is smaller than the dimension of the vector corresponding to the target image, and the compressed convolutional neural network model is a model constructed by a convolutional neural network after performing a sparse processing on the convolutional layer by discarding neurons in the convolutional layer and performing a pruning processing on the fully connected layer based on a preset compression threshold;

[0126] An image feature determination module 430 is used to extract a second image feature corresponding to the target image by using a preset image feature extraction rule, and determine the image feature corresponding to the target image based on the extracted second image feature and the first image feature, wherein the features extracted by using the image feature extraction rule include features not included in the features extracted by using the compressed convolutional neural network model;

[0127] The identification module 440 is used to determine whether the target image is an image with preset risks based on the similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image.

[0128] In the embodiment of the present invention, the first image feature extraction module 420 includes:

[0129] A preprocessing unit, used to preprocess the target image, and input the preprocessed target image into the convolution layer of the compressed convolutional neural network model for convolution processing, so as to obtain a first preset dimension vector corresponding to the target image;

[0130] A neuron processing unit, used to input the first preset dimensional vector into the pooling layer of the compressed convolutional neural network model for maximum pooling processing to obtain a second preset dimensional vector corresponding to the target image, and input the second preset dimensional vector into a preset activation function to obtain a third preset dimensional vector corresponding to the target image, wherein the preset activation function is used to discard neurons of the convolution layer by controlling the function value of the negative domain;

[0131] The compression flattening processing unit is used to perform compression flattening processing on the third preset dimensional vector in the fully connected layer of the compressed convolutional neural network model based on at least two pooling processing parameters of preset sizes to obtain the preset dimensional vector corresponding to the target image, and determine the first image feature of the target image according to the preset dimensional vector.

[0132] In the embodiment of the present invention, the first image feature extraction module 420 discards the convolutional layer neurons, including:

[0133] A method of randomly discarding neurons in the convolutional layer of a compressed convolutional neural network model according to a preset probability value.

[0134] In the embodiment of the present invention, the image feature determination module 430 includes:

[0135] An image unit determination unit, configured to determine an image unit corresponding to each pixel point based on adjacent pixels of a plurality of pixels specified in a target image according to a preset image feature extraction rule;

[0136] An image block determining unit, configured to determine an image block based on a plurality of image units;

[0137] An image block feature vector determination unit is used to scan the target image based on each image block with one image unit as a step length to obtain a feature vector corresponding to each image block;

[0138] The second image feature determination unit is used to determine the second image feature of the target image according to the feature vector corresponding to each image block.

[0139] In the embodiment of the present invention, the method for obtaining the risk image features corresponding to the risk image preset in the identification module 440 includes:

[0140] Feature information is extracted based on the risk image set by the user to obtain risk image features corresponding to the preset risk image; or, risk image features corresponding to the preset risk image are obtained based on an open source risk image feature information library.

[0141] In the embodiment of the present invention, the method for determining the similarity between the image feature corresponding to the target image and the risk image feature corresponding to the preset risk image in the recognition module 440 includes:

[0142] Calculate the cosine similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image; or calculate the Pearson correlation coefficient between the image features corresponding to the target image and the risk image features corresponding to the preset risk image.

[0143] An embodiment of the present invention provides an image filtering device, which obtains a target image to be detected for risk, inputs the target image into a pre-trained compressed convolutional neural network model, compresses and flattens the target image through the compressed convolutional neural network model, and extracts features of the target image in a manner of converting the target image into one or more preset dimensional vectors to obtain a first image feature of the target image, extracts a second image feature corresponding to the target image through a preset image feature extraction rule, and determines image features corresponding to the target image based on the extracted second image features and the first image features, and determines whether the target image is an image with preset risks based on the similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image. In this way, by using the compressed convolutional neural network model to compress and flatten the target image, specifically by discarding the convolutional layer neurons to perform sparse processing on the convolutional layer and pruning the fully connected layer, the redundant parameters of the convolutional layer in the convolutional neural network and the redundant nodes in the fully connected layer that have little effect on the output of the neural network are eliminated while ensuring the accuracy of model recognition, thereby greatly reducing the parameters of the convolutional neural network, reducing the memory capacity of the compressed convolutional neural network model deployed to the user's local client, and improving the model's operating efficiency, which is conducive to successfully deploying the model to the client. In addition, in the embodiments of this specification, two image feature extraction methods are used to extract the first image feature and the second image feature respectively. By combining the two image feature extraction methods, the acquired image features can more comprehensively reflect the target image, which is conducive to improving the accuracy of risk identification.

[0144] Embodiment 5

[0145] Figure 5 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention is provided.

[0146] The electronic device 500 includes but is not limited to: a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, a processor 510, and a power supply 511. Those skilled in the art will appreciate that Figure 5 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In the embodiments of the present invention, the electronic device includes but is not limited to a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted terminal, a wearable device, and a pedometer.

[0147] Among them, the processor 510 is used to: obtain a target image to be detected for risk; input the target image into a pre-trained compressed convolutional neural network model, compress and flatten the target image through the compressed convolutional neural network model, and extract features of the target image by converting the target image into one or more preset dimensional vectors to obtain a first image feature of the target image, where the preset dimension is smaller than the dimension of the vector corresponding to the target image, and the compressed convolutional neural network model is a model constructed by a convolutional neural network after the convolutional layer is sparsely processed by discarding convolutional layer neurons and the fully connected layer is pruned based on a preset compression threshold; extract a second image feature corresponding to the target image through a preset image feature extraction rule, and determine the image feature corresponding to the target image based on the extracted second image feature and the first image feature, wherein the features extracted through the image feature extraction rule include features not included in the features extracted through the compressed convolutional neural network model; based on the similarity between the image feature corresponding to the target image and the risk image feature corresponding to the preset risk image, determine whether the target image is an image with a preset risk.

[0148] Processor 510 is also used to: pre-process the target image, and input the pre-processed target image into the convolution layer of the compressed convolutional neural network model for convolution processing to obtain a first preset dimensional vector corresponding to the target image; input the first preset dimensional vector into the pooling layer of the compressed convolutional neural network model for maximum pooling processing to obtain a second preset dimensional vector corresponding to the target image, and input the second preset dimensional vector into a preset activation function to obtain a third preset dimensional vector corresponding to the target image, and the preset activation function is used to discard neurons in the convolution layer by controlling the function value of the negative domain; in the fully connected layer of the compressed convolutional neural network model, based on at least two preset sizes of pooling processing parameters, compress and flatten the third preset dimensional vector to obtain the preset dimensional vector corresponding to the target image, and determine the first image feature of the target image according to the preset dimensional vector.

[0149] The processor 510 is further used to: set a method for randomly discarding neurons in the convolutional layer in the compressed convolutional neural network model according to a preset probability value.

[0150] Processor 510 is also used to: determine the image unit corresponding to each pixel point based on adjacent pixels of multiple pixels specified in the target image through a preset image feature extraction rule; determine the image block based on multiple image units; scan the target image based on each image block with one image unit as a step size to obtain a feature vector corresponding to each image block; and determine the second image feature of the target image based on the feature vector corresponding to each image block.

[0151] In addition, the processor 510 is also used to: extract feature information according to the risk image set by the user to obtain risk image features corresponding to the preset risk image; or, obtain risk image features corresponding to the preset risk image according to an open source risk image feature information library.

[0152] In addition, the processor 510 is also used to: calculate the cosine similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image; or calculate the Pearson correlation coefficient between the image features corresponding to the target image and the risk image features corresponding to the preset risk image.

[0153] The embodiment of the present invention provides an electronic device, which obtains a target image to be detected for risk, inputs the target image into a pre-trained compressed convolutional neural network model, compresses and flattens the target image through the compressed convolutional neural network model, extracts features of the target image by converting the target image into one or more preset dimensional vectors, obtains a first image feature of the target image, extracts a second image feature corresponding to the target image through a preset image feature extraction rule, determines an image feature corresponding to the target image based on the extracted second image feature and the first image feature, and determines whether the target image is an image with a preset risk based on the similarity between the image feature corresponding to the target image and the risk image feature corresponding to the preset risk image. In this way, by using the compressed convolutional neural network model to compress and flatten the target image, specifically by discarding the convolutional layer neurons, the convolutional layer is sparsely processed and the fully connected layer is pruned, thereby eliminating redundant parameters of the convolutional layer in the convolutional neural network and redundant nodes in the fully connected layer with less effect on the output of the neural network while ensuring the recognition accuracy of the model, thereby greatly reducing the parameters of the convolutional neural network, reducing the memory capacity of the compressed convolutional neural network model deployed to the user's local client, improving the model operation efficiency, and thus facilitating the successful deployment of the model to the client. In addition, in the embodiments of this specification, two image feature extraction methods are used to extract the first image feature and the second image feature respectively. By combining the two image feature extraction methods, the acquired image features can more comprehensively reflect the target image, which is conducive to improving the accuracy of risk identification.

[0154] It should be understood that in the embodiment of the present invention, the radio frequency unit 501 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving downlink data from the base station, it is sent to the processor 510 for processing; in addition, uplink data is sent to the base station. Generally, the radio frequency unit 501 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 501 can also communicate with the network and other electronic devices through a wireless communication system.

[0155] The electronic device provides users with wireless broadband Internet access through the network module 502, such as helping users to send and receive emails, browse web pages, and access streaming media.

[0156] The audio output unit 503 can convert the audio data received by the RF unit 501 or the network module 502 or stored in the memory 509 into an audio signal and output it as sound. Moreover, the audio output unit 503 can also provide audio output related to a specific function performed by the electronic device 500 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 503 includes a speaker, a buzzer, a receiver, etc.

[0157] The input unit 504 is used to receive audio or video signals. The input unit 504 may include a graphics processor (GPU) 5041 and a microphone 5042. The graphics processor 5041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 506. The image frame processed by the graphics processor 5041 can be stored in the memory 509 (or other storage medium) or sent via the radio frequency unit 501 or the network module 502. The microphone 5042 can receive sound and can process such sound into audio data. The processed audio data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 501 in the case of a telephone call mode.

[0158] The electronic device 500 also includes at least one sensor 505, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 5061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 5061 and / or the backlight when the electronic device 500 is moved to the ear. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used to identify the posture of the electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the sensor 505 can also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.

[0159] The display unit 506 is used to display information input by the user or information provided to the user. The display unit 506 may include a display panel 5061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0160] The user input unit 507 can be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the electronic device. Specifically, the user input unit 507 includes a touch panel 5071 and other input devices 5072. The touch panel 5071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 5071 or near the touch panel 5071 using any suitable object or accessory such as a finger, stylus, etc.). The touch panel 5071 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 510, receives the command sent by the processor 510 and executes it. In addition, the touch panel 5071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 5071, the user input unit 507 may also include other input devices 5072. Specifically, other input devices 5072 may include but are not limited to a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0161] Furthermore, the touch panel 5071 may be overlaid on the display panel 5061. When the touch panel 5071 detects a touch operation on or near it, it transmits the information to the processor 510 to determine the type of the touch event. Then, the processor 510 provides a corresponding visual output on the display panel 5061 according to the type of the touch event. Figure 5 In the figure, the touch panel 5071 and the display panel 5061 are used as two independent components to realize the input and output functions of the electronic device. However, in some embodiments, the touch panel 5071 and the display panel 5061 can be integrated to realize the input and output functions of the electronic device, which is not limited here.

[0162] The interface unit 508 is an interface for connecting an external device to the electronic device 500. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 508 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the electronic device 500 or may be used to transmit data between the electronic device 500 and an external device.

[0163] The memory 509 can be used to store software programs and various data. The memory 509 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 509 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0164] The processor 510 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 509 and calling data stored in the memory 509, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. The processor 510 may include one or more processing units; preferably, the processor 510 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 510.

[0165] The electronic device 500 may also include a power supply 511 (such as a battery) for supplying power to each component. Preferably, the power supply 511 may be logically connected to the processor 510 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0166] Preferably, an embodiment of the present invention further provides an electronic device, comprising a processor 510, a memory 509, and a computer program stored in the memory 509 and executable on the processor 510. When the computer program is executed by the processor 510, each process of the above-mentioned image filtering method and model training method embodiments is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0167] Embodiment 6

[0168] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned image filtering method and model training method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0169] An embodiment of the present invention provides a computer-readable storage medium, which obtains a target image to be detected for risk, inputs the target image into a pre-trained compressed convolutional neural network model, compresses and flattens the target image through the compressed convolutional neural network model, and extracts features of the target image by converting the target image into one or more preset dimensional vectors to obtain a first image feature of the target image, extracts a second image feature corresponding to the target image through a preset image feature extraction rule, and determines image features corresponding to the target image based on the extracted second image features and the first image features, and determines whether the target image is an image with preset risks based on the similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image. In this way, by using the compressed convolutional neural network model to compress and flatten the target image, specifically by discarding the convolutional layer neurons to perform sparse processing on the convolutional layer and pruning the fully connected layer, the redundant parameters of the convolutional layer in the convolutional neural network and the redundant nodes in the fully connected layer that have little effect on the output of the neural network are eliminated while ensuring the accuracy of model recognition, thereby greatly reducing the parameters of the convolutional neural network, reducing the memory capacity of the compressed convolutional neural network model deployed to the user's local client, and improving the model's operating efficiency, which is conducive to successfully deploying the model to the client. In addition, in the embodiments of this specification, two image feature extraction methods are used to extract the first image feature and the second image feature respectively. By combining the two image feature extraction methods, the acquired image features can more comprehensively reflect the target image, which is conducive to improving the accuracy of risk identification.

[0170] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0171] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0174] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0175] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0176] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0177] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity or device including the element.

[0178] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0179] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. An image filtering method, characterized in that: The method comprises: Acquire a target image to be detected for risk; The target image is input into a pre-trained compressed convolutional neural network model, and the target image is compressed and flattened by the compressed convolutional neural network model, and the target image is converted into one or more preset dimension vectors to extract features of the target image, so as to obtain a first image feature of the target image, wherein the preset dimension is smaller than the dimension of the vector corresponding to the target image, and the compressed convolutional neural network model is a model constructed by a convolutional neural network after the convolutional layer is sparsely processed by discarding neurons in the convolutional layer and the fully connected layer is pruned based on a preset compression threshold; Extracting a second image feature corresponding to the target image by a preset image feature extraction rule, and determining an image feature corresponding to the target image based on the extracted second image feature and the first image feature, wherein the features extracted by the image feature extraction rule include features not included in the features extracted by the compressed convolutional neural network model, and the image features extracted by the preset image feature extraction rule include: at least one of an image contour feature, an image color feature, and an image perceptual hash value feature; Based on the similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image, it is determined whether the target image is an image with a preset risk.

2. The method according to claim 1, characterized in that The step of inputting the target image into a pre-trained compressed convolutional neural network model, compressing and flattening the target image through the compressed convolutional neural network model, and extracting features of the target image by converting the target image into one or more preset dimensional vectors to obtain a first image feature of the target image includes: Preprocessing the target image, and inputting the preprocessed target image into the convolution layer of the compressed convolutional neural network model for convolution processing to obtain a first preset dimension vector corresponding to the target image; Inputting the first preset dimensional vector into the pooling layer of the compressed convolutional neural network model for maximum pooling processing to obtain a second preset dimensional vector corresponding to the target image, and inputting the second preset dimensional vector into a preset activation function to obtain a third preset dimensional vector corresponding to the target image, wherein the preset activation function is used to discard neurons of the convolutional layer by controlling the function value of the negative number domain to make it infinitely close to 0; In the fully connected layer of the compressed convolutional neural network model, based on at least two pooling processing parameters of preset sizes, the third preset dimensional vector is compressed and flattened to obtain the preset dimensional vector corresponding to the target image, and the first image feature of the target image is determined based on the preset dimensional vector. The compression and flattening processing includes pooling processing and splicing processing on the third preset dimensional vector.

3. The method according to claim 1, characterized in that: The method of discarding neurons in the convolutional layer includes: A method of randomly discarding neurons in the convolutional layer of the compressed convolutional neural network model according to a preset probability value.

4. The method according to claim 1, characterized in that: The extracting the second image feature corresponding to the target image by using a preset image feature extraction rule includes: Determine the image unit corresponding to each pixel point based on adjacent pixels of a plurality of pixels specified in the target image by using a preset image feature extraction rule; determining an image block based on a plurality of image units; Taking one image unit as a step length, scanning the target image based on each image block to obtain a feature vector corresponding to each image block; Determine a second image feature of the target image according to a feature vector corresponding to each image block; The step of scanning the target image based on each image block with one image unit as the step length to obtain a feature vector corresponding to each image block includes: The target image is scanned with each image block, with a scanning step length of one image unit, to obtain a feature vector corresponding to each image block, wherein the image unit is an image unit obtained by segmenting the target image.

5. The method according to claim 1, characterized in that The method for acquiring the risk image features corresponding to the preset risk image includes: Extract feature information according to the risk image set by the user to obtain risk image features corresponding to the preset risk image; or According to the open source risk image feature information library, the risk image features corresponding to the preset risk images are obtained.

6. The method according to claim 1, characterized in that The method for determining the similarity between the image feature corresponding to the target image and the risk image feature corresponding to the preset risk image includes: Calculate the cosine similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image; or calculate the Pearson correlation coefficient between the image features corresponding to the target image and the risk image features corresponding to the preset risk image.

7. A model training method, characterized in that: The method comprises: Acquire image samples with label information, where the label information is used to mark risk images and normal images; Preprocessing the image sample, and inputting the preprocessed image sample into the convolution layer of the compression convolution neural network model for convolution processing, to obtain a first preset dimension vector corresponding to the image sample, the compression convolution neural network model is used to compress and flatten the image sample, to extract features of the image sample by converting the image sample into one or more preset dimension vectors, to obtain a first image feature of the image sample, wherein the preset dimension is smaller than the dimension of the vector corresponding to the image sample; Inputting the first preset dimensional vector into the pooling layer of the compressed convolutional neural network model for maximum pooling processing to obtain a second preset dimensional vector corresponding to the image sample, and inputting the second preset dimensional vector into a preset activation function to obtain a third preset dimensional vector corresponding to the image sample, wherein the preset activation function is used to discard neurons of the convolutional layer by controlling the function value of the negative number domain to make it infinitely close to 0; In the fully connected layer of the compressed convolutional neural network model, based on at least two pooling processing parameters of preset sizes, the third preset dimensional vector is compressed and flattened to obtain the preset dimensional vector corresponding to the image sample, and the first image feature of the image sample is determined according to the preset dimensional vector corresponding to the image sample, wherein the compression and flattening processing includes pooling and splicing processing on the third preset dimensional vector; Based on the image samples and a preset loss function, the compressed convolutional neural network model is trained to obtain a trained compressed convolutional neural network model.

8. An image filtering device, characterized in that: The device comprises: A target image acquisition module is used to acquire a target image to be detected for risk; A first image feature extraction module, used for inputting the target image into a pre-trained compressed convolutional neural network model, compressing and flattening the target image through the compressed convolutional neural network model, and extracting features of the target image by converting the target image into one or more preset dimension vectors to obtain a first image feature of the target image, wherein the preset dimension is smaller than the dimension of the vector corresponding to the target image, and the compressed convolutional neural network model is a model constructed by a convolutional neural network after performing sparse processing on the convolutional layer by discarding neurons in the convolutional layer and pruning the fully connected layer based on a preset compression threshold; An image feature determination module, configured to extract a second image feature corresponding to the target image by using a preset image feature extraction rule, and determine an image feature corresponding to the target image based on the extracted second image feature and the first image feature, wherein the features extracted by using the image feature extraction rule include features not included in the features extracted by using the compressed convolutional neural network model, and the image features extracted by using the preset image feature extraction rule include at least one of: an image contour feature, an image color feature, and an image perceptual hash value feature; The recognition module is used to determine whether the target image is an image with preset risks based on the similarity between the image features corresponding to the target image and the risk image features corresponding to the preset risk image.

9. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the image filtering method according to any one of claims 1 to 6 or the steps of a model training method according to claim 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image filtering method described in any one of claims 1 to 6 or the steps of a model training method described in claim 7 are implemented.