Sensitive content identification method

Abstract

The invention provides a sensitive content recognition method, and the method comprises the steps: determining the sensitivity score of a to-be-recognized object through the feature data of the to-be-recognized object; and based on the sensitivity score of the to-be-identified object, judging whether the to-be-identified object is sensitive content or not. Therefore, it can be avoided that whether the object to be recognized is the sensitive content or not is directly determined through matching or mismatching, and the one-step recognition mode is avoided. Meanwhile, the sensitivity degree of the to-be-recognized object can be determined by setting the sensitivity score, and the recognition precision is improved in a finer-grained recognition mode.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, and in particular, to a sensitive content recognition method. BACKGROUND

[0002] With the rapid development and wide application of cloud storage services, more and more individual and enterprise users store their data in cloud disks. However, some sensitive data will inevitably be stored in the cloud disk, and once these sensitive data are spread, they may bring serious consequences to the platform.

[0003] Currently, the recognition of sensitive data often pre-sets a set of sensitive keywords or Message-Digest Algorithm 5 (MD5) values. When data is uploaded or shared, the content is scanned and matched, and once a sensitive word is found, it is shielded or encrypted. However, keyword matching can only identify known sensitive words or known sensitive file MD5, and cannot effectively identify variants, synonyms, etc., with low recognition accuracy. SUMMARY

[0004] The present disclosure provides a sensitive content recognition method. The sensitivity score of the to-be-recognized object is determined based on the feature data of the to-be-recognized object, and it is judged whether the to-be-recognized object is sensitive content based on the sensitivity score. Thereby, sensitive content can be identified in a more fine-grained manner, and the recognition efficiency is improved.

[0005] In one aspect, the present embodiment provides a sensitive content recognition method, which includes: determining a sensitivity score of a to-be-recognized object based on feature data of the to-be-recognized object; and determining whether the to-be-recognized object is sensitive content based on the sensitivity score of the to-be-recognized object, so as to process the to-be-recognized object based on a preset policy.

[0006] In the embodiments of the present disclosure, the feature data of the to-be-recognized object is extracted, and the feature data at least includes a text feature vector, an image feature vector and / or a video feature vector; and the feature data is standardized processed, so as to determine the sensitivity score of the to-be-recognized object based on the standardized processed feature data.

[0007] In the embodiments of the present disclosure, the sensitivity score of the to-be-recognized object is determined based on the feature data of the to-be-recognized object, including: in response to the to-be-recognized object being a text segment, inputting the text feature vector into a bidirectional long short-term memory network to obtain hidden layer vectors of a plurality of words, each text feature at least including a plurality of words; determining a sensitivity weight of each word based on the hidden layer vector of each word; and determining the sensitivity score of the text segment based on each sensitivity weight and each hidden layer vector.

[0008] In an embodiment of the present disclosure, the sensitivity score of the to-be-identified object is determined based on the feature data of the to-be-identified object, comprising: in response to the to-be-identified object being an image, obtaining a plurality of sensitive feature maps based on an image feature vector; determining an initial sensitivity score of the image and a spatial attention weight of each sensitive feature map based on the plurality of sensitive feature maps; and determining the sensitivity score of the image based on the initial sensitivity score and the plurality of spatial attention weights.

[0009] In an embodiment of the present disclosure, the plurality of sensitive feature maps are obtained based on the image feature vector, comprising: determining a sensitive feature extractor based on a pre-trained convolutional neural network; and obtaining the plurality of sensitive feature maps based on the sensitive feature extractor.

[0010] In an embodiment of the present disclosure, the sensitivity score of the to-be-identified object is determined based on the feature data of the to-be-identified object, comprising: in response to the to-be-identified object being a video, determining the sensitivity scores of a plurality of images based on a video feature vector, the video comprising at least the plurality of images; and aggregating the sensitivity scores of the plurality of images to determine the sensitivity score of the video.

[0011] In an embodiment of the present disclosure, the sensitivity score of the to-be-identified object is determined based on the sensitivity score of the to-be-identified object, comprising: determining a mean of sensitivity scores and a standard deviation of sensitivity scores based on the sensitivity scores of a plurality of to-be-identified objects; determining an abnormality judgment threshold based on the mean of sensitivity scores and the standard deviation of sensitivity scores; and in response to the sensitivity score of the to-be-identified object being greater than the abnormality judgment threshold, determining that the to-be-identified object is sensitive content.

[0012] In an embodiment of the present disclosure, further comprising: in response to determining that the to-be-identified object is sensitive content, encrypting the to-be-identified object so as to store the encrypted to-be-identified object.

[0013] In an embodiment of the present disclosure, the to-be-identified object is encrypted, comprising: in response to determining that the to-be-identified object is a to-be-encrypted object, determining a data set to which the to-be-identified object belongs, the data set comprising at least a plurality of text segments, a plurality of images, and / or a plurality of videos; dividing a plurality of sensitivity scores into a plurality of sub-regions based on the sensitivity scores of the plurality of text segments, the sensitivity scores of the plurality of images, and / or the sensitivity scores of the plurality of videos included in the data set; determining a plurality of semantic mapping data of each to-be-encrypted object based on the plurality of sub-regions; and encrypting each semantic mapping data to encrypt the to-be-encrypted object.

[0014] In the embodiments of the present disclosure, the plurality of semantic mapping data of each object to be encrypted is determined based on the plurality of sub-regions, including: dividing the object to be encrypted into a plurality of sub-objects to be encrypted based on the plurality of sub-regions; determining the encryption granularity of each sub-region; and determining the semantic mapping data of each sub-object to be encrypted based on the sensitivity score of the sub-object to be encrypted and the encryption granularity of the sub-region.

[0015] In another aspect, the embodiments of the present disclosure provide a network device, comprising: a memory for storing computer readable instructions; and a processor for running the above computer readable instructions, so that the network device performs the above sensitive content identification method.

[0016] In yet another aspect, the embodiments of the present disclosure provide a computer program product, comprising a computer program which, when executed by a processor, implements the above sensitive content identification method. BRIEF DESCRIPTION OF DRAWINGS

[0017] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. The drawings provided in the present disclosure are for the purpose of illustration and explanation only and thus do not limit the present disclosure. In the drawings, the same reference symbols generally refer to the same parts or steps throughout the specification.

[0018] Figure 1 An environmental application diagram according to an embodiment of the present disclosure is schematically illustrated.

[0019] Figure 2 A flowchart of a sensitive content identification method according to an embodiment of the present disclosure is schematically illustrated.

[0020] Figure 3 A flowchart of another sensitive content identification method according to an embodiment of the present disclosure is schematically illustrated.

[0021] Figure 4 A flowchart of determining a sensitivity score of an object to be identified according to an embodiment of the present disclosure is schematically illustrated.

[0022] Figure 5 Another schematic diagram of determining a sensitivity score of an object to be identified according to an embodiment of the present disclosure is schematically illustrated.

[0023] Figure 6 A flowchart of another sensitive content identification method according to an embodiment of the present disclosure is schematically illustrated.

[0024] Figure 7 A flowchart of determining whether an object to be identified is sensitive content according to an embodiment of the present disclosure is schematically illustrated.

[0025] Figure 8 A schematic diagram of encrypting a to-be-identified object is illustratively shown according to an embodiment of the present disclosure.

[0026] Figure 9 A flowchart of determining a plurality of semantic mapping data of each to-be-encrypted object is illustratively shown according to an embodiment of the present disclosure.

[0027] Figure 10 A block diagram of a network device is illustratively shown according to an embodiment of the present disclosure.

[0028] Figure 11 A block diagram of a computer program product is illustratively shown according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] In order to make the purposes, technical solutions and advantages of the present disclosure more obvious, the example embodiments according to the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the example embodiments described herein.

[0030] Figure 1 An environmental application diagram is illustratively shown according to an embodiment of the present disclosure.

[0031] As shown in Figure 1 , a computing device 101 and a terminal device 102. The computing device 101 and the terminal device 102 can perform real-time data transmission.

[0032] In some embodiments, the terminal device 102 can upload a file to the computing device 101, and the computing device 101 can receive the uploaded file and perform sensitive identification and storage on the file. The computing device 101 can be a server or other network device with computing function. The terminal device 102 can include a smartphone, a smart tablet, a laptop, or other network device.

[0033] In some embodiments, the terminal device 102 can also upload a file to a network device with storage function, for example, to a cloud disk. The computing platform can perform sensitive identification on the file uploaded to the cloud disk. In this case, the computing platform can be composed of a plurality of computing devices 101 with computing function.

[0034] In some embodiments, the computing device 101 can perform preliminary classification on the file uploaded by the user and perform preprocessing on the file. Further, based on the preprocessed file, the computing device 101 can perform sensitive content identification on the file. When the file is identified to include sensitive content, the computing device 101 can process according to a preset policy.

[0035] In some cases, the method of identifying sensitive content by using keywords or file MD5 matching often needs to pre-set a set of sensitive keywords or file MD5 values, and when uploading or sharing data, the content is scanned and matched, and once the sensitive word is found, it is shielded or encrypted. However, keyword matching can only identify known sensitive words or known sensitive file MD5, and cannot effectively identify variants, synonyms, etc., resulting in a large number of sensitive data being missed. And fixed identification rules are difficult to update in time, further leading to a decline in identification effect. At the same time, using the above method can only determine whether the content is sensitive, and cannot evaluate the severity of the sensitivity, which is easy to cause "one-size-fits-all" protection measures due to individual differences.

[0036] Based on this, the embodiments of the present disclosure provide a sensitive content identification method, which determines the sensitivity score of the to-be-identified object through the feature data of the to-be-identified object. And based on the sensitivity score of the to-be-identified object, it is judged whether the to-be-identified object is sensitive content. Thus, it can be avoided to directly determine whether the to-be-identified object is sensitive content by "matching" or "not matching", and the "one-size-fits-all" identification method is avoided. At the same time, by setting the sensitivity score, the sensitivity of the to-be-identified object can also be determined, so as to improve the identification accuracy in a more fine-grained identification manner.

[0037] Next, the present document will be combined with Figures 2 to 9 The above how the computing device 101 identifies sensitive content in the file will be described in detail.

[0038] Figure 2 The flow chart of a sensitive content identification method according to an embodiment of the present disclosure is schematically shown.

[0039] As Figure 2 shown, the sensitive content identification method of the embodiments of the present disclosure includes S201 and S202: S201, determining the sensitivity score of the to-be-identified object based on the feature data of the to-be-identified object.

[0040] In the embodiments of the present disclosure, the computing device can identify the sensitive content of the to-be-identified object uploaded by the user. Further, the computing device can determine the sensitivity score of the identified sensitive content. By determining the sensitivity score, different sensitive content is valued, thereby avoiding "one-size-fits-all" for sensitive content.

[0041] In some embodiments, the to-be-identified objects can include text segments, images, videos. For example, when a user uploads a text file, the to-be-identified objects can include multiple text segments. For another example, when a user uploads an image file, the to-be-identified objects can include multiple images, or multiple images and multiple text segments. For another example, when a user uploads a video file, the to-be-identified objects can include multiple images, or multiple images and multiple text segments.

[0042] Figure 3 A flowchart of another sensitive content identification method according to an embodiment of the present disclosure is schematically shown.

[0043] In an embodiment of the present disclosure, as shown in Figure 3 S301 and S302 can be included: S301, extracting feature data of the to-be-identified objects, the feature data at least including a text feature vector, an image feature vector and / or a video feature vector.

[0044] In an embodiment of the present disclosure, when a user uploads files, the computing device can preliminarily classify these files in real time, and distinguish different types of data such as text segments, images and videos. In some embodiments, the computing device can also preprocess the uploaded data. The preprocessing can include data cleaning and formatting. For example, text segments can be converted into a uniform format, images can be compressed and standardized, and videos can extract key frames for standardized processing, so as to facilitate subsequent analysis and processing.

[0045] In some embodiments, after preprocessing the to-be-identified objects, feature data of the to-be-identified objects can also be extracted. For example, for text segments in a text file, the computing device can use natural language processing techniques, such as Bidirectional Encoder Representations from Transformers (BERT) model, to extract text feature vectors from the text segments for representing key information and semantic features. For image data and video key frame data, computer vision techniques (such as convolutional neural networks) can be used to analyze image content, and extract image feature vectors and video feature vectors for representing key visual features.

[0046] S302, performing standardized processing on the feature data, so as to determine a sensitivity score of the to-be-identified objects based on the standardized processed feature data.

[0047] In embodiments of the present disclosure, the text feature vector, the image feature vector, and the video feature vector can be normalized. For example, normalization, standardization, logarithmic transformation, and the like can be used to make the dimensions of the feature vector on a "fair scale" to participate in the calculation. Normalizing the feature data can eliminate scale interference, optimize the training process, enhance the generalization ability, and ultimately improve the accuracy and robustness of sensitive content recognition, especially for implicit sensitive content, such as the recognition of text without explicit sensitive words but with semantic violations.

[0048] In some embodiments, the extracted text feature vector may include multi-dimensional data. For example, the text feature vector where R represents the set of real numbers, indicating that each element of the feature vector is a real number. t may represent the dimension of the text feature vector, that is, how many numerical elements this vector has. For example, when the text is represented by a 300-dimensional word embedding, d t = 300, meaning that f t is a real number vector with a length of 300.

[0049] In some embodiments, the computing device can normalize the text feature vector extracted based on the text. For example, L2 normalization can be performed to project the text feature vector onto a unit hypersphere, which can be achieved by the following formula:

[0050] In some embodiments, the computing device can apply a Box-Cox transformation to the normalized text feature vector to improve the normality and / or equal variance of the text feature vector. For example, the Box-Cox transformation can be determined by the following formula:

[0051] where λ is a transformation parameter that can be obtained by maximum likelihood estimation; is the i-th dimension of the text feature; is the i-th dimension of the text feature vector after the Box-Cox transformation.

[0052] In some embodiments, the image feature vector may be scaled to a preset interval [0, 1] to unify the numerical range of the feature and improve the stability, training efficiency, and prediction accuracy of the model. Similar to the text feature vector, represents the number of dimensions of the image feature.

[0053] ​In some embodiments, the image feature vector can be scaled to the interval [0, 1] by Min-Max normalization, which can be implemented by the following formula:

[0054] wherein, denotes the i-th dimensional image feature.

[0055] In some embodiments, for each dimensional image feature, the computing device can improve the symmetry and / or normality of the feature distribution by Yeo-Johnson transformation, which can be implemented by the following formula:

[0056] wherein, the parameter is the parameter of Box-Cox transformation, used to control the form of data transformation; is the i-th dimensional image feature; is the i-th dimensional image feature after Yeo-Johnson transformation.

[0057] In some embodiments, the normalization of the video feature vector of video key frame extraction is similar to the normalization process of the above-mentioned video feature vector, and is not described here. First, Min-Max normalization is performed, and then Yeo-Johnson transformation is applied, and the transformation formula is the same as above.

[0058] According to embodiments of the present disclosure, by standardizing the extracted feature data, the scale difference, distribution deviation and other interference in the feature vector of different dimensions can be eliminated, and the accuracy and efficiency can be improved.

[0059] In embodiments of the present disclosure, the standardized text features or image features can be input into a network model to determine the sensitivity score of the text, image or video.

[0060] For example, the network model can be input with N t text segments to determine the sensitivity score of each text segment. Among them, may represent a total of N t text segments, is a text feature vector, which can represent the standardized feature vector of the i-th text segment. For another example, the network model can be input with N v images to determine the sensitivity score of each image. Among them, may represent a total of N v images, is a normalized feature vector of the i-th image. For example, a feature vector of an image can be input into the network model N k videos, to determine a sensitivity score of each text segment. Wherein, the sensitivity score of the i-th text segment can be determined by N k videos, wherein the number of videos is M i is the number of key frames of the i-th video, is a feature vector of an image of a video, and can represent a normalized feature vector of the j-th key frame of the i-th video.

[0061] In an embodiment of the present disclosure, as shown in Figure 4 S201 can include S401, S402 and S403.

[0062] Figure 4 A flowchart for determining a sensitivity score of an object to be recognized according to an embodiment of the present disclosure is schematically shown.

[0063] S401, in response to the object to be recognized being a text segment, inputting a text feature vector into a bidirectional long short-term memory network to obtain hidden layer vectors of a plurality of words, each text feature comprising at least the plurality of words.

[0064] In some embodiments, in the case that the object to be recognized is a text segment, for each text segment i, its corresponding text feature vector determines a sensitivity score .

[0065] For example, the text feature vector can be input into a bidirectional long short-term memory network to obtain a hidden layer representation , wherein can represent a hidden layer vector of the j-th word in the i-th text segment, n i is the number of words of the i-th text segment, d h can represent the hidden layer dimension of the unidirectional long short-term memory network. R 2dh represents a 2R h dimensional real vector space.

[0066] S402, based on the hidden layer vector of each word, determining a sensitivity weight of each word.

[0067] ​In an embodiment of the present disclosure, when calculating the sensitivity weight, a high-order mutual information (HOMI) of word embedding is designed. Wherein, the sensitivity weight of the jth word in the ith text segment at time step t can be realized by the following formula:

[0068] wherein, and are attention parameters; represents transpose, which performs dot product operation with h t,i,j to obtain a scalar value, which represents the similarity between the hidden layer vector and the attention parameter; is a balance factor; is a sensitive word dictionary; is a word w i,j The high-order mutual information of the sensitive word dictionary can be calculated by the information measure:

[0069] wherein, is the order parameter of the information measure; is the probability distribution of the word frequency estimation; w i,j represents the jth word of the ith text segment.

[0070] S403, based on each sensitivity weight and each hidden layer vector, determine the sensitivity score of the text segment.

[0071] In an embodiment of the present disclosure, for a certain text segment i, the sensitivity score of the text segment i can be determined by using the product of the sensitivity weight of the word and the hidden layer vector of the word in a weighted average manner, which can be realized by the following formula:

[0072] wherein, is a learnable weight vector; is a Sigmoid function, which can map the sensitivity score to the interval [0, 1] to facilitate subsequent calculation; is the sensitivity score.

[0073] In an embodiment of the present disclosure, as shown in Figure 5 , the above S201 can include S501, S502 and S503.

[0074] Figure 5 Another flowchart for determining the sensitivity score of the to-be-recognized object according to an embodiment of the present disclosure is schematically shown.

[0075] S501、responding to the object to be identified being an image, obtaining a plurality of sensitive feature maps based on the image feature vector.

[0076] In some embodiments, in the case of the object to be identified being an image, for each image i, its corresponding image feature vector determining the sensitivity score .

[0077] In embodiments of the present disclosure, before S501, there can further be included determining a sensitive feature extractor based on a pre-trained convolutional neural network; and obtaining the plurality of sensitive feature maps based on the sensitive feature extractor.

[0078] In some embodiments, the pre-trained convolutional neural network such as Visual Geometry Group Network (VGG), Residual Network (ResNet), etc. can be used to extract the image feature vector to obtain an initial feature map .

[0079] Further, the pre-trained convolutional neural network can be fine-tuned by using a sensitive image training set , so as to learn the sensitive features. After fine-tuning, a sensitive feature extractor can be obtained. Then based on the sensitive feature extractor , the image feature vector can be mapped into a multi-scale sensitive feature map, i.e. a multi-layer sensitive feature map.

[0080] Exemplarily, the loss function can be as follows:

[0081] wherein, is the sensitive probability predicted by the model.

[0082]

[0083] wherein, is the l-th layer sensitive feature map of the image i; and L is the number of layers of the sensitive feature map.

[0084] S502, determining the initial sensitivity score of the image and the spatial attention weight of each sensitive feature map based on the plurality of sensitive feature maps.

[0085] In embodiments of the present disclosure, for the image i, the global feature descriptor of the image i can be obtained by global average pooling, which can be realized by the following public formula:

[0086] where H L and W L denote the height and width of the Lth sensitive feature map, respectively. is a multi-scale sensitive feature map, which is derived from the above.

[0087] Further, the global feature descriptor is utilized to compute the initial sensitivity score of image i which can be realized by the following public:

[0088] where, and are learnable parameters, and the function is a nonlinear activation function. The function is:

[0089] where, The function can map the real number field to the interval [0, 1], producing an output value similar to a probability.

[0090] In some embodiments, the spatial attention weight of a sensitive feature map can be determined by calculating an adaptive threshold . By determining the adaptive threshold to capture the global statistical features of image i. Wherein the adaptive threshold can be determined by the following formula:

[0091] where, is a learnable parameter.

[0092] Further, the spatial attention weight can be determined by the following formula:

[0093]

[0094] where, , and are learnable convolution kernel parameters; is a convolution operation; is the order of fractional differentiation; is a Gabor filter bank; is a fractional differential operator.

[0095] S503. Based on the initial sensitivity score and multiple spatial attention weights, determine the sensitivity score of the image.

[0096] In some embodiments, the sensitivity score of image i It can integrate initial sensitivity Weighted determination of spatial attention across multiple sensitive feature maps:

[0097] in, This indicates element-wise multiplication.

[0098] In embodiments of this disclosure, such as Figure 6 As shown, S201 may include S601 and S602.

[0099] Figure 6 The flowchart illustrating another method for determining the sensitivity score of an object to be identified according to an embodiment of the present disclosure is shown in the illustration.

[0100] S601. In response to the object to be identified being a video, based on the video feature vector, determine the sensitivity scores of multiple images, wherein the video includes at least multiple images.

[0101] In some embodiments, when the object to be identified is a video, for each video i, a sensitivity score is determined for each video i. At this time, image feature vectors can be used first. Determine the sensitivity score of the j-th keyframe of the i-th video. That is, first determine the sensitivity score of the j-th image. Among them, the sensitivity score of the j-th image is determined. You can refer to this. Figure 5 The method.

[0102] S602. Aggregate the sensitivity scores of multiple images to determine the sensitivity score of the video.

[0103] In some embodiments, after determining the sensitivity score of the j-th image in the i-th video... In this case, the sensitivity scores of multiple images can be aggregated to determine the sensitivity score of the i-th video.

[0104] In some embodiments, the sensitivity scores of keyframes can be aggregated using the following formula:

[0105] in, Let be the fuzzy measure function for the keyframe, which satisfies the conditions of monotonicity and continuity. It can be generated by fuzzy density function and fuzzy measure parameters The calculation yielded:

[0106] in, Indicate the importance of the j-th keyframe; This is a fuzzy metric parameter used to control the interaction between keyframes.

[0107] Therefore, the sensitivity score of the i-th video can be determined using the above formula. .

[0108] According to embodiments of this disclosure, sensitive content in data is adaptively identified and quantified by dynamically learning data features, thereby enabling real-time updates to identification rules and overcoming the limitations of static rule-based methods. By introducing attention mechanisms such as higher-order mutual information, semantic features of the data can be automatically extracted to capture implicit sensitive information. Furthermore, by setting sensitivity scores, the sensitivity of the data can be assessed in a fine-grained manner, providing more precise guidance for subsequent privacy protection.

[0109] S202. Based on the sensitivity score of the object to be identified, determine whether the object to be identified is sensitive content, so as to process the object to be identified based on the preset strategy.

[0110] In the embodiments of this disclosure, when a sensitivity score is determined for an object to be identified, it can be used to determine whether the object is sensitive content. Determining whether an object is sensitive content based on its sensitivity score can be done by comparing the sensitivity score with a sensitivity threshold. For example, if the sensitivity score is greater than the sensitivity threshold, the object can be determined to be sensitive content. Conversely, if the sensitivity score is less than the sensitivity threshold, the object can be determined not to be sensitive content.

[0111] In embodiments of this disclosure, the sensitivity threshold can be preset or dynamically changed. In some embodiments, the sensitivity threshold can be dynamically changed. By setting a dynamically changing sensitivity threshold, the sensitivity threshold can be updated in a timely manner when the characteristics of sensitive data change over time, thereby ensuring the recognition effect.

[0112] According to embodiments of this disclosure, a sensitive content identification method is provided. This method determines a sensitivity score for the object to be identified using its feature data. Based on the sensitivity score, it determines whether the object is sensitive content. This avoids directly determining whether an object is sensitive content based on a "match" or "non-match," avoiding a "one-size-fits-all" approach. Furthermore, by setting a sensitivity score, the degree of sensitivity of the object can be determined, improving the accuracy of identification with a more granular approach.

[0113] Figure 7 The flowchart illustrating the determination of whether an object to be identified is sensitive content according to an embodiment of the present disclosure is schematically shown.

[0114] In embodiments of this disclosure, such as Figure 7 As shown, the above S202 may include S701, S702 and S703: S701. Based on the sensitivity scores of multiple objects to be identified, determine the mean of the sensitivity scores and the standard deviation of the sensitivity scores.

[0115] In the embodiments of this disclosure, the sensitivity scores of all objects to be identified included in the current batch can be determined. For example, if a cloud drive contains 10,000 text files, and 100 text files are processed at a time, then the current batch can include 100 text files. Further, the sensitivity scores of these 100 text files can be determined. It is understood that a text file can include multiple text segments, meaning the sensitivity scores of multiple text segments can be determined.

[0116] In embodiments of this disclosure, the mean sensitivity score and standard deviation of the sensitivity score for the current batch can be determined by using the sensitivity scores of multiple objects to be identified.

[0117] In some embodiments, the current batch may include multiple text files, images, and / or videos. In some cases, the mean and standard deviation of the sensitivity scores can be determined based on the multiple text files included in the current batch. Alternatively, the mean and standard deviation of the sensitivity scores can be determined based on the multiple images included in the current batch. Alternatively, the mean and standard deviation of the sensitivity scores can be determined based on the multiple videos included in the current batch.

[0118] For example, to calculate the mean sensitivity score and the standard deviation of the sensitivity score for multiple images included in the current batch, the mean sensitivity score is... It can be determined using the following formula:

[0119] Standard deviation of sensitivity score It can be determined using the following formula:

[0120] in, N t The number of images included in the current batch; Sensitivity score for the images included in the current batch.

[0121] S702. Determine the anomaly detection threshold based on the mean and standard deviation of the sensitivity scores.

[0122] In embodiments of this disclosure, an initial anomaly detection threshold may also be determined. For example, when calculating the mean sensitivity score and standard deviation of the sensitivity scores corresponding to the multiple images included in the current batch, the mean sensitivity score of the images included in the current batch may be used as the initial anomaly detection threshold.

[0123] Furthermore, the mean and standard deviation of the sensitivity scores are used to determine the anomaly detection threshold, which is then used to update the initial anomaly detection threshold. The anomaly detection threshold can be determined using the following formula:

[0124] in, It can be used to control the smoothness of anomaly detection threshold updates; It can be used to control the degree to which the anomaly detection threshold deviates from the mean.

[0125] S703. In response to the sensitivity score of the object to be identified being greater than the anomaly determination threshold, the object to be identified is determined to be sensitive content.

[0126] In some embodiments, when determining whether an image is sensitive content, the average sensitivity score of multiple images included in the current batch can be used as the initial anomaly detection threshold. Further, the initial anomaly detection threshold is updated using the average sensitivity score and the standard deviation of the sensitivity scores determined from the multiple images included in the current batch, resulting in the anomaly detection threshold. Furthermore, the anomaly detection threshold will be... The sensitivity score is compared with that of each image in the current batch. If an image's sensitivity score is greater than an anomaly detection threshold, the image is determined to be sensitive content. For example, whether an image is sensitive content can be determined using the following formula:

[0127] This indicates that the i-th image is identified as sensitive content. This indicates that the i-th image is normal.

[0128] In some embodiments, when determining whether a text fragment is sensitive content, the above formula can be used to calculate the mean and standard deviation of the sensitivity scores corresponding to multiple text fragments included in the current batch, and thereby determine the anomaly detection threshold for the text fragment. Furthermore, an anomaly detection threshold is set for text fragments. By assigning a sensitivity score to each text segment, it is determined whether each text segment is sensitive content. Similarly, when determining whether a video is sensitive content, an anomaly detection threshold can be determined for that video. Furthermore, anomaly detection thresholds are set for videos. Each video is assigned a sensitivity score to determine whether it is sensitive content.

[0129] In some embodiments, hyperparameters can be adjusted according to specific tasks and data characteristics. and The value of . Larger . This will make the anomaly detection threshold update smoother and more stable, while smaller... This allows the anomaly detection threshold to adapt more quickly to changes in data distribution. The strictness of anomaly detection was controlled, resulting in a larger... This will make the judgment criteria more lenient and smaller. This makes the judgment more stringent.

[0130] In some embodiments, for the object to be identified, if If so, it can be marked as sensitive content. After identifying sensitive content, it can be processed according to preset strategies (such as blocking, replacement, and warning).

[0131] According to embodiments of this disclosure, the anomaly detection threshold is updated in real time based on the sensitivity scores of multiple objects to be identified. This enables dynamic updating of the identification rules, avoiding the limitations of static rules and thus improving the accuracy of identification.

[0132] In some cases, after sensitive content is identified, it is often subject to manual review in the background. That is, content moderators with administrator privileges manually review uploaded or shared data. In this case, the administrator can view the user's original privacy data files, which can easily lead to the leakage of personal privacy and cause adverse effects. Based on this, the sensitive content identification method provided in the embodiments of this disclosure further includes: in response to determining that the object to be identified is sensitive content, encrypting the object to be identified so as to store the encrypted object to be identified.

[0133] After identifying sensitive content, it is necessary to protect its privacy. This ensures data confidentiality while preserving as much semantic information and statistical characteristics as possible to support subsequent analysis and data mining tasks. Furthermore, encrypting sensitive content can further protect the privacy of user data.

[0134] Figure 8 The flowchart illustrating the encryption of the object to be identified according to an embodiment of the present disclosure is shown in the illustration.

[0135] In embodiments of this disclosure, such as Figure 8 As shown, it may include S801, S802, S803, and S804: S801. In response to determining that the object to be identified is an object to be encrypted, determine the dataset to which the object to be identified belongs, wherein the dataset includes at least multiple text fragments, multiple images and / or multiple videos.

[0136] In the embodiments of this disclosure, after identifying sensitive content, it is often necessary to protect the privacy of the entire dataset. The aim is to preserve the semantic information and statistical characteristics of the data as much as possible while ensuring data confidentiality, in order to support subsequent analysis and mining tasks. For example, if a text fragment is determined to be sensitive content, the text file to which the text fragment belongs can be identified, and this text file can be the dataset.

[0137] S802. Based on the sensitivity scores of multiple text fragments, multiple images, and / or multiple videos included in the dataset, divide the multiple sensitivity scores into multiple sub-regions.

[0138] In embodiments of this disclosure, multiple data records in the dataset can be obtained. Sensitivity score Data Records It can be text fragments, images, or videos. Furthermore, it can be based on data records. Sensitivity score The distribution divides the data space into multiple sub-regions.

[0139] In some embodiments, it may be assumed that the dataset The sensitivity score set is Therefore, we can first sort the multiple sensitivity scores in the sensitivity score set. For example, sorting the sensitivity score set S in ascending order will yield the sorted sensitivity score set. ,in, For example, sort the sensitivity score set S in descending order.

[0140] Furthermore, it can be based on a predefined granularity. We obtain m-1 partition points, which can be determined by the following formula:

[0141] in, This indicates rounding down. This method of selecting the dividing point ensures that each sub-region contains approximately the same number of data records. This means dividing the total number of records n into m equal parts, and then taking the position of the kth part.

[0142] Furthermore, the data space can be divided into m sub-regions, R, based on multiple partitioning points. k It can be determined by the following formula:

[0143] in, , This partitioning method ensures that data records with similar sensitivity scores are assigned to the same sub-region.

[0144] S803. Based on multiple sub-regions, determine multiple semantic mapping data for each object to be encrypted.

[0145] Figure 9 The flowchart illustrating the determination of multiple semantic mapping data for each object to be encrypted according to an embodiment of the present disclosure is shown in the illustration.

[0146] In embodiments of this disclosure, such as Figure 9 As shown, the above S803 may include S901, S902 and S903: S901. Based on multiple sub-regions, the object to be encrypted is divided into multiple sub-objects to be encrypted.

[0147] In embodiments of this disclosure, the object to be encrypted can be divided into multiple sub-objects to be encrypted. For example, a data record can be... Divide into m parts, each part being x i,j Where j represents a data record. For the j-th part, then x i,j It can be a sub-object to be encrypted.

[0148] S902. Determine the encryption granularity for each sub-region.

[0149] In embodiments of this disclosure, each sub-region R can be... k Its encryption granularity can be calculated using the following formula. :

[0150] in, Subregion R kThe number of data records in the data; Represents data records Sensitivity score.

[0151] S903. Based on the sensitivity score of the sub-object to be encrypted and the encryption granularity of the sub-region, determine the semantic mapping data of each sub-object to be encrypted.

[0152] In embodiments of this disclosure, for each data record It can be determined by the sensitivity score of the sub-object to be encrypted, the encryption granularity of the sub-region, and the data record. Whether it is sensitive content, determine multiple semantic mapping data. .Can A data record Semantic mapping data of multiple sub-objects to be encrypted It can be determined using the following formula:

[0153] in, This indicates that the mean is 0 and the variance is 0. Gaussian noise; It is a hyperparameter for controlling noise intensity; It is the encryption granularity of the sub-region; It is a sensitivity score; y i It is determined by the content in S703; x i,j The sub-object to be encrypted.

[0154] In the embodiments of this disclosure, Gaussian noise takes into account not only the encryption granularity of sub-regions. Data recording was also taken into consideration. Self-sensitivity score This allows for more sensitive data recording. To obtain stronger privacy protection.

[0155] S804. Encrypt each semantic mapping data to encrypt the object to be encrypted.

[0156] In embodiments of this disclosure, for each semantically mapped data record Homomorphic encryption algorithms (such as Paillier encryption) can be used for encryption:

[0157] Where E is a homomorphic encryption function.

[0158] In embodiments of this disclosure, encrypted data can be recorded. The data is stored in the cloud for subsequent data recovery and verification, serving as evidence of sensitive files. When an authorized user needs to access the data, they can use the private key to decrypt the ciphertext. Decrypting the ciphertext using the private key can be represented as:

[0159] In the embodiments of this disclosure, after identifying sensitive content, the computing device can process it according to preset rules. For example, it can automatically block sensitive images in real time, replace sensitive text, or issue warnings to administrators, awaiting further manual review. Furthermore, the computing device can also collect feedback on operations and user evaluations, and continuously optimize algorithm performance and user experience based on this information. Simultaneously, the model is updated regularly to adapt to new data patterns and changes in the social environment.

[0160] According to embodiments of this disclosure, an adaptive partitioned encryption strategy can dynamically adjust the encryption granularity based on the data's sensitivity score distribution. This maximizes data usability while ensuring privacy and security. Furthermore, through semantic preservation mapping technology, noise related to the original data features is injected during encryption to ensure that the encrypted data retains a certain semantic structure, facilitating subsequent analysis and mining. Thus, adaptive adjustment of data encryption granularity and preservation of semantic structure can be achieved, minimizing the impact on data usability while providing strong privacy protection.

[0161] Figure 10 A block diagram of a network device according to an embodiment of the present disclosure is schematically illustrated; like Figure 10 As shown, the network device 1000 of this embodiment includes a memory 1001 and a processor 1002.

[0162] The memory 1001 is used to store computer-readable instructions. The processor 1002 is used to execute the computer-readable instructions, causing the network device to perform the sensitive content identification method.

[0163] Figure 11 A block diagram illustrating a computer program product according to an embodiment of the present disclosure is shown schematically. like Figure 11 As shown, a computer program product 1100 according to an embodiment of this disclosure includes a computer program 1101, which, when executed by a processor, implements the sensitive content identification method as described above.

[0164] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0165] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0166] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0167] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0168] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described above can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0169] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0170] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for identifying sensitive content, characterized in that, The method includes: Based on the feature data of the object to be identified, a sensitivity score for the object to be identified is determined; and Based on the sensitivity score of the object to be identified, it is determined whether the object to be identified is sensitive content, so as to process the object to be identified based on a preset strategy.

2. The sensitive content identification method according to claim 1, characterized in that, Also includes: Extract the feature data of the object to be identified, wherein the feature data includes at least text feature vectors, image feature vectors, and / or video feature vectors; as well as The feature data is standardized in order to determine the sensitivity score of the object to be identified based on the standardized feature data.

3. The sensitive content identification method according to claim 2, characterized in that, The process of determining the sensitivity score of the object to be identified based on its feature data includes: In response to the fact that the object to be identified is a text fragment, the text feature vector is input into a bidirectional long short-term memory network to obtain hidden layer vectors of multiple words, and each text feature includes at least multiple words; Based on the hidden layer vector of each word, determine the sensitivity weight of each word; and The sensitivity score of the text segment is determined based on each of the sensitivity weights and each of the hidden layer vectors.

4. The sensitive content identification method according to claim 2, characterized in that, The process of determining the sensitivity score of the object to be identified based on its feature data includes: In response to the fact that the object to be identified is an image, multiple sensitive feature maps are obtained based on the image feature vector; Based on the multiple sensitive feature maps, an initial sensitivity score for the image and a spatial attention weight for each of the sensitive feature maps are determined; and The sensitivity score of the image is determined based on the initial sensitivity score and multiple spatial attention weights.

5. The sensitive content identification method according to claim 4, characterized in that, The process of obtaining multiple sensitive feature maps based on the image feature vector includes: Based on a pre-trained convolutional neural network, a sensitive feature extractor is determined; and Based on the sensitive feature extractor, multiple sensitive feature maps are obtained.

6. The sensitive content identification method according to claim 4, characterized in that, The process of determining the sensitivity score of the object to be identified based on its feature data includes: In response to the object to be identified being a video, sensitivity scores for multiple images are determined based on the video feature vector, wherein the video comprises at least multiple images; and The sensitivity scores of multiple images are aggregated to determine the sensitivity score of the video.

7. The sensitive content identification method according to any one of claims 3, 4, and 6, characterized in that, The step of determining whether the object to be identified is sensitive content based on the sensitivity score of the object to be identified includes: Based on the sensitivity scores of multiple objects to be identified, the mean sensitivity score and the standard deviation of the sensitivity score are determined; Based on the mean of the sensitivity scores and the standard deviation of the sensitivity scores, an anomaly detection threshold is determined; and If the sensitivity score of the object to be identified is greater than the anomaly determination threshold, the object to be identified is determined to be sensitive content.

8. The sensitive content identification method according to claim 7, characterized in that, Also includes: In response to determining that the object to be identified is sensitive content, the object to be identified is encrypted so that the encrypted object to be identified can be stored.

9. The sensitive content identification method according to claim 8, characterized in that, The encryption of the object to be identified includes: In response to determining that the object to be identified is an object to be encrypted, the dataset to which the object to be identified belongs is determined, the dataset including at least multiple text fragments, multiple images and / or multiple videos; Based on the sensitivity scores of multiple text segments, multiple images, and / or multiple videos included in the dataset, the multiple sensitivity scores are divided into multiple sub-regions; Based on the multiple sub-regions, determine multiple semantic mapping data for each of the objects to be encrypted; and Each of the semantic mapping data is encrypted to encrypt the object to be encrypted.

10. The sensitive content identification method according to claim 9, characterized in that, The step of determining multiple semantic mapping data for each of the objects to be encrypted based on multiple sub-regions includes: Based on the multiple sub-regions, the object to be encrypted is divided into multiple sub-objects to be encrypted; Determine the encryption granularity for each of the sub-regions; and Based on the sensitivity score of the sub-object to be encrypted and the encryption granularity of the sub-region, the semantic mapping data of each sub-object to be encrypted is determined.

Images