Sensitive information feature extraction method, device, electronic device and medium

Abstract

The embodiments of the present disclosure disclose a method, device, electronic device and medium for extracting features of sensitive information. A specific implementation of the method includes: performing information preprocessing on each user's sensitive information to generate preprocessed user information and obtain a preprocessed user information set, and performing information preprocessing on each scene description information to generate preprocessed scene information and obtain a preprocessed scene information set; extracting feature information of each preprocessed user information to generate user feature information and obtain a user feature information set; extracting feature information of each preprocessed scene information to obtain a scene feature information set; and generating an information feature set based on the user feature information set and the scene feature information set. This implementation is related to artificial intelligence and can more accurately achieve feature extraction for user sensitive information and scene description information.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a method, device, electronic device, and medium for extracting features of sensitive information. Background Art

[0002] At present, in the era of big data, the leakage of user sensitive information will cause serious security incidents. Therefore, the protection of user's personal sensitive information has become an important concern of people today. The identification, classification and grading of sensitive information is the premise of personal information security protection. For downstream tasks such as identification, classification and grading of sensitive information, most existing technologies design different models for different downstream tasks. However, these downstream tasks based on sensitive information need to process multimodal sensitive information. For the feature extraction of user sensitive information, the method usually adopted is: directly input user sensitive information into the feature extraction model to generate user feature information.

[0003] However, the inventors have found that when the above method is used to extract features, the following technical problems often occur:

[0004] For the multimodal form of user sensitive information, the feature extraction model can only extract single-modal user feature information, and the extracted user feature information is not comprehensive enough, resulting in the extraction of user feature information being inaccurate and inefficient.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose sensitive information feature extraction methods, devices, electronic devices, computer-readable media and program products to solve the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a method for extracting features of sensitive information, comprising: performing information preprocessing on each user sensitive information in a sample data set to generate preprocessed user information and obtain a preprocessed user information set, and performing information preprocessing on each scene description information in a corresponding scene description information set to generate preprocessed scene information and obtain a preprocessed scene information set, wherein the above-mentioned sample data set includes user sensitive information in a multimodal form; extracting feature information of each preprocessed user information in the above-mentioned preprocessed user information set to generate user feature information and obtain a user feature information set; extracting feature information of each preprocessed scene information in the above-mentioned preprocessed scene information set based on the user feature information set to generate scene feature information and obtain a scene feature information set; generating an information feature set based on the above-mentioned user feature information set and the above-mentioned scene feature information set.

[0009] Optionally, the user sensitive information in the above-mentioned sample data set includes: user attribute information and user attribute content information; and the above-mentioned information preprocessing of each user sensitive information in the sample data set to generate preprocessed user information includes: information preprocessing of the user attribute information included in the above-mentioned user sensitive information to obtain preprocessed user attribute information, and information preprocessing of the user attribute content information included in the above-mentioned user sensitive information to obtain preprocessed attribute content information; based on the above-mentioned preprocessed user attribute information and the above-mentioned preprocessed attribute content information, generating preprocessed user information for the above-mentioned user sensitive information.

[0010] Optionally, the extracting characteristic information of each preprocessed user information in the preprocessed user information set to generate user characteristic information includes: extracting attribute content characteristic information corresponding to the preprocessed attribute content information included in the preprocessed user information as the user characteristic information.

[0011] Optionally, the above-mentioned extracting feature information of each preprocessed scene information in the above-mentioned preprocessed scene information set according to the user feature information set to generate scene feature information includes: determining the user feature information corresponding to the above-mentioned preprocessed scene information as the target user feature information; inputting the preprocessed user attribute information corresponding to the above-mentioned target user feature information and the above-mentioned preprocessed scene information into the first scene feature information generation model based on the attention mechanism to generate the first initial scene feature information; generating the scene feature information for the above-mentioned preprocessed scene information according to the above-mentioned first initial scene feature information.

[0012] Optionally, the above-mentioned scene feature information for the pre-processed scene information is generated based on the above-mentioned first initial scene feature information, including: combining the above-mentioned target user feature information and the above-mentioned pre-processed scene information to obtain first combination information; inputting the above-mentioned first combination information into the second scene feature information generation model to generate second initial scene feature information; combining the above-mentioned first initial scene feature information, the above-mentioned second initial scene feature information and the above-mentioned pre-processed scene information to obtain second combination information as the scene feature information for the above-mentioned pre-processed scene information.

[0013] Optionally, the above-mentioned extraction of attribute content feature information corresponding to the preprocessed attribute content information included in the above-mentioned preprocessed user information as user feature information includes: in response to determining that the modal form of the user sensitive information corresponding to the above-mentioned preprocessed user information is in text form, inputting the above-mentioned preprocessed user information into a bidirectional temporal neural network model to generate hidden layer output information for each moment; using the first attention mechanism model, performing weighted summation processing on the obtained various hidden layer output information to generate attribute content feature information as user feature information.

[0014] Optionally, the above-mentioned extraction of attribute content feature information corresponding to the preprocessed attribute content information included in the above-mentioned preprocessed user information as user feature information includes: in response to determining that the modal form of the user sensitive information corresponding to the above-mentioned preprocessed user information is in image form, performing multi-dimensional convolution on the above-mentioned preprocessed user information to obtain multiple convolution results; splicing each of the above-mentioned multiple convolution results to obtain a spliced ​​result; and inputting the above-mentioned splicing result into an image feature information generation model to generate the attribute content feature information corresponding to the above-mentioned preprocessed attribute content information as user feature information.

[0015] Optionally, the above-mentioned extraction of attribute content feature information corresponding to the preprocessed attribute content information included in the above-mentioned preprocessed user information as user feature information includes: in response to determining that the modal form of the user sensitive information corresponding to the above-mentioned preprocessed user information is in video form, for each frame image in the frame image sequence corresponding to the above-mentioned preprocessed user information, executing the following processing steps: performing multi-dimensional convolution on the above-mentioned image to obtain multiple convolution results; splicing each of the above-mentioned multiple convolution results to obtain a spliced ​​result; inputting the above-mentioned splicing result into an image feature information generation model to generate the attribute content feature information corresponding to the above-mentioned preprocessed attribute content information as target attribute content feature information; inputting the obtained target attribute content feature information sequence into a temporal neural network model to output a hidden layer output result set at each moment; using the first attention mechanism model, performing weighted sum processing on each hidden layer output information in the above-mentioned hidden layer output result set to generate attribute content feature information as user feature information.

[0016] Optionally, the above-mentioned extraction of attribute content feature information corresponding to the preprocessed attribute content information included in the above-mentioned preprocessed user information as user feature information includes: in response to determining that the modal form of the user sensitive information corresponding to the above-mentioned preprocessed user information is in audio form, using an audio feature information generation model to generate attribute content feature information for the above-mentioned preprocessed user information as user feature information.

[0017] Optionally, the method further includes: inputting the information feature set into a classification model corresponding to the target classification task to generate a classification result.

[0018] In a second aspect, some embodiments of the present disclosure provide a sensitive information feature extraction device, comprising: a preprocessing unit, configured to perform information preprocessing on each user sensitive information in a sample data set to generate preprocessed user information and obtain a preprocessed user information set, and to perform information preprocessing on each scene description information in a corresponding scene description information set to generate preprocessed scene information and obtain a preprocessed scene information set, wherein the above-mentioned sample data set includes user sensitive information in a multimodal form; a first extraction unit, configured to extract feature information of each preprocessed user information in the above-mentioned preprocessed user information set to generate user feature information and obtain a user feature information set; a second extraction unit, configured to extract feature information of each preprocessed scene information in the above-mentioned preprocessed scene information set according to the user feature information set to generate scene feature information and obtain a scene feature information set; a generation unit, configured to generate an information feature set based on the above-mentioned user feature information set and the above-mentioned scene feature information set.

[0019] Optionally, the user sensitive information in the above-mentioned sample data set includes: user attribute information and user attribute content information; and the preprocessing unit can be configured to: perform information preprocessing on the user attribute information included in the above-mentioned user sensitive information to obtain preprocessed user attribute information, and perform information preprocessing on the user attribute content information included in the above-mentioned user sensitive information to obtain preprocessed attribute content information; generate preprocessed user information for the above-mentioned user sensitive information based on the above-mentioned preprocessed user attribute information and the above-mentioned preprocessed attribute content information.

[0020] Optionally, the first extraction unit may be configured to extract attribute content feature information corresponding to the pre-processed attribute content information included in the pre-processed user information as user feature information.

[0021] Optionally, the second extraction unit can be configured to: determine user feature information corresponding to the above-mentioned preprocessed scene information as target user feature information; input the preprocessed user attribute information corresponding to the above-mentioned target user feature information and the above-mentioned preprocessed scene information into the first scene feature information generation model based on the attention mechanism to generate first initial scene feature information; and generate scene feature information for the above-mentioned preprocessed scene information based on the above-mentioned first initial scene feature information.

[0022] Optionally, the second extraction unit can be configured to: combine the above-mentioned target user feature information and the above-mentioned pre-processed scene information to obtain first combination information; input the above-mentioned first combination information into the second scene feature information generation model to generate second initial scene feature information; combine the above-mentioned first initial scene feature information, the above-mentioned second initial scene feature information and the above-mentioned pre-processed scene information to obtain second combination information as scene feature information for the above-mentioned pre-processed scene information.

[0023] Optionally, the first extraction unit can be configured to: in response to determining that the modal form of the user sensitive information corresponding to the above-mentioned preprocessed user information is in text form, input the above-mentioned preprocessed user information into a bidirectional temporal neural network model to generate hidden layer output information for each moment; use the first attention mechanism model to perform weighted sum processing on the obtained hidden layer output information to generate attribute content feature information as user feature information.

[0024] Optionally, the first extraction unit can be configured to: in response to determining that the modal form of the user sensitive information corresponding to the above-mentioned preprocessed user information is in image form, perform multi-dimensional convolution on the above-mentioned preprocessed user information to obtain multiple convolution results; splice each of the above-mentioned multiple convolution results to obtain a spliced ​​result; input the above-mentioned splicing result into the image feature information generation model to generate attribute content feature information corresponding to the above-mentioned preprocessed attribute content information as user feature information.

[0025] Optionally, the first extraction unit can be configured to: in response to determining that the modal form of the user sensitive information corresponding to the above-mentioned pre-processed user information is in video form, for each frame image in the frame image sequence corresponding to the above-mentioned pre-processed user information, perform the following processing steps: perform multi-dimensional convolution on the above-mentioned image to obtain multiple convolution results; splice each of the above-mentioned multiple convolution results to obtain a spliced ​​result; input the above-mentioned splicing result into an image feature information generation model to generate attribute content feature information corresponding to the above-mentioned pre-processed attribute content information as target attribute content feature information; input the obtained target attribute content feature information sequence into a temporal neural network model to output a set of hidden layer output results at each moment; use the first attention mechanism model to perform weighted sum processing on each hidden layer output information in the above-mentioned hidden layer output result set to generate attribute content feature information as user feature information.

[0026] Optionally, the first extraction unit can be configured to: in response to determining that the modal form of the user sensitive information corresponding to the above-mentioned preprocessed user information is in audio form, generate attribute content feature information for the above-mentioned preprocessed user information as user feature information using an audio feature information generation model.

[0027] Optionally, the device also includes: inputting the above-mentioned information feature set into a classification model corresponding to the target classification task to generate a classification result.

[0028] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0029] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.

[0030] In a fifth aspect, some embodiments of the present disclosure provide a computer program product, including a computer program, which implements the method described in any implementation manner in the above-mentioned first aspect when executed by a processor.

[0031] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the sensitive information feature extraction method of some embodiments of the present disclosure can more accurately realize the feature extraction of user sensitive information and scene description information. Specifically, the reason why the relevant classification results are not accurate enough is that for the multimodal form of user sensitive information, the feature extraction model can only extract single-modal user feature information, and the extracted user feature information is not comprehensive enough, resulting in the extraction of user feature information not being accurate and efficient enough. Based on this, the sensitive information feature extraction method of some embodiments of the present disclosure first performs information preprocessing on each user sensitive information in the sample data set to generate preprocessed user information and obtain a preprocessed user information set, and performs information preprocessing on each scene description information in the corresponding scene description information set to generate preprocessed scene information and obtain a preprocessed scene information set. Among them, the above-mentioned sample data set includes user sensitive information in multimodal form. Here, by preprocessing the user sensitive information and scene description information, the feature extraction of subsequent information is facilitated. Then, the feature information of each preprocessed user information in the above-mentioned preprocessed user information set can be accurately extracted to generate user feature information and obtain a user feature information set. Here, the extracted user feature information set facilitates the generation of subsequent information feature sets. Furthermore, based on the user feature information set, the feature information of each preprocessed scene information in the preprocessed scene information set can be accurately extracted to generate scene feature information and obtain a scene feature information set. Here, the obtained scene feature information set is used to generate a subsequent training sample set. Finally, based on the user feature information set and the scene feature information set, an information feature set is generated. Here, by extracting the feature information set of the scene information set corresponding to the sample data set (i.e., the scene feature information set), an information feature set with richer feature information can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0033] Figure 1-Figure 2 is a schematic diagram of an application scenario of a sensitive information feature extraction method according to some embodiments of the present disclosure;

[0034] Figure 3 is a flow chart of some embodiments of the sensitive information feature extraction method according to the present disclosure;

[0035] Figure 4 is a flow chart of other embodiments of the sensitive information feature extraction method according to the present disclosure;

[0036] Figure 5 is a schematic diagram of the structure of some embodiments of the sensitive information feature extraction device according to the present disclosure;

[0037] Figure 6 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0038] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0039] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0040] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0041] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0042] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0043] With regard to the collection, storage, and use of user information (such as sensitive user information, scenario description information, and personal privacy information) involved in this disclosure, before performing the corresponding operations, the relevant organizations or individuals shall fulfill their obligations, including conducting personal information security impact assessments, fulfilling the obligation to inform the personal information subject, and obtaining the authorization and consent of the personal information subject in advance.

[0044] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0045] Figure 1-Figure 2 It is a schematic diagram of an application scenario of a sensitive information feature extraction method according to some embodiments of the present disclosure.

[0046] exist Figure 1-Figure 2In the application scenario, first, the electronic device 101 can perform information preprocessing on each user sensitive information in the sample data set 102 to generate preprocessed user information, obtain preprocessed user information set 104, and perform information preprocessing on each scene description information in the corresponding scene description information set 103 to generate preprocessed scene information, and obtain preprocessed scene information set 105. Among them, the sample data set 102 includes user sensitive information in multimodal form. In this application scenario, the sample data set 102 may include: first user sensitive information 1021, second user sensitive information 1022 and third user sensitive information 1023. Specifically, the modal form corresponding to the first user sensitive information 1021 is image form. The modal form corresponding to the second user sensitive information 1022 is audio form. The modal form corresponding to the third user sensitive information 1023 is video form. The pre-processed user information set 104 may include: first pre-processed user information 1041 corresponding to the first user sensitive information 1021, second pre-processed user information 1042 corresponding to the second user sensitive information 1022, and third pre-processed user information 1043 corresponding to the third user sensitive information 1023. The scene description information set 103 may include: first scene description information 1031, second scene description information 1032, and third scene description information 1033. The pre-processed scene information set 105 may include: first pre-processed scene information 1051 corresponding to the first scene description information 1031, second pre-processed scene information 1052 corresponding to the second scene description information 1032, and third pre-processed scene information 1053 corresponding to the third scene description information 1033. Then, the electronic device 101 may extract feature information of each pre-processed user information in the pre-processed user information set 104 to generate user feature information and obtain the user feature information set 106. In this application scenario, the user feature information set 106 may include: first user feature information 1061 corresponding to the first user sensitive information 1021, second user feature information 1062 corresponding to the second user sensitive information 1022, and third user feature information 1063 corresponding to the third user sensitive information 1023. Next, the electronic device 101 may extract feature information of each pre-processed scene information in the pre-processed scene information set 105 according to the user feature information set 106 to generate scene feature information and obtain the scene feature information set 107. In this application scenario, the scene feature information set 107 may include: first scene feature information 1071 corresponding to the first scene description information 1031, second scene feature information 1072 corresponding to the second scene description information 1032, and third scene feature information 1073 corresponding to the third scene description information 1033. Finally, the electronic device 101 may generate an information feature set 108 according to the user feature information set 106 and the scene feature information set 107.In this application scenario, the information feature set 108 includes: a first information feature 1081 for the first scene feature information 1071 and the first user feature information 1061, a second information feature 1082 for the second scene feature information 1072 and the second user feature information 1062, and a third information feature 1083 for the third scene feature information 1073 and the third user feature information 1063.

[0047] It should be noted that the electronic device 101 can be hardware or software. When the electronic device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or it can be implemented as a single server or a single terminal device. When the electronic device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here.

[0048] It should be understood that Figure 1-Figure 2 The number of electronic devices in the embodiment is only for illustration. Any number of electronic devices may be provided according to implementation requirements.

[0049] Continue to refer Figure 3 , shows a process 300 of some embodiments of the sensitive information feature extraction method according to the present disclosure. The sensitive information feature extraction method comprises the following steps:

[0050] Step 301, preprocessing each user sensitive information in the sample data set to generate preprocessed user information and obtain a preprocessed user information set, and preprocessing each scene description information in the corresponding scene description information set to generate preprocessed scene information and obtain a preprocessed scene information set.

[0051] In some embodiments, the execution entity of the above sensitive information feature extraction method (for example Figure 1The electronic device 101 shown can perform information preprocessing on each user sensitive information in the sample data set to generate preprocessed user information and obtain a preprocessed user information set, and perform information preprocessing on each scene description information in the corresponding scene description information set to generate preprocessed scene information and obtain a preprocessed scene information set. Among them, the above-mentioned sample data set includes user sensitive information in multimodal form. That is, each user sensitive information in the sample data set can be user sensitive information of various modalities. For example, each modality can include: image modality, audio modality, text modality and video modality. User sensitive information can be various aspects of user information. In practice, the above-mentioned user sensitive information can include: user's personal privacy information. For example, personal privacy information can be user taxi data. There is a one-to-one correspondence between the scene description information in the above-mentioned scene description information set and the user sensitive information in the sample data set. The scene description information can be scene information that describes the scene corresponding to the user sensitive information. Among them, the scene description information can be information in text form.

[0052] As an example, for privacy data in which user sensitive information is in text mode, the above-mentioned execution subject can use a word segmentation tool to segment the user sensitive information to obtain a word set. Then, the word embedding tool Word2Vec is used to encode the words in the word set to obtain an encoding vector as the preprocessed user information. For privacy data in which user sensitive information is in image mode, the above-mentioned execution subject can perform normalization on the user sensitive information to obtain a normalized result. Then, the size corresponding to the normalized result is re-adjusted, and the result is cropped to obtain a cropped result as the preprocessed user information.

[0053] As another example, the execution subject may segment the scene description information to obtain a scene word set. Then, the scene words in the scene word set are encoded by the word embedding tool Word2Vec to obtain a scene encoding vector set. Finally, the scene encoding vector sets are combined to obtain combined information as preprocessed scene information.

[0054] In some optional implementations of some embodiments, the user sensitive information in the sample data set includes: user attribute information and user attribute content information. The user attribute information may be the attribute names involved in the user sensitive information. The user attribute content information may be the attribute content corresponding to the attribute names in the user sensitive information.

[0055] Optionally, the above-mentioned preprocessing of each user sensitive information in the sample data set to generate preprocessed user information may include the following steps:

[0056] In the first step, the execution subject may perform information preprocessing on the user attribute information included in the user sensitive information to obtain preprocessed user attribute information, and perform information preprocessing on the user attribute content information included in the user sensitive information to obtain preprocessed attribute content information.

[0057] Specifically, for privacy data whose user attribute information is in text mode, the above-mentioned execution subject can use the word segmentation tool to segment the user attribute information to obtain a word set. Then, the word embedding tool Word2Vec is used to encode the words in the word set to obtain the encoding vector as the preprocessing user attribute information. Similarly, for privacy data whose user attribute content information is in text mode, the user attribute content information can also be preprocessed in this way, and the details will not be repeated.

[0058] In a second step, the execution subject may generate pre-processed user information for the user sensitive information based on the pre-processed user attribute information and the pre-processed attribute content information.

[0059] As an example, the execution subject may fuse the pre-processed user attribute information and the pre-processed attribute content information to generate fused information as the pre-processed user information for the user sensitive information.

[0060] Step 302: extract characteristic information of each pre-processed user information in the pre-processed user information set to generate user characteristic information and obtain a user characteristic information set.

[0061] In some embodiments, the execution subject may extract characteristic information of each pre-processed user information in the pre-processed user information set to generate user characteristic information and obtain a user characteristic information set. The user characteristic information may represent characteristic information corresponding to user sensitive information, that is, reflect characteristic content corresponding to user sensitive information.

[0062] As an example, for privacy data whose user sensitive information is in text mode, the above-mentioned execution subject can input the corresponding pre-processed user information into the Bert (Bidirectional Encoder Representations from Transformers) encoding model to output user feature information. For privacy data whose user sensitive information is in image mode, the above-mentioned execution subject can input the corresponding pre-processed user information into a multi-layer serially connected convolutional neural network model to generate user feature information.

[0063] In some optional implementations of some embodiments, the execution subject may extract attribute content feature information corresponding to the pre-processed attribute content information included in the pre-processed user information as user feature information.

[0064] Optionally, the extracting of attribute content feature information corresponding to the pre-processed attribute content information included in the pre-processed user information as user feature information may include the following steps:

[0065] In the first step, in response to determining that the modal form of the user sensitive information corresponding to the pre-processed user information is a video form, for each frame image in the frame image sequence corresponding to the pre-processed user information, the following processing steps are performed:

[0066] Sub-step 1: Perform multi-dimensional convolution on the above image to obtain multiple convolution results.

[0067] As an example, the above execution entity can use multiple convolution kernels of different dimensions to perform multi-dimensional convolution on the image to generate multiple convolution results.

[0068] Sub-step 2: splicing each of the multiple convolution results to obtain a spliced ​​result.

[0069] Sub-step 3, inputting the above stitching result into the image feature information generation model to generate the attribute content feature information corresponding to the above pre-processed attribute content information as the target attribute content feature information. The image feature information generation model can be a neural network model for generating image feature information (i.e., image feature information in vector form). In practice, the image feature information generation model can be a fully connected (FC) model with multiple layers of serial connections.

[0070] The second step is to input the obtained target attribute content feature information sequence into the time series neural network model to output the hidden layer output result set at each moment. In practice, the time series neural network model can be a long short-term memory network (LSTM, Long Short-Term Memory) model. Each moment in each moment has a one-to-one correspondence with the hidden layer output result in the hidden layer output result set. The specific hidden layer output results are not repeated here.

[0071] In the third step, the first attention mechanism model is used to perform weighted sum processing on each hidden layer output information in the above hidden layer output result set to generate attribute content feature information as user feature information. The first attention mechanism model can be a neural network model based on the attention mechanism. In practice, the first attention mechanism model can be a multi-head attention mechanism model.

[0072] As an example, first, the execution subject can use the first attention mechanism model to generate weights for each hidden layer output information. Then, the MiG hidden layer output information in each hidden layer output information is multiplied by the corresponding weight to obtain each multiplication result. Finally, each multiplication result is added to obtain the addition result as the user feature information.

[0073] In some optional implementations of some embodiments, in response to determining that the modal form of the user sensitive information corresponding to the above-mentioned pre-processed user information is in audio form, the above-mentioned execution subject can use an audio feature information generation model to generate attribute content feature information for the above-mentioned pre-processed user information as user feature information. Among them, the audio feature information generation model can be a model for generating audio feature information. In practice, the modal form of the corresponding user sensitive information is in audio form, and the pre-processed user information can be a Mel-type spectrum. The above-mentioned audio feature information generation model can be a multi-layer residual network (Residual Network, ResNet).

[0074] Step 303: extracting feature information of each preprocessed scene information in the preprocessed scene information set according to the user feature information set to generate scene feature information and obtain a scene feature information set.

[0075] In some embodiments, the execution subject may extract feature information of each pre-processed scene information in the pre-processed scene information set according to the user feature information set to generate scene feature information and obtain a scene feature information set. The scene feature information may represent feature information of the scene content corresponding to the scene description information.

[0076] As an example, for each pre-processed scene information, first, the execution subject may concatenate the pre-processed scene information and the corresponding user feature information to obtain concatenated information, and then input the concatenated information into a recurrent neural network (RNN) to generate scene feature information.

[0077] In some optional implementations of some embodiments, the extracting, according to the user feature information set, feature information of each pre-processed scene information in the pre-processed scene information set to generate scene feature information may include the following steps:

[0078] In the first step, the execution subject may determine user feature information corresponding to the pre-processing scenario information as target user feature information.

[0079] In the second step, the execution subject may input the preprocessed user attribute information corresponding to the target user feature information and the preprocessed scene information into the first scene feature information generation model based on the attention mechanism to generate the first initial scene feature information. The first scene feature information generation model may be a model for generating scene feature information. In practice, the first scene feature information generation model based on the attention mechanism may be a multi-head attention mechanism neural network model.

[0080] As an example, the above-mentioned execution entity can use the word segmentation set corresponding to the pre-processed user attribute information as the value (value) in the first scene feature information generation model, and use the word segmentation set corresponding to the pre-processed scene information as the key (key) and query information (query) in the first scene feature information generation model to generate the first initial scene feature information.

[0081] The third step is to generate scene feature information for the pre-processed scene information based on the first initial scene feature information.

[0082] As an example, the execution entity may determine the first initial scene feature information as the scene feature information for the pre-processed scene information.

[0083] In some optional implementations of some embodiments, generating the scene feature information for the pre-processed scene information according to the first initial scene feature information may include the following steps:

[0084] In the first step, the execution entity may combine the target user feature information and the pre-processing scenario information to obtain first combination information.

[0085] In the second step, the execution subject may input the first combination information into a second scene feature information generation model to generate second initial scene feature information. The second scene feature information generation model may be a model for generating scene feature information. In practice, the second scene feature information generation model may be a fully connected network.

[0086] In a third step, the execution entity may combine the first initial scene feature information, the second initial scene feature information and the pre-processed scene information to obtain second combined information as scene feature information for the pre-processed scene information.

[0087] Step 304: Generate an information feature set based on the user feature information set and the scene feature information set.

[0088] In some embodiments, the execution subject may generate an information feature set according to the user feature information set and the scene feature information set, wherein the information features in the information feature set have corresponding user feature information and scene feature information.

[0089] As an example, the execution subject may perform corresponding information combination of user feature information in the user feature information set and scene feature information in the scene feature information set to obtain a feature combination information set as the information feature set.

[0090] In some optional implementations of some embodiments, the execution subject may input the information feature set into a classification model corresponding to a target classification task to generate a classification result. The target classification task may be a pre-set task for classification. In practice, the target classification task may be a classification task for personal sensitive information or a classification task for personal sensitive information. The model structure of the classification model is dynamically set according to the target classification task. Here, the specific training method is not described in detail.

[0091] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the sensitive information feature extraction method of some embodiments of the present disclosure can more accurately realize the feature extraction of user sensitive information and scene description information. Specifically, the reason why the relevant classification results are not accurate enough is that for the multimodal form of user sensitive information, the feature extraction model can only extract single-modal user feature information, and the extracted user feature information is not comprehensive enough, resulting in the extraction of user feature information not being accurate and efficient enough. Based on this, the sensitive information feature extraction method of some embodiments of the present disclosure first performs information preprocessing on each user sensitive information in the sample data set to generate preprocessed user information and obtain a preprocessed user information set, and performs information preprocessing on each scene description information in the corresponding scene description information set to generate preprocessed scene information and obtain a preprocessed scene information set. Among them, the above-mentioned sample data set includes user sensitive information in multimodal form. Here, by preprocessing the user sensitive information and scene description information, the feature extraction of subsequent information is facilitated. Then, the feature information of each preprocessed user information in the above-mentioned preprocessed user information set can be accurately extracted to generate user feature information and obtain a user feature information set. Here, the extracted user feature information set facilitates the generation of subsequent information feature sets. Furthermore, based on the user feature information set, the feature information of each preprocessed scene information in the preprocessed scene information set can be accurately extracted to generate scene feature information and obtain a scene feature information set. Here, the obtained scene feature information set is used to generate a subsequent training sample set. Finally, based on the user feature information set and the scene feature information set, an information feature set is generated. Here, by extracting the feature information set of the scene information set corresponding to the sample data set (i.e., the scene feature information set), an information feature set with richer feature information can be obtained.

[0092] Further references Figure 4 , shows a process 400 of another embodiment of the sensitive information feature extraction method according to the present disclosure. The sensitive information feature extraction method comprises the following steps:

[0093] Step 401, preprocessing each user sensitive information in the sample data set to generate preprocessed user information and obtain a preprocessed user information set, and preprocessing each scene description information in the corresponding scene description information set to generate preprocessed scene information and obtain a preprocessed scene information set.

[0094] Step 402, in response to determining that the modal form of the user sensitive information corresponding to the above-mentioned pre-processed user information is in text form, the above-mentioned pre-processed user information is input into a bidirectional temporal neural network model to generate hidden layer output information for each moment.

[0095] In some embodiments, in response to determining that the modal form of the user sensitive information corresponding to the pre-processed user information is in text form, the execution subject (eg Figure 1 The electronic device 101 shown in the figure can input the above pre-processed user information into the bidirectional temporal neural network model to generate hidden layer output information for each moment. In practice, the bidirectional temporal neural network model can be a bidirectional long short-term memory network (Bi-LSTM, Bi-directional-Long Short-Term Memory).

[0096] Step 403, using the first attention mechanism model, performs weighted sum processing on the obtained hidden layer output information to generate attribute content feature information as user feature information.

[0097] In some embodiments, the execution subject may use the first attention mechanism model to perform weighted summation processing on the obtained hidden layer output information to generate attribute content feature information as user feature information. The first attention mechanism model may be a Transformer model.

[0098] As an example, the execution subject may use the first attention mechanism model to determine the weight information corresponding to each hidden output information. Then, each hidden output information is multiplied by the corresponding weight information to generate a multiplication result and obtain a multiplication result set. Finally, each multiplication result in the multiplication result set is added to obtain attribute content feature information as user feature information.

[0099] Step 404, in response to determining that the modal form of the user sensitive information corresponding to the above-mentioned pre-processed user information is an image form, multi-dimensional convolution is performed on the above-mentioned pre-processed user information to obtain multiple convolution results.

[0100] In some embodiments, in response to determining that the modal form of the user sensitive information corresponding to the pre-processed user information is an image form, the execution subject may perform multi-dimensional convolution on the pre-processed user information to obtain multiple convolution results. The convolution results may represent the convolution feature information corresponding to the corresponding user sensitive information.

[0101] As an example, the execution entity may use a plurality of different convolution kernels to perform multi-dimensional convolution on the pre-processed user information to obtain a plurality of convolution results.

[0102] Step 405: splice each of the multiple convolution results to obtain a spliced ​​result.

[0103] In some embodiments, the execution entity may concatenate each of the multiple convolution results to obtain a concatenated result.

[0104] Step 406: input the stitching result into the image feature information generation model to generate attribute content feature information corresponding to the pre-processed attribute content information as user feature information.

[0105] In some embodiments, the execution subject may input the splicing result into an image feature information generation model to generate attribute content feature information corresponding to the preprocessed attribute content information as user feature information. The image feature information generation model may be a model for generating image feature information. In practice, the image feature information generation model may be a multi-layer serially connected fully connected layer.

[0106] Step 407: extracting feature information of each preprocessed scene information in the preprocessed scene information set according to the user feature information set to generate scene feature information and obtain a scene feature information set.

[0107] Step 408: Generate an information feature set based on the user feature information set and the scene feature information set.

[0108] In some embodiments, the specific implementation of steps 401, 407-408 and the technical effects thereof can be referred to in Figure 3 Steps 301, 303-304 in the corresponding embodiment are not described in detail here.

[0109] from Figure 4 It can be seen that Figure 2 Compared with the description of some corresponding embodiments, Figure 4 The process 400 of the sensitive information feature extraction method in some corresponding embodiments can accurately extract features of user sensitive information in text form and image form.

[0110] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a sensitive information feature extraction device. These device embodiments are similar to Figure 2 Corresponding to the method embodiments shown, the sensitive information feature extraction device can be specifically applied to various electronic devices.

[0111] like Figure 5As shown, a sensitive information feature extraction device 500 includes: a preprocessing unit 501, a first extraction unit 502, a second extraction unit 503 and a generation unit 504. The preprocessing unit 501 is configured to perform information preprocessing on each user sensitive information in the sample data set to generate preprocessed user information and obtain a preprocessed user information set, and to perform information preprocessing on each scene description information in the corresponding scene description information set to generate preprocessed scene information and obtain a preprocessed scene information set, wherein the sample data set includes user sensitive information in a multimodal form; the first extraction unit 502 is configured to extract feature information of each preprocessed user information in the preprocessed user information set to generate user feature information and obtain a user feature information set; the second extraction unit 503 is configured to extract feature information of each preprocessed scene information in the preprocessed scene information set according to the user feature information set to generate scene feature information and obtain a scene feature information set; the generation unit 504 is configured to generate an information feature set according to the user feature information set and the scene feature information set.

[0112] In some optional implementations of some embodiments, the user sensitive information in the above-mentioned sample data set includes: user attribute information and user attribute content information; and the preprocessing unit 501 can be further configured to: perform information preprocessing on the user attribute information included in the above-mentioned user sensitive information to obtain preprocessed user attribute information, and perform information preprocessing on the user attribute content information included in the above-mentioned user sensitive information to obtain preprocessed attribute content information; generate preprocessed user information for the above-mentioned user sensitive information based on the above-mentioned preprocessed user attribute information and the above-mentioned preprocessed attribute content information.

[0113] In some optional implementations of some embodiments, the first extraction unit 502 may be further configured to extract attribute content feature information corresponding to the pre-processed attribute content information included in the pre-processed user information as user feature information.

[0114] In some optional implementations of some embodiments, the second extraction unit 503 can be further configured to: determine user feature information corresponding to the above-mentioned preprocessed scene information as target user feature information; input the preprocessed user attribute information corresponding to the above-mentioned target user feature information and the above-mentioned preprocessed scene information into the first scene feature information generation model based on the attention mechanism to generate first initial scene feature information; and generate scene feature information for the above-mentioned preprocessed scene information based on the above-mentioned first initial scene feature information.

[0115] In some optional implementations of some embodiments, the second extraction unit 503 can be further configured to: combine the above-mentioned target user feature information and the above-mentioned pre-processed scene information to obtain first combination information; input the above-mentioned first combination information into the second scene feature information generation model to generate second initial scene feature information; combine the above-mentioned first initial scene feature information, the above-mentioned second initial scene feature information and the above-mentioned pre-processed scene information to obtain second combination information as scene feature information for the above-mentioned pre-processed scene information.

[0116] In some optional implementations of some embodiments, the first extraction unit 502 can be further configured to: in response to determining that the modal form of the user sensitive information corresponding to the above-mentioned preprocessed user information is in text form, input the above-mentioned preprocessed user information into a bidirectional temporal neural network model to generate hidden layer output information for each moment; use the first attention mechanism model to perform weighted summation processing on the obtained hidden layer output information to generate attribute content feature information as user feature information.

[0117] In some optional implementations of some embodiments, the first extraction unit 502 may be further configured to: in response to determining that the modal form of the user sensitive information corresponding to the preprocessed user information is in image form, perform multi-dimensional convolution on the preprocessed user information to obtain multiple convolution results; splice each of the multiple convolution results to obtain a spliced ​​result; and input the spliced ​​result into an image feature information generation model to generate attribute content feature information corresponding to the preprocessed attribute content information as user feature information.

[0118] In some optional implementations of some embodiments, the first extraction unit 502 can be further configured to: in response to determining that the modal form of the user sensitive information corresponding to the above-mentioned pre-processed user information is in video form, for each frame image in the frame image sequence corresponding to the above-mentioned pre-processed user information, perform the following processing steps: perform multi-dimensional convolution on the above-mentioned image to obtain multiple convolution results; splice each of the above-mentioned multiple convolution results to obtain a spliced ​​result; input the above-mentioned splicing result into an image feature information generation model to generate attribute content feature information corresponding to the above-mentioned pre-processed attribute content information as target attribute content feature information; input the obtained target attribute content feature information sequence into a temporal neural network model to output a set of hidden layer output results at each moment; use the first attention mechanism model to perform weighted sum processing on each hidden layer output information in the above-mentioned hidden layer output result set to generate attribute content feature information as user feature information.

[0119] In some optional implementations of some embodiments, the first extraction unit 502 can be further configured to: in response to determining that the modal form of the user sensitive information corresponding to the above-mentioned preprocessed user information is in audio form, generate an audio feature information generation model to generate attribute content feature information for the above-mentioned preprocessed user information as user feature information.

[0120] In some optional implementations of some embodiments, the sensitive information feature extraction device 500 further includes: an input unit. The input unit may be further configured to: input the above information feature set into a classification model corresponding to the target classification task to generate a classification result.

[0121] It is understandable that the units recorded in the sensitive information feature extraction device 500 are similar to those in the reference Figure 2 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the sensitive information feature extraction device 500 and the units contained therein, and will not be described in detail here.

[0122] Reference below Figure 6 , which shows an electronic device (eg, Figure 1 Schematic diagram of the structure of the electronic device 101)600. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0123] like Figure 6 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory 602 or a program loaded from a storage device 608 into a random access memory 603. Various programs and data required for the operation of the electronic device 600 are also stored in the random access memory 603. The processing device 601, the read-only memory 602, and the random access memory 603 are connected to each other via a bus 604. An input / output interface 605 is also connected to the bus 604.

[0124] Typically, the following devices may be connected to the input / output interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 6 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0125] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a read-only memory 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0126] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0127] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0128] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: performs information preprocessing on each user sensitive information in the sample data set to generate preprocessed user information and obtain a preprocessed user information set, and performs information preprocessing on each scene description information in the corresponding scene description information set to generate preprocessed scene information and obtain a preprocessed scene information set, wherein the sample data set includes user sensitive information in a multimodal form; extracts feature information of each preprocessed user information in the preprocessed user information set to generate user feature information and obtain a user feature information set; extracts feature information of each preprocessed scene information in the preprocessed scene information set according to the user feature information set to generate scene feature information and obtain a scene feature information set; generates an information feature set according to the user feature information set and the scene feature information set.

[0129] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0131] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be arranged in a processor, for example, may be described as: a processor includes a preprocessing unit, a first extraction unit, a second extraction unit and a generation unit. The names of these units do not constitute a limitation on the units themselves in certain cases, for example, the generation unit may also be described as "a unit for generating an information feature set based on the above-mentioned user feature information set and the above-mentioned scene feature information set".

[0132] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0133] Some embodiments of the present disclosure also provide a computer program product, including a computer program, which implements any of the above-mentioned sensitive information feature extraction methods when executed by a processor.

[0134] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A sensitive information feature extraction method, comprising: Performing information preprocessing on each user sensitive information in the sample data set to generate preprocessed user information to obtain a preprocessed user information set, and performing information preprocessing on each scene description information in the corresponding scene description information set to generate preprocessed scene information to obtain a preprocessed scene information set, wherein the sample data set includes user sensitive information in a multimodal form, and the user sensitive information in the sample data set includes: user attribute information and user attribute content information, and performing information preprocessing on each user sensitive information in the sample data set to generate preprocessed user information includes: performing information preprocessing on the user attribute information included in the user sensitive information to obtain preprocessed user attribute information, and performing information preprocessing on the user attribute content information included in the user sensitive information to obtain preprocessed attribute content information; generating preprocessed user information for the user sensitive information according to the preprocessed user attribute information and the preprocessed attribute content information; Extracting characteristic information of each preprocessed user information in the preprocessed user information set to generate user characteristic information, and obtaining the user characteristic information set, wherein the extracting characteristic information of each preprocessed user information in the preprocessed user information set to generate user characteristic information includes: extracting attribute content characteristic information corresponding to the preprocessed attribute content information included in the preprocessed user information as the user characteristic information; Extracting feature information of each preprocessed scene information in the preprocessed scene information set according to the user feature information set to generate scene feature information, thereby obtaining a scene feature information set; An information feature set is generated according to the user feature information set and the scene feature information set.

2. The method according to claim 1, wherein: The step of extracting feature information of each preprocessed scene information in the preprocessed scene information set according to the user feature information set to generate scene feature information includes: Determining user characteristic information corresponding to the pre-processed scene information as target user characteristic information; Inputting the preprocessed user attribute information corresponding to the target user feature information and the preprocessed scene information into a first scene feature information generation model based on an attention mechanism to generate first initial scene feature information; Scene feature information for the pre-processed scene information is generated according to the first initial scene feature information.

3. The method according to claim 2, wherein: The step of generating scene feature information for the pre-processed scene information according to the first initial scene feature information includes: Combining the target user feature information and the pre-processed scene information to obtain first combined information; Inputting the first combination information into a second scene feature information generation model to generate second initial scene feature information; The first initial scene feature information, the second initial scene feature information and the pre-processed scene information are combined to obtain second combined information as scene feature information for the pre-processed scene information.

4. The method according to claim 1, wherein: The extracting, as user characteristic information, attribute content feature information corresponding to the pre-processed attribute content information included in the pre-processed user information includes: In response to determining that the modal form of the user sensitive information corresponding to the pre-processed user information is in text form, inputting the pre-processed user information into a bidirectional temporal neural network model to generate hidden layer output information for each moment; The first attention mechanism model is used to perform weighted sum processing on the obtained hidden layer output information to generate attribute content feature information as user feature information.

5. The method according to claim 1, wherein: The extracting, as user characteristic information, attribute content feature information corresponding to the pre-processed attribute content information included in the pre-processed user information includes: In response to determining that the modal form of the user sensitive information corresponding to the pre-processed user information is an image form, performing multi-dimensional convolution on the pre-processed user information to obtain multiple convolution results; Splicing each of the multiple convolution results to obtain a spliced ​​result; The stitching result is input into an image feature information generation model to generate attribute content feature information corresponding to the pre-processed attribute content information as user feature information.

6. The method according to claim 1, wherein: The extracting, as user characteristic information, attribute content feature information corresponding to the pre-processed attribute content information included in the pre-processed user information includes: In response to determining that the modal form of the user sensitive information corresponding to the pre-processed user information is in video form, for each frame image in the frame image sequence corresponding to the pre-processed user information, the following processing steps are performed: Performing multi-dimensional convolution on the image to obtain multiple convolution results; Splicing each of the multiple convolution results to obtain a spliced ​​result; Inputting the stitching result into an image feature information generation model to generate attribute content feature information corresponding to the pre-processed attribute content information as target attribute content feature information; The obtained target attribute content feature information sequence is input into the time series neural network model to output the hidden layer output result set at each moment; The first attention mechanism model is used to perform weighted sum processing on each hidden layer output information in the hidden layer output result set to generate attribute content feature information as user feature information.

7. The method according to claim 1, wherein: The extracting, as user characteristic information, attribute content feature information corresponding to the pre-processed attribute content information included in the pre-processed user information includes: In response to determining that the modal form of the user sensitive information corresponding to the pre-processed user information is an audio form, an audio feature information generation model is used to generate attribute content feature information for the pre-processed user information as user feature information.

8. The method according to claim 1, wherein: The method further comprises: The information feature set is input into the classification model corresponding to the target classification task to generate a classification result.

9. A sensitive information feature extraction device, comprising: The preprocessing unit is configured to perform information preprocessing on each user sensitive information in the sample data set to generate preprocessed user information and obtain a preprocessed user information set, and to perform information preprocessing on each scene description information in the corresponding scene description information set to generate preprocessed scene information and obtain a preprocessed scene information set, wherein the sample data set includes user sensitive information in a multimodal form, and the user sensitive information in the sample data set includes: user attribute information and user attribute content information, and the information preprocessing on each user sensitive information in the sample data set to generate preprocessed user information includes: performing information preprocessing on the user attribute information included in the user sensitive information to obtain preprocessed user attribute information, and performing information preprocessing on the user attribute content information included in the user sensitive information to obtain preprocessed attribute content information; and generating preprocessed user information for the user sensitive information according to the preprocessed user attribute information and the preprocessed attribute content information; A first extraction unit is configured to extract feature information of each pre-processed user information in the pre-processed user information set to generate user feature information and obtain a user feature information set; The second extraction unit is configured to extract feature information of each pre-processed scene information in the pre-processed scene information set according to the user feature information set to generate scene feature information and obtain the scene feature information set, wherein the extracting feature information of each pre-processed user information in the pre-processed user information set to generate user feature information includes: extracting attribute content feature information corresponding to the pre-processed attribute content information included in the pre-processed user information as user feature information; The generating unit is configured to generate an information feature set according to the user feature information set and the scene feature information set.

10. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images