Method, device, equipment and storage medium for identifying sensitive information

Abstract

The present application discloses a method, device, equipment and storage medium for identifying sensitive information, which belongs to the field of Internet technology. The method comprises: obtaining comment information of multimedia data; inputting the comment information into a pre-trained sensitive information identification model to obtain a sensitive determination result, wherein the sensitive determination result is used to indicate whether the multimedia data corresponding to the comment information has sensitive information; obtaining the target comment information having sensitive information as a sensitive determination result and the target multimedia data to which the target comment information belongs, and sending the identification information of the target multimedia data and the target comment information to the review terminal. The present application can mine more multimedia data containing sensitive information by identifying the comment information of multimedia data.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a method, apparatus, device and storage medium for identifying sensitive information. Background Art

[0002] With the development of Internet technology, people can upload various multimedia data, that is, various works, such as songs, short videos, etc., on various network platforms. Some of these works may contain sensitive information, so general network platforms will set up a review process to identify various works on the platform. If sensitive information is found in a work, the corresponding work can be processed, for example, songs with sensitive information in the lyrics will be removed from the shelves.

[0003] The audit technology set up in the network platform generally directly obtains the content data of the work, for example, obtaining the lyrics text in the song, obtaining the video frames in the video, and then detects whether the corresponding content data contains sensitive information, for example, detecting whether the lyrics text in the song contains sensitive words, and then judging whether the work contains sensitive information based on the detection results.

[0004] In the process of implementing this application, the inventor found that the prior art has at least the following problems:

[0005] Existing auditing technologies can only detect works that contain sensitive words. For some works that do not contain sensitive words but still express sensitive information, existing auditing technologies cannot accurately identify them. Summary of the invention

[0006] The embodiments of the present application provide a method, apparatus, device and storage medium for identifying sensitive information, which can mine more works that may contain sensitive information by detecting the comment information corresponding to the multimedia data. The technical solution is as follows:

[0007] In one aspect, a method for identifying sensitive information is provided, the method comprising:

[0008] Obtain comment information of multimedia data;

[0009] Inputting the comment information into a pre-trained sensitive information recognition model to obtain a sensitivity determination result, wherein the sensitivity determination result is used to indicate whether the multimedia data corresponding to the comment information contains sensitive information;

[0010] The target comment information having sensitive information and the target multimedia data to which the target comment information belongs are obtained as a result of the sensitivity determination, and the identification information of the target multimedia data and the target comment information are sent to the review terminal.

[0011] Optionally, the pre-trained sensitive information identification model includes an input layer, an embedding layer, a bidirectional long short-term memory network layer, an attention layer and an output layer.

[0012] Optionally, inputting the comment information into a pre-trained sensitive information recognition model to obtain a sensitivity determination result includes:

[0013] Performing word segmentation on the comment information to obtain at least one phrase corresponding to the comment information;

[0014] Inputting the at least one phrase into the input layer to obtain a phrase identifier corresponding to the at least one phrase;

[0015] Inputting a phrase identifier corresponding to the at least one phrase into the embedding layer to obtain a phrase vector corresponding to the at least one phrase;

[0016] Inputting the phrase vector corresponding to the at least one phrase into the bidirectional long short-term memory network layer to obtain a feature vector corresponding to the at least one phrase;

[0017] Inputting the feature vector corresponding to the at least one phrase into the attention layer to obtain the feature vector corresponding to the comment information;

[0018] The feature vector corresponding to the comment information is input into the output layer to obtain the sensitivity determination result corresponding to the comment information.

[0019] Optionally, after obtaining the sensitivity determination result as target comment information having sensitive information and target multimedia data to which the target comment information belongs, the method further includes:

[0020] Determining the number of target comment information corresponding to each target multimedia data;

[0021] Based on the number corresponding to each target multimedia data, the target multimedia data are ranked in order of the number from most to least, to obtain ranking information of each target multimedia data;

[0022] The ranking information of each target multimedia data is sent to the audit terminal.

[0023] Optionally, the sending the identification information of the target multimedia data and the target comment information to the review terminal includes:

[0024] Determining the number of target comment information corresponding to each target multimedia data;

[0025] The identification information of the target multimedia data and the target comment information whose number of corresponding target comment information exceeds the number threshold are sent to the review terminal.

[0026] Optionally, obtaining the target comment information having sensitive information as a result of the sensitivity determination and the target multimedia data to which the target comment information belongs includes:

[0027] Whenever the preset cycle duration is reached, the corresponding sensitivity determination result within the current cycle duration is determined to be the target comment information having sensitive information and the target multimedia data to which the target comment information belongs.

[0028] Optionally, after sending the identification information of the target multimedia data and the target comment information to the review terminal, the method further includes:

[0029] Receiving a data acquisition request sent by the audit terminal, wherein the data acquisition request carries first identification information of the multimedia data to be queried;

[0030] The target multimedia data corresponding to the first identification information is sent to the audit terminal.

[0031] Optionally, the method further includes:

[0032] Receiving a multimedia data deletion notification sent by the audit terminal, wherein the processing notification carries second identification information of the multimedia data to be deleted;

[0033] Delete the target multimedia data corresponding to the second identification information.

[0034] On the other hand, a device for identifying sensitive works is provided, the device comprising:

[0035] An acquisition module, configured to acquire comment information of multimedia data;

[0036] a determination module configured to input the comment information into a pre-trained sensitive information recognition model to obtain a sensitivity determination result, wherein the sensitivity determination result is used to indicate whether the multimedia data corresponding to the comment information contains sensitive information;

[0037] The first sending module is configured to obtain target comment information with a sensitivity determination result of having sensitive information, and target multimedia data to which the target comment information belongs, and send identification information of the target multimedia data and the target comment information to the review terminal.

[0038] Optionally, the pre-trained sensitive information identification model includes an input layer, an embedding layer, a bidirectional long short-term memory network layer, an attention layer and an output layer.

[0039] Optionally, the determining module is configured to:

[0040] Performing word segmentation on the comment information to obtain at least one phrase corresponding to the comment information;

[0041] Inputting the at least one phrase into the input layer to obtain a phrase identifier corresponding to the at least one phrase;

[0042] Inputting a phrase identifier corresponding to the at least one phrase into the embedding layer to obtain a phrase vector corresponding to the at least one phrase;

[0043] Inputting the phrase vector corresponding to the at least one phrase into the bidirectional long short-term memory network layer to obtain a feature vector corresponding to the at least one phrase;

[0044] Inputting the feature vector corresponding to the at least one phrase into the attention layer to obtain the feature vector corresponding to the comment information;

[0045] The feature vector corresponding to the comment information is input into the output layer to obtain the sensitivity determination result corresponding to the comment information.

[0046] Optionally, the device further includes a second sending module, configured to:

[0047] Determining the number of target comment information corresponding to each target multimedia data;

[0048] Based on the number corresponding to each target multimedia data, the target multimedia data are ranked in order of the number from most to least, to obtain ranking information of each target multimedia data;

[0049] The ranking information of each target multimedia data is sent to the audit terminal.

[0050] Optionally, the first sending module is configured to:

[0051] Determining the number of target comment information corresponding to each target multimedia data;

[0052] The identification information of the target multimedia data and the target comment information whose number of corresponding target comment information exceeds the number threshold are sent to the review terminal.

[0053] Optionally, the acquisition module is configured as:

[0054] Whenever the preset cycle duration is reached, the corresponding sensitivity determination result within the current cycle duration is determined to be the target comment information having sensitive information and the target multimedia data to which the target comment information belongs.

[0055] Optionally, the device further includes a third sending module, configured to:

[0056] Receiving a data acquisition request sent by the audit terminal, wherein the data acquisition request carries first identification information of the multimedia data to be queried;

[0057] The target multimedia data corresponding to the first identification information is sent to the audit terminal.

[0058] Optionally, the device further includes a processing module configured to:

[0059] Receiving a multimedia data deletion notification sent by the audit terminal, wherein the processing notification carries second identification information of the multimedia data to be deleted;

[0060] Delete the target multimedia data corresponding to the second identification information.

[0061] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operations performed by the method for identifying sensitive information as described above.

[0062] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement the operations performed by the method for identifying sensitive information as described above.

[0063] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0064] By obtaining the comment information corresponding to the multimedia data and detecting whether the comment information is a sensitive comment, the sensitive comment and the corresponding multimedia data are sent to the review terminal to complete the review of the multimedia data. By adopting this application, more multimedia data that does not contain sensitive words but has sensitive information can be mined by detecting the comment information of the multimedia data. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0066] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0067] Figure 2 is a flow chart of a method for identifying sensitive information provided by an embodiment of the present application;

[0068] Figure 3 is a schematic diagram of the structure of a device for identifying sensitive information provided in an embodiment of the present application;

[0069] Figure 4 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0070] Figure 5 A schematic diagram of the structure of the sensitive information identification model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0072] Figure 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application. Figure 1 , the method provided in this application can be implemented by the audit terminal and the server. The audit terminal and the user terminal have communication functions and can access the Internet. The terminal can be a mobile phone, a tablet computer, a smart wearable device, a desktop computer, a laptop computer, etc. The server can establish communication with the terminal. The server can be a single server or a server group. If it is a single server, the server can be responsible for all the processing in the following scheme. If it is a server group, different servers in the server group can be responsible for different processing in the following scheme. The specific processing distribution can be arbitrarily set by the technical personnel according to actual needs, and will not be repeated here.

[0073] Users publish various multimedia data, i.e. various works, such as articles, songs, short videos, etc., in the corresponding network platform through various applications. Users in the network platform can browse the corresponding works, and can also leave messages and comments under the corresponding works. In addition, the network platform is provided with an audit link, which can audit the multimedia data uploaded by users, for example, whether there are sensitive words in the lyrics of the song. While some works may not contain obvious sensitive words, they may also contain sensitive information. Generally, there will be more extreme or sensitive comments in the comments corresponding to such works. Therefore, the method for identifying sensitive information provided by the present application can identify the comment information after the user operates the terminal to send the comment information to the server, and then send the work corresponding to the sensitive comment to the audit terminal, and the audit terminal will audit whether the work contains sensitive information. In the embodiment of the present application, taking the multimedia data as a song work as an example, the corresponding application is a music application, and the scheme is described in detail. Other situations are similar and will not be repeated.

[0074] Figure 2 is a flow chart of a method for identifying sensitive information provided by an embodiment of the present application. Figure 2 , the embodiment comprises:

[0075] Step 201: Obtain comment information of multimedia data.

[0076] In implementation, the user can select a song in the music player application and enter the song's playback interface to play the song. A comment option is set in the playback interface, and the user can click the comment option to enter the comment page to comment on the currently playing song. After the user completes the comment on the currently playing song, he can click the corresponding send option to send a comment request to the server, wherein the comment request carries the user's comment content, i.e., the comment information of the multimedia data, the song ID corresponding to the currently playing song, and the account ID of the user to log in to the music application. After the server receives the comment request sent by the user terminal, it can obtain the comment content carried in the comment request.

[0077] Step 202: Input the comment information into a pre-trained sensitive information recognition model to obtain a sensitivity determination result.

[0078] The sensitivity determination result is used to indicate whether the multimedia data corresponding to the comment information contains sensitive information.

[0079] In implementation, a pre-trained sensitive information identification model can be set in the server. After the server obtains the user's comment content, the obtained comment content can be input into the sensitive information identification model. The sensitive information identification model can distinguish the input comment content and output a sensitive information discrimination value, that is, a judgment result, wherein the sensitive information discrimination value can be a probability value that the comment content belongs to sensitive information. If it is a probability value, the technician can set a probability threshold. When the obtained sensitive information discrimination value exceeds the probability threshold, it can be determined that the corresponding comment content belongs to sensitive information. The sensitive information discrimination value can be binary information. For example, when the output comment content judgment result is that the comment is a sensitive comment, the sensitive information discrimination value can be 1. When the output comment content judgment result is not that the comment is a sensitive comment, the sensitive information discrimination value can be.

[0080] Among them, the pre-trained sensitive information identification model includes an input layer, an embedding layer, a bidirectional long short-term memory network layer, an attention layer and an output layer. The comment information is input into the pre-trained sensitive information identification model, and the processing to obtain the sensitive judgment result can be as follows: the comment information is segmented to obtain at least one phrase corresponding to the comment information; at least one phrase is input into the input layer to obtain a phrase identifier corresponding to at least one phrase; the phrase identifier corresponding to at least one phrase is input into the embedding layer to obtain a phrase vector corresponding to at least one phrase; the phrase vector corresponding to at least one phrase is input into the bidirectional long short-term memory network layer to obtain a feature vector corresponding to at least one phrase; the feature vector corresponding to at least one phrase is input into the attention layer to obtain a feature vector corresponding to the comment information; the feature vector corresponding to the comment information is input into the output layer to obtain a sensitive judgment result corresponding to the comment information.

[0081] In implementation, before inputting the comment content into the pre-trained sensitive information identification model, the comment content may be segmented to obtain at least one phrase corresponding to the comment content. For example, if the comment content is "this song sounds nice", the phrases after the segmentation process may be "this", "first", "song", "sounds", "nice". Then, each phrase after the segmentation process is input into the input layer of the pre-trained sensitive information identification model. In the input layer, the phrase identifier corresponding to each input phrase may be determined according to the preset correspondence between the phrase and the identifier, and then the phrase identifier corresponding to each phrase may be output, wherein each phrase may correspond to a unique phrase identifier. After obtaining the phrase identifier of each phrase, the corresponding phrase identifier may be input into the embedding layer. In the embedding layer of the pre-trained sensitive information identification model, the corresponding phrase vector may be determined according to each phrase identifier, and the dimension of the phrase vector may be preset by a technician. Then, the corresponding phrase vector may be input into the bidirectional long short-term memory network layer, and feature extraction may be performed on each phrase vector in the bidirectional long short-term memory network layer to obtain a feature vector corresponding to each phrase vector. Then, the feature vector corresponding to each phrase can be input into the attention layer, which outputs the feature vector corresponding to the comment content. Finally, the feature vector corresponding to the comment content can be input into the output layer, which outputs the sensitive information discrimination value corresponding to the comment content. Figure 5 As shown, Figure 5The figure is a schematic diagram of the structure of the sensitive information identification model, which includes an input layer, an embedding layer, a bidirectional long short-term memory network (Bi-LSTM) layer, an attention layer, and an output layer. X1-Xt is the phrase identifier corresponding to each phrase after the word segmentation of the comment content. After X1-Xt is input into the embedding layer, the phrase vector e1-et corresponding to each phrase identifier can be constructed in the embedding layer according to the word2vec algorithm. Then the obtained phrase vector e1-et can be input into the bidirectional long short-term memory network layer, and the feature vector of each phrase vector, i.e., h1-ht, can be obtained in the bidirectional long short-term memory network layer. Then h1-ht can be input into the attention layer, and the input h1-ht can be weighted and summed in the attention layer to obtain the feature vector corresponding to the comment content. Finally, the feature vector corresponding to the comment content can be input into the output layer to obtain the sensitive information discrimination value y of the feature vector corresponding to the comment content.

[0082] The training process of the sensitive information recognition model can be as follows:

[0083] Obtain sample comment data and the sensitive information discrimination value corresponding to the sample comment data. The sample comment data may be the comment content of the user's history, such as "fighting for freedom", "advocating human rights", "this author has incorrect values", etc., and then set the sensitive information discrimination value for the corresponding comment content. The sensitive information discrimination value may indicate whether the corresponding comment content is a sensitive comment, or whether the comment content can reflect that the song contains sensitive information. When training the sensitive information recognition model, the sample comment data may be input into the sensitive information recognition model training, and the sensitive information discrimination value of the sample comment data may be output. Then, the sensitive information recognition model may be trained based on the sensitive information discrimination value of the output sample comment data and the benchmark sensitive information discrimination value corresponding to the sample comment data. After the sensitive information recognition model is trained with a large amount of sample comment data and the sensitive information discrimination value corresponding to the sample comment data, a trained sensitive information recognition model may be obtained.

[0084] Step 203: Obtain target comment information with a sensitivity determination result of having sensitive information, and target multimedia data to which the target comment information belongs, and send identification information of the target multimedia data and the target comment information to the review terminal.

[0085] In implementation, a comment storage list may be set in the server to store the user's comment content, as shown in Table 1. Other options related to the comment content may also be set in the comment storage list, such as the commented song identifier corresponding to the comment content, the account identifier corresponding to the comment content, etc. In addition, a sensitive indication option for reflecting whether the comment is a sensitive comment may also be set in the comment storage table. For example, when the sensitive information recognition model determines that the user's comment content is a sensitive comment, the corresponding comment content may be stored in the comment storage list, and the corresponding sensitive indication option may be set to 1. When the sensitive information recognition model determines that the user's comment content is not a sensitive comment, the corresponding comment content may be stored in the comment storage list, and the corresponding sensitive indication option may be set to 0. When the sensitive information recognition model in the server identifies that the comment content is sensitive content, the corresponding comment content may be stored in the comment storage list, and the corresponding sensitive indication option may be set to 1. The server may obtain the comment content (i.e., the target comment information) with the sensitive indication option set to 1 in the comment storage list, and may obtain the identification information of the multimedia data corresponding to the target comment information according to the comment storage list, and then send the target comment information and the identification information of the corresponding multimedia data to the review terminal.

[0086] Optionally, whenever a preset cycle duration is reached, the corresponding sensitivity determination result within the current cycle duration is determined to be the target comment information having sensitive information and the target multimedia data to which the target comment information belongs.

[0087] The cycle duration can be one day or 12 hours, etc., which can be set by a technician. The cycle duration is not limited here.

[0088] During implementation, whenever a preset cycle duration is reached, the server may send all comment contents with a sensitive indication option of 1 within the current cycle duration, as well as identification information of multimedia data corresponding to the comment contents, to the review terminal. The review terminal may display the corresponding comment contents and identification information of multimedia data corresponding to the comment contents in the form of a list on the review terminal.

[0089] Song ID Comment user ID Commenting on User Content Comment time Sensitive Indicators Song_123 User_1001 That's the power 2020.1.16 18:41 1 Song_124 User_1002 It sounds nice. 2020.1.16 18:41 0 Song_125 User_1003 Not bad, not bad 2020.1.16 18:41 0 Song_126 User_1004 This society is not free 2020.1.16 18:41 1

[0090] Table 1

[0091] Optionally, the multimedia data can be sorted according to the number of sensitive comments corresponding to each multimedia data, and the corresponding processing is as follows: determine the number of target comment content corresponding to each target multimedia data; based on the number corresponding to each target multimedia data, rank the target multimedia data in order of number from most to least to obtain the ranking information of each target multimedia data; send the ranking information of each target multimedia data to the review terminal.

[0092] In implementation, whenever the preset cycle duration is reached, the server can obtain the comment content of all sensitive indication options as 1 within the current cycle duration, and the identification information of the multimedia data corresponding to the comment content, that is, obtain all sensitive comments detected by the sensitive information recognition model within the current cycle duration, and the song identification corresponding to the sensitive comments, and then store the sensitive comments and the corresponding song identification in the high-risk song library. Then, the song identifications are sorted in the order of most to least according to the number of sensitive comments corresponding to each song identification in the high-risk song library to obtain the ranking information of the song identification. Then, the ranking information and the corresponding song identification and the sensitive comments corresponding to the song identification are sent to the audit terminal. After the audit terminal receives the ranking information and the corresponding song identification and the sensitive comments corresponding to the song identification sent by the server, the audit terminal can display the song identification according to the ranking information, wherein the number and content of the corresponding sensitive comments can also be displayed after each song identification.

[0093] Optionally, when sending the identification information of the target multimedia data and the target comment information to the review terminal, the number of target comment information corresponding to each target multimedia data can also be determined; then the identification information and target comment information of the target multimedia data whose number of corresponding target comment information exceeds a number threshold are sent to the review terminal.

[0094] The number threshold indicates a threshold of the number of target comment information corresponding to the multimedia data, and can be preset by a computer personnel.

[0095] In implementation, after obtaining all sensitive comments detected by the sensitive information recognition model within the current cycle time, as well as the song identifiers corresponding to the sensitive comments, and storing the sensitive comments and the corresponding song identifiers in the high-risk song library, the sensitive comments corresponding to each song identifier in the high-risk song library can be counted to obtain the number of sensitive comments corresponding to each song identifier, and then the song identifiers whose number of sensitive comments exceeds the number threshold are determined, and then the song identifiers whose number of sensitive comments exceeds the number threshold and the corresponding sensitive comments are sent to the review terminal.

[0096] Optionally, the auditor may determine whether the multimedia data contains sensitive information by re-auditing the multimedia data, and the corresponding processing may be as follows: receiving a data acquisition request sent by the audit terminal, wherein the data acquisition request carries first identification information of the multimedia data to be queried; and sending target multimedia data corresponding to the first identification information to the audit terminal.

[0097] In implementation, after the audit terminal receives the sensitive comments sent by the server and the song identifiers corresponding to the sensitive comments, the sensitive comments and the song identifiers corresponding to the sensitive comments can be displayed. The auditor can click on the song identifier to send a data acquisition request to the server, wherein the data acquisition request carries first identification information, and the first identification information is the song identifier clicked by the auditor. After the server receives the first identification information sent by the audit terminal, the target multimedia data corresponding to the first identification information can be sent to the audit terminal, such as the lyrics in the song, or the audio of the song. The auditor can determine whether there is sensitive information in the song by reviewing the lyrics in the song or the audio of the song.

[0098] Optionally, when the reviewer determines that sensitive information exists in the multimedia data, the server may perform the following processing: receive a multimedia data deletion notification sent by the review terminal, wherein the processing notification carries second identification information of the multimedia data to be deleted; and delete the target multimedia data corresponding to the second identification information.

[0099] In implementation, a processing option may be set after the multimedia data identification displayed in the audit terminal. When the auditor finds that there is sensitive information in the multimedia data, he can click the corresponding processing option. After that, the audit terminal can send a deletion notice to the server, wherein the deletion notice may carry the second identification information, that is, the song identification corresponding to the processing option clicked by the auditor. After the server receives the deletion notice, it can delete the song corresponding to the song identification according to the song identification carried in the deletion notice. In addition, the audit terminal can also set the deletion time corresponding to the sensitive work, and send a deletion notice carrying the deletion time to the server. After the server receives the deletion notice, it can delete the song corresponding to the song identification according to the deletion time and song identification carried in the deletion notice. After the deletion time, the corresponding song can be put on the shelf again. For example, if the deletion time carried in the deletion notice is 3 days, the server can count the time after deleting the corresponding song. When the deletion time reaches 3 days, the server can re-upload the corresponding song.

[0100] In the embodiment of the present application, the review of multimedia data is completed by obtaining the comment information corresponding to the multimedia data, detecting whether the comment information is a sensitive comment, and then sending the sensitive comment and the corresponding multimedia data to the review terminal. By adopting the present application, more multimedia data that does not contain sensitive words but has sensitive information can be mined by detecting the comment information of the multimedia data.

[0101] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0102] Figure 3is a schematic diagram of a structure of a device for identifying sensitive works provided in an embodiment of the present application. The device may be the server in the above embodiment, such as Figure 3 As shown, the device comprises:

[0103] An acquisition module 310 is configured to acquire comment information of multimedia data;

[0104] The determination module 320 is configured to input the comment information into a pre-trained sensitive information recognition model to obtain a sensitivity determination result, wherein the sensitivity determination result is used to indicate whether the multimedia data corresponding to the comment information contains sensitive information;

[0105] The first sending module 330 is configured to obtain target comment information with a sensitivity determination result of having sensitive information, and target multimedia data to which the target comment information belongs, and send identification information of the target multimedia data and the target comment information to the review terminal.

[0106] Optionally, the pre-trained sensitive information identification model includes an input layer, an embedding layer, a bidirectional long short-term memory network layer, an attention layer and an output layer.

[0107] Optionally, the determining module is configured to:

[0108] Performing word segmentation on the comment information to obtain at least one phrase corresponding to the comment information;

[0109] Inputting the at least one phrase into the input layer to obtain a phrase identifier corresponding to the at least one phrase;

[0110] Inputting a phrase identifier corresponding to the at least one phrase into the embedding layer to obtain a phrase vector corresponding to the at least one phrase;

[0111] Inputting the phrase vector corresponding to the at least one phrase into the bidirectional long short-term memory network layer to obtain a feature vector corresponding to the at least one phrase;

[0112] Inputting the feature vector corresponding to the at least one phrase into the attention layer to obtain the feature vector corresponding to the comment information;

[0113] The feature vector corresponding to the comment information is input into the output layer to obtain the sensitivity determination result corresponding to the comment information.

[0114] Optionally, the device further includes a second sending module, configured to:

[0115] Determining the number of target comment information corresponding to each target multimedia data;

[0116] Based on the number corresponding to each target multimedia data, the target multimedia data are ranked in order of the number from most to least, to obtain ranking information of each target multimedia data;

[0117] The ranking information of each target multimedia data is sent to the audit terminal.

[0118] Optionally, the first sending module 330 is configured to:

[0119] Determining the number of target comment information corresponding to each target multimedia data;

[0120] The identification information of the target multimedia data and the target comment information whose number of corresponding target comment information exceeds the number threshold are sent to the review terminal.

[0121] Optionally, the acquisition module is configured as:

[0122] Whenever the preset cycle duration is reached, the corresponding sensitivity determination result within the current cycle duration is determined to be the target comment information having sensitive information and the target multimedia data to which the target comment information belongs.

[0123] Optionally, the device further includes a third sending module, configured to:

[0124] Receiving a data acquisition request sent by the audit terminal, wherein the data acquisition request carries first identification information of the multimedia data to be queried;

[0125] The target multimedia data corresponding to the first identification information is sent to the audit terminal.

[0126] Optionally, the device further includes a processing module configured to:

[0127] Receiving a multimedia data deletion notification sent by the audit terminal, wherein the processing notification carries second identification information of the multimedia data to be deleted;

[0128] Delete the target multimedia data corresponding to the second identification information.

[0129] It should be noted that: the device for identifying sensitive works provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device for identifying sensitive works provided in the above embodiment and the method embodiment for identifying sensitive works belong to the same concept. The specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0130] Figure 4It is a structural diagram of a computer device provided in an embodiment of the present application. The computer device 400 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central processing units, CPU) 401 and one or more memories 402, wherein the memory 402 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 401 to implement the methods provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the server may also include other components for implementing device functions, which will not be described in detail here.

[0131] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions, which can be executed by a processor in a terminal to complete the method of storing sensitive works in the above embodiment. The computer-readable storage medium can be non-transitory. For example, the computer-readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0132] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0133] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for identifying sensitive information, characterized in that: The method comprises: Receiving a comment request sent by a user terminal, wherein the comment request carries comment information of multimedia data; Obtaining comment information of the multimedia data; Performing word segmentation on the comment information to obtain at least one phrase corresponding to the comment information; The pre-trained sensitive information recognition model includes an input layer, an embedding layer, a bidirectional long short-term memory network layer, an attention layer, and an output layer, and the at least one phrase is input into the input layer to obtain a phrase identifier corresponding to the at least one phrase; Inputting a phrase identifier corresponding to the at least one phrase into the embedding layer to obtain a phrase vector corresponding to the at least one phrase; Inputting the phrase vector corresponding to the at least one phrase into the bidirectional long short-term memory network layer to obtain a feature vector corresponding to the at least one phrase; Inputting the feature vector corresponding to the at least one phrase into the attention layer to obtain the feature vector corresponding to the comment information; Inputting the feature vector corresponding to the comment information into the output layer to obtain a sensitivity determination result corresponding to the comment information, wherein the sensitivity determination result is used to indicate whether the multimedia data corresponding to the comment information contains sensitive information; Obtaining target comment information with sensitive information as a result of a sensitivity determination and target multimedia data to which the target comment information belongs; determining the number of target comment information corresponding to each target multimedia data, sending identification information and target comment information of target multimedia data whose number of corresponding target comment information exceeds a number threshold to a review terminal, so that the review terminal displays the target comment information and identification information of the target multimedia data to which the target comment information belongs; Receiving a data acquisition request sent by the audit terminal, wherein the data acquisition request carries first identification information of the multimedia data to be queried, and the first identification information is an identification of the multimedia data clicked by the auditor; The target multimedia data corresponding to the first identification information is sent to the audit terminal to audit and determine whether the target multimedia data contains sensitive information.

2. The method according to claim 1, characterized in that After obtaining the target comment information having sensitive information as a result of the sensitivity determination and the target multimedia data to which the target comment information belongs, the method further includes: Determine the number of target comment information corresponding to each target multimedia data; Based on the number corresponding to each target multimedia data, the target multimedia data are ranked in order of the number from most to least, to obtain ranking information of each target multimedia data; The ranking information of each target multimedia data is sent to the audit terminal.

3. The method according to claim 1, characterized in that Obtaining target comment information having sensitive information as a result of a sensitivity determination and target multimedia data to which the target comment information belongs, including: Whenever the preset cycle duration is reached, the corresponding sensitivity determination result within the current cycle duration is determined to be the target comment information having sensitive information and the target multimedia data to which the target comment information belongs.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Receiving a multimedia data deletion notification sent by the audit terminal, wherein the processing notification carries second identification information of the multimedia data to be deleted; Delete the target multimedia data corresponding to the second identification information.

5. A device for identifying sensitive works, characterized in that: The device comprises: The acquisition device is configured to receive a comment request sent by a user terminal, wherein the comment request carries comment information of the multimedia data; and acquire the comment information of the multimedia data; The determination module is configured to perform word segmentation processing on the comment information to obtain at least one phrase corresponding to the comment information; the pre-trained sensitive information identification model includes an input layer, an embedding layer, a bidirectional long short-term memory network layer, an attention layer and an output layer, the at least one phrase is input into the input layer to obtain a phrase identifier corresponding to the at least one phrase; the phrase identifier corresponding to the at least one phrase is input into the embedding layer to obtain a phrase vector corresponding to the at least one phrase; the phrase vector corresponding to the at least one phrase is input into the bidirectional long short-term memory network layer to obtain a feature vector corresponding to the at least one phrase; the feature vector corresponding to the at least one phrase is input into the attention layer to obtain a feature vector corresponding to the comment information; the feature vector corresponding to the comment information is input into the output layer to obtain a sensitivity determination result corresponding to the comment information, wherein the sensitivity determination result is used to indicate whether the multimedia data corresponding to the comment information has sensitive information; The first sending module is configured to obtain target comment information with sensitive information as determined by a sensitivity determination result and target multimedia data to which the target comment information belongs; determine the number of target comment information corresponding to each target multimedia data, and send identification information and target comment information of target multimedia data whose number of corresponding target comment information exceeds a number threshold to the review terminal, so that the review terminal displays the target comment information and the identification information of the target multimedia data to which the target comment information belongs; The third sending module is configured to receive a data acquisition request sent by the audit terminal, wherein the data acquisition request carries first identification information of the multimedia data to be queried, and the first identification information is the identification of the multimedia data clicked by the auditor; and send target multimedia data corresponding to the first identification information to the audit terminal to audit and determine that the target multimedia data contains sensitive information.

6. The device according to claim 5, characterized in that The device also includes a second sending module, configured to: Determine the number of target comment information corresponding to each target multimedia data; Based on the number corresponding to each target multimedia data, the target multimedia data are ranked in order of the number from most to least, to obtain ranking information of each target multimedia data; The ranking information of each target multimedia data is sent to the audit terminal.

7. The device according to claim 5 or 6, characterized in that The device also includes a processing module configured to: Receiving a multimedia data deletion notification sent by the audit terminal, wherein the processing notification carries second identification information of the multimedia data to be deleted; Delete the target multimedia data corresponding to the second identification information.

8. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operations performed by the method for identifying sensitive information as described in any one of claims 1 to 4.

9. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operations performed by the method for identifying sensitive information as described in any one of claims 1 to 4.

Images