File content review method, system, computing device, and storage medium

Abstract

Provided in the present application are a file content review method, a system, a computing device, and a storage medium. The method comprises: a file content review system first acquiring a first file, the first file being a file that requires content review; then extracting a first statement from the first file on the basis of a first keyword; and, then determining a second statement on the basis of the first statement, the similarity between the second statement and the first statement satisfying a similarity threshold, such that risk information of the second statement can be used as risk information of the first statement, the risk information being used for indicating whether a corresponding statement contains risky content. By means of measuring the similarity between a statement extracted from a file requiring content review and a statement marked with risk information, the method can accurately evaluate whether the file content contains risky content.

Description

File content review method, system, computing device and storage medium

[0001] The present application claims priority to the Chinese patent application No. 202410650398.7, filed on May 23, 2024, and entitled "File content review method, system, computing device and storage medium", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of artificial intelligence (AI), and in particular to a file content review method, system, computing device and storage medium. BACKGROUND

[0003] Identification module

[0004] Files can be used to record various types of data. Current file content review methods mainly rely on keyword identification technology. The specific scheme is as follows: the user pre-configures the keywords to be identified. When the file needs to be reviewed, the file is scanned in full text, and the positions of the keywords configured by the user appearing in the file are marked.

[0005] It can be seen that the above keyword identification method can only perform content identification based on pre-set keywords, and the keyword setting method depends on the user being able to define accurate keywords. The identification result obtained by the above identification method cannot indicate whether the file content has a risk item, that is, the above identification method cannot effectively identify and judge the risk of the file content. SUMMARY

[0006] The present application provides a file content review method, system, computing device and storage medium, which can accurately evaluate whether the file content has a risk.

[0007] In a first aspect, the present application provides a file content review method, which is executed by a file content review system. The method comprises: first acquiring a first file, the first file being a file that needs to be reviewed; then extracting a first sentence from the first file according to a first keyword; and then determining a second sentence according to the first sentence, the similarity between the first sentence and the second sentence satisfying a similarity threshold, so as to take the risk information of the second sentence as the risk information of the first sentence. The risk information is used to indicate whether the corresponding sentence is risk content.

[0008] In the present solution, the file content review system first extracts a sentence containing a corresponding keyword from a file based on the keyword, and then calculates the similarity between the extracted sentence and other sentences labeled with risk information. If the similarity between the extracted sentence and a certain sentence labeled with risk information meets a similarity threshold, the two sentences are considered similar, and the same risk information is labeled for the two sentences, that is, the risk information of the sentence labeled with risk information is taken as the risk information of the currently extracted sentence. This way can effectively identify the risk content in the file.

[0009] In a possible implementation, the similarity threshold can be determined in the following manner: the file content review system obtains a plurality of sentences containing the second sentence, the plurality of sentences having the same risk information, and then converts the plurality of sentences into a plurality of feature vectors, the plurality of sentences and the plurality of feature vectors corresponding one-to-one; then, the similarity between each two of the plurality of sentences is calculated according to the plurality of feature vectors, to obtain a plurality of similarities, and then the similarity threshold is determined according to the plurality of similarities.

[0010] That is, the file content review system can calculate the similarity between each two of a plurality of sentences having the same risk information, and then determine the similarity threshold according to the plurality of calculated similarities, such as taking the average of the plurality of similarities as the similarity threshold, or taking the minimum value of the plurality of similarities as the similarity threshold, and so on.

[0011] In a possible implementation, the plurality of sentences all contain a keyword belonging to a first word category, and the first sentence includes a first keyword also belonging to the first word category. That is, the file content review system distinguishes different word categories, and calculates the similarity between sentences having the same risk information and containing keywords belonging to the same word category, and then determines the similarity threshold corresponding to the word category according to the calculated similarity. The similarity between sentences not containing keywords belonging to the same word category is not calculated, so as to avoid affecting the accuracy of the similarity threshold. Each word category can calculate the corresponding similarity threshold in a similar manner. When the file content review system extracts a sentence containing a keyword from a file, it can obtain other sentences containing keywords of the same word category and having risk information according to the keyword, and then calculate the similarity between the extracted sentence and the other sentences, and compare it with the similarity threshold corresponding to the word category, so as to accurately determine the risk information of the extracted sentence.

[0012] In a possible implementation, the file content review system can extract the first sentence in the following manner: first, a scanning rule is acquired, the scanning rule being used to indicate that a sentence meeting a sentence range is extracted, then the first file is divided into a plurality of sentences meeting the sentence range, the plurality of sentences meeting the sentence range including the first sentence, and then the first sentence is extracted according to the first keyword.

[0013] In this solution, the file content review system first divides a file that needs to be reviewed into a plurality of sentences meeting a sentence range indicated by a scanning rule, and then determines whether each divided sentence contains a corresponding keyword, and only in the case where a sentence contains a corresponding keyword, the sentence is extracted from the file. That is, the extracted sentence meets both the requirement of the sentence range and the requirement of the keyword (contains a corresponding keyword). The application does not make a specific limitation on the sentence range, which can be one sentence, a plurality of consecutive sentences, a paragraph, a plurality of consecutive paragraphs, X consecutive characters, and the like. The sentence range can be given in the scanning rule or be a default setting of the file content review system.

[0014] In a possible implementation, the scanning rule and the first sentence meet one of the following conditions:

[0015] (1) The scanning rule includes the first keyword and a first identification type, the first identification type being used to indicate that a sentence containing the first keyword and meeting the sentence range is extracted, and the first sentence contains the first keyword;

[0016] (2) The scanning rule includes a plurality of keywords and a second identification type, the second identification type being used to indicate that a sentence containing the plurality of keywords and meeting the sentence range is extracted, and the first sentence contains the plurality of keywords;

[0017] (3) The scanning rule includes a keyword set and a third identification type, the keyword set including the first keyword, the third identification type being used to indicate that a sentence containing a number of keywords in the keyword set greater than or equal to a hit number threshold and meeting the sentence range is extracted, the hit number threshold being less than a number of keywords in the keyword set, the first sentence containing the first keyword, and the first sentence containing a number of keywords in the keyword set greater than or equal to the hit number threshold.

[0018] The above (1) to (3) are actually three setting modes of the scanning rule. Mode (1) is that a keyword and the first identification type are specified in the scanning rule, at this time the scanning rule indicates extracting the sentence with the keyword and satisfying the sentence range, that is, for any sentence in the file satisfying the sentence range, as long as the keyword specified in the scanning rule appears in the sentence, the sentence can be extracted. Mode (2) is that multiple keywords and the second identification type are specified in the scanning rule, at this time the scanning rule indicates extracting the sentence with the multiple keywords and satisfying the sentence range, that is, for the sentence in the file satisfying the sentence range, only when the multiple keywords specified in the scanning rule appear in the sentence at the same time, the sentence can be extracted. Mode (3) is that a keyword set (including N keywords, N is a positive integer greater than 1) and the third identification type are specified in the scanning rule, at this time the scanning rule indicates extracting the sentence with M keywords of the N keywords and satisfying the sentence range, that is, for the sentence in the file satisfying the sentence range, as long as any M keywords of the N keywords appear in the sentence (M is a hit quantity threshold), the sentence can be extracted. M is a positive integer, M is less than N, M can be given in the scanning rule or can be a system default setting (such as default M = N-P, P is a positive integer).

[0019] In a possible implementation, the scanning rule further includes auxiliary judgment information, the auxiliary judgment information is used to indicate the professional field and / or regulation corresponding to the scanning rule. After the system extracts the first sentence in the first file according to the scanning rule and determines the risk information of the first sentence, the system can feed back the review result of the first file to the user, the review result includes the risk information of the first sentence and the above-mentioned auxiliary judgment information, so as to improve the explainability of the review result.

[0020] In a possible implementation, the file content review system can further obtain a second file, the second file is a file that needs content review. In the case that the file format of the second file does not conform to the file type configured by the file content review system, the scanning time of the second file exceeds the first time threshold, or the waiting time of the second file exceeds the second time threshold, the system feeds back to the user that the second file cannot be processed temporarily.

[0021] That is, a protection mechanism is provided here, when the file that needs content review does not conform to the file type configured by the system, the scanning time of the file is too long, or the waiting time of the file is too long, the system can feed back to the user that the file cannot be processed temporarily, and stop processing the file to avoid affecting the processing process of other files, which helps to protect the reliability of the system.

[0022] In a second aspect, the present application also provides a file content review system, which comprises function modules for performing the method of the first aspect or any possible implementation manner of the first aspect.

[0023] In a third aspect, the present application also provides a computing device, comprising a processor and a memory. The processor is configured to execute instructions stored in the memory, so that the computing device performs the operation steps of the method of the first aspect or any possible implementation manner of the first aspect.

[0024] In a fourth aspect, the present application also provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory. The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the operation steps of the method of the first aspect or any possible implementation manner of the first aspect.

[0025] In a fifth aspect, the present application also provides a chip system, comprising a processor and a power supply circuit. The power supply circuit is configured to supply power to the processor. The processor is configured to perform the operation steps of the method of the first aspect or any possible implementation manner of the first aspect.

[0026] In a sixth aspect, the present application also provides a computer readable storage medium, comprising computer program instructions. When the computer program instructions are executed by a computing device, the computing device performs the operation steps of the method of the first aspect or any possible implementation manner of the first aspect.

[0027] In a seventh aspect, the present application also provides a computer program product comprising instructions. When the instructions are run by a computing device, the computing device performs the operation steps of the method of the first aspect or any possible implementation manner of the first aspect.

[0028] On the basis of the implementation manners of the aspects provided by the present application, further combinations can be made to provide more implementation manners. BRIEF DESCRIPTION OF DRAWINGS

[0029] FIG. 1 is a system architecture diagram provided by the present application;

[0030] FIG. 2 is a file parsing process diagram provided by the present application;

[0031] FIG. 3 is a structure diagram of a semantic recognition model provided by the present application;

[0032] FIG. 4 is a stage diagram of a file content review provided by the present application;

[0033] FIG. 5 is a flow diagram of a file content review method provided by the present application;

[0034] FIG. 6 is a structural schematic diagram of a computing device provided by the present application;

[0035] FIG. 7 is a structural schematic diagram of a computing device cluster provided by the present application;

[0036] FIG. 8 is a schematic diagram of two computing devices connected through a network provided by the present application. DETAILED DESCRIPTION

[0037] In order to solve the problem that the prior art cannot effectively identify and judge the risk of file content, the present application provides a file content review system. First, the file content review system extracts a sentence from a file based on a keyword, and then takes the risk information of a similar sentence of the sentence as the risk information of the sentence. Specifically, the file content review system first extracts a sentence from a file that needs content review based on a keyword, and then calculates the similarity between the extracted sentence and other sentences with risk information. The risk information is used to indicate whether the corresponding sentence is risk content. If the similarity between the extracted sentence and a certain sentence with risk information meets a similarity threshold, the two sentences are considered to be similar sentences, and the risk information of the sentence with risk information is taken as the risk information of the current extracted sentence. This way can effectively identify the risk content in the file.

[0038] The file content review system will be described in detail below.

[0039] Please refer to FIG. 1, which is a system architecture diagram provided by the present application, including a client 100 and a file content review system 200. The client 100 and the file content review system 200 have a communication connection, which can be a wired connection or a wireless connection. The number of clients 100 that establish a communication connection with the file content review system 200 can be one or more (FIG. 1 takes one client 100 as an example), which is not specifically limited by the present application.

[0040] The client 100 is used to realize human-computer interaction and can be deployed on a terminal device or a computing device. The terminal device can be a smart phone, a wearable device, a notebook computer, a tablet computer, a vehicle-mounted device, or a smart conference device, etc. The computing device can be a server, a personal computer (PC), etc., which is not specifically limited by the present application.

[0041] In some implementations, the client 100 can be an application (APP) client / mobile client running on a mobile terminal such as a smart phone, a wearable device, etc., or a software or application running on a computing device (such as a PC client), or a web application / website accessed based on a web browser, or a front-end console of a cloud platform, which is not limited in the present application.

[0042] The file content review system 200 is used to provide the function of file content review, and can be deployed on a computing device, a computing device cluster composed of multiple computing devices, or a terminal device. The computing device can be a physical server, a virtual machine, a container, or an edge computing device, etc. The virtual machine refers to a complete computer system running in a completely isolated environment by software simulation with complete hardware system functions. When creating a virtual machine in a computing device, part of the hard disk and memory capacity of the entity machine needs to be used as the hard disk and memory capacity of the virtual machine. Each virtual machine has an independent basic input / output system (CMOS), hard disk, and operating system, and can be operated like an entity machine. The container is a portable software unit that can combine an application and all its dependencies into a software package that is not limited by the underlying host operating system, so that the complex environment does not need to be built again, simplifying the application development to deployment process. The edge computing device refers to a device closer to the data source and the end user, with low latency and high bandwidth characteristics, such as a smart router, an edge server, etc., which is not limited in the present application. The terminal device can refer to the previous description, which is not repeated here.

[0043] Optionally, the client 100 and the file content review system 200 can be deployed on the same terminal device or computing device; or the client 100 is deployed on the terminal device, and the file content review system 200 is deployed on a single computing device or a computing device cluster. It should be understood that the above examples are only for illustration, and the deployment of the client 100 and the file content review system 200 can be determined according to the actual application scenario.

[0044] Further, the file content review system 200 can be divided into multiple modules according to functions. FIG. 1 exemplarily shows a division manner of the file content review system 200, which includes an acquisition module 210, an extraction module 220, a determination module 230, a sending module 240, and a training module 250, and further includes a semantic recognition model 260, a scanning rule library 270, and a sentence sample library 280, which are introduced as follows.

[0045] 1. The acquisition module 210 is used to acquire a file that needs content review.

[0046] The user can send one or more files (i.e., single or batch upload files) to the file content review system 200 through the client 100, so that the acquisition module 210 in the file content review system 200 obtains the files that need to be reviewed. In addition to receiving the files that need to be reviewed sent by the user through the client 100, the acquisition module 210 can also obtain the files that need to be reviewed from other modules or systems (such as a file system, a database system, etc.).

[0047] The present application does not limit the file type of the file that needs to be reviewed. For example, the file type can include a plain text file (TXT), a Word document (DOCX), a portable document format (PDF) document, a presentation (PowerPoint, PPT), an Excel file, a picture format (such as JPG, PNG, GIF, etc.) file, and the like.

[0048] The acquisition module 210 is also used to obtain scanning rules. For example, the acquisition module 210 can receive one or more scanning rules sent by the user through the client 100, or can obtain one or more scanning rules from other modules or systems, which are not limited by the present application.

[0049] The scanning rule is used to indicate a keyword that needs to be identified. The scanning rule can include the following three setting modes (1) to (3).

[0050] (1) The scanning rule includes a single keyword and a first identification type, and the first identification type is used to indicate that a sentence containing the keyword and meeting the sentence range is extracted.

[0051] It should be understood that mode (1) specifies a certain keyword and a first identification type in the scanning rule. At this time, the scanning rule means that the sentence with the keyword and meeting the sentence range is extracted. That is, for any sentence meeting the sentence range in the file, as long as the sentence appears the keyword specified in the scanning rule, the sentence can be extracted.

[0052] The sentence range refers to the range when identifying the keyword. The sentence range can be a sentence (identified by a period), a plurality of continuous sentences, a paragraph (identified by a line break), X continuous paragraphs (X is a positive integer), Y continuous characters (Y is a positive integer), and the like, which are not limited by the present application. The specific sentence range can be specified in the scanning rule, or the sentence range can not be specified in the scanning rule (at this time, the default sentence range is adopted), which are not limited by the present application. That is, if the single keyword given in the scanning rule appears in a certain sentence meeting the sentence range, the sentence can be extracted.

[0053] For example, assume that the scanning rule includes the keyword "artificial intelligence", the first recognition type, and the sentence range "one sentence", which means that the keyword "artificial intelligence" needs to be recognized in one sentence. If the file is scanned based on the scanning rule, all the sentences in which the keyword "artificial intelligence" appears in the file will be extracted.

[0054] (2) The scanning rule includes multiple keywords and the second recognition type, and the second recognition type is used to indicate that sentences containing the multiple keywords and satisfying the sentence range are extracted.

[0055] It should be understood that in mode (2), the multiple keywords and the second recognition type are specified in the scanning rule, and at this time, the scanning rule means that sentences having the multiple keywords and satisfying the sentence range are extracted, that is, for sentences in the file satisfying the sentence range, only when the multiple keywords specified in the scanning rule appear in the sentence at the same time, the sentence is extracted. The sentence range can be referred to the description in mode (1), which will not be described here.

[0056] For example, assume that the scanning rule includes the keywords "artificial intelligence" and "image" and the second recognition type, and the sentence range is "one sentence" by default, which means that the keywords "artificial intelligence" and "image" need to be recognized at the same time in one sentence. If the file is scanned based on the scanning rule, all the sentences in which the keywords "artificial intelligence" and "image" appear in the file will be extracted.

[0057] (3) The scanning rule includes a keyword set and the third recognition type, the keyword set includes N keywords, N is a positive integer greater than 1, and the third recognition type is used to indicate that sentences containing keywords in the keyword set in a number greater than or equal to M and satisfying the sentence range are extracted.

[0058] It should be understood that in mode (3), a keyword set and the third recognition type are specified in the scanning rule, and at this time, the scanning rule means that sentences having M keywords in the keyword set and satisfying the sentence range are extracted, that is, for sentences in the file satisfying the sentence range, as long as any M keywords of the N keywords appear in the sentence, the sentence can be extracted. The value of M is referred to as a hit number threshold, and M is a positive integer less than N. M can be given in the scanning rule or can be a default setting of the file content review system 200 (such as default M = N-P, P is a positive integer), which is not limited in the present application.

[0059] For example, assume that the scanning rule includes a keyword set composed of the three keywords "artificial intelligence", "image", and "cat" and the third identification type, and the scanning rule also specifies a hit quantity threshold M of 2 and a sentence range of "50 characters" by default, which means that at least two keywords in the keyword set need to be identified within 50 characters. If the file is scanned based on the scanning rule, all sentences in the file that contain at least two keywords in the keyword set and have a character number within 50 characters will be extracted.

[0060] Optionally, in addition to specifying specific keywords in the scanning rule, the scanning rule can also indicate the category to which the keywords to be identified belong (in which case the specific keywords are not specified) and the identification type. The identification type can refer to the first identification type, the second identification type, and the third identification type introduced above, which will not be described here.

[0061] For example, assume that the file content review system 200 divides the risk action category, the character object category, the code category, and the like into word categories, and one or more keywords are set under each word category, which is not limited by the present application. The acquisition module 210 receives a scanning rule sent by the user through the client 100, which indicates the risk action category and the character object category, and the identification type given in the scanning rule is the second identification type, and the default sentence range is assumed to be one sentence, which means that the sentence containing the keywords belonging to the risk action category and the keywords belonging to the character object category and meeting the sentence range needs to be extracted.

[0062] Assume that the file uploaded by the user to the file content review system 200 through the client 100 contains the following sentence "conflicts occurred during the visit, and A character beat B character". Based on the above scanning rule, the file is scanned, it can be determined that the sentence includes the keyword "beat" belonging to the risk action category, and also includes the keywords "A character" and "B character" belonging to the character object category, and the sentence meets the one-sentence sentence range requirement, so the sentence can be extracted from the file.

[0063] Optionally, the scanning rule can also include auxiliary judgment information, which is used to indicate the professional field and / or regulation corresponding to the scanning rule. When the file content review system 200 scans the file uploaded by the user based on the scanning rule, the auxiliary judgment information can be fed back to the user, so that the user knows the professional field and / or regulation corresponding to the scanning rule adopted by the file content review system 200, thereby improving the explainability of the file content review result output by the file content review system 200.

[0064] For example, the user can give the scanning rules in the form of Table 1 on the client 100, each row in the table represents a scanning rule, and each scanning rule can include a serial number, a professional field / regulation, a keyword, an identification type, a sentence range (optional), and the like. When the acquisition module 210 receives the scanning rules sent by the user through the client 100, it can store them in the scanning rule library 270 of FIG. 1.

[0065] Table 1

[0066] In the scanning rule with serial number 001, the professional field given is the image recognition field, and the scanning rule also gives two keywords "artificial intelligence" and "image", the specified identification type is "second identification type", and the specified sentence range is "one sentence", which means that a sentence containing the two keywords "artificial intelligence" and "image" and meeting the sentence range of "one sentence" needs to be extracted. In the scanning rule with serial number 002, the specific regulation given is "Article XX of Enterprise Information Management: Prohibit disclosure of employee login account and password", and the scanning rule also gives four keywords "account", "password", "login", and "employee", as well as the third identification type, and also specifies N=2 and the sentence range of "one paragraph", which means that a sentence containing at least two of the above four keywords and meeting the identification range of "one paragraph" needs to be extracted.

[0067] It should be noted that the scanning rules given in the above example are only for illustration and do not constitute a specific limitation, and other scanning rules can also be set in actual application scenarios, and the scanning rules can also include other fields.

[0068] Optionally, when the user sends a file that needs content review to the file content review system 200 through the client 100, the user can also indicate the professional field corresponding to the file, and the file content review system 200 can use the scanning rules of the corresponding professional field to process the file. If the user does not indicate the professional field corresponding to the file, the file content review system 200 can use part or all of the scanning rules stored in the scanning rule library 270 of FIG. 1 to process the file.

[0069] 2, the extraction module 220: for scanning the file according to the scanning rule, so as to extract the sentence including the corresponding keyword from the file.

[0070] Specifically, when the acquisition module 210 acquires the file that needs content review, it sends the file to the extraction module 220.

[0071] As shown in FIG. 2, the extraction module 220 first determines the file type of the file, such as whether the file is a picture format file (e.g., JPG, PNG, GIF, etc.), or a file of another type (e.g., EXCEL, Word, PDF, PPT, etc.), and then determines whether the file contains a picture. If the file does not contain a picture, the text in the file can be directly output. If the file contains a picture, the picture in the file needs to be first rotated and corrected so that the picture is in a normal position (picture skewing can adversely affect text extraction), and then an Optical Character Recognition (OCR) algorithm is used to convert the text in the picture into a text format, and then the text corresponding to the file (including the text converted from the picture in the file and the original text in the file) is output. The OCR is a technology for converting text in an image into editable and searchable text, which will not be described in detail here.

[0072] Then, the extraction module 220 scans the text corresponding to the file based on the scanning rule, so as to extract the sentence containing the corresponding keyword from the file. Specifically, the extraction module 220 first acquires the scanning rule, which is used to indicate that the sentence meeting the sentence range and containing the corresponding keyword is extracted. The scanning rule and the sentence range can be referred to the foregoing description, and will not be described herein. Then, the extraction module 220 divides the text corresponding to the file according to the sentence range indicated by the scanning rule, that is, divides the text into a plurality of sentences meeting the sentence range, and then determines whether each divided sentence contains the corresponding keyword specified in the scanning rule. If the corresponding keyword is contained, the sentence is extracted. The extracted sentences are the sentences of the risk information to be identified, and the risk information of the sentences will be determined by the determination module 230 (which will be described later).

[0073] For example, it is assumed that the scanning rule includes the two keywords “artificial intelligence” and “image” and the second identification type, and the sentence range is by default “one sentence”. At this time, the scanning rule indicates that the sentence containing the two keywords “artificial intelligence” and “image” and meeting the “one sentence” sentence range is extracted. Based on the scanning rule, the extraction module 220 first divides the file to be content reviewed into a plurality of sentences (i.e., a plurality of sentences meeting the “one sentence” sentence range), that is, scans from the beginning of the file, and each time a period is found, a sentence is divided. A plurality of sentences can be divided in total. Then, the extraction module 220 determines whether each of the plurality of sentences contains the two keywords “artificial intelligence” and “image”. If yes, the sentence is extracted from the file. If no, the sentence is not extracted from the file.

[0074] For another example, assuming that the scanning rule includes a keyword set composed of the keywords "artificial intelligence", "image", and "cat" and the third recognition type, and the scanning rule also specifies that the hit quantity threshold M is 2 and the sentence range is "one paragraph" by default, the scanning rule indicates that the sentences containing at least two keywords in the keyword set and satisfying the "one paragraph" sentence range are extracted. Based on the scanning rule, the extraction module 220 first divides the file to be content reviewed into a plurality of sentences satisfying the "one paragraph" sentence range, i.e., scans from the beginning of the file, and divides a sentence each time a line break is found, so that a plurality of sentences satisfying the "one paragraph" requirement can be divided. Then, the extraction module 220 determines whether each of the plurality of sentences contains at least two keywords of "artificial intelligence", "image", and "cat", and if so, extracts the sentence from the file, and if not, does not extract the sentence from the file.

[0075] 3. The training module 250 is configured to train the semantic recognition model 260.

[0076] As shown in FIG. 1, the sentence sample library 280 is configured to provide a plurality of sentence samples for training the semantic recognition model 260, and the sentence samples satisfy the scanning rule. Each sentence sample can be a sentence, a plurality of consecutive sentences, a paragraph, X consecutive paragraphs, Y consecutive characters, etc., which are not limited in the present application. Each sentence sample has a corresponding risk information label, which is used to indicate whether the corresponding sentence sample is risk content. The risk information label of the sentence sample can be manually labeled, for example, first manually determine which keywords exist in the sentence sample, and then determine whether the sentence sample is risk content according to the semantics of the keywords in the sentence sample, and then label the corresponding risk information label for the sentence sample.

[0077] For example, the risk information label can be divided into two types of high-risk label and low-risk label, the high-risk label is used to indicate that the corresponding sentence sample is risk content (there is real risk), and the low-risk label is used to indicate that the corresponding sentence sample is not risk content (although the keywords in the scanning rule are hit, but it is not real risk content, which is a false hit). The labeling personnel can determine whether the sentence sample is risk content according to experience, and then label the corresponding risk information label for the sentence sample.

[0078] For another example, the risk information label can be divided into three types, i.e., a high-risk label, a medium-risk label and a low-risk label. The high-risk label is used to indicate that the corresponding sentence sample is risk content (there is real risk) and the risk degree is high. The medium-risk label is used to indicate that the corresponding sentence sample is risk content (there is real risk) and the risk degree is low. That is, the high-risk label and the medium-risk label are both used to indicate risk content, but the risk degree is different. The specific risk degree (high risk or medium risk) can be determined by manual experience. The low-risk label is used to indicate that the corresponding sentence sample is not risk content (the sentence is a false positive keyword). The annotator can determine whether the sentence sample is risk content according to manual experience, and then annotate the corresponding risk information label for the sentence sample.

[0079] Optionally, the user can send the scanning rule to the file content review system 200, or send the marked sentence sample to the file content review system 200. The file content review system 200 stores the sentence sample sent by the user in the sentence sample library 280 in FIG. 1, and then uses the sentence sample sent by the user to train the semantic recognition model 260. Alternatively, the user only sends the scanning rule to the file content review system 200, but does not actively send the sentence sample to the file content review system 200. At this time, the file content review system 200 can use the existing sentence sample in the sentence sample library 280 in FIG. 1 (these sentence samples can be annotated and uploaded by other personnel) to train the semantic recognition model 260.

[0080] For example, it is assumed that the user sends the scanning rule No. 001 in Table 1 to the file content review system 200 through the client 100. The scanning rule corresponds to the image field, and is used to indicate the extraction of a sentence containing the keywords "artificial intelligence" and "image" and satisfying "one sentence". The user can also obtain some sentence samples containing the keywords "artificial intelligence" and "image" by himself, annotate the corresponding risk information label for these sentence samples according to manual experience, and then send the marked sentence sample to the file content review system 200, so that the file content review system 200 trains the semantic recognition model 260 based on the sentence sample sent by the user.

[0081] For another example, assume that the user sends the scanning rule with serial number 002 in Table 1 to the file content review system 200 through the client 100, where the scanning rule is used to indicate extracting a sentence containing at least two keywords in a keyword set and meeting the sentence range of "a paragraph", and the keyword set specifically includes the four keywords of account number, password, login, and employee. The user can not actively send a sentence sample to the file content review system 200, and at this time the file content review system 200 can obtain a sentence sample meeting the scanning rule from the sentence sample library 280 in FIG. 1 to train the semantic recognition model 260. The sentence sample meeting the scanning rule can be uploaded to the sentence sample library 280 by other personnel, which is not limited in the present application.

[0082] Then, the training module 250 trains the semantic recognition model 260 based on the plurality of sentence samples in the sentence sample library 280. The type of the semantic recognition model 260 can be a language model such as a Transformer network, a bi-directional encoder representations from transformers (BERT), etc., which is not limited in the present application.

[0083] The training process is described in detail below.

[0084] First, the training module 250 obtains a plurality of sentence samples with the same risk information, i.e., the risk information of the plurality of sentence samples indicates that they are all risk content or are not risk content, and then inputs the plurality of sentence samples into the semantic recognition model 260.

[0085] The semantic recognition model 260 converts the plurality of sentence samples into a plurality of feature vectors, and the plurality of feature vectors correspond one-to-one to the plurality of sentence samples. As for converting the sentence sample into a plurality of feature vectors, a method commonly used in the field of natural language processing (NLP) such as word embedding, term frequency-inverse document frequency (TF-IDF), etc. can be used, which is not limited in the present application.

[0086] Then, the semantic recognition model 260 calculates the similarity between each pair of the plurality of sentence samples according to the plurality of feature vectors, that is, the similarity between two sentence samples is calculated by calculating the similarity between the feature vectors corresponding to the two sentence samples. As for calculating the similarity between two feature vectors, it can be calculated in the form of Euclidean distance (Euclidean distance measures the straight-line distance between two vectors, and the smaller the value, the more similar the two vectors are), Manhattan distance, cosine similarity (cosine similarity measures the similarity of the direction of two vectors, and the value ranges between [-1, 1], and the closer the value is to 1, the more similar the two vectors are), etc., and the present application does not limit this.

[0087] After calculating the similarity between each pair of the plurality of sentence samples, a plurality of similarities can be obtained, and then the semantic recognition model 260 determines a similarity threshold value according to the plurality of similarities, such as taking the arithmetic mean of the plurality of similarities as the similarity threshold value, or taking the minimum value in the plurality of similarities as the similarity threshold value, etc.

[0088] Optionally, a plurality of different threshold values (random or artificial) can be set, and then the model score obtainable under each threshold value can be evaluated, and finally the threshold value with the highest score is selected as the similarity threshold value.

[0089] Specifically, first, a threshold value is selected, and for each sentence sample in the plurality of sentence samples labeled with risk information, the similarity between the sentence sample and other sentence samples in the plurality of sentence samples except the sentence sample is calculated (the way of calculating the similarity between sentence samples can refer to the foregoing description). If the similarity between the sentence sample and a certain sentence sample meets the threshold value, the semantic recognition model 260 predicts the risk information of the above certain sentence sample as the risk information of the sentence sample, and then judges whether the risk information marked by the sentence sample is consistent with the risk information predicted by the semantic recognition model 260 for the sentence sample. If consistent, it is considered to be accurate, and if inconsistent, it is considered to be inaccurate. According to the above method, the number of sentence samples whose risk information is accurately judged under the threshold value can be counted, and then the number of sentence samples whose risk information is accurately judged is taken as the model score under the threshold value. The model scores under other threshold values can also be determined in the above manner. By comparing the model scores under different threshold values, the threshold value with the highest score is finally taken as the similarity threshold value.

[0090] Optionally, a plurality of different word categories can be divided, such as a number category, a risk action category, a character object category, a region category, etc., and the present application does not limit this. For each of the plurality of word categories, the training module 250 obtains a plurality of sentence samples having the same risk information and containing keywords belonging to the word category, to train the semantic recognition model 260. The semantic recognition model 260 converts the plurality of sentence samples into a plurality of feature vectors, and then calculates the similarity between each pair of the plurality of sentence samples according to the plurality of feature vectors, and further determines the similarity threshold corresponding to the word category according to the plurality of similarities obtained. In the above manner, the semantic recognition model 260 can determine the corresponding similarity threshold for each word category.

[0091] Optionally, as shown in FIG. 3, the semantic recognition model 260 includes a common module and a plurality of grouping modules (for example, a first grouping module and a second grouping module are taken as examples), and the plurality of grouping modules and the plurality of word categories correspond one by one. Among them, the common module is used to convert the input sentence sample into a corresponding feature vector, and then input the feature vector of the sentence sample into the corresponding grouping module according to the word category to which the keyword in the sentence sample belongs; the grouping module is used to determine the similarity threshold corresponding to the word category according to the feature vectors of the plurality of sentence samples (containing keywords belonging to the word category corresponding to the grouping module) input by the common module (for reference, the foregoing description).

[0092] For example, the training module 250 obtains a plurality of sentence samples containing keywords belonging to the first word category, and inputs them into the common module in the semantic recognition model 260. The common module converts the plurality of sentence samples into corresponding feature vectors, respectively. Since the plurality of sentence samples all contain keywords belonging to the first word category, the common module will input the feature vectors of the plurality of sentence samples into the first grouping module (corresponding to the first word category). Then, the first grouping module calculates the similarity between each pair of the feature vectors of the plurality of sentence samples, obtains a plurality of similarities, and further determines the similarity threshold corresponding to the first word category according to the plurality of similarities, such as taking the arithmetic mean of the plurality of similarities as the similarity threshold corresponding to the first word category, or taking the minimum value in the plurality of similarities as the similarity threshold corresponding to the first word category, etc. The similarity thresholds corresponding to other word categories except the first word category can also be determined in the above manner, which is not described here.

[0093] 4. The determining module 230 is configured to determine the risk information of the sentence to be recognized, i.e., the sentence extracted from the file requiring content review by the extracting module 220.

[0094] Specifically, after the training module 250 trains the semantic recognition model 260, the determination module 230 can use the semantic recognition model 260 to determine the risk information of the sentence of the to-be-recognized risk information. Assuming that the extraction module 220 extracts a first sentence from a file that needs to be reviewed, the determination module 230 can input the first sentence into the semantic recognition model 260. The semantic recognition model 260 first converts the first sentence into a feature vector, and then calculates the similarity between the feature vector and the feature vector corresponding to the sentence sample learned by the semantic recognition model 260 before. If the similarity between the feature vector of the first sentence and the feature vector of the second sentence (any sentence sample learned by the semantic recognition model 260 before) satisfies the similarity threshold described above, the semantic recognition model 260 determines that the second sentence is a similar sentence of the first sentence, and further determines the risk information marked by the second sentence as the risk information of the first sentence. That is, if the risk information marked by the second sentence indicates that the second sentence is risk content, the semantic recognition model 260 determines that the first sentence is also risk content, and if the risk information marked by the second sentence indicates that the second sentence is not risk content, the semantic recognition model 260 determines that the first sentence is also not risk information.

[0095] Optionally, as introduced in the foregoing, the semantic recognition model 260 can determine a corresponding similarity threshold for each word category during the training phase. After the semantic recognition model 260 is trained, assuming that the extraction module 220 extracts a first sentence from a file that needs to be reviewed, and the first sentence contains a first keyword belonging to a first word category, the determination module 230 can input the first sentence into the semantic recognition model 260 to determine the risk information of the first sentence. The semantic recognition model 260 first converts the first sentence into a feature vector, and then obtains the similarity threshold corresponding to the word category to which the first keyword belongs according to the word category, and calculates the similarity between the feature vector of the first sentence and the feature vector corresponding to the sentence sample containing the word category learned by the semantic recognition model 260 before. If the similarity between the feature vector of the first sentence and the feature vector of the second sentence (any sentence sample containing the word category learned by the semantic recognition model 260 before) can satisfy the similarity threshold corresponding to the word category, the semantic recognition model 260 determines that the second sentence is a similar sentence of the first sentence, and further determines the risk information marked by the second sentence as the risk information of the first sentence.

[0096] Optionally, the determination module 230 can also calculate the index of the semantic recognition model 260, and then the sending module 240 sends the index to the user. The first keyword can be a keyword in the scanning rule sent by the user through the client 100, or can be another keyword, which is not limited in the present application.

[0097] The above indicators can include at least one of a sensitivity (Recall) and a hit rate (Precision):

[0098] (1)

[0099] (2)

[0100] wherein C is the number of sentences in the plurality of sentences containing the first keyword that are determined by the semantic recognition model 260 to be risk content, K is the number of sentences in the sentences determined by the semantic recognition model 260 to be risk content that are not actually risk content, and S is the number of sentences in the sentences determined by the semantic recognition model 260 not to be risk content in the above plurality of sentences that are actually risk content.

[0101] For example, assuming that the first keyword is "assault", 100 sentences containing the keyword "assault" are input into the semantic recognition model 260, and the semantic recognition model 260 outputs risk information for the 100 sentences, which indicates whether the corresponding sentence is risk content. As shown in Table 2, 80 sentences are determined by the semantic recognition model 260 to be risk content, and 20 sentences are determined by the semantic recognition model 260 not to be risk content (i.e., non-risk content).

[0102] Table 2

[0103] Among the 80 sentences determined by the model to be risk content, it is found through manual review that the risk information of 70 sentences is correct, and the risk information of the remaining 10 sentences is incorrect, i.e., the 10 sentences are actually non-risk content but are incorrectly determined by the semantic recognition model 260 to be risk content. Among the 20 sentences determined by the model to be non-risk content, it is found through manual review that the risk information of 18 sentences is correct, and the risk information of the remaining 2 sentences is incorrect, i.e., the 2 sentences are actually risk content but are incorrectly determined by the semantic recognition model 260 to be non-risk content.

[0104] Based on the sensitivity calculation formula (1), the sensitivity corresponding to the first keyword can be determined. Based on the hit rate calculation formula (2), the hit rate corresponding to the first keyword can be determined. The calculation processes of the sensitivity and the hit rate corresponding to other keywords are similar, and will not be described in detail here. Then, at least one of the sensitivity and the hit rate corresponding to the first keyword can be sent by the sending module 240 to the client 100, so that the user can view the sensitivity and / or the hit rate of the semantic recognition model 260 in identifying the first keyword on the client 100. These performance indicators can assist the user in setting appropriate scanning rules.

[0105] Optionally, in a case where the sensitivity corresponding to the first keyword is lower than a sensitivity threshold and / or the hit rate corresponding to the first keyword is lower than a hit rate threshold, the training module 250 can obtain more sentence samples containing the first keyword to train the semantic recognition model 260, so as to improve the sensitivity corresponding to the first keyword and / or the hit rate corresponding to the first keyword. The sensitivity threshold and the hit rate threshold can be set by a user according to actual use requirements, and the present application does not make any limitation in this regard.

[0106] Optionally, if there is a sentence in the file that is indicated as risk content by the semantic recognition model 260, the file can be determined as a risk file (or the risk level of the file is determined as high); if there is no sentence in the file that is indicated as risk content by the semantic recognition model 260, the file can be determined as a safe file (or the risk level of the file is determined as low).

[0107] 5. The sending module 240 is configured to send the review result of the file to the client 100.

[0108] The review result of the file includes the sentence containing the corresponding keyword extracted from the file by the extraction module 220 according to the scanning rule, and also includes the risk information output by the semantic recognition model 260 for the extracted sentence in the file. For how to extract the sentence containing the keyword from the file according to the scanning rule and determine the risk information of the sentence, reference can be made to the foregoing description, and no further elaboration is made herein.

[0109] Optionally, the sending module 240 can send the review result of the file to the client 100 where the reviewer (a kind of user) is located, the reviewer issues an evaluation opinion for the review result, the evaluation opinion is used to indicate whether the risk information in the review result is accurate, and then the evaluation opinion is fed back to the file content review system 200.

[0110] For example, it is assumed that the extraction module 220 extracts a sentence containing the keyword "assault" from the file based on the scanning rule, the determination module 230 inputs the sentence into the semantic recognition model 260, and the model outputs the risk information indicating that the sentence is not risk content. The sending module 240 carries the sentence and the risk information of the sentence in the review result of the file, and then sends the review result to the client 100 where the reviewer is logged in. The reviewer views the risk information of the sentence given in the review result on the client 100, and the reviewer determines according to experience that the sentence is actually risk content, so the reviewer sends an evaluation opinion through the client 100, and the evaluation opinion indicates that the risk information of the sentence given in the review result is inaccurate.

[0111] ​​​​​​Optionally, a new sentence sample can be constructed according to the risk information of the sentence output by the semantic recognition model 260 and the review opinion given by the reviewer on the risk information, and then the new sentence sample is added to the sentence sample library 280 for training the semantic recognition model 260, so as to improve the accuracy of the semantic recognition model 260 in identifying risk content.

[0112] For example, for the sentence containing the keyword "artificial intelligence" extracted from the file by the extraction module 220, the risk information output by the semantic recognition model 260 for the sentence indicates that the sentence is risk content, and the review opinion given by the reviewer indicates that the risk information of the sentence is accurate, so the sentence can be taken as a new sentence sample, and the risk information label of the sentence sample indicates that the sentence sample is risk content. For the sentence containing the keyword "password" extracted from the file, the risk information output by the semantic recognition model 260 for the sentence indicates that the sentence is not risk content, but the review opinion given by the reviewer indicates that the risk information of the sentence is inaccurate, so the sentence can be taken as another sentence sample, and the risk information label of the sentence sample indicates that the sentence sample is risk content.

[0113] The number of reviewers and the review process are not specifically limited in the present application.

[0114] For example, FIG. 4 exemplarily shows a stage diagram of file content review. The first stage is to review the file uploaded by the demand person (a kind of user) by the system. First, the file content review system 200 receives one or more files uploaded by the demand person, which need content review, and then the file content review system 200 automatically starts to review the file (for reference to the introduction of the extraction module 220 and the determination module 230), and records the review result of the file (to be manually checked), which includes the sentences containing the keywords extracted from the file and the risk information of the sentences.

[0115] The second stage is the review stage. It is assumed that two reviewers are set in the review stage, i.e., a first-level reviewer and a second-level reviewer, who are responsible for reviewing the review results of the files output by the file content review system 200. First, the first-level reviewer checks the review results of the files, which can be a review detail page of a single file or a result summary page of all files. In the review detail page of a single file, the keyword hits in the file can be checked, and the risk information of the keyword-containing sentence can also be checked. The first-level reviewer can check whether the risk information of the keyword-containing sentence given by the file content review system 200 is accurate, and then give a review opinion and submit it. In the result summary page of all files, the first-level reviewer can manually check whether the file risk level determined by the file content review system 200 is accurate, and also check whether the file content review system 200 has made a determination on all the files uploaded by the demander.

[0116] When the first-level reviewer finishes the review, the second-level reviewer can check the review results of the files, which can be a review detail page of a single file or a result summary page of all files, and also can check the review opinions of the first-level reviewer. The review detail page of a single file and the result summary page of all files can be referred to the foregoing description, and will not be described herein. Optionally, if the review opinions of the first-level reviewer and the second-level reviewer are inconsistent, the review opinion of the second-level reviewer is used as the final review opinion or the second stage of FIG. 4 is restarted.

[0117] The third stage is the file processing and return result. According to the review opinions of the reviewers, it is determined whether the file passes the review. If the review opinion indicates that there is no risk content in the file, it is determined that the file passes the review. If the review opinion indicates that there is risk content in the file, it is determined that the file fails the review. Then, it is fed back to the demander whether the file passes the review.

[0118] The fourth stage is the operation analysis. Through the operation analysis, the accuracy of the file content review system 200 in identifying the risk content can be improved. Specifically, by recording the risk information of the sentence output by the semantic recognition model 260 in the file content review system 200 and the review opinions of the reviewers on the model output, the performance indicators of the semantic recognition model 260 can be calculated, including the sensitivity and / or hit rate of various keywords (which can be referred to the foregoing description). If the sensitivity and / or hit rate of the semantic recognition model 260 for a certain keyword does not meet the standard, the semantic recognition model 260 can be continuously trained using the sentence sample containing the keyword.

[0119] Optionally, the file content review system 200 can set a fuse protection mechanism, which is used to feed back the indication information of temporarily unable to process the file to the demander by the sending module 240 in the case of system downtime, unsupported file type, scanning timeout or waiting timeout, so as to guarantee the stable operation of the whole file content review system 200 and improve the system reliability.

[0120] For example, it is assumed that the demander sends a file to the file content review system 200 through the client 100, and then the extraction module 220 in the file content review system 200 judges the file type of the file and finds that the file type of the file is a file type that is not supported by the file content review system 200 at present, so the sending module 240 can feed back to the client 100 that the file cannot be processed temporarily.

[0121] For another example, it is assumed that the demander sends a large number of files to the file content review system 200 through the client 100, and then the extraction module 220 in the file content review system 200 scans these files one by one based on the scanning rules in the scanning rule library 270 of FIG. 1. It is assumed that the scanning time of a certain file exceeds the first time threshold (may be due to system busy, file data volume too large, etc.), then the scanning of the file can be stopped, and the sending module 240 feeds back to the client 100 that the file cannot be processed temporarily, but does not affect the processing of other files. It is assumed that the waiting time of a certain file exceeds the second time threshold (has not been scanned), then the sending module 240 feeds back to the client 100 that the file cannot be processed temporarily. As for the first time threshold and the second time threshold, the present application does not make specific limitation, which can be reasonably set according to actual use demand.

[0122] It should be noted that the semantic recognition model 260 and / or the sentence sample library 280 in FIG. 1 can be set inside the file content review system 200, or can be set outside the file content review system 200, which is not limited by the present application.

[0123] It should also be noted that the file content review system 200 of FIG. 1 is only illustratively divided into the obtaining module 210, the extracting module 220, the determining module 230, the sending module 240 and the training module 250 according to functions. In fact, the file content review system 200 of FIG. 1 can also include more or fewer modules, for example, one of the above modules can be split into multiple functional modules, two or more of the above modules can be combined into one functional module, other functional modules can be added to the file content review system 200 of FIG. 1, and the present application does not make specific limitations thereon. The obtaining module 210, the extracting module 220, the determining module 230, the sending module 240 and the training module 250 can all be implemented by software or by hardware. Illustratively, the implementation of the extracting module 220 will be introduced below. Similarly, the implementation of the other modules can be referred to the implementation of the extracting module 220.

[0124] As an example of a software functional unit, the extracting module 220 can include code running on a computing instance. The computing instance can include at least one of a physical host (computing device), a virtual machine, and a container. Further, the computing instance can be one or more. For example, the extracting module 220 can include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers for running the code can be distributed in the same region, or in different regions. Further, the multiple hosts / virtual machines / containers for running the code can be distributed in the same availability zone (AZ), or in different AZs, each AZ including one data center or multiple data centers in close geographical proximity. Generally, one region can include multiple AZs. Similarly, the multiple hosts / virtual machines / containers for running the code can be distributed in the same virtual private cloud (VPC), or in multiple VPCs. Generally, one VPC is set in one region, and communication between two VPCs in the same region, or between VPCs in different regions, needs to be set in each VPC to set a communication gateway to realize the interconnection between VPCs.

[0125] As an example of a hardware functional unit, the extraction module 220 can include at least one computing device, such as a server. Alternatively, the extraction module 220 can also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), etc. The PLD can be implemented by a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0126] The plurality of computing devices included in the extraction module 220 can be distributed in the same region or in different regions. The plurality of computing devices included in the extraction module 220 can be distributed in the same AZ or in different AZs. Similarly, the plurality of computing devices included in the extraction module 220 can be distributed in the same VPC or in multiple VPCs. The plurality of computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0127] Based on the foregoing introduction of FIG. 1, the following describes an embodiment of the file content review method provided by the present application.

[0128] Referring to FIG. 5, FIG. 5 is a flowchart of a file content review method provided by the present application, including steps S501-S504.

[0129] S501, the file content review system 200 acquires a first file, the first file being a file that needs content review.

[0130] Optionally, the first file can be sent by a user to the file content review system 200 through the client 100, or can be acquired by the file content review system 200 from other modules or systems (such as a file system, a database system, etc.), which is not limited by the present application.

[0131] Optionally, the first file can be a text file (TXT), a Word document (DOCX), a portable document format (PDF) document, a presentation (PowerPoint, PPT), an Excel file, or a picture format (such as JPG, PNG, GIF, etc.) file, etc. The application does not limit the file type of the first file.

[0132] S502, the file content review system 200 extracts the first sentence from the first file according to the first keyword.

[0133] Optionally, the file content review system 200 obtains a scanning rule, the scanning rule is used to indicate that the sentence meeting the sentence range is extracted, and then the first file is divided into a plurality of sentences (including the first sentence) meeting the sentence range, and the first sentence is extracted according to the first keyword. The above-mentioned sentence range can be a sentence (identified by a period), a plurality of continuous sentences, a paragraph (identified by a line break), X paragraphs (X is a positive integer), Y characters (Y is a positive integer), etc. The application does not make specific limitations. The specific sentence range can be specified in the scanning rule, or the sentence range can not be specified in the scanning rule (at this time, the file content review system 200 uses the default sentence range), and the application does not make limitations.

[0134] Optionally, the scanning rule and the first sentence meet one of the following conditions:

[0135] (1) The scanning rule includes the first keyword and the first identification type, the first identification type is used to indicate that the sentence containing the first keyword and meeting the sentence range is extracted, and the first sentence contains the first keyword;

[0136] (2) The scanning rule includes a plurality of keywords and a second identification type, the second identification type is used to indicate that the sentence containing the plurality of keywords and meeting the sentence range is extracted, and the first sentence contains the plurality of keywords;

[0137] (3) The scanning rule includes a keyword set and a third identification type, the keyword set includes the first keyword, the third identification type is used to indicate that the sentence containing the number of keywords in the keyword set greater than or equal to the hit number threshold and meeting the sentence range is extracted, the hit number threshold is less than the number of keywords in the keyword set, the first sentence contains the first keyword, and the number of keywords in the keyword set contained in the first sentence is greater than or equal to the hit number threshold.

[0138] The above (1)-(3) are actually three setting modes of the scanning rule. Mode (1) is that a keyword and the first identification type are specified in the scanning rule, at this time the scanning rule indicates extracting the sentence with the keyword and satisfying the sentence range, that is, for any sentence in the file satisfying the sentence range, the sentence can be extracted as long as the keyword specified in the scanning rule appears in the sentence. Mode (2) is that multiple keywords and the second identification type are specified in the scanning rule, at this time the scanning rule indicates extracting the sentence with the multiple keywords and satisfying the sentence range, that is, for the sentence in the file satisfying the sentence range, the sentence can be extracted only if the multiple keywords specified in the scanning rule appear in the sentence at the same time. Mode (3) is that a keyword set (including N keywords, N is a positive integer greater than 1) and the third identification type are specified in the scanning rule, at this time the scanning rule indicates extracting the sentence with M keywords (M is a hit quantity threshold, M is a positive integer, M is less than N) of the N keywords and satisfying the sentence range, that is, for the sentence in the file satisfying the sentence range, the sentence can be extracted as long as any M keywords of the N keywords appear in the sentence. M can be given in the scanning rule or can be a default setting of the file content review system 200 (such as default M=N-P, P is a positive integer), which is not limited in the present application.

[0139] Optionally, the scanning rule further includes auxiliary judgment information, and the auxiliary judgment information is used to indicate the professional field and / or regulation corresponding to the scanning rule. The auxiliary judgment information can be referred to the foregoing description, and details are not described herein.

[0140] S503, the file content review system 200 determines a second sentence according to the first sentence, the similarity between the second sentence and the first sentence satisfies a similarity threshold, and risk information of the second sentence is used as risk information of the first sentence, the risk information being used to indicate whether the corresponding sentence is risk content.

[0141] It should be understood that the first sentence is a sentence extracted from a file requiring content review, and the risk information of the first sentence needs to be determined. The file content review system 200 stores a plurality of sentence samples (including the second sentence) marked with risk information, and the risk information is used to indicate whether the corresponding sentence sample is risk content.

[0142] The file content review system 200 can determine the risk information of the first sentence by calculating the similarity between the first sentence and the sentence samples. If the similarity between the first sentence and the second sentence satisfies the similarity threshold, the second sentence is considered as a similar sentence of the first sentence, and the risk information of the second sentence is taken as the risk information of the first sentence, which is equivalent to labeling the first sentence and the second sentence with the same risk information label. That is, if the risk information marked by the second sentence indicates that the second sentence is risk content, the semantic recognition model 260 determines that the first sentence is also risk content, and if the risk information marked by the second sentence indicates that the second sentence is not risk content, the semantic recognition model 260 determines that the first sentence is also not risk information.

[0143] Optionally, before step S503, the file content review system 200 can determine the similarity threshold in the following manner. First, a plurality of sentences (including the second sentence) having the same risk information are obtained, and then the plurality of sentences are converted into a plurality of feature vectors, and the plurality of sentences and the plurality of feature vectors are one-to-one corresponding. How to convert a sentence into a corresponding feature vector can be referred to the foregoing description, which will not be repeated here. Then, the file content review system 200 calculates the similarity between the plurality of sentences according to the plurality of feature vectors, obtains a plurality of similarities, and then determines the similarity threshold according to the plurality of similarities, such as taking the average of the plurality of similarities as the similarity threshold, or taking the minimum value of the plurality of similarities as the similarity threshold, and the like.

[0144] Optionally, the plurality of sentences all contain keywords belonging to a first word category, the first sentence includes a first keyword, and the first keyword also belongs to the first word category. That is, the file content review system 200 distinguishes different word categories, and calculates the similarity between the sentences having the same risk information and containing keywords belonging to the same word category, and then determines the similarity threshold corresponding to the word category according to the calculated similarity. Each word category can calculate the corresponding similarity threshold in a similar manner. When the file content review system 200 extracts the first sentence containing the first keyword from the file, other sentences having the same word category and risk information are obtained according to the word category to which the first keyword belongs, and then the similarity between the first sentence and the other sentences is calculated, and the calculated similarity is compared with the similarity threshold corresponding to the word category, so as to accurately determine the risk information of the first sentence.

[0145] How the file content review system 200 determines the risk information of the sentence can also be referred to the semantic recognition model 260 described above, which will not be repeated here.

[0146] S504, the file content review system 200 sends the review result of the first file to the user, and the review result of the first file includes the first sentence and the risk information of the first sentence.

[0147] It should be understood that in addition to the risk information of the first sentence, the review result of the first file can also include the risk information of other sentences in the first file, and the extraction manner of the other sentences and the manner of determining the risk information are similar to the first sentence, which will not be described here. As for the presentation manner of the risk information, the present application does not make any limitation.

[0148] For example, as described in the foregoing, the first sentence includes the first keyword. If the risk information of the first sentence received by the client 100 from the file content review system 200 indicates that the first sentence is risk content, the client 100 can mark the first keyword in the first sentence in the first file with red when displaying the first file, so as to indicate that the first sentence containing the first keyword is risk content. If the risk information of the first sentence indicates that the first sentence is not risk content, the client 100 can mark the first keyword in the first sentence in the first file with green when displaying the first file, so as to indicate that the first sentence containing the first keyword is not risk content. Other sentences in the first file can also be presented in a similar manner to indicate whether the sentence is risk content.

[0149] It should be noted that in addition to distinguishing risk content and non-risk content by different colors, risk content and non-risk content can also be distinguished by different fonts, symbols or other manners, which are not limited by the present application.

[0150] Optionally, the file review result of the first file also includes auxiliary judgment information, and the auxiliary judgment information is derived from the scanning rule in step S501. The auxiliary judgment information is used to indicate the professional field and / or regulation corresponding to the scanning rule. Combined with the auxiliary judgment information, the user can better understand the risk information of the first sentence.

[0151] Optionally, the file content review system 200 can also obtain a second file, and the second file is also a file that needs content review. In the case that the file format of the second file does not conform to the file type configured by the file content review system 200, the scanning time of the second file exceeds the first time threshold, or the waiting time of the second file exceeds the second time threshold, the user is fed back that the second file cannot be processed temporarily. Here, the fuse protection mechanism introduced in the foregoing can be referred to, which will not be described here.

[0152] In summary, in the file content review method provided in the present application, the file content review system 200 first extracts a sentence from the file based on a keyword, and then calculates the similarity between the extracted sentence and other sentences marked with risk information. If the similarity between the extracted sentence and a certain sentence marked with risk information meets the similarity threshold, the two sentences are considered to be similar sentences, and the risk information of the sentence marked with risk information is taken as the risk information of the currently extracted sentence. This way can effectively identify the risk content in the file.

[0153] Referring to FIG. 6, the present application further provides a computing device 600, which includes a bus 602, a processor 604, a memory 606 and a communication interface 608. The processor 604, the memory 606 and the communication interface 608 communicate with each other through the bus 602. The computing device 600 can be a server, a notebook computer, a tablet computer, a desktop computer, an edge device, a smart phone, a smart screen, etc., and the present application does not make specific limitations, nor does the present application limit the number of processors and memories in the computing device 600.

[0154] The bus 602 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is shown in FIG. 6, but it does not mean that there is only one bus or only one type of bus. The bus 602 can include a path for transmitting information between various components (e.g., the memory 606, the processor 604, the communication interface 608) of the computing device 600.

[0155] The processor 604 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0156] The memory 606 can include volatile memory (volatile memory), such as random access memory (RAM). The processor 604 can also include non-volatile memory (non-volatile memory), such as read-only memory (ROM), flash memory, a hard disk drive (HDD) or a solid state drive (SSD).

[0157] The executable program code is stored in the memory 606. The processor 604 executes the executable program code to realize the functions of the acquisition module 210, the extraction module 220, the determination module 230, the sending module 240 and the training module 250 in FIG. 1, respectively, to realize the method steps in FIG. 5 of the present application.

[0158] The communication interface 608 uses a transceiver module such as but not limited to a network interface card and a transceiver to realize the communication between the computing device 600 and other devices or communication networks.

[0159] It should be understood that the computing device 600 according to the present application can correspond to the file content review system 200 shown in FIG. 1 of the present application, and each module of the computing device 600 is used to realize the operation steps in the file content review method of FIG. 5. In addition, the computing device 600 can correspond to the file content review system 200 shown in FIG. 1, which is used to realize the functions of each module in the file content review system 200. For the sake of brevity, it will not be repeated here.

[0160] As a possible implementation, the computing device 600 can also include a chip system including a processor and a power supply circuit for supplying power to the processor, which is used to execute the operation steps in the file content review method of FIG. 5. For the sake of brevity, it will not be repeated here. Among them, the processor can be realized by CPU, also can be realized by GPU, DPU, NPU, XPU, SoC, offload card, acceleration card and other computing devices or AI chips.

[0161] As a possible implementation, the computing device 600 can include multiple types of processors 604, that is, the computing device 600 is a heterogeneous device, for example, the computing device 600 includes CPU and GPU, and at least one of the processors 604 in the computing device 600 can execute the operation steps in the method of FIG. 5. For the sake of brevity, it will not be repeated here.

[0162] As shown in FIG. 7, the present application also provides a computing device cluster, which includes at least one computing device 600. The memory 606 in one or more computing devices 600 in the computing device cluster can store the same instructions for implementing the method of FIG. 5.

[0163] In some possible implementation manners, the memory 606 in one or more computing devices 600 in the computing device cluster can also respectively store partial instructions for implementing the method of FIG. 5. In other words, the combination of one or more computing devices 600 can collectively execute the instructions for implementing the method of FIG. 5.

[0164] It should be noted that the memory 606 in different computing devices 600 in the computing device cluster can store different instructions, respectively for performing partial functions of the file content review system 200 of FIG. 1. That is, the instructions stored in the memory 606 in different computing devices 600 can implement the functions of one or more of the obtaining module 210, the extracting module 220, the determining module 230, the sending module 240 and the training module 250.

[0165] In some possible implementation manners, one or more computing devices in the computing device cluster can be connected through a network. The network can be a wide area network or a local area network, etc. FIG. 8 shows one possible implementation manner. As shown in FIG. 8, two computing devices 600A and 600B are connected through a network. Specifically, the communication interface in each computing device is connected to the network. In this type of possible implementation manner, the memory 606 in the computing device 600A stores instructions for performing the functions of the obtaining module 210 and the sending module 240. Meanwhile, the memory 606 in the computing device 600B stores instructions for performing the functions of the extracting module 220, the determining module 230 and the training module 250.

[0166] It should be understood that the functions of the computing device 600A shown in FIG. 8 can also be completed by multiple computing devices 600. Similarly, the functions of the computing device 600B can also be completed by multiple computing devices 600.

[0167] The present application also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to the connection manner of the computing device cluster described with reference to FIG. 8. The difference is that the memory 606 in one or more computing devices 600 in the computing device cluster can store the same instructions for implementing the method of FIG. 5.

[0168] In some possible implementations, partial instructions for implementing the method of FIG. 5 can also be respectively stored in the memory 606 of one or more computing devices 600 in the computing device cluster. In other words, the combination of one or more computing devices 600 can collectively execute the instructions for implementing the method of FIG. 5.

[0169] The present application also provides a computer readable storage medium. The computer readable storage medium can be any available medium or data storage device that can be accessed by a computing device, such as a data center containing one or more available media. The available media can be a magnetic medium, such as a floppy diskette, a hard disk drive, a magnetic tape, an optical medium, such as a DVD, or a semiconductor medium, such as a solid state hard drive, etc. The computer readable storage medium includes instructions that instruct a computing device to perform the method steps in FIG. 5.

[0170] The present application also provides a computer program product containing instructions. The computer program product can be a software or program product containing instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device is caused to perform the operation steps in the file content review method of FIG. 5.

[0171] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those of ordinary skill in the art should understand: they can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A file content review method characterized by, The method is executed by a file content review system, and the method comprises: obtaining a first file, the first file being a file requiring content review; extracting a first sentence from the first file according to a first keyword; determining a second sentence according to the first sentence, the similarity between the first sentence and the second sentence satisfying a similarity threshold; taking risk information of the second sentence as risk information of the first sentence, the risk information being used to indicate whether a corresponding sentence is risk content.

2. The method of claim 1, wherein, Before the step of determining the second sentence according to the first sentence, the method further comprises: obtaining a plurality of sentences including the second sentence, the plurality of sentences having the same risk information; converting the plurality of sentences into a plurality of feature vectors, the plurality of sentences and the plurality of feature vectors corresponding to each other; calculating the similarity between each two of the plurality of sentences according to the plurality of feature vectors, to obtain a plurality of similarities; determining the similarity threshold according to the plurality of similarities.

3. The method of claim 2, wherein, The plurality of sentences all contain a keyword belonging to a first word category, and the first sentence includes the first keyword, the first keyword belonging to the first word category.

4. The method according to any one of claims 1 to 3, characterized in that, The step of extracting the first sentence from the first file according to the first keyword comprises: obtaining a scanning rule, the scanning rule being used to indicate that a sentence satisfying a sentence range is extracted; dividing the first file into a plurality of sentences satisfying the sentence range, the plurality of sentences satisfying the sentence range including the first sentence; extracting the first sentence according to the first keyword.

5. The method of claim 4, wherein, The scanning rule and the first sentence satisfy one of the following conditions: the scanning rule includes the first keyword and a first identification type, the first identification type being used to indicate that a sentence containing the first keyword and satisfying the sentence range is extracted, and the first sentence contains the first keyword; the scanning rule includes a plurality of keywords including the first keyword and a second identification type, the second identification type being used to indicate that a sentence containing the plurality of keywords and satisfying the sentence range is extracted, and the first sentence contains the plurality of keywords; the scanning rule includes a keyword set including the first keyword and a third identification type, the third identification type being used to indicate that a sentence containing a number of keywords in the keyword set greater than or equal to a hit number threshold and satisfying the sentence range is extracted, the hit number threshold being less than the number of keywords in the keyword set, and the first sentence contains the number of keywords in the keyword set greater than or equal to the hit number threshold.

6. A file content review system characterized by, comprise: an obtaining module, configured to obtain a first file, the first file being a file requiring content review; an extracting module, configured to extract a first sentence from the first file according to a first keyword; a determining module, configured to determine a second sentence according to the first sentence, the similarity between the first sentence and the second sentence satisfying a similarity threshold; The determination module is further configured to determine the risk information of the second sentence as the risk information of the first sentence, the risk information being used to indicate whether the corresponding sentence is risk content.

7. The system of claim 6, wherein, The obtaining module is further configured to obtain a plurality of sentences including the second sentence, the plurality of sentences having the same risk information. The determination module is further configured to: convert the plurality of sentences into a plurality of feature vectors, the plurality of sentences and the plurality of feature vectors corresponding to each other; calculate similarities between the plurality of sentences according to the plurality of feature vectors, to obtain a plurality of similarities; determine the similarity threshold according to the plurality of similarities.

8. The system of claim 7, wherein, The plurality of sentences all include a first keyword belonging to a first word category, and the first sentence includes the first keyword belonging to the first word category.

9. The system of any one of claims 6 to 8, wherein, The extraction module is specifically configured to: obtain a scanning rule, the scanning rule being used to indicate extraction of a sentence satisfying a sentence range; divide the first file into a plurality of sentences satisfying the sentence range, the plurality of sentences satisfying the sentence range including the first sentence; extract the first sentence according to the first keyword.

10. The system of claim 9, wherein, The scanning rule and the first sentence satisfy one of the following conditions: the scanning rule includes the first keyword and a first identification type, the first identification type being used to indicate extraction of a sentence containing the first keyword and satisfying the sentence range, and the first sentence contains the first keyword; the scanning rule includes a plurality of keywords including the first keyword and a second identification type, the second identification type being used to indicate extraction of a sentence containing the plurality of keywords and satisfying the sentence range, and the first sentence contains the plurality of keywords; the scanning rule includes a keyword set including the first keyword and a third identification type, the third identification type being used to indicate extraction of a sentence containing a number of keywords in the keyword set greater than or equal to a hit number threshold and satisfying the sentence range, the hit number threshold being less than the number of keywords in the keyword set, and the first sentence contains the number of keywords in the keyword set greater than or equal to the hit number threshold.

11. A computing device, comprising: The computing device includes a processor and a memory; The processor is configured to execute instructions stored in the memory, so that the computing device performs the method of any one of claims 1 to 5.

12. A cluster of computing devices, characterized in that, The computing device includes a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method of any one of claims 1 to 5.

13. A chip system, characterized by The chip system includes a processor and a power supply circuit, the power supply circuit being used to supply power to the processor, and the processor being used to execute the method of any one of claims 1 to 5.

14. A computer program product comprising instructions, characterized in that, When the instructions are executed by the computing device cluster, the computing device cluster performs the method of any one of claims 1 to 5.

15. A computer readable storage medium, characterized in that, comprising computer program instructions which, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method of any one of claims 1 to 5.

Images