File detection method, device, processor and electronic device

Abstract

The application discloses a file detection method and device, a processor and an electronic device. It relates to the field of financial technology. The method comprises the following steps: obtaining a target file to be detected, determining a detection requirement of the target file, determining a text detection model according to the detection requirement, wherein the text detection model is a first text detection model or a second text detection model; inputting the target file into the text detection model, outputting a detection result containing M candidate detection labels; screening a comparison file associated with the M candidate detection labels from a database, obtaining a reference detection label carried by the comparison file, and adjusting the M candidate detection labels by using the reference detection label to obtain a detection label of the target file. Through the application, the problem of low recognition accuracy in identifying the detection label of the file in the related art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of financial technology, in particular, to a file detection method and device, a processor and an electronic device. BACKGROUND

[0002] With the rapid development of network information, unstructured text data has increased dramatically, while providing more available information for people, it has become more difficult for people to obtain the information they need from a large amount of text data. Therefore, how to mine important information from massive information is a problem that researchers urgently need to solve.

[0003] Therefore, in the related art, a large amount of text information is mainly screened by manual and recognition model acquisition methods to obtain the required information. However, when a large amount of text information is obtained, due to the large amount of information, it is necessary to consume a large amount of manpower and time to collect and classify text data by manual methods. At the same time, the training method of the recognition model also has many problems: when information recognition operations are performed on information in different fields, the recognition model needs to be retrained, which will waste a lot of time cost and it is difficult to obtain the required recognition result.

[0004] In view of the problem of low recognition accuracy in detecting the detection label of the file in the related art, no effective solution has been proposed so far. SUMMARY

[0005] The main purpose of the present application is to provide a file detection method, device, processor and electronic device to solve the problem of low recognition accuracy in detecting the detection label of the file in the related art.

[0006] In order to achieve the above purpose, according to one aspect of the present application, a file detection method is provided. The method comprises: obtaining a target file to be detected, and determining the detection requirements of the target file; determining a text detection model according to the detection requirements, wherein the text detection model is a first text detection model or a second text detection model, the first text detection model is used to detect chapter text in the file, and the second text detection model is used to detect paragraph text in the file; inputting the target file into the text detection model, and outputting a detection result containing M candidate detection labels, wherein each candidate detection label is used to label the detection rule that the target file meets, the text detection model is obtained by training a preset text detection model with a plurality of sample data, and the sample data includes a sample file and a detection label associated with the sample file; filtering out a comparison file associated with the M candidate detection labels from a database, obtaining a reference detection label carried by the comparison file, and adjusting the M candidate detection labels using the reference detection label to obtain a detection label of the target file.

[0007] Optionally, determining the text detection model according to the detection requirement comprises: identifying the detection requirement to obtain a label granularity of the target file, wherein the label granularity comprises a first granularity and a second granularity, the first granularity indicates that the file needs to meet a detection rule of a chapter text, and the second granularity indicates that the file needs to meet a detection rule of a paragraph text; in a case where the label granularity of the target file is the first granularity, determining the first text detection model as the text detection model; and in a case where the label granularity of the target file is the second granularity, determining the second text detection model as the text detection model.

[0008] Optionally, before the target file to be detected is obtained and the detection requirement of the target file is determined, the method further comprises: determining a domain type to which the target file belongs, and determining a plurality of preset detection rules according to the domain type, wherein the preset detection rules are detection rules that need to be met by files in the domain type; and adding the plurality of preset detection rules to an initial text detection model to obtain a preset text detection model, wherein the initial text detection model comprises a general file detection rule.

[0009] Optionally, the text detection model is obtained by: obtaining N sample files and X detection labels associated with each sample file, processing the N sample files and the X detection labels associated with each sample file to obtain N sample data, randomly dividing the N sample data to obtain Y sample data and Z sample data, wherein X=Y+Z, and X, Y and Z are positive integers, taking the Y sample data as training set data, training the preset text detection model by using the training set data to obtain a trained text detection model, taking the Z sample data as test set data, determining a model index of the trained text detection model according to the test set data, and in a case where the model index meets a preset requirement, determining the trained text detection model as the text detection model, wherein the model index at least comprises one of a model precision rate and a model recall rate.

[0010] Optionally, taking the Z sample data as test set data and determining a model index of the trained text detection model according to the test set data comprises: inputting Z sample files in the Z sample data into the trained text detection model to obtain training labels associated with each sample file, comparing the training labels with detection labels associated with each sample file, in a case where the training labels are consistent with the detection labels associated with each sample file, determining a sample file corresponding to the training labels as a correct prediction sample, in a case where the training labels are inconsistent with the detection labels associated with each sample file, determining a sample file corresponding to the training labels as an incorrect prediction sample, and calculating the model precision rate and the model recall rate by using a value of the correct prediction sample, a value of the incorrect prediction sample and the Z sample data.

[0011] Optionally, inputting the target file into the text detection model, and outputting the detection result containing M candidate detection labels includes: determining an evaluation probability value of each candidate detection label, to obtain M evaluation probability values, wherein the evaluation probability value is used to represent the accuracy of each candidate detection label; sorting the M evaluation probability values in descending order to obtain a probability arrangement, selecting the evaluation probability values before the preset position in the probability arrangement, combining the candidate detection labels associated with the evaluation probability values before the preset position, and determining the combined candidate detection labels as the detection result.

[0012] Optionally, adjusting the M candidate detection labels by using the reference detection label to obtain the detection label of the target file includes: feeding the reference detection label and the M candidate detection labels to the detection personnel, and obtaining the detection label fed back by the detection personnel; and determining the detection label fed back by the detection personnel as the detection label of the target file.

[0013] Optionally, adjusting the M candidate detection labels by using the reference detection label to obtain the detection label of the target file includes: performing intersection calculation on the reference detection label and the M candidate detection labels to obtain the detection label of the target file.

[0014] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a file detection device is provided. The device comprises: an acquisition unit configured to acquire a target file to be detected, and determine a detection requirement of the target file, and determine a text detection model according to the detection requirement, wherein the text detection model is a first text detection model or a second text detection model, the first text detection model is used to detect chapter text in a file, and the second text detection model is used to detect paragraph text in a file; an input unit configured to input the target file into the text detection model, and output a detection result containing M candidate detection labels, wherein each candidate detection label is used to label a detection rule met by the target file, the text detection model is obtained by training a preset text detection model by using a plurality of sample data, and the sample data includes a sample file and a detection label associated with the sample file; and a screening unit configured to screen out a comparison file associated with the M candidate detection labels from a database, obtain a reference detection label carried by the comparison file, and adjust the M candidate detection labels by using the reference detection label to obtain a detection label of the target file.

[0015] According to another aspect of the embodiment of the present application, a processor is also provided, which is used to run a program, wherein the program controls a device in which a non-volatile storage medium is located to perform a file detection method when the program is running.

[0016] According to another aspect of the embodiments of the present application, an electronic device is provided, comprising one or more processors and a memory; the memory stores computer readable instructions, and the processor is configured to execute the computer readable instructions, wherein the computer readable instructions perform a file detection method when executed.

[0017] According to the present application, the following steps are adopted: obtaining a target file to be detected, and determining a detection requirement of the target file; determining a text detection model according to the detection requirement, wherein the text detection model is a first text detection model or a second text detection model, the first text detection model is used to detect chapter text in the file, and the second text detection model is used to detect paragraph text in the file; inputting the target file into the text detection model, and outputting a detection result containing M candidate detection labels, wherein each candidate detection label is used to label a detection rule met by the target file, the text detection model is obtained by training a preset text detection model by using a plurality of sample data, and the sample data includes a sample file and a detection label associated with the sample file; selecting, from a database, a comparison file associated with the M candidate detection labels, obtaining a reference detection label carried by the comparison file, and adjusting the M candidate detection labels by using the reference detection label to obtain a detection label of the target file, thereby solving the problem of low recognition accuracy in recognizing the detection label of the file in the related art, and improving the detection accuracy of the model in detecting the label of the file by obtaining the detection requirement of the target file, obtaining the text detection model by using the detection requirement, outputting the candidate detection label by using the text detection model, and obtaining the detection label by screening the candidate detection label. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application illustrate the embodiments of the present application and, together with the description, serve to explain the application. In the drawings:

[0019] Figure 1 is a flowchart of a file detection method according to an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of a detection label according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a file detection model according to an embodiment of the present application Figure 1 ;

[0022] Figure 4 is a schematic diagram of a file detection model according to an embodiment of the present application Figure 2 ;

[0023] Figure 5is a schematic diagram of an optional file detection method provided by an embodiment of the application;

[0024] Figure 6 is a schematic diagram of a file detection device provided by an embodiment of the application;

[0025] Figure 7 is a schematic diagram of an electronic device provided by an embodiment of the application. DETAILED DESCRIPTION

[0026] It should be noted that the embodiments and features in the application can be combined with each other without conflict. The application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] In order to enable those skilled in the art to better understand the application scheme, the technical solutions in the embodiments of the application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the application. Obviously, the described embodiments are only a part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the application.

[0028] It should be noted that the terms "first", "second" and the like in the specification and claims of the application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0029] It should be noted that the related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, analyzed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties.

[0030] The application will be described below in combination with the preferred implementation steps, Figure 1 is a flowchart of a file detection method provided by an embodiment of the application, as Figure 1 shown, the method comprises the following steps:

[0031] In step S101, a target file to be detected is obtained, and a detection requirement of the target file is determined. A text detection model is determined according to the detection requirement, wherein the text detection model is a first text detection model or a second text detection model. The first text detection model is used to detect chapter text in a file, and the second text detection model is used to detect paragraph text in a file.

[0032] Specifically, the target file refers to a file that needs to be labeled. The target file can include relevant regulatory rule files in or outside a country. When a target file needs to be selected from multiple regulatory rule files, multiple regulatory rule files are obtained from a database in the field. Since the number of regulatory rule files is large, if manual selection is relied on, a large amount of human and time costs will be consumed. Therefore, each file is labeled, so that each regulatory rule file is associated with multiple labels. When the target file needs to be obtained, the target file can be obtained through the labeled labels.

[0033] It should be noted that the text detection model can use a BERT (Bidirectional Encoder Representations from Transformers) algorithm for data processing. The BERT algorithm is a language processing algorithm based on a transformer structure. The BERT model is improved so that the text detection model obtained by using the BERT model is suitable for the field of the target file. Specifically, before the BERT model is improved, the text recognition model obtained by using the BERT model can only recognize simple text, such as whether the text in the text is positive or negative. After the BERT model is improved by adding multiple regulatory rules in the financial field, the text detection model can detect whether the file in the financial field meets the regulatory rules. The first text detection model is obtained by adding low-precision regulatory rules to the BERT model, and the second text detection model is obtained by adding high-precision regulatory rules to the BERT model. The model is processed by using regulatory rules with different precisions, so that the text detection model outputs labels with different precisions.

[0034] In step S102, the target file is input into the text detection model, and a detection result containing M candidate detection labels is output, wherein each candidate detection label is used to label a detection rule met by the target file. The text detection model is obtained by training a preset text detection model with a plurality of sample data, and the sample data includes a sample file and a detection label associated with the sample file.

[0035] Specifically, the target file to be labeled is input into the trained text detection model, and the detection label associated with the file is obtained by screening the text detection model. Each candidate detection label obtained by screening is associated with an evaluation probability value, wherein the evaluation probability value refers to the accuracy of each detection label. The higher the evaluation probability value, the more the target file meets the detection rule corresponding to the detection label.

[0036] For example, when the detection requirement of the target file is low precision, the target file is input into the first text detection model, and a plurality of candidate detection labels associated with the target file are output by the first text detection model. The detection result of the target file is obtained according to the evaluation probability value of each candidate detection label, wherein the detection rule associated with the detection label is more; when the detection requirement of the target file is high precision, the target file is input into the second text detection model, and a plurality of detection labels associated with the target file are output by the second text detection model. The detection result of the target file is obtained according to the evaluation probability value of each candidate detection label, wherein the detection rule associated with the detection label is less.

[0037] In step S103, the contrast file associated with the M candidate detection labels is screened from the database, the reference detection label carried by the contrast file is obtained, and the M candidate detection labels are adjusted using the reference detection label to obtain the detection label of the target file.

[0038] After obtaining a plurality of candidate detection labels, further screening is needed to confirm the detection label of the target file. Specifically, the contrast file associated with the candidate detection label is screened from the database according to the candidate detection label. For example, if the candidate detection label is "financial transaction", the file related to "financial transaction" can be screened from the database as the contrast file, and the reference detection label carried by the contrast file also contains the label of "financial transaction".

[0039] Further, the reference detection label of the contrast file is compared with the candidate detection label, and the candidate detection label is added or deleted to obtain the detection label of the target file, for example, Figure 2 is a schematic diagram of a detection label provided by an embodiment of the present application, such as Figure 2As shown, an overseas financial regulatory document is input into a text detection model as a target file, a plurality of candidate detection labels are obtained from the text detection model, which are respectively architecture responsibility setting, policy system construction, risk assessment, suspicious transaction identification and early warning, etc., and a relevant comparison file is retrieved from a file database according to the candidate detection label, the comparison file is associated with a plurality of reference detection labels, which are respectively architecture responsibility setting, permission setting, risk assessment, suspicious transaction identification and early warning, and human resource allocation, etc., the comparison file and the candidate detection label are sent to a detection personnel, and five detection labels are obtained from the comparison file and the candidate detection label by the detection personnel, which are respectively architecture responsibility setting, policy system construction, risk assessment, suspicious transaction identification and early warning, and customer understanding, etc., and the labels are determined as the detection labels of the target file. By comparing the two detection labels, the detection rules applicable to the target file can be improved, and the staff can handle related work more conveniently.

[0040] The file detection method provided by the embodiment of the present application comprises the following steps: obtaining a target file to be detected, and determining a detection requirement of the target file; determining a text detection model according to the detection requirement, wherein the text detection model is a first text detection model or a second text detection model, the first text detection model is used to detect chapter text in a file, and the second text detection model is used to detect paragraph text in a file; inputting the target file into the text detection model, and outputting a detection result containing M candidate detection labels, wherein each candidate detection label is used to mark a detection rule met by the target file, the text detection model is obtained by training a preset text detection model by using a plurality of sample data, and the sample data comprises a sample file and a detection label associated with the sample file; screening a comparison file associated with the M candidate detection labels from a database, obtaining a reference detection label carried by the comparison file, and adjusting the M candidate detection labels by using the reference detection label, to obtain a detection label of the target file. The method solves the problem of low recognition accuracy in recognizing the detection label of a file in the related art. By obtaining the detection requirement of the target file, then obtaining the text detection model by using the detection requirement, outputting the candidate detection label by using the text detection model, and obtaining the detection label by screening and processing the candidate detection label, the detection accuracy of the model in label detection of a file is improved.

[0041] The type of the text detection model needs to be determined according to the required label granularity. Optionally, in the file detection method provided in the embodiments of the present application, determining the text detection model according to the detection requirement includes: identifying the detection requirement to obtain the label granularity of the target file, wherein the label granularity includes a first granularity and a second granularity, the first granularity represents that the file needs to meet the detection rule of the chapter text, and the second granularity represents that the file needs to meet the detection rule of the paragraph text; in the case where the label granularity of the target file is the first granularity, the first text detection model is determined as the text detection model; and in the case where the label granularity of the target file is the second granularity, the second text detection model is determined as the text detection model.

[0042] Specifically, the low precision degree refers to the first granularity, and the high precision degree refers to the second granularity. Since the detection label range obtained by marking the chapter is larger than the detection label range obtained by marking the paragraph, when the detection requirement of the target file is identified as the first granularity, it indicates that the detection precision of the detection label required by the target file is low, and a low-precision chapter detection model can be selected, that is, the first text detection model is used as the text detection model; when the detection requirement of the target file is identified as the second granularity, it indicates that the detection precision of the detection label required by the target file is high, and a high-precision paragraph detection model can be selected, that is, the second text detection model is used as the text detection model.

[0043] In order to obtain the text detection model corresponding to the target file, optionally, in the file detection method provided in the embodiments of the present application, before obtaining the target file to be detected and determining the detection requirement of the target file, the method further includes: determining the domain type to which the target file belongs, and determining a plurality of preset detection rules according to the domain type, wherein the preset detection rules are detection rules required to be met by the file in the domain type; adding the plurality of preset detection rules to an initial text detection model to obtain a preset text detection model, wherein the initial text detection model contains a general file detection rule.

[0044] Specifically, since the BERT model cannot be applied to each target file corresponding to the type to which it belongs, the BERT model needs to be modified before the file detection of a certain target file. Specifically, after identifying the domain type to which the target file belongs, the preset detection rule of the domain type is obtained, and the BERT model is processed according to the preset detection rule to obtain the required text detection model.

[0045] For example, when the domain type to which the target file belongs is the financial consumption domain, the preset detection rules of the financial consumption domain can include rules such as finance, consumption, and abnormal data. Before the target file is input into the text detection model, the plurality of preset detection rules are input into the initial text detection model, so that the text detection model is suitable for identification of file labels in the financial consumption domain.

[0046] The text detection model needs to be trained by using a plurality of sample files. Alternatively, in the file detection method provided in the embodiments of the present application, the text detection model is trained in the following manner: N sample files and X detection labels associated with each sample file are obtained, the N sample files and the X detection labels associated with each sample file are processed to obtain N sample data; the N sample data is randomly divided to obtain Y sample data and Z sample data, where X = Y + Z, and X, Y, and Z are positive integers; the Y sample data is used as training set data, the preset text detection model is trained by using the training set data to obtain a trained text detection model; the Z sample data is used as test set data, the model index of the trained text detection model is determined according to the test set data, and in the case where the model index meets a preset requirement, the trained text detection model is determined as the text detection model, wherein the model index at least includes one of the following: model precision and model recall.

[0047] It should be noted that, Figure 3 is a schematic diagram of a file detection model provided in the embodiments of the present application Figure 1 As shown in Figure 3 , the text detection model includes a first text detection model and a second text detection model. The first text detection model is composed of an encoder and a decoder. The encoder converts each input sentence into a vector, and then encodes the vector sentence by using a position encoding module to obtain the context information of each sentence. Then, the sentence is processed by a deep learning attention module, and the processed data is transmitted to the decoder module. The decoder performs decoding operation on the data transmitted by the encoder by using a masking deep learning attention module and a deep learning attention module respectively, and outputs the detection label of the target file. Figure 4 is a schematic diagram of a file detection model provided in the embodiments of the present application Figure 2 As shown in Figure 4 , the second text detection model identifies the label of the processed paragraph by using a self-attention mechanism after vectorizing and position encoding the input paragraph, and then obtains the detection label of the target file.

[0048] Figure 5 is a schematic diagram of an optional file detection method provided in the embodiments of the present application, as Figure 5It is to be noted that, before the target file is input into the text detection model, a preprocessing operation such as file cleaning needs to be performed, and after the candidate detection label output by the text detection model is obtained, the candidate detection label needs to be evaluated and adjusted, and finally the detection label of the target file is obtained.

[0049] Specifically, the training of the file detection model needs to use a plurality of sample files and corresponding true detection labels, and the plurality of sample files are randomly divided into two groups to obtain training set data and test set data. After the sample files in the training set data are input into the preset text detection model, the simulated detection label of the sample files is output, and the simulated detection label and the true detection label corresponding to the sample files are processed. The processing result is used to adjust the parameters in the text detection model, and thus a trained text detection model is obtained.

[0050] Further, the trained text detection model is verified by using a plurality of sample files in the test set data. After the sample files in the test set data are input into the trained text detection model, the model detection label is output. The simulated detection label and the true detection label corresponding to the sample files are calculated and processed to obtain a model index. When the model index does not meet the preset requirement, the parameters of the trained text detection model are adjusted until the model index meets the preset requirement, and the adjusted text detection model is used as the text detection model.

[0051] The model index is calculated from the test set data and the data output by the text detection model. Alternatively, in the file detection method provided in the embodiments of the present application, Z sample data are used as test set data, and the model index of the trained text detection model is determined according to the test set data, including: inputting the Z sample files in the Z sample data into the trained text detection model to obtain the training label associated with each sample file; comparing the training label with the detection label associated with each sample file, in the case where the training label and the detection label associated with each sample file are consistent, the sample file corresponding to the training label is determined as a prediction correct sample; in the case where the training label and the detection label associated with each sample file are inconsistent, the sample file corresponding to the training label is determined as a prediction error sample; the number of prediction correct samples, the number of prediction error samples, and the Z sample data are used for calculation to obtain the model precision and the model recall rate.

[0052] Specifically, the comparison between the simulated detection label output by the text detection model and the real detection label can obtain the model index. For example, if the detection label of a sample file includes the real detection label and the simulated detection label output by the text detection model, the number of correct predictions, the total number of predicted labels A, and the total number of real labels A are calculated according to the two sets of labels. The number of correct predictions refers to the consistency of the results obtained by the simulated detection label and the real detection label. The total number of predicted labels A refers to the total number of simulated detection labels A obtained by the model, including the number of correct predictions and the number of incorrect predictions. The total number of actual A refers to the total number of real detection labels A of the sample, including the number of correct predictions and the number of incorrect predictions. The model precision is calculated according to the number of correct predictions and the total number of predicted labels A. The model recall rate is calculated using the number of correct predictions and the total number of real labels A. When the model precision is greater than the first preset threshold or the model recall rate is less than the second preset threshold, the trained text detection model is determined as the text detection model, that is, the training is completed.

[0053] The embodiment quantitatively displays the pros and cons of model training, can accurately know the effect of model training, and further obtains a text detection model with better effect.

[0054] The detection result includes a plurality of evaluation probability values. In the file detection method provided in the embodiment of the present application, the target file is input into the text detection model, and the detection result including M candidate detection labels is output. The evaluation probability value of each candidate detection label is determined to obtain M evaluation probability values, wherein the evaluation probability value is used to represent the accuracy of each candidate detection label. The M evaluation probability values are sorted in descending order to obtain a probability arrangement. The evaluation probability values before the preset position in the probability arrangement are selected. The candidate detection labels associated with the evaluation probability values before the preset position are combined, and the combined candidate detection labels are determined as the detection result.

[0055] The detection label of the target file is determined according to the evaluation probability value of each candidate detection label as the determination standard. Specifically, after obtaining the evaluation probability value of each candidate detection label, each evaluation probability value is sorted in descending order. Since the evaluation probability value refers to the degree of conformity of the target file to the detection rule corresponding to the candidate detection label, the higher the evaluation probability value, the more the target file conforms to the detection rule. Therefore, the evaluation probability values before the preset position in the arrangement are selected after sorting in descending order, and the candidate detection labels are selected. For example, the evaluation probability values in the top fifty percent of the arrangement can be selected, and the candidate detection labels corresponding to the evaluation probability values in the top fifty percent of the arrangement are obtained. The candidate detection labels are combined as the detection result.

[0056] The acquisition manner of the detection label can include multiple manners. Optionally, in the file detection method provided in the embodiments of the present application, the M candidate detection labels are adjusted by using the reference detection label to obtain the detection label of the target file, which includes: performing intersection calculation on the reference detection label and the M candidate detection labels to obtain the detection label of the target file.

[0057] Specifically, when multiple comparison files and multiple reference detection labels associated with each comparison file are acquired, the intersection of the multiple reference detection labels and the candidate detection labels is taken as the detection label of the target file. For example, when the reference detection labels of the comparison files are A, B, C and D, and the candidate detection labels determined according to the text detection model are B, C, D and E, the intersection calculation is performed on the eight detection labels, and the three detection labels B, C and D are determined as the detection label of the target file.

[0058] In order to improve the accuracy of determining the detection label of the target file, the opinions of the detection personnel can be referred to. Optionally, in the file detection method provided in the embodiments of the present application, the M candidate detection labels are adjusted by using the reference detection label to obtain the detection label of the target file, which includes: feeding the reference detection label and the M candidate detection labels to the detection personnel, acquiring the detection label fed back by the detection personnel, and determining the detection label fed back by the detection personnel as the detection label of the target file.

[0059] Specifically, the comparison file refers to a file associated with multiple same labels of the candidate detection label. After the comparison file similar to the candidate detection label is filtered out according to the multiple candidate detection labels, the detection label of the target file can be further determined according to the comparison file. Specifically, the detection personnel can view the candidate detection label, the comparison file and the target file to further obtain the detection label fed back by the detection personnel. For example, an overseas financial regulatory file is input into a text detection model as a target file, and four candidate detection labels A, B1, C and D are obtained by the text detection model. Then, a relevant comparison file is retrieved from a file database according to the candidate detection label, and the comparison file is associated with five reference detection labels A, B2, C, D and F, including information technology application, bank card withdrawal exception, bank card risk assessment, transaction warning and bank card permission setting. After the comparison file and the candidate detection label are sent to the detection personnel, four detection labels A, B, C and D are obtained by the detection personnel based on the comparison file and the candidate detection label. Therefore, the labels are determined as the detection label of the target file, wherein B1 and B2 are different direction detection labels under the same type.

[0060] It is noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0061] The embodiment of the present application also provides a file detection device. It is noted that the file detection device of the embodiment of the present application can be used to execute the file detection method provided by the embodiment of the present application. The file detection device provided by the embodiment of the present application is introduced as follows.

[0062] Figure 6 is a schematic diagram of the file detection device provided by the embodiment of the present application, as shown in the figure, the device comprises an acquisition unit 60, an input unit 61 and a screening unit 62. Figure 6

[0063] The acquisition unit 60 is configured to acquire a target file to be detected, determine a detection requirement of the target file, and determine a text detection model according to the detection requirement, wherein the text detection model is a first text detection model or a second text detection model, the first text detection model is used to detect chapter text in a file, and the second text detection model is used to detect paragraph text in a file.

[0064] The input unit 61 is configured to input the target file into the text detection model, and output a detection result containing M candidate detection labels, wherein each candidate detection label is used to mark a detection rule met by the target file, the text detection model is obtained by training a preset text detection model by using a plurality of sample data, and the sample data comprises a sample file and a detection label associated with the sample file.

[0065] The screening unit 62 is configured to screen out, from a database, a comparison file associated with the M candidate detection labels, acquire a reference detection label carried by the comparison file, and adjust the M candidate detection labels by using the reference detection label to obtain a detection label of the target file.

[0066] Optionally, in the file detection device provided by the embodiment of the present application, the acquisition unit 60 comprises an identification module, a first determination module and a second determination module. The identification module is configured to identify the detection requirement to obtain a label granularity of the target file, wherein the label granularity comprises a first granularity and a second granularity, the first granularity represents that the file needs to meet a detection rule of chapter text, and the second granularity represents that the file needs to meet a detection rule of paragraph text. The first determination module is configured to determine the first text detection model as the text detection model in a case where the label granularity of the target file is the first granularity. The second determination module is configured to determine the second text detection model as the text detection model in a case where the label granularity of the target file is the second granularity.

[0067] ​Optionally, in the file detection device provided by the embodiment of the present application, the device further comprises a determination unit configured to determine a domain type to which the target file belongs before acquiring the target file to be detected and determining detection requirements of the target file, and determine a plurality of preset detection rules according to the domain type, wherein the preset detection rules are detection rules required to be met by files in the domain type; and an adding unit configured to add the plurality of preset detection rules to an initial text detection model to obtain a preset text detection model, wherein the initial text detection model comprises a general file detection rule.

[0068] Optionally, in the file detection device provided by the embodiment of the present application, the text detection model is obtained by the following method: an acquisition module configured to acquire N sample files and X detection labels associated with each sample file, process the N sample files and the X detection labels associated with each sample file to obtain N sample data; a division module configured to randomly divide the N sample data to obtain Y sample data and Z sample data, wherein X=Y+Z, and X, Y and Z are positive integers; a training module configured to use the Y sample data as training set data, train the preset text detection model by using the training set data to obtain a trained text detection model; and a third determination module configured to use the Z sample data as test set data, determine a model index of the trained text detection model according to the test set data, and determine the trained text detection model as the text detection model in a case where the model index meets a preset requirement, wherein the model index at least comprises one of a model precision rate and a model recall rate.

[0069] Optionally, in the file detection device provided by the embodiment of the present application, the input unit 61 comprises: an input module configured to input Z sample files in the Z sample data to the trained text detection model to obtain training labels associated with each sample file; a comparison module configured to compare the training labels with the detection labels associated with each sample file, and determine a sample file corresponding to a training label as a prediction correct sample in a case where the training label is consistent with the detection label associated with each sample file; a fourth determination module configured to determine a sample file corresponding to a training label as a prediction error sample in a case where the training label is inconsistent with the detection label associated with each sample file; and a first calculation module configured to calculate the model precision rate and the model recall rate by using a value of the prediction correct sample, a value of the prediction error sample and the Z sample data.

[0070] Optionally, in the file detection device provided by the embodiment of the present application, the input unit 61 comprises: a fifth determination module, configured to determine an evaluation probability value of each candidate detection label, to obtain M evaluation probability values, wherein the evaluation probability value is used to represent the accuracy of each candidate detection label; and a sorting module, configured to sort the M evaluation probability values in descending order to obtain a probability arrangement, select the evaluation probability values before a preset position in the probability arrangement, combine the candidate detection labels associated with the evaluation probability values before the preset position, and determine the combined candidate detection labels as the detection result.

[0071] Optionally, in the file detection device provided by the embodiment of the present application, the screening unit 62 comprises: a feedback module, configured to feed back the reference detection label and the M candidate detection labels to the detection personnel, and obtain the detection label fed back by the detection personnel; and a sixth determination module, configured to determine the detection label fed back by the detection personnel as the detection label of the target file.

[0072] Optionally, in the file detection device provided by the embodiment of the present application, the screening unit 62 comprises: a second calculation module, configured to calculate the intersection of the reference detection label and the M candidate detection labels to obtain the detection label of the target file.

[0073] The file detection device provided by the embodiment of the present application comprises: an obtaining unit 60, configured to obtain a target file to be detected, and determine the detection requirement of the target file, and determine a text detection model according to the detection requirement, wherein the text detection model is a first text detection model or a second text detection model, the first text detection model is used to detect chapter text in a file, and the second text detection model is used to detect paragraph text in a file; an input unit 61, configured to input the target file into the text detection model, and output a detection result comprising M candidate detection labels, wherein each candidate detection label is used to label the detection rule met by the target file, the text detection model is obtained by training a preset text detection model by using a plurality of sample data, and the sample data comprises a sample file and a detection label associated with the sample file; and a screening unit 62, configured to screen a comparison file associated with the M candidate detection labels from a database, obtain a reference detection label carried by the comparison file, and adjust the M candidate detection labels by using the reference detection label to obtain the detection label of the target file. The file detection device solves the problem of low recognition accuracy in recognizing the detection label of a file in the related art. By obtaining the detection requirement of the target file, the text detection model is obtained according to the detection requirement, the candidate detection label is output by using the text detection model, the detection label is obtained after the screening processing of the candidate detection label, and the effect of improving the detection accuracy of the model in the label detection of the file is achieved.

[0074] The detection device of the file comprises a processor and a memory, the acquisition unit 60, the input unit 61, the screening unit 62 and the like are stored in the memory as program units, and the corresponding functions are realized by the processor executing the program units stored in the memory.

[0075] The processor comprises a core, and the core calls the corresponding program units in the memory.

[0076] The memory can comprise a non-permanent memory in a computer readable medium, a random access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory comprises at least one memory chip.

[0077] The embodiment of the present application provides a computer readable storage medium, which stores a program, and the program is executed by a processor to realize the detection method of the file.

[0078] The embodiment of the present application provides a processor, and the processor is used for running a program, wherein the program is executed to realize the detection method of the file.

[0079] Figure 7 It is a schematic diagram of an electronic device according to the embodiment of the present application, as shown in the figure, Figure 7 The embodiment of the present application provides an electronic device, and the electronic device 70 comprises a processor, a memory and a program stored in the memory and capable of running on the processor, and the processor is used for running computer readable instructions, wherein the computer readable instructions are executed to realize the detection method of the file.

[0080] The present application also provides a computer program product, which is suitable for executing the detection method of the file when being executed on a data processing device.

[0081] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product.

[0082] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0083] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0084] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0085] In one typical configuration, the computing device includes one or more processors (CPU's), input / output interfaces, network interfaces, and memory.

[0086] The memory can include non-persistent memory and / or persistent memory, such as flash memory, read-only memory (ROM), and / or volatile or non-volatile random access memory (RAM), among others. The memory is an example of computer-readable media.

[0087] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0088] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but also other elements not explicitly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0089] Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, system or computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0090] The above merely provides embodiments of the present application and is not intended to limit the present application. Various modifications and changes can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method of detecting a file, characterized by, The method comprises the following steps: obtaining a target file to be detected, and determining a detection requirement of the target file, and determining a text detection model according to the detection requirement, wherein the text detection model is a first text detection model or a second text detection model, the first text detection model is used for detecting chapter text in a file, the second text detection model is used for detecting paragraph text in a file, and the target file comprises a regulatory rule file outside or inside a country; inputting the target file into the text detection model, and outputting a detection result containing M candidate detection labels, wherein each candidate detection label is used for labeling a detection rule met by the target file, the text detection model is obtained by training a preset text detection model by using a plurality of sample data, and the sample data comprises a sample file and a detection label associated with the sample file; filtering out a comparison file associated with the M candidate detection labels from a database, obtaining a reference detection label carried by the comparison file, and adjusting the M candidate detection labels by using the reference detection label to obtain a detection label of the target file; wherein, before obtaining the target file to be detected and determining the detection requirement of the target file: determining a domain type to which the target file belongs, determining a plurality of preset detection rules according to the domain type, wherein the preset detection rules are detection rules required to be met by files in the domain type; adding the plurality of preset detection rules to an initial text detection model to obtain the preset text detection model, wherein the initial text detection model contains general file detection rules; adjusting the M candidate detection labels by using the reference detection label to obtain the detection label of the target file, comprising: performing intersection calculation on the reference detection label and the M candidate detection labels to obtain the detection label of the target file.

2. The method of claim 1, wherein, According to the detection requirement, the text detection model is determined, which comprises: identifying the detection requirement to obtain a label granularity of the target file, wherein the label granularity comprises a first granularity and a second granularity, the first granularity represents that a file needs to meet a chapter text detection rule, and the second granularity represents that a file needs to meet a paragraph text detection rule; in a case where the label granularity of the target file is the first granularity, the first text detection model is determined as the text detection model; in a case where the label granularity of the target file is the second granularity, the second text detection model is determined as the text detection model.

3. The method of claim 1, wherein, The text detection model is obtained by the following method: obtaining N sample files and X detection labels associated with each sample file, processing the N sample files and the X detection labels associated with each sample file to obtain N sample data; randomly dividing the N sample data to obtain Y sample data and Z sample data, wherein X=Y+Z, X, Y and Z are positive integers; using the Y sample data as training set data, training the preset text detection model by using the training set data to obtain a trained text detection model; The Z sample data are used as test set data. The model metrics of the trained text detection model are determined based on the test set data. If the model metrics meet the preset requirements, the trained text detection model is determined as the text detection model. The model metrics include at least one of the following: model precision and model recall.

4. The method of claim 3, wherein, Using the Z sample data as test set data, the model metrics of the trained text detection model are determined based on the test set data, including: Input the Z sample files from the Z sample data into the trained text detection model to obtain the training labels associated with each sample file; The training label is compared with the detection label associated with each sample file. If the training label matches the detection label associated with each sample file, the sample file corresponding to the training label is determined as the correctly predicted sample. If the training label is inconsistent with the detection label associated with each sample file, the sample file corresponding to the training label is identified as a mispredicted sample. The model precision and model recall are calculated using the values ​​of the correctly predicted samples, the values ​​of the incorrectly predicted samples, and the Z sample data.

5. The method of claim 1, wherein, The target file is input into the text detection model, and the output includes detection results containing M candidate detection tags, including: The evaluation probability value of each candidate detection label is determined, resulting in M ​​evaluation probability values, wherein the evaluation probability values ​​are used to characterize the accuracy of each candidate detection label; The M evaluation probability values ​​are sorted in descending order to obtain a probability arrangement. The evaluation probability values ​​that are before a preset position in the probability arrangement are selected, and the candidate detection labels associated with the evaluation probability values ​​before the preset position are combined. The combined candidate detection labels are determined as the detection results.

6. The method of claim 1, wherein, The M candidate detection tags are adjusted using the reference detection tags to obtain the detection tags for the target file, including: The reference detection label and the M candidate detection labels are fed back to the detection personnel, and the detection labels fed back by the detection personnel are obtained; The detection tags provided by the testing personnel are determined as the detection tags for the target file.

7. An apparatus for detecting a file, characterized by comprising: include: An acquisition unit is used to acquire a target file to be detected, determine the detection requirements of the target file, and determine a text detection model according to the detection requirements. The text detection model is either a first text detection model or a second text detection model. The first text detection model is used to detect the chapter text in the file, and the second text detection model is used to detect the paragraph text in the file. The target file includes regulatory rule documents from overseas or within China. The input unit is used to input the target file into the text detection model and output a detection result containing M candidate detection labels. Each candidate detection label is used to label the detection rules that the target file conforms to. The text detection model is obtained by training a preset text detection model with multiple sample data. The sample data includes sample files and detection labels associated with the sample files. The screening unit is configured to screen a comparison file associated with the M candidate detection labels from a database, acquire a reference detection label carried by the comparison file, and adjust the M candidate detection labels by using the reference detection label to obtain a detection label of the target file. The determining unit is configured to, before acquiring a target file to be detected and determining a detection requirement of the target file: determine a field type to which the target file belongs, determine a plurality of preset detection rules according to the field type, wherein the preset detection rules are detection rules required to be met by a file in the field type; and the adding unit is configured to add the plurality of preset detection rules to an initial text detection model to obtain the preset text detection model, wherein the initial text detection model contains a general file detection rule. The screening unit includes a second calculation module configured to perform intersection calculation on the reference detection label and the M candidate detection labels to obtain the detection label of the target file.

8. A processor, comprising: The processor is configured to run a program, and the program performs the file detection method in any one of claims 1 to 6 when the program is run.

9. An electronic device, comprising: The one or more processors and the memory are configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the file detection method in any one of claims 1 to 6.

Images