Data processing method and device, equipment, storage medium and program product

Abstract

The invention discloses a data processing method and device, equipment, a storage medium and a program product. The invention relates to the technical field of data processing. The method comprises the steps of obtaining a target text set; performing feature extraction on each text in the target text set to obtain a plurality of candidate features; determining first evaluation information and second evaluation information of each candidate feature based on the target text set; wherein the first evaluation information and the second evaluation information are both used for reflecting the contribution degree of the candidate features to category division; and screening out a target feature from the plurality of candidate features based on the first evaluation information and the second evaluation information. According to the data processing method provided by the embodiment of the invention, the candidate features are screened based on the first evaluation information and the second evaluation information reflecting the contribution degree to category division, so that the screened features are more beneficial to detection of abnormal transaction texts; the reliability of the screened features can be improved, and the detection efficiency and accuracy of the abnormal transaction text can be improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of data processing technology, and in particular, to a data processing method, apparatus, device, storage medium, and program product. Background Art

[0002] In recent years, with the widespread application of new technologies such as mobile payment, big data, and cloud computing in the financial field, it has not only driven the improvement of financial service efficiency and promoted the inclusion of digital finance, but also brought about a series of financial security issues, among which abnormal transactions are an important area for financial institutions to monitor compliance. Existing abnormal transaction monitoring technologies have problems such as low monitoring efficiency and inaccurate monitoring. Summary of the invention

[0003] The embodiments of the present invention provide a data processing method, apparatus, device, storage medium and program product, which can improve the reliability of screened features, thereby improving the detection efficiency and accuracy of abnormal transaction texts.

[0004] In a first aspect, an embodiment of the present invention provides a data processing method, characterized by comprising:

[0005] Acquire a target text set; wherein the target text set includes text subsets of multiple transaction categories;

[0006] Extracting features from each text in the target text set to obtain multiple candidate features;

[0007] Determining first evaluation information and second evaluation information of each candidate feature based on the target text set; wherein the first evaluation information and the second evaluation information are both used to reflect the contribution of the candidate feature to the category division;

[0008] A target feature is screened out from the plurality of candidate features based on the first evaluation information and the second evaluation information.

[0009] In a second aspect, an embodiment of the present invention further provides a data processing device, characterized in that it includes:

[0010] A target text set acquisition module, used to acquire a target text set; wherein the target text set includes text subsets of multiple transaction categories;

[0011] A candidate feature extraction module is used to extract features from each text in the target text set to obtain multiple candidate features;

[0012] An evaluation information determination module, used to determine first evaluation information and second evaluation information of each candidate feature based on the target text set; wherein the first evaluation information and the second evaluation information are both used to reflect the contribution of the candidate feature to the classification;

[0013] A target feature screening module is used to screen a target feature from the plurality of candidate features based on the first evaluation information and the second evaluation information.

[0014] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the data processing method as described in any one of the embodiments of the present invention is implemented.

[0015] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a data processing method as described in any one of the embodiments of the present invention.

[0016] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements a data processing method as described in any one of the embodiments of the present invention.

[0017] The embodiment of the present invention discloses a data processing method, device, equipment, storage medium and program product, the method comprising: obtaining a target text set; wherein the target text set includes a text subset of multiple transaction categories; performing feature extraction on each text in the target text set to obtain multiple candidate features; determining first evaluation information and second evaluation information of each candidate feature based on the target text set; wherein the first evaluation information and the second evaluation information are both used to reflect the contribution of the candidate feature to the category division; and filtering out the target feature from the multiple candidate features based on the first evaluation information and the second evaluation information. The data processing method provided by the embodiment of the present invention filters the candidate features based on the first evaluation information and the second evaluation information reflecting the contribution to the category division, so that the filtered features are more conducive to the detection of abnormal transaction texts, and can improve the reliability of the filtered features and improve the detection efficiency and accuracy of abnormal transaction texts. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A flowchart of a data processing method provided by an embodiment of the present invention;

[0020] Figure 2A flowchart of a data processing method provided by an embodiment of the present invention;

[0021] Figure 3 A schematic diagram of the structure of a data processing device provided by an embodiment of the present invention;

[0022] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0024] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. The acquisition, storage, use, processing, etc. of data in the technical solution of this application comply with the relevant provisions of national laws and regulations. It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. The purpose is only to illustrate the feasibility of the implementation of the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0025] Figure 1 The present invention provides a flowchart of a data processing method, which is applicable to extracting key features from abnormal transaction texts. The method can be executed by a data processing device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a PC or a server. Figure 1 As shown, the method includes:

[0026] S110, obtaining a target text set.

[0027] The target text set includes text subsets of multiple transaction categories. In this embodiment, the target text set may be an already identified abnormal transaction text set, for example, an abnormal transaction text may be a text formed by a transaction report with financial security issues. In this embodiment, the texts in the target text set may be divided into transaction categories in advance, and each transaction category includes at least one text. Exemplarily, the target text set C includes the following transaction categories: c1, c2, c3, ... c j, each transaction category corresponds to a text subset, that is, each category includes multiple texts, which can be expressed as: c j ={d1,d2,…d n}.

[0028] S120, extracting features from each text in the target text set to obtain a plurality of candidate features.

[0029] The candidate features may be represented by words extracted from each text in the target text set.

[0030] Specifically, feature extraction is performed on each text in the target text set to obtain multiple candidate features by: preprocessing each text in the target text set; performing word segmentation on each preprocessed text to obtain multiple words; and determining the multiple words as multiple candidate features.

[0031] Among them, the preprocessing method may include: data cleaning, etc. Cleaning each text may include the following methods: removing special characters, numbers, spaces, line breaks, punctuation marks, single words and other meaningless words (such as pause words, etc.). After preprocessing each text in the target text set, the preprocessed text is segmented according to semantics, so that each text is divided into individual words to form a word set, which is the subsequent feature set.

[0032] S130, determining first evaluation information and second evaluation information of each candidate feature based on the target text set.

[0033] The first evaluation information and the second evaluation information are both used to reflect the contribution of the candidate feature to the classification. For example, the first evaluation information can be represented by information gain, and the second evaluation information can be represented by Fisher score.

[0034] In this embodiment, the method of determining the first evaluation information and the second evaluation information of each candidate feature based on the target text set may be: for each candidate feature, multiple statistical parameters of the candidate feature in the target text set are determined, and the first evaluation information and the second evaluation information thereof are determined based on at least one of the multiple statistical parameters. The multiple statistical parameters may include: the ratio between the number of texts in the text subsets under each transaction category where the candidate feature appears and the number of texts in the target text set where the candidate feature does not appear; the ratio between the number of texts in the text subsets under each transaction category where the candidate feature does not appear and the number of texts in the target text set; the number of texts in which the candidate feature appears in the target text set; the number of texts in which the candidate feature does not appear in the target text set, etc.

[0035] Optionally, the method for determining the first evaluation information of each candidate feature based on the target text set may be: for each transaction category, determining the ratio of the number of texts contained in the text subset under the transaction category and the target text set as the first ratio; for each candidate feature, determining the ratio between the number of texts containing the candidate feature in the text subset under the transaction category and the number of texts in the target text set, and the ratio between the number of texts not containing the candidate feature in the text subset under the transaction category and the number of texts in the target text set, as the second ratio and the third ratio, respectively; determining the ratio between the number of texts containing the candidate feature in the target text set and the number of texts in the target text set, and the ratio between the number of texts not containing the candidate feature in the target text set and the number of texts in the target text set, as the fourth ratio and the fifth ratio, respectively; and determining the first evaluation information of the candidate feature under each transaction category according to the first ratio, the second ratio, the third ratio, the fourth ratio and the fifth ratio.

[0036] The first ratio represents the ratio of the number of texts contained in the text subsets under each transaction category to the total number of texts (i.e., the number of texts contained in the target text set). It can also be understood as the probability of the texts of each transaction category appearing in the total text. For example, the jth transaction category is represented by c j , then the corresponding first ratio is expressed as P(c j ). The second ratio can be understood as the ratio of the number of texts containing a candidate feature in the text subset under each transaction category to the total number of texts, or it can be understood as the conditional probability of containing a candidate feature and belonging to a transaction category. Assume that t i represents the i-th candidate feature, then it contains feature t i and belongs to c j The conditional probability of the class, i.e. the second proportion, can be expressed as P(c j |t i ). The third ratio can be understood as the ratio of the number of texts that do not contain a candidate feature in the text subset under each transaction category to the total number of texts, or it can be understood as the conditional probability of not containing a candidate feature and belonging to a transaction category, for example: not containing feature t i and belongs to c j The conditional probability of the class, i.e. the second proportion, can be expressed as The fourth ratio can be understood as the ratio of the number of texts containing a certain candidate feature in the target text set to the total number of texts, or it can be understood as the probability of a certain candidate feature appearing in the target text set, which can be expressed as P(t i ). The fifth ratio can be understood as the ratio of the number of texts in the target text set that do not contain a certain candidate feature to the total number of texts, or it can be understood as the probability of the target text set not containing a certain candidate feature, which can be expressed as

[0037] In this embodiment, the process of determining the first evaluation information of the candidate feature under each transaction category according to the first ratio, the second ratio, the third ratio, the fourth ratio and the fifth ratio may be: performing a logarithmic operation on the first ratio and then fusing it with the first ratio to obtain a first fusion result; performing a logarithmic operation on the second ratio and then fusing it with the second ratio and the fourth ratio to obtain a second fusion result; performing a logarithmic operation on the third ratio and then fusing it with the third ratio and the fifth ratio to obtain a third fusion result; finally, the first fusion result, the second fusion result and the third fusion result are accumulated to obtain the first evaluation information of the candidate feature under a certain transaction category. For example, the candidate feature t i In transaction category c j The calculation formula of the first evaluation information can be expressed as:

[0038]

[0039] In this embodiment, the second evaluation information can be determined by the inter-class difference information and intra-class difference information of the candidate feature. The inter-class difference information can reflect the degree of difference in the distribution center between different transaction categories on a certain candidate feature. The greater the inter-class difference information, the greater the contribution of the candidate feature to the division of transaction categories. The intra-class difference information can reflect the degree of dispersion between texts on the candidate feature within the same transaction category.

[0040] Optionally, the method of determining the second evaluation information of each candidate feature based on the target text set may be: for each candidate feature, determining the inter-class difference information and intra-class difference information of the candidate feature based on the target text set; and determining the second evaluation information of the candidate feature based on the inter-class difference information and the intra-class difference information.

[0041] The inter-class difference information and the intra-class difference information of the candidate features may be determined according to the frequency of the candidate features appearing in the target text set and the frequency of the candidate features appearing in each text subset.

[0042] Specifically, for each candidate feature, the method of determining the inter-class difference information and intra-class difference information of the candidate feature based on the target text set can be: determining the frequency of occurrence of the candidate feature in each text subset, which is determined as the first frequency; determining the mean of the occurrence frequency of the candidate feature in each text subset and the mean of the occurrence frequency in the target text set, which are respectively used as the first mean and the second mean; determining the number of texts that contain the candidate feature in the target text set and each text subset, which are used as the first text number and the second text number; determining the inter-class difference information and intra-class difference information of the candidate feature based on at least one of the first frequency, the first mean, the second mean, the first text number and the second text number.

[0043] The frequency of occurrence of the candidate feature in the text subset may be determined by counting the frequency of occurrence of the candidate feature in each text in the text subset, accumulating the frequency, and obtaining the frequency of occurrence of the candidate feature in the text subset. For example, assuming that the frequency of occurrence of the i-th candidate feature in the j-th transaction category is denoted as y (i) , there are three texts in the text subset of the jth transaction category, and the frequencies of the i-th candidate feature appearing in these three texts are: 10 times, 15 times, and 20 times respectively. Then the frequency of occurrence of this candidate feature in the text subset is: y (i) =10+15+20=45. The mean value of the frequency of occurrence of the candidate feature in the text subset can be determined by first determining the frequency of occurrence of the candidate feature in each text in the text subset, and then averaging these frequencies. The mean value of the frequency of occurrence of the i-th candidate feature in the text subset corresponding to the j-th transaction category, i.e., the first mean value, is expressed as The method for determining the mean of the frequency of occurrence of the candidate feature in the target text set can be: first determine the frequency of occurrence of the candidate feature in each text in the target text set, and then calculate the average of these frequencies. The mean of the frequency of occurrence of the i-th candidate feature in the target text set, that is, the second mean, is represented by x (i) The number of texts in the target text set that contain the i-th candidate feature, that is, the first text number, can be expressed as n, and the number of texts in the text subset of the j-th transaction category that contain the i-th candidate feature, that is, the second text number, can be expressed as n j .

[0044] Specifically, the method of determining the inter-class difference information and intra-class difference information of the candidate feature based on at least one of the first frequency, the first mean, the second mean, the first number of texts and the second number of texts can be: determining the inter-class difference information of the candidate feature according to the first mean, the second mean, the first number of texts and the second number of texts; determining the intra-class difference information of the candidate feature according to the first frequency, the second mean set and the first number of texts.

[0045] In this embodiment, the calculation formula for determining the inter-class difference information of the i-th candidate feature according to the first mean, the second mean, the first text number, and the second text number can be expressed as: The calculation formula for determining the intra-class difference information of the i-th candidate feature according to the first frequency and the first text number of the second mean set can be expressed as:

[0046] In this embodiment, the second evaluation information of the candidate feature may be determined according to the inter-class difference information and the intra-class difference information by taking the quotient of the inter-class difference information and the intra-class difference information to obtain the second evaluation information of the candidate feature.

[0047] Specifically, the calculation formula of the second evaluation information of the i-th candidate feature can be expressed as:

[0048] S140: Filter out a target feature from a plurality of candidate features based on the first evaluation information and the second evaluation information.

[0049] In this embodiment, the process of screening out the target feature from multiple candidate features based on the first evaluation information and the second evaluation information may be: first, the first evaluation information and the second evaluation information are processed in a preset manner to obtain the target evaluation information, and then the target feature is screened out from multiple candidate features based on the target evaluation information.

[0050] In this embodiment, the method of filtering out the target feature from multiple candidate features based on the first evaluation information and the second evaluation information can be: determining the target evaluation information of each candidate feature according to the first evaluation information and the second evaluation information; sorting the target evaluation information in reverse order, and extracting the candidate features corresponding to a set number of target evaluation information ranked first as the target feature; or, sorting the target evaluation information in ascending order, and extracting the candidate features corresponding to a set number of target evaluation information ranked last as the target feature.

[0051] Optionally, a method for determining target evaluation information of each candidate feature based on the first evaluation information and the second evaluation information may be: accumulating the first evaluation information under each transaction category; and fusing the accumulated first evaluation information with the second evaluation information to obtain target evaluation information.

[0052] Specifically, for the i-th candidate feature, the first evaluation information of the candidate feature under each transaction category is accumulated, and then the accumulated first evaluation information is multiplied by the second evaluation information to obtain the target evaluation information. The formula for accumulating the first evaluation information of the i-th candidate feature under each transaction category can be expressed as:

[0053]

[0054] Optionally, a method for determining target evaluation information of each candidate feature based on the first evaluation information and the second evaluation information may be: fusing the first evaluation information under each transaction category with the second evaluation information respectively to obtain intermediate evaluation information under each transaction category; and determining the largest intermediate evaluation information as the target evaluation information.

[0055] Specifically, for the ith candidate feature, the first evaluation information of the candidate feature under each transaction category is multiplied by the second evaluation information to obtain the intermediate evaluation information under each transaction category; finally, the largest intermediate evaluation information is determined as the target evaluation information.

[0056] In this embodiment, after obtaining the target evaluation information of each candidate feature, the target evaluation information is sorted in reverse order or in ascending order, and finally a set number of candidate features ranked first or a set number of candidate features ranked last are extracted as target features.

[0057] Optionally, after selecting a target feature from multiple candidate features based on the first evaluation information and the second evaluation information, the method further includes the following steps: obtaining a text to be recognized; performing anomaly detection on the text to be recognized based on the target feature to obtain a detection result of the text to be recognized.

[0058] The detection result includes that the text to be recognized is a normal transaction text or that the text to be recognized is an abnormal transaction text. The process of performing abnormal detection on the text to be recognized based on the target feature can be: first, the text to be recognized is preprocessed by cleaning, and then the preprocessed text to be recognized is segmented to obtain multiple words; multiple words are matched with the target feature, if the target feature that exceeds the first set ratio matches part of the multiple words, the text to be recognized is an abnormal transaction text, and if the target feature that is less than the second set ratio matches part of the multiple words, the text to be recognized is a normal transaction text. The first set ratio is greater than the second set ratio, for example: the first set ratio is set to 50%, and the second set ratio is set to 10%.

[0059] The technical solution of this embodiment is to obtain a target text set; wherein the target text set includes text subsets of multiple transaction categories; extract features from each text in the target text set to obtain multiple candidate features; determine the first evaluation information and the second evaluation information of each candidate feature based on the target text set; wherein the first evaluation information and the second evaluation information are both used to reflect the contribution of the candidate feature to the category division; and select the target feature from the multiple candidate features based on the first evaluation information and the second evaluation information. The data processing method provided by the embodiment of the present invention selects the candidate features based on the first evaluation information and the second evaluation information reflecting the contribution to the category division, so that the selected features are more conducive to the detection of abnormal transaction texts, and can improve the reliability of the selected features and improve the detection efficiency and accuracy of abnormal transaction texts.

[0060] Based on the above embodiments, Figure 2 is a flow chart of a data processing method provided by an embodiment of the present invention, such as Figure 2 As shown, the method comprises the following steps:

[0061] S210, obtaining a target text set.

[0062] S220, preprocessing each text in the target text set; performing word segmentation on each preprocessed text to obtain a plurality of words; and determining the plurality of words as a plurality of candidate features.

[0063] S230, determining a first ratio, a second ratio, a third ratio, a fourth ratio and a fifth ratio based on the target text set, and determining first evaluation information of the candidate features under each transaction category according to the first ratio, the second ratio, the third ratio, the fourth ratio and the fifth ratio.

[0064] S240, determining the frequency of occurrence of the candidate feature in each text subset, as the first frequency; determining the mean of the frequency of occurrence of the candidate feature in each text subset and the mean of the frequency of occurrence in the target text set, as the first mean and the second mean, respectively; determining the number of texts in the target text set and each text subset that contain the candidate feature, as the first text number and the second text number.

[0065] S250, determining inter-class difference information of the candidate feature according to the first mean, the second mean, the first text number and the second text number; determining intra-class difference information of the candidate feature according to the first frequency, the second mean and the first text number.

[0066] S260, taking the quotient of the inter-class difference information and the intra-class difference information to obtain second evaluation information of the candidate feature.

[0067] S270, respectively merge the first evaluation information under each transaction category with the second evaluation information to obtain intermediate evaluation information under each transaction category; determine the largest intermediate evaluation information as the target evaluation information.

[0068] S280, sort the target evaluation information in reverse order, and extract candidate features corresponding to a set number of target evaluation information ranked first as target features.

[0069] Figure 3 is a structural diagram of a data processing device provided by an embodiment of the present invention, such as Figure 3 As shown, the device comprises:

[0070] The target text set acquisition module 310 is used to acquire a target text set; wherein the target text set includes text subsets of multiple transaction categories;

[0071] A candidate feature extraction module 320 is used to extract features from each text in the target text set to obtain multiple candidate features;

[0072] An evaluation information determination module 330 is used to determine first evaluation information and second evaluation information of each candidate feature based on the target text set; wherein the first evaluation information and the second evaluation information are both used to reflect the contribution of the candidate feature to the classification;

[0073] The target feature screening module 340 is used to screen out a target feature from a plurality of candidate features based on the first evaluation information and the second evaluation information.

[0074] Optionally, the candidate feature extraction module 320 is further used for:

[0075] Preprocess each text in the target text set;

[0076] Perform word segmentation on each preprocessed text to obtain multiple words;

[0077] The plurality of words are determined as a plurality of candidate features.

[0078] Optionally, the evaluation information determination module 330 is further configured to:

[0079] For each transaction category, determining the ratio of the number of texts contained in the text subset under the transaction category and the target text set as a first ratio;

[0080] For each candidate feature, determine the ratio between the number of texts in the text subset under the transaction category that contain the candidate feature and the number of texts in the target text set, and the ratio between the number of texts in the text subset under the transaction category that do not contain the candidate feature and the number of texts in the target text set, as the second ratio and the third ratio, respectively;

[0081] Determine the ratio between the number of texts in the target text set that contain the candidate feature and the number of texts in the target text set, and the ratio between the number of texts in the target text set that do not contain the candidate feature and the number of texts in the target text set, as the fourth ratio and the fifth ratio, respectively;

[0082] First evaluation information of the candidate feature under each transaction category is determined according to the first ratio, the second ratio, the third ratio, the fourth ratio and the fifth ratio.

[0083] Optionally, the evaluation information determination module 330 is further configured to:

[0084] For each candidate feature, determine the inter-class difference information and intra-class difference information of the candidate feature based on the target text set;

[0085] The second evaluation information of the candidate feature is determined according to the inter-class difference information and the intra-class difference information.

[0086] Optionally, the evaluation information determination module 330 is further configured to:

[0087] Determine the occurrence frequency of the candidate feature in each text subset and determine it as the first frequency;

[0088] Determine the mean of the occurrence frequency of the candidate features in each text subset and the mean of the occurrence frequency in the target text set, as the first mean and the second mean respectively;

[0089] Determine the number of texts in the target text set and each text subset that contain the candidate feature as the first text number and the second text number;

[0090] The inter-class difference information and the intra-class difference information of the candidate feature are determined based on at least one of the first frequency, the first mean, the second mean, the first text number, and the second text number.

[0091] Optionally, the evaluation information determination module 330 is further configured to:

[0092] Determine inter-class difference information of the candidate feature according to the first mean, the second mean, the first text number, and the second text number;

[0093] Intra-class difference information of the candidate feature is determined according to the first frequency, the second mean, and the first text number.

[0094] Optionally, the evaluation information determination module 330 is further configured to:

[0095] The inter-class difference information and the intra-class difference information are taken as the quotient to obtain the second evaluation information of the candidate feature.

[0096] Optionally, the target feature screening module 340 is further used to:

[0097] Determine target evaluation information of each candidate feature according to the first evaluation information and the second evaluation information;

[0098] Sort the target evaluation information in reverse order, and extract candidate features corresponding to a set number of target evaluation information ranked first as target features; or,

[0099] The target evaluation information is sorted in positive order, and candidate features corresponding to a set number of target evaluation information ranked later are extracted as target features.

[0100] Optionally, the target feature screening module 340 is further used to:

[0101] Accumulating the first evaluation information under each transaction category;

[0102] The accumulated first evaluation information is fused with the second evaluation information to obtain target evaluation information.

[0103] Optionally, the target feature screening module 340 is further used to:

[0104] The first evaluation information under each transaction category is respectively integrated with the second evaluation information to obtain intermediate evaluation information under each transaction category;

[0105] The maximum intermediate evaluation information is determined as the target evaluation information.

[0106] Optionally, it also includes: a text detection module, used to:

[0107] Get the text to be recognized;

[0108] Anomaly detection is performed on the text to be recognized based on the target features to obtain the detection result of the text to be recognized.

[0109] The above device can execute the methods provided by all the above embodiments of the present invention, and has the corresponding functional modules and beneficial effects of executing the above methods. For technical details not described in detail in this embodiment, please refer to the methods provided by all the above embodiments of the present invention.

[0110] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0111] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0112] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0113] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a data processing method.

[0114] In some embodiments, the data processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the data processing method in any other appropriate manner (e.g., by means of firmware).

[0115] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0116] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0117] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0118] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0119] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0120] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0121] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the data processing method provided in any embodiment of the present application.

[0122] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0123] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data processing method, characterized in that: include: Acquire a target text set; wherein the target text set includes text subsets of multiple transaction categories; Extracting features from each text in the target text set to obtain multiple candidate features; Determining first evaluation information and second evaluation information of each candidate feature based on the target text set; wherein the first evaluation information and the second evaluation information are both used to reflect the contribution of the candidate feature to the category division; A target feature is screened out from the plurality of candidate features based on the first evaluation information and the second evaluation information.

2. The method according to claim 1, characterized in that: Perform feature extraction on each text in the target text set to obtain multiple candidate features, including: Preprocessing each text in the target text set; Perform word segmentation on each preprocessed text to obtain multiple words; The plurality of words are determined as a plurality of candidate features.

3. The method according to claim 1, characterized in that Determining first evaluation information of each of the candidate features based on the target text set includes: For each transaction category, determining a ratio of the number of texts contained in the text subset under the transaction category and the target text set as a first ratio; For each candidate feature, determining a ratio between the number of texts in the text subset under the transaction category that contain the candidate feature and the number of texts in the target text set, and a ratio between the number of texts in the text subset under the transaction category that do not contain the candidate feature and the number of texts in the target text set, as a second ratio and a third ratio, respectively; Determine the ratio between the number of texts in the target text set that contain the candidate feature and the number of texts in the target text set and the ratio between the number of texts in the target text set that do not contain the candidate feature and the number of texts in the target text set, as a fourth ratio and a fifth ratio, respectively; The first evaluation information of the candidate feature under each of the transaction categories is determined according to the first ratio, the second ratio, the third ratio, the fourth ratio and the fifth ratio.

4. The method according to claim 1, characterized in that: Determining second evaluation information of each of the candidate features based on the target text set includes: For each candidate feature, determining inter-class difference information and intra-class difference information of the candidate feature based on the target text set; The second evaluation information of the candidate feature is determined according to the inter-class difference information and the intra-class difference information.

5. The method according to claim 4, characterized in that For each candidate feature, determining inter-class difference information and intra-class difference information of the candidate feature based on the target text set includes: Determine the occurrence frequency of the candidate feature in each of the text subsets, and determine it as a first frequency; Determine the mean of the occurrence frequency of the candidate feature in each of the text subsets and the mean of the occurrence frequency in the target text set, as the first mean and the second mean respectively; Determine the number of texts in the target text set and each of the text subsets that contain the candidate feature as the first text number and the second text number; The inter-class difference information and the intra-class difference information of the candidate feature are determined based on at least one of the first frequency, the first mean, the second mean, the first text number, and the second text number.

6. The method according to claim 5, characterized in that Determining inter-class difference information and intra-class difference information of the candidate feature based on at least one of the first frequency, the first mean, the second mean, the first text number, and the second text number includes: Determine inter-class difference information of the candidate feature according to the first mean, the second mean, the first number of texts, and the second number of texts; Intra-class difference information of the candidate feature is determined according to the first frequency, the second mean, and the first number of texts.

7. The method according to claim 4, characterized in that Determining second evaluation information of the candidate feature according to the inter-class difference information and the intra-class difference information includes: The inter-class difference information and the intra-class difference information are taken as a quotient to obtain second evaluation information of the candidate feature.

8. The method according to claim 3, characterized in that Filtering a target feature from the plurality of candidate features based on the first evaluation information and the second evaluation information includes: Determine target evaluation information of each of the candidate features according to the first evaluation information and the second evaluation information; sorting the target evaluation information in reverse order, and extracting candidate features corresponding to a set number of target evaluation information ranked first as target features; or, The target evaluation information is sorted in positive order, and candidate features corresponding to a set number of target evaluation information ranked later are extracted as target features.

9. The method according to claim 8, characterized in that Determining target evaluation information of each of the candidate features according to the first evaluation information and the second evaluation information includes: Accumulating the first evaluation information under each of the transaction categories; The accumulated first evaluation information is fused with the second evaluation information to obtain target evaluation information.

10. The method according to claim 8, characterized in that Determining target evaluation information of each of the candidate features according to the first evaluation information and the second evaluation information includes: Merging the first evaluation information under each of the transaction categories with the second evaluation information respectively to obtain intermediate evaluation information under each of the transaction categories; The maximum intermediate evaluation information is determined as the target evaluation information.

11. The method according to claim 1, characterized in that: After selecting a target feature from the plurality of candidate features based on the first evaluation information and the second evaluation information, the method further includes: Get the text to be recognized; Anomaly detection is performed on the text to be recognized based on the target feature to obtain a detection result of the text to be recognized.

12. A data processing device, characterized in that: include: A target text set acquisition module, used to acquire a target text set; wherein the target text set includes text subsets of multiple transaction categories; A candidate feature extraction module is used to extract features from each text in the target text set to obtain multiple candidate features; An evaluation information determination module, used to determine first evaluation information and second evaluation information of each candidate feature based on the target text set; wherein the first evaluation information and the second evaluation information are both used to reflect the contribution of the candidate feature to the classification; A target feature screening module is used to screen a target feature from the plurality of candidate features based on the first evaluation information and the second evaluation information.

13. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the data processing method according to any one of claims 1 to 11 when executing the computer program.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the data processing method as described in any one of claims 1 to 11 is implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the data processing method according to any one of claims 1 to 11.

Images