A data processing method, device, apparatus, storage medium, and program product

Abstract

A data processing method and device, equipment, storage medium and program product are disclosed. The present application relates to the technical field of data processing. The method comprises obtaining a target text set; performing feature extraction on each text in the target text set to obtain a plurality of candidate features; determining first evaluation information and second evaluation information of each candidate feature based on the target text set; wherein the first evaluation information and the second evaluation information are used to reflect the contribution degree of the candidate feature to the category division; and selecting target features from the plurality of candidate features based on the first evaluation information and the second evaluation information. The data processing method provided by the embodiment of the present application selects candidate features based on the first evaluation information and the second evaluation information reflecting the contribution degree to the category division, so that the selected features are more conducive to the detection of abnormal transaction texts, and the reliability of the selected features can be improved, and the detection efficiency and accuracy of abnormal transaction texts can be improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of data processing, and in particular, to a data processing method, device, equipment, storage medium and program product. BACKGROUND

[0002] In recent years, with the wide application of new technologies such as mobile payment, big data and cloud computing in the financial field, the efficiency of financial services has been improved, the digital financial inclusion has been promoted, and a series of financial security problems have also been brought about, among which abnormal transactions are an important field in the monitoring compliance of financial institutions. The existing abnormal transaction monitoring technology has problems such as low monitoring efficiency and inaccurate monitoring. SUMMARY

[0003] Embodiments of the present application provide a data processing method, device, equipment, storage medium and program product, which can improve the reliability of the screened features, thereby improving the detection efficiency and accuracy of abnormal transaction texts.

[0004] In a first aspect, embodiments of the present application provide a data processing method, characterized in that it comprises:

[0005] obtaining a target text set; wherein the target text set comprises text subsets of multiple transaction categories;

[0006] performing feature extraction on each text in the target text set to obtain a plurality of candidate features;

[0007] determining first evaluation information and second evaluation information of each candidate feature based on the target text set; wherein the first evaluation information and the second evaluation information are both used to reflect the contribution degree of the candidate feature to category division;

[0008] screening target features from the plurality of candidate features based on the first evaluation information and the second evaluation information.

[0009] In a second aspect, embodiments of the present application also provide a data processing device, characterized in that it comprises:

[0010] a target text set obtaining module for obtaining a target text set; wherein the target text set comprises text subsets of multiple transaction categories;

[0011] a candidate feature extraction module for performing feature extraction on each text in the target text set to obtain a plurality of candidate features;

[0012] an evaluation information determination module for determining first evaluation information and second evaluation information of each candidate feature based on the target text set; wherein the first evaluation information and the second evaluation information are both used to reflect the contribution degree of the candidate feature to category division;

[0013] a target feature screening module configured to screen a target feature from the plurality of candidate features based on the first evaluation information and the second evaluation information.

[0014] In a third aspect, an embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the data processing method according to any one of the embodiments of the present application when executing the program.

[0015] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and the program is executable on a processor to implement the data processing method according to any one of the embodiments of the present application.

[0016] In a fifth aspect, an embodiment of the present application also provides a computer program product, which includes a computer program, and the computer program is executable on a processor to implement the data processing method according to any one of the embodiments of the present application.

[0017] The embodiments of the present application disclose a data processing method, device, equipment, storage medium and program product, and the method includes: obtaining a target text set; wherein the target text set includes text subsets of multiple transaction categories; performing feature extraction on each text in the target text set to obtain a plurality of candidate features; determining first evaluation information and second evaluation information of each candidate feature based on the target text set; wherein the first evaluation information and the second evaluation information are both used to reflect the contribution degree of the candidate feature to category division; and screening a target feature from the plurality of candidate features based on the first evaluation information and the second evaluation information. The data processing method provided by the embodiments of the present application screens the candidate features based on the first evaluation information and the second evaluation information reflecting the contribution degree to category division, so that the screened features are more conducive to the detection of abnormal transaction texts, and the reliability of the screened features can be improved, and the detection efficiency and accuracy of abnormal transaction texts can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0019] Figure 1 a flowchart of a data processing method provided by an embodiment of the present application;

[0020] Figure 2A flowchart of a data processing method provided in an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of the structure of a data processing device provided in an embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0023] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0024] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first" and "second" are used only to distinguish descriptions and should not be construed as indicating or implying relative importance. The acquisition, storage, use, and processing of data in the technical solutions of this application comply with relevant national laws and regulations. It should be noted that in the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solutions of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0025] Figure 1 This is a flowchart illustrating a data processing method provided in an embodiment of the present invention. This method is applicable to extracting key features from abnormal transaction text. The method can be executed by a data processing device, which can be implemented in software and / or hardware, optionally through an electronic device such as a mobile terminal, PC, or server. Figure 1 As shown, the method includes:

[0026] S110, Obtain the target text set.

[0027] The target text set includes subsets of text from various transaction categories. In this embodiment, the target text set can be a collection of identified abnormal transaction texts, such as texts generated from transaction reports with financial security issues. In this embodiment, the texts in the target text set can be pre-categorized into transaction categories, with each transaction category including at least one text. For example, the target text set C includes the following transaction categories: c1, c2, c3, ... c jEach transaction category corresponds to a text subset, i.e., each category includes multiple texts, which can be expressed as: c j = {d1, d2, … d n}.

[0028] S120, feature extraction is performed on each text in the target text set to obtain a plurality of candidate features.

[0029] The candidate features can be represented by words extracted from each text in the target text set.

[0030] Specifically, the manner in which each text in the target text set is subjected to feature extraction to obtain a plurality of candidate features can be: preprocessing each text in the target text set; performing word segmentation processing on each preprocessed text to obtain a plurality of words; and determining the plurality of words as the plurality of candidate features.

[0031] The preprocessing manner can include data cleaning, etc. Cleaning each text can include the following manner: removing special characters, numbers, spaces, line breaks, punctuation marks, single characters, and other meaningless words (such as pause words, etc.). After preprocessing each text in the target text set, the preprocessed text is subjected to word segmentation processing according to semantics, thereby dividing each text into individual words to form a word set, i.e., a subsequent feature set.

[0032] S130, determining first evaluation information and second evaluation information of each candidate feature based on the target text set.

[0033] The first evaluation information and the second evaluation information are both used to reflect the contribution degree of the candidate feature to the category division. For example, the first evaluation information can be represented by information gain, and the second evaluation information can be represented by Fisher score.

[0034] In this embodiment, the manner in which the first evaluation information and the second evaluation information of each candidate feature are determined based on the target text set can be: for each candidate feature, determining a plurality of statistical parameters of the candidate feature in the target text set, and determining the first evaluation information and the second evaluation information of the candidate feature based on at least one of the plurality of statistical parameters. The plurality of statistical parameters can include: the ratio between the number of texts in which the candidate feature appears in the text subset under each transaction category and the number of texts in which the candidate feature appears in the target text set; the ratio between the number of texts in which the candidate feature does not appear in the text subset under each transaction category and the number of texts in which the candidate feature appears in the target text set; the number of texts in which the candidate feature appears in the target text set; the number of texts in which the candidate feature does not appear in the target text set, etc.

[0035] Optionally, the manner of determining the first evaluation information of each candidate feature based on the target text set can be: for each transaction category, determining the proportion of the number of texts contained in the text subset under the transaction category and the target text set as a first proportion; for each candidate feature, determining the proportion between the number of texts containing the candidate feature in the text subset under the transaction category and the number of texts in the target text set, and the proportion between the number of texts not containing the candidate feature in the text subset under the transaction category and the number of texts in the target text set as a second proportion and a third proportion respectively; determining the proportion between the number of texts containing the candidate feature in the target text set and the number of texts in the target text set, and the proportion between the number of texts not containing the candidate feature in the target text set and the number of texts in the target text set as a fourth proportion and a fifth proportion respectively; and determining the first evaluation information of the candidate feature under each transaction category according to the first proportion, the second proportion, the third proportion, the fourth proportion and the fifth proportion.

[0036] wherein the first proportion represents the proportion of the number of texts contained in the text subset under each transaction category to the total number of texts (i.e. the number of texts contained in the target text set), and can also be understood as the probability of the texts of each transaction category appearing in the total texts, for example: the jth transaction category is represented as c j , and the corresponding first proportion is represented as P(c j ). The second proportion can be understood as the proportion between the number of texts containing a certain candidate feature in the text subset under each transaction category and the total number of texts, and can also be understood as the conditional probability of containing a certain candidate feature and belonging to a certain transaction category, assuming that t i represents the ith candidate feature, and the conditional probability of containing the feature t i and belonging to c j category, i.e. the second proportion, can be represented as P(c j |t i ). The third proportion can be understood as the proportion between the number of texts not containing a certain candidate feature in the text subset under each transaction category and the total number of texts, and can also be understood as the conditional probability of not containing a certain candidate feature and belonging to a certain transaction category, for example: the conditional probability of not containing the feature t i and belonging to c j category, i.e. the second proportion, can be represented as The fourth proportion can be understood as the proportion between the number of texts containing a certain candidate feature in the target text set and the total number of texts, and can also be understood as the probability of containing a certain candidate feature in the target text set, and can be represented as P(t i ). The fifth proportion can be understood as the proportion between the number of texts not containing a certain candidate feature in the target text set and the total number of texts, and can also be understood as the probability of not containing a certain candidate feature in the target text set, and can be represented as

[0037] In this embodiment, the process of determining the first evaluation information of the candidate feature under each transaction category according to the first proportion, the second proportion, the third proportion, the fourth proportion and the fifth proportion can be: performing logarithmic operation on the first proportion and fusing the first proportion to obtain a first fusion result; performing logarithmic operation on the second proportion and fusing the second proportion and the fourth proportion to obtain a second fusion result; performing logarithmic operation on the third proportion and fusing the third proportion and the fifth proportion to obtain a third fusion result; and finally accumulating the first fusion result, the second fusion result and the third fusion result to obtain the first evaluation information of the candidate feature under a certain transaction category. For example, the first evaluation information of the candidate feature t i The first evaluation information of the candidate feature under the transaction category c j may be expressed as:

[0038]

[0039] In this embodiment, the second evaluation information can be determined by the inter-class difference information and the intra-class difference information of the candidate feature. The inter-class difference information can reflect the difference degree of the distribution center of the candidate feature between different transaction categories, and the greater the inter-class difference information, the greater the contribution of the candidate feature to the division of the transaction category. The intra-class difference information can reflect the dispersion degree of the texts in the same transaction category on the candidate feature.

[0040] Optionally, the manner of determining the second evaluation information of each candidate feature based on the target text set can be: for each candidate feature, determining the inter-class difference information and the intra-class difference information of the candidate feature based on the target text set; and determining the second evaluation information of the candidate feature according to the inter-class difference information and the intra-class difference information.

[0041] The inter-class difference information and the intra-class difference information of the candidate feature can be determined according to the frequency of the candidate feature appearing in the target text set and the frequency of the candidate feature appearing in each text subset.

[0042] Specifically, for each candidate feature, the manner of determining the inter-class difference information and the intra-class difference information of the candidate feature based on the target text set can be: determining the frequency of the candidate feature appearing in each text subset as a first frequency; determining the mean value of the frequency of the candidate feature appearing in each text subset and the mean value of the frequency of the candidate feature appearing in the target text set as a first mean value and a second mean value respectively; determining the number of texts containing the candidate feature in the target text set and each text subset as a first text number and a second text number; and determining the inter-class difference information and the intra-class difference information of the candidate feature based on at least one of the first frequency, the first mean value, the second mean value, the first text number and the second text number.

[0043] The determination manner of the occurrence frequency of the candidate feature in the text subset can be: the occurrence frequency of the candidate feature in each text in the text subset is counted, and the occurrence frequency of the candidate feature in the text subset is obtained by accumulating the frequency. For example, the occurrence frequency of the ith candidate feature in the jth transaction category is represented as y (i) , there are three texts in the text subset of the jth transaction category, and the occurrence frequency of the ith candidate feature in the three texts is 10 times, 15 times and 20 times respectively. The occurrence frequency of the candidate feature in the text subset is y (i) = 10 + 15 + 20 = 45. The determination manner of the mean value of the occurrence frequency of the candidate feature in the text subset can be: the occurrence frequency of the candidate feature in each text in the text subset is determined first, and then the mean value of the frequency is obtained. The mean value of the occurrence frequency of the ith candidate feature in the text subset corresponding to the jth transaction category, i.e. the first mean value, is represented as The determination manner of the mean value of the occurrence frequency of the candidate feature in the target text set can be: the occurrence frequency of the candidate feature in each text in the target text set is determined first, and then the mean value of the frequency is obtained. The mean value of the occurrence frequency of the ith candidate feature in the target text set, i.e. the second mean value, is represented as x (i) . The number of texts containing the ith candidate feature in the target text set, i.e. the first number of texts, can be represented as n, and the number of texts containing the ith candidate feature in the text subset of the jth transaction category, i.e. the second number of texts, can be represented as n j .

[0044] Specifically, the manner of determining the inter-class difference information and the intra-class difference information of the candidate feature based on at least one of the first frequency, the first mean value, the second mean value, the first number of texts and the second number of texts can be: determining the inter-class difference information of the candidate feature according to the first mean value, the second mean value, the first number of texts and the second number of texts; and determining the intra-class difference information of the candidate feature according to the first frequency, the second mean value and the first number of texts.

[0045] In this embodiment, the calculation formula of the inter-class difference information of the ith candidate feature according to the first mean value, the second mean value, the first number of texts and the second number of texts can be represented as: The calculation formula of the intra-class difference information of the ith candidate feature according to the first frequency, the second mean value and the first number of texts can be represented as:

[0046] In this embodiment, the manner of determining the second evaluation information of the candidate feature according to the inter-class difference information and the intra-class difference information can be: multiplying the inter-class difference information and the intra-class difference information to obtain the second evaluation information of the candidate feature.

[0047] Specifically, the calculation formula of the second evaluation information of the ith candidate feature can be represented as:

[0048] S140, Based on the first evaluation information and the second evaluation information, the target feature is selected from multiple candidate features.

[0049] In this embodiment, the process of selecting target features from multiple candidate features based on the first evaluation information and the second evaluation information can be as follows: first, the first evaluation information and the second evaluation information are processed in a preset manner to obtain target evaluation information, and then the target features are selected from multiple candidate features based on the target evaluation information.

[0050] In this embodiment, the method of selecting target features from multiple candidate features based on the first evaluation information and the second evaluation information may be as follows: determine the target evaluation information of each candidate feature according to the first evaluation information and the second evaluation information; sort the target evaluation information in reverse order and extract the candidate features corresponding to the first set number of target evaluation information as target features; or, sort the target evaluation information in ascending order and extract the candidate features corresponding to the last set number of target evaluation information as target features.

[0051] Optionally, the target evaluation information for each candidate feature can be determined based on the first evaluation information and the second evaluation information by: accumulating the first evaluation information for each transaction category; and then fusing the accumulated first evaluation information with the second evaluation information to obtain the target evaluation information.

[0052] Specifically, for the i-th candidate feature, the first evaluation information of this candidate feature under each transaction category is accumulated, and then the accumulated first evaluation information is multiplied by the second evaluation information to obtain the target evaluation information. The formula for accumulating the first evaluation information of the i-th candidate feature under each transaction category can be expressed as:

[0053]

[0054] Optionally, the method for determining the target evaluation information for each candidate feature based on the first evaluation information and the second evaluation information may be: to merge the first evaluation information under each transaction category with the second evaluation information to obtain the intermediate evaluation information under each transaction category; and to determine the largest intermediate evaluation information as the target evaluation information.

[0055] Specifically, for the i-th candidate feature, the first evaluation information of the candidate feature under each transaction category is multiplied by the second evaluation information to obtain the intermediate evaluation information under each transaction category; finally, the largest intermediate evaluation information is determined as the target evaluation information.

[0056] In this embodiment, after obtaining the target evaluation information of each candidate feature, the target evaluation information is sorted in reverse order or in ascending order, and finally a certain number of candidate features with high ranking or a certain number of candidate features with low ranking are extracted as target features.

[0057] Optionally, after the target feature is selected from the plurality of candidate features based on the first evaluation information and the second evaluation information, the method further includes the following steps: obtaining the text to be recognized; and performing abnormality detection on the text to be recognized based on the target feature to obtain a detection result of the text to be recognized.

[0058] The detection result includes that the text to be recognized is normal transaction text or that the text to be recognized is abnormal transaction text. The process of performing abnormality detection on the text to be recognized based on the target feature can be: first, pre-processing the text to be recognized, such as cleaning, and then performing word segmentation processing on the pre-processed text to be recognized to obtain a plurality of words; matching the plurality of words with the target feature, if more than a first set proportion of target features match part of the plurality of words, then the text to be recognized is abnormal transaction text, and if less than a second set proportion of target features match part of the plurality of words, then the text to be recognized is normal transaction text. The first set proportion is greater than the second set proportion, for example, the first set proportion is set to 50%, and the second set proportion is set to 10%.

[0059] The technical scheme of this embodiment includes the following steps: obtaining a target text set; the target text set includes text subsets of a plurality of transaction categories; performing feature extraction on each text in the target text set to obtain a plurality of candidate features; determining first evaluation information and second evaluation information of each candidate feature based on the target text set; the first evaluation information and the second evaluation information are both used to reflect the contribution degree of the candidate feature to category division; and selecting a target feature from the plurality of candidate features based on the first evaluation information and the second evaluation information. The data processing method provided by the embodiment of the application selects the candidate features based on the first evaluation information and the second evaluation information reflecting the contribution degree to category division, so that the selected features are more conducive to the detection of abnormal transaction text, and the reliability of the selected features can be improved, and the detection efficiency and accuracy of abnormal transaction text can be improved.

[0060] On the basis of the above-mentioned embodiments, Figure 2 is a flowchart of a data processing method provided by an embodiment of the application, as Figure 2 shown, the method includes the following steps:

[0061] S210, obtaining a target text set.

[0062] S220, pre-processing each text in the target text set; performing word segmentation processing on each pre-processed text to obtain a plurality of words; and determining the plurality of words as a plurality of candidate features.

[0063] S230, determine the first proportion, the second proportion, the third proportion, the fourth proportion and the fifth proportion based on the target text set, and determine the first evaluation information of the candidate feature under each transaction category according to the first proportion, the second proportion, the third proportion, the fourth proportion and the fifth proportion.

[0064] S240, determine the occurrence frequency of the candidate feature in each text subset as the first frequency; determine the mean value of the occurrence frequency of the candidate feature in each text subset and the mean value of the occurrence frequency in the target text set as the first mean value and the second mean value respectively; and determine the number of texts containing the candidate feature in the target text set and each text subset as the first text number and the second text number.

[0065] S250, determine the inter-class difference information of the candidate feature according to the first mean value, the second mean value, the first text number and the second text number; and determine the intra-class difference information of the candidate feature according to the first frequency, the second mean value and the first text number.

[0066] S260, multiply the inter-class difference information and the intra-class difference information to obtain the second evaluation information of the candidate feature.

[0067] S270, fuse the first evaluation information under each transaction category with the second evaluation information respectively to obtain the intermediate evaluation information under each transaction category; and determine the maximum intermediate evaluation information as the target evaluation information.

[0068] S280, sort the target evaluation information in reverse order, and extract the candidate features corresponding to the target evaluation information in the front of the set number as the target features.

[0069] Figure 3 is a structural schematic diagram of a data processing device provided by an embodiment of the present application, as shown in the figure, the device comprises: Figure 3

[0070] a target text set acquisition module 310, configured to acquire a target text set; wherein the target text set comprises text subsets of multiple transaction categories;

[0071] a candidate feature extraction module 320, configured to perform feature extraction on each text in the target text set to obtain multiple candidate features;

[0072] an evaluation information determination module 330, configured to determine the first evaluation information and the second evaluation information of each candidate feature based on the target text set; wherein the first evaluation information and the second evaluation information are both used to reflect the contribution degree of the candidate feature to the category division;

[0073] a target feature screening module 340, configured to screen the target features from the multiple candidate features based on the first evaluation information and the second evaluation information.​

[0074] Optionally, the candidate feature extraction module 320 is further configured to:

[0075] preprocess each text in the target text set;

[0076] perform word segmentation processing on each preprocessed text to obtain a plurality of words;

[0077] determine the plurality of words as a plurality of candidate features.

[0078] Optionally, the evaluation information determination module 330 is further configured to:

[0079] for each transaction category, determine a proportion of the number of texts contained between the text subset under the transaction category and the target text set as a first proportion;

[0080] for each candidate feature, determine a proportion between the number of texts containing the candidate feature in the text subset under the transaction category and the number of texts in the target text set and a proportion between the number of texts not containing the candidate feature in the text subset under the transaction category and the number of texts in the target text set as a second proportion and a third proportion, respectively;

[0081] determine a proportion between the number of texts containing the candidate feature in the target text set and the number of texts in the target text set and a proportion between the number of texts not containing the candidate feature in the target text set and the number of texts in the target text set as a fourth proportion and a fifth proportion, respectively;

[0082] determine the first evaluation information of the candidate feature under each transaction category according to the first proportion, the second proportion, the third proportion, the fourth proportion and the fifth proportion.

[0083] Optionally, the evaluation information determination module 330 is further configured to:

[0084] for each candidate feature, determine inter-class difference information and intra-class difference information of the candidate feature based on the target text set;

[0085] determine the second evaluation information of the candidate feature according to the inter-class difference information and the intra-class difference information.

[0086] Optionally, the evaluation information determination module 330 is further configured to:

[0087] determine the frequency of occurrence of the candidate feature in each text subset as a first frequency;

[0088] determine the mean of the frequency of occurrence of the candidate feature in each text subset and the mean of the frequency of occurrence in the target text set as a first mean and a second mean, respectively;

[0089] determine the number of texts containing the candidate feature in the target text set and each text subset as the first text number and the second text number;

[0090] determine the inter-class difference information and the intra-class difference information of the candidate feature based on at least one of the first frequency, the first average value, the second average value, the first text number and the second text number.

[0091] Optionally, the evaluation information determination module 330 is further configured to:

[0092] determine the inter-class difference information of the candidate feature according to the first average value, the second average value, the first text number and the second text number;

[0093] determine the intra-class difference information of the candidate feature according to the first frequency, the second average value and the first text number.

[0094] Optionally, the evaluation information determination module 330 is further configured to:

[0095] multiply the inter-class difference information and the intra-class difference information to obtain the second evaluation information of the candidate feature.

[0096] Optionally, the target feature screening module 340 is further configured to:

[0097] determine the target evaluation information of each candidate feature according to the first evaluation information and the second evaluation information;

[0098] perform reverse sequence sorting on the target evaluation information, and extract candidate features corresponding to the target evaluation information with a set number of top rankings as the target features; or

[0099] perform forward sequence sorting on the target evaluation information, and extract candidate features corresponding to the target evaluation information with a set number of last rankings as the target features.

[0100] Optionally, the target feature screening module 340 is further configured to:

[0101] accumulate the first evaluation information under each transaction category;

[0102] fuse the accumulated first evaluation information and the second evaluation information to obtain the target evaluation information.

[0103] Optionally, the target feature screening module 340 is further configured to:

[0104] fuse the first evaluation information under each transaction category and the second evaluation information respectively to obtain intermediate evaluation information under each transaction category;

[0105] determine the maximum intermediate evaluation information as the target evaluation information.

[0106] Optionally, the method further comprises a text detection module configured to:

[0107] obtaining to-be-recognized text;

[0108] performing anomaly detection on the to-be-recognized text based on the target feature, to obtain a detection result of the to-be-recognized text.

[0109] The device can perform the method provided by all the foregoing embodiments of the application, and has the corresponding function modules and beneficial effects of performing the foregoing method. Technical details not described in the embodiment can be referred to the method provided by all the foregoing embodiments of the application.

[0110] Figure 4 A structural diagram of an electronic device 10 that can be used to implement embodiments of the application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the application described and / or claimed in this document.

[0111] As shown in Figure 4 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0112] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, a speaker, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0113] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, and the like. The processor 11 performs various methods and processes described above, such as a data processing method.

[0114] In some embodiments, the data processing method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the data processing method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the data processing method by any other suitable means, such as by means of firmware.

[0115] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0116] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0117] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0118] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0119] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0120] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0121] The embodiment of the present application further provides a computer program product comprising a computer program which, when executed by a processor, implements the data processing method provided by any embodiment of the present application.

[0122] The computer program product can be written in one or more programming languages or combinations of languages including object-oriented languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0123] Note that the above only describes the preferred embodiments of the present application and the applied technical principles. Those skilled in the art will understand that the present application is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A data processing method, characterized by, The method comprises the following steps: acquiring a target text set, wherein the target text set comprises text subsets of multiple transaction categories; extracting features from each text in the target text set to obtain multiple candidate features; determining first evaluation information and second evaluation information of each candidate feature based on the target text set, wherein the first evaluation information and the second evaluation information are used to reflect the contribution degree of the candidate feature to category division; screening a target feature from the multiple candidate features based on the first evaluation information and the second evaluation information; determining the first evaluation information of each candidate feature based on the target text set comprises the following steps: for each transaction category, determining the proportion of the number of texts contained between the text subset under the transaction category and the target text set as a first proportion; for each candidate feature, determining the proportion between the number of texts containing the candidate feature in the text subset under the transaction category and the number of texts in the target text set, and the proportion between the number of texts not containing the candidate feature in the text subset under the transaction category and the number of texts in the target text set, as a second proportion and a third proportion respectively; determining the proportion between the number of texts containing the candidate feature in the target text set and the number of texts in the target text set, and the proportion between the number of texts not containing the candidate feature in the target text set and the number of texts in the target text set, as a fourth proportion and a fifth proportion respectively; determining the first evaluation information of the candidate feature under each transaction category according to the first proportion, the second proportion, the third proportion, the fourth proportion and the fifth proportion; determining the second evaluation information of each candidate feature based on the target text set comprises the following steps: for each candidate feature, determining the inter-class difference information and the intra-class difference information of the candidate feature based on the target text set; multiplying the inter-class difference information and the intra-class difference information to obtain the second evaluation information of the candidate feature; for each candidate feature, determining the inter-class difference information and the intra-class difference information of the candidate feature based on the target text set comprises the following steps: determining the frequency of the candidate feature in each text subset as a first frequency; determining the mean value of the frequency of the candidate feature in each text subset and the mean value of the frequency of the candidate feature in the target text set as a first mean value and a second mean value respectively; determining the number of texts containing the candidate feature in the target text set and each text subset as a first text number and a second text number; determining the inter-class difference information of the candidate feature according to the first mean value, the second mean value, the first text number and the second text number; determining the intra-class difference information of the candidate feature according to the first frequency, the second mean value and the first text number.

2. The method of claim 1, wherein, The method comprises the following steps: preprocessing each text in the target text set; performing word segmentation processing on each preprocessed text to obtain multiple words; determining the multiple words as multiple candidate features.

3. The method of claim 1, wherein, screening a target feature from the plurality of candidate features based on the first evaluation information and the second evaluation information, comprising: determining target evaluation information of each of the candidate features according to the first evaluation information and the second evaluation information; performing reverse order sorting on the target evaluation information, and extracting candidate features corresponding to a set number of target evaluation information at the front of the sorting as target features; or performing forward order sorting on the target evaluation information, and extracting candidate features corresponding to a set number of target evaluation information at the rear of the sorting as target features.

4. The method of claim 3, wherein, determining target evaluation information of each of the candidate features according to the first evaluation information and the second evaluation information, comprising: accumulating the first evaluation information under each of the transaction categories; fusing the accumulated first evaluation information and the second evaluation information to obtain target evaluation information.

5. The method of claim 3, wherein, determining target evaluation information of each of the candidate features according to the first evaluation information and the second evaluation information, comprising: fusing the first evaluation information under each of the transaction categories with the second evaluation information respectively to obtain intermediate evaluation information under each of the transaction categories; determining the maximum intermediate evaluation information as target evaluation information.

6. The method of claim 1, wherein, After screening a target feature from the plurality of candidate features based on the first evaluation information and the second evaluation information, further comprising: obtaining a to-be-recognized text; performing abnormality detection on the to-be-recognized text based on the target feature to obtain a detection result of the to-be-recognized text.

7. A data processing apparatus for performing the data processing method of any one of claims 1 to 5, characterized in that, comprising: a target text set acquisition module configured to obtain a target text set; wherein the target text set comprises text subsets of multiple transaction categories; a candidate feature extraction module configured to perform feature extraction on each text in the target text set to obtain a plurality of candidate features; an evaluation information determination module configured to determine first evaluation information and second evaluation information of each of the candidate features based on the target text set; wherein the first evaluation information and the second evaluation information are both used to reflect the contribution degree of the candidate feature to category division; a target feature screening module configured to screen a target feature from the plurality of candidate features based on the first evaluation information and the second evaluation information.

8. An electronic device, comprising: A computer program product comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the data processing method of any one of claims 1-6 when executing the computer program.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the data processing method of any one of claims 1-6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the data processing method of any one of claims 1-6.

Images