A data mining method

Abstract

The present invention relates to the field of data processing technology, and specifically discloses a data mining method, the method comprising obtaining regional logistics information, determining regional purchasing power based on the regional logistics information, and determining a collection index based on the regional purchasing power; randomly obtaining a preset number of file data containing user information uploaded by regional personnel; wherein the preset number is determined by the collection index; performing content recognition on the file data, and generating target data based on the content recognition result; inputting the target data into a trained data analysis model to generate an evaluation report; wherein the evaluation report at least includes keywords. The present invention locates users through logistics data, obtains file data based on user wishes, and performs data mining based on the obtained file data, thereby improving user participation in the data mining process and having high authenticity.

Description

Technical Field

[0001] The invention relates to the technical field of data processing, in particular to a data mining method. Background Art

[0002] Data mining is a hot topic in the field of artificial intelligence and database research. Data mining refers to the non-trivial process of revealing implicit, previously unknown and potentially valuable information from a large amount of data in a database. Data mining is a decision support process that is mainly based on artificial intelligence, machine learning, pattern recognition, statistics, databases, visualization technology, etc. It analyzes corporate data in a highly automated manner, makes inductive inferences, and mines potential patterns to help decision makers adjust market strategies, reduce risks, and make correct decisions.

[0003] Existing data mining technologies are generally based on user data. This mining process is actually very dependent on user data, which will inevitably cause user disgust. The authenticity of the data mined based on user disgust is difficult to guarantee. Summary of the invention

[0004] The purpose of the present invention is to provide a data mining method to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A data mining method, comprising:

[0007] Acquire regional logistics information, determine regional purchasing power based on the regional logistics information, and determine acquisition indicators based on the regional purchasing power;

[0008] Randomly obtain a preset number of file data containing user information uploaded by regional personnel; wherein the preset number is determined by the collection index;

[0009] Performing content recognition on the file data, and generating target data according to the content recognition result;

[0010] The target data is input into a trained data analysis model to generate an evaluation report; wherein the evaluation report at least includes keywords.

[0011] As a further limitation of the technical solution of the present invention: the steps of obtaining regional logistics information, determining regional purchasing power according to the regional logistics information, and determining the acquisition index according to the regional purchasing power include:

[0012] Obtain the logistics data of any logistics service provider with the region as the destination, and obtain the market share of the logistics service provider;

[0013] Calculate regional logistics data based on the logistics data and the market share; the regional logistics data includes total value, total weight and transportation cost;

[0014] The regional purchasing power is determined according to the regional logistics data, and the acquisition index is determined according to the regional purchasing power.

[0015] As a further limitation of the technical solution of the present invention: the step of randomly acquiring a preset number of file data containing user information uploaded by regional personnel includes:

[0016] Generate a file acquisition request containing rewards based on the statistical feedback ratio and randomly send it to the regional personnel in the area;

[0017] The personnel in the receiving area upload the file data containing the user information, calculate the number of files, compare the number of files with the reference number, and determine the revised number; wherein the revised number includes a symbol;

[0018] Determine a correction file according to the correction quantity, and correct the file data according to the correction file;

[0019] Among them, the reward is related to the file type, and the regional personnel belong to the recipient in the logistics information.

[0020] As a further limitation of the technical solution of the present invention: the receiving area personnel uploads file data containing user information, calculates the number of files, compares the number of files with the reference number, and determines the corrected number by the steps of:

[0021] Receiving file data containing user information uploaded by personnel in the receiving area, identifying the format of the file data, and determining the unit value according to the format of the file data;

[0022] Obtaining the quantity of each type of file data, and calculating the number of files according to the quantity and the unit value;

[0023] Calculating the difference between the number of files and a preset reference number to obtain a corrected number;

[0024] The types of the file data include text data, image data and audio data.

[0025] As a further limitation of the technical solution of the present invention: the step of performing content recognition on the file data and generating target data according to the content recognition result comprises:

[0026] When the file data is text data, traverse the text information in the text file to obtain the separator position;

[0027] Convert the text information into a string array based on the delimiter position and calculate the total number of strings;

[0028] Content recognition is performed on the string array according to the total number of strings, and target data is generated according to the content recognition result.

[0029] As a further limitation of the technical solution of the present invention: the step of performing content recognition on the string array according to the total number of strings and generating target data according to the content recognition result comprises:

[0030] Comparing the total number of character strings with a preset stratification value, and grading the character strings according to the comparison result;

[0031] When the total number of the character strings is below the minimum hierarchical value, extracting keywords from the character strings;

[0032] When the total number of the character string is below the maximum hierarchical value, performing part-of-speech analysis on the character string to obtain a feature word, and marking the character string as a key sentence according to the feature word;

[0033] When the total number of the character strings is between the minimum stratification value and the maximum stratification value, the character strings are subjected to part-of-speech analysis to obtain feature words, and the feature words are extracted according to the preset part-of-speech structure to generate feature sentences.

[0034] As a further limitation of the technical solution of the present invention: the step of performing content recognition on the file data and generating target data according to the content recognition result further includes:

[0035] When the file data is audio data, decompressing the audio data to generate a pure waveform file;

[0036] The pure waveform file is silenced and framed to generate multiple segments of speech;

[0037] Extracting pure acoustic features from the multiple speech segments to generate a feature matrix;

[0038] The text data is confirmed based on the feature matrix, and content recognition is performed on the text data to obtain target data.

[0039] As a further limitation of the technical solution of the present invention: the step of performing content recognition on the file data and generating target data according to the content recognition result further includes:

[0040] When the file data is image data, the image data is input into a trained block model to obtain a sub-image group;

[0041] Performing content recognition on the sub-image group, and when text data exists in the sub-image, marking the sub-image and extracting corresponding text data;

[0042] The text data is connected and content recognition is performed, and target data is generated according to the content recognition result.

[0043] As a further limitation of the technical solution of the present invention: the step of performing content recognition on the file data and generating target data according to the content recognition result further includes:

[0044] When the file data is video data, extracting audio data and image data from the video data;

[0045] The audio data and the image data are converted into text data, and content recognition is performed on the text data to generate target data.

[0046] As a further limitation of the technical solution of the present invention: the method further comprises:

[0047] Establishing a connection channel with a sensitive word library, traversing the target data in units of single words, locating sensitive words and obtaining corresponding sensitive addresses;

[0048] Acquire a word to be checked based on the sensitive address, establish a connection channel with a sensitive word library, determine whether the word to be checked is a sensitive word based on the sensitive word library, and calculate the number of sensitive words with a sensitive level according to the determination result;

[0049] The risk level of the target data is determined according to the number of sensitive words containing the sensitivity level, and when the risk level reaches a preset level threshold, the target data is sent to the manual end.

[0050] Compared with the prior art, the beneficial effects of the present invention are: the present invention locates users through logistics data, obtains file data based on user wishes, and performs data mining based on the acquired file data, thereby improving user participation in the data mining process with higher authenticity. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0052] Figure 1 A flowchart of a data mining method is shown.

[0053] Figure 2 A first sub-process flowchart of the data mining method is shown.

[0054] Figure 3A second sub-flow block diagram of the data mining method is shown.

[0055] Figure 4 A third sub-process flowchart of the data mining method is shown.

[0056] Figure 5 A fourth sub-process flowchart of the data mining method is shown.

[0057] Figure 6 A fifth sub-process flowchart in the data mining method is shown. DETAILED DESCRIPTION

[0058] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0059] Example 1

[0060] Figure 1 A flow chart of a data mining method is shown. In an embodiment of the present invention, a data mining method is provided. The method includes steps S100 to S400:

[0061] Step S100: Acquire regional logistics information, determine regional purchasing power according to the regional logistics information, and determine acquisition indicators according to the regional purchasing power;

[0062] Logistics information represents the most real needs of a region. Data extraction and mining based on logistics information can ensure the authenticity of data mining results to the greatest extent.

[0063] Step S200: randomly acquiring a preset number of file data containing user information uploaded by regional personnel; wherein the preset number is determined by the collection index;

[0064] For a region, there is a lot of logistics information, and it is updated every moment. If a comprehensive analysis method is used, the computing pressure will be too high. In fact, when the number of samples obtained reaches a certain threshold, the authenticity of the data mined from these samples has received the result of the comprehensive analysis. Therefore, the process of step S200 is to obtain a certain amount of file data and perform data mining from point to surface.

[0065] Step S300: performing content recognition on the file data, and generating target data according to the content recognition result;

[0066] Step S300 is the core step in the technical solution of the present invention, which performs content recognition on the file data. After content recognition, the original file data is converted into target data. It can be imagined that the size of the target data is much smaller than the file data. In other words, the target data is the focus of the file data.

[0067] Step S400: input the target data into the trained data analysis model to generate an evaluation report; wherein the evaluation report at least includes keywords.

[0068] The target data is a text composed of keywords or key sentences, which is the output of the data mining process. Integrating the target data and then generating a corresponding evaluation report and converting it into a form that is easier for users to read is the function that step S400 wants to accomplish.

[0069] Figure 2 The first sub-process flowchart of the data mining method is shown, wherein the steps of obtaining regional logistics information, determining regional purchasing power according to the regional logistics information, and determining acquisition indicators according to the regional purchasing power include steps S101 to S103:

[0070] Step S101: Obtaining logistics data of any logistics service provider with the region as the destination, and obtaining the market share of the logistics service provider;

[0071] There are many logistics service providers, and it is somewhat difficult to obtain the data of each logistics service provider. However, it is easy to reach a cooperative service with a single logistics service provider. Based on the data of any logistics service provider and its corresponding market share, the overall logistics information of the area can be roughly calculated.

[0072] Step S102: Calculating regional logistics data according to the logistics data and the market share; the regional logistics data includes total value, total weight and transportation cost;

[0073] There are a lot of regional logistics data. In addition to total value, total weight and transportation cost, some other indicators such as total volume or total guarantee can also be used as a type of regional logistics data.

[0074] Step S103: Determine the regional purchasing power according to the regional logistics data, and determine the collection index according to the regional purchasing power.

[0075] The implementation process of step S103 is not difficult. The key point is the relationship between regional logistics data, regional purchasing power and collection indicators. This relationship is an empirical formula fitted according to actual conditions. It is one of the most basic problems in statistics and the technical solution of the present invention will not be elaborated here.

[0076] Figure 3The second sub-flow chart of the data mining method is shown, wherein the step of randomly acquiring a preset number of file data containing user information uploaded by regional personnel includes steps S201 to S203:

[0077] Step S201: Generate a file acquisition request containing a reward according to the statistical feedback ratio, and randomly send it to the regional personnel in the area;

[0078] If you want to obtain 100 pieces of file data, then the file acquisition requests sent are definitely more than 100, and may even be thousands or tens of thousands of file acquisition requests. The function of step S201 is to determine the number of requests based on the statistical feedback ratio.

[0079] Step S202: receiving file data containing user information uploaded by personnel in the receiving area, calculating the number of files, comparing the number of files with the reference number, and determining a revised number; wherein the revised number includes a symbol;

[0080] Step S203: determining a correction file according to the correction quantity, and correcting the file data according to the correction file;

[0081] Among them, the reward is related to the file type, and the regional personnel belong to the recipient in the logistics information.

[0082] The preset feedback ratio is generally related to some historical data. For example, in the historical records, 100 pieces of file data may be received after sending 1,000 requests. When 1,000 requests are sent and only 90 pieces of file data are received, it means that there are certain problems. The difference of 10 pieces of file data also represents certain information. Therefore, the data mined from the 90 pieces of file data needs to be corrected.

[0083] Figure 4 The third sub-flow diagram of the data mining method is shown, wherein the receiving area personnel uploads file data containing user information, calculates the number of files, compares the number of files with the reference number, and determines the corrected number, including the following steps:

[0084] Step S2021: receiving file data containing user information uploaded by regional personnel, identifying the format of the file data, and determining the unit value according to the format of the file data;

[0085] Step S2022: Obtain the number of file data of each type, and calculate the number of files according to the number and the unit value;

[0086] Step S2023: Calculate the difference between the number of files and a preset reference number to obtain a corrected number;

[0087] The types of the file data include text data, image data and audio data.

[0088] The information fed back by a user is different. It can be imagined that image information is generally more important than text data. From a computer perspective, one image information is equivalent to multiple text information. Therefore, the calculation of the file volume needs to be unified. First, determine the unit value corresponding to each type of file, and then calculate the number of files based on the number of files of various types.

[0089] Figure 5 The fourth sub-flow chart of the data mining method is shown, wherein the step of performing content recognition on the file data and generating target data according to the content recognition result comprises steps S301 to S303:

[0090] Step S301: when the file data is text data, traverse the text information in the text file to obtain the separator position;

[0091] Step S302: converting the text information into a string array based on the separator position, and calculating the total number of strings;

[0092] Step S303: performing content recognition on the string array according to the total number of strings, and generating target data according to the content recognition result.

[0093] Steps S301 to S303 provide a method for identifying text data. The text data identification process is the bottom-end identification process of the technical solution of the present invention. Other types of files will eventually be converted into text data for content identification. The text data processing process is to first convert the text data into multiple short sentences, and then perform content identification on the multiple short sentences. The conversion process requires the identification of delimiters.

[0094] Figure 6 The fifth sub-flow chart of the data mining method is shown, wherein the step of performing content recognition on the string array according to the total number of strings and generating target data according to the content recognition result includes steps S3031 to S3034:

[0095] Step S3031: comparing the total number of character strings with a preset stratification value, and grading the character strings according to the comparison result;

[0096] Step S3032: when the total number of the character strings is below the minimum hierarchical value, extracting keywords from the character strings;

[0097] Step S3033: When the total number of the character strings is below the maximum hierarchical value, the character strings are subjected to part-of-speech analysis to obtain feature words, and the character strings are marked as key sentences according to the feature words;

[0098] Step S3034: When the total number of the character string is between the minimum stratification value and the maximum stratification value, the character string is subjected to part-of-speech analysis to obtain feature words, and the feature words are extracted according to the preset part-of-speech structure to generate a feature sentence.

[0099] Steps S3031 to S3034 are further descriptions of the text data recognition process, wherein there are three main types of character strings: one is a very short character string, of which there are only a limited number, and therefore, only keywords need to be extracted; another is a very long character string, and for these character strings, when a character string has a large number of feature words, it is marked as a key sentence; finally, for the character string in the middle, which is also the largest type of character string, the original character string is converted into a feature sentence composed of feature words according to the results of part-of-speech analysis; a feature sentence is just a stack of feature words and does not necessarily have to constitute a meaningful sentence.

[0100] Furthermore, the step of performing content recognition on the file data and generating target data according to the content recognition result further includes:

[0101] When the file data is audio data, decompressing the audio data to generate a pure waveform file;

[0102] The pure waveform file is silenced and framed to generate multiple segments of speech;

[0103] Extracting pure acoustic features from the multiple speech segments to generate a feature matrix;

[0104] The text data is confirmed based on the feature matrix, and content recognition is performed on the text data to obtain target data.

[0105] General audio inputs are all time domain waveforms. We know that sound is actually a kind of wave. Common formats such as mp3 and wmv are compressed formats and must be converted into uncompressed pure waveform files.

[0106] Silence removal is a common operation on time domain waveforms, the purpose of which is to reduce interference. This is a common signal processing technology. The time domain waveform must be framed, that is, the waveform is cut into small segments, each small segment is called a frame, and the framing operation is usually implemented using a moving window function. Of course, there are some preprocessing operations before framing, which is not the content of the present invention and will not be described in detail here.

[0107] The waveform has almost no descriptive ability in the time domain, so the waveform must be converted. The most common transformation method is to extract the MFCC features and turn each frame of the waveform into a vector. This vector is 12-dimensional and can also be called a matrix. This extraction method is based on the physiological characteristics of the human ear. In this way, the audio information is converted into calculable data. Of course, in practical applications, there are many details, which will not be elaborated here.

[0108] Confirming text information based on feature matrix is ​​a relatively mature existing technology. In summary, it is to identify frames into states, which is the difficulty, and then combine the states into phonemes, and finally, combine the phonemes into words. There are many algorithms that can complete these processes, and the present invention will not describe them in detail; the focus of the present invention is to send the text information to the text recognition module. The purpose of this step is to convert audio information into text information, and use the text recognition module to perform information recognition. The advantage of this is that changing the performance of the text recognition module will change the performance of the audio recognition module, which is convenient for later maintenance and management.

[0109] Specifically, the step of performing content recognition on the file data and generating target data according to the content recognition result further includes:

[0110] When the file data is image data, the image data is input into a trained block model to obtain a sub-image group;

[0111] Performing content recognition on the sub-image group, and when text data exists in the sub-image, marking the sub-image and extracting corresponding text data;

[0112] The text data is connected and content recognition is performed, and target data is generated according to the content recognition result.

[0113] The recognition of image data mainly involves the recognition of text box contents. The function of the above content is to extract the contents of a text box in a picture and convert it into text data for content recognition.

[0114] Example 2

[0115] Different from the first embodiment, in the embodiment of the present invention, the step of performing content recognition on the file data and generating target data according to the content recognition result further includes:

[0116] When the file data is video data, extracting audio data and image data from the video data;

[0117] The audio data and the image data are converted into text data, and content recognition is performed on the text data to generate target data.

[0118] For video, it is a combination of audio and image. Under the premise that the technical solution of the present invention can recognize audio and image, the recognition process of video only has one more step of conversion process.

[0119] In addition, the data mining method further comprises:

[0120] Establishing a connection channel with a sensitive word library, traversing the target data in units of single words, locating sensitive words and obtaining corresponding sensitive addresses;

[0121] Acquire a word to be checked based on the sensitive address, establish a connection channel with a sensitive word library, determine whether the word to be checked is a sensitive word based on the sensitive word library, and calculate the number of sensitive words with a sensitive level according to the determination result;

[0122] The risk level of the target data is determined according to the number of sensitive words containing the sensitivity level, and when the risk level reaches a preset level threshold, the target data is sent to the manual end.

[0123] The target data is the data sent by the user. There is a high possibility that illegal information may exist in these data, so sensitive detection is required. First, sensitive word detection is performed on the text, and then sensitive word detection is performed.

[0124] There is no unique way to confirm the words to be checked by sensitive words. Here is one confirmation method: get two words forward from the sensitive word, and then get two words backward, and combine the five characters. There are four two-character words, three three-character words, two four-character words and one five-character word. These words are all subjected to sensitive word detection as described in the text recognition module; of course, you can also get one word forward from the sensitive word, and then get one word backward, which is all possible; all sensitive words are detected, and finally the number of sensitive words is generated.

[0125] The functions that can be achieved by the above-mentioned data mining method are all completed by a computer device, and the computer device includes one or more processors and one or more memories, and at least one program code is stored in the one or more memories. The program code is loaded and executed by the one or more processors to achieve the functions of the data mining method.

[0126] The processor takes out instructions from the memory one by one, analyzes the instructions, and then completes the corresponding operations according to the instruction requirements, generating a series of control commands, so that the various parts of the computer can automatically, continuously and coordinately move to become an organic whole, realize the input of programs, the input of data, and the calculation and output of results. The arithmetic operations or logical operations generated in this process are all completed by the operator; the memory includes a read-only memory (ROM), which is used to store computer programs, and a protection device is provided outside the memory.

[0127] Exemplarily, the computer program may be divided into one or more modules, one or more modules are stored in a memory and executed by a processor to implement the present invention. One or more modules may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program in a terminal device.

[0128] Those skilled in the art will understand that the description of the above service equipment is merely an example and does not constitute a limitation on the terminal equipment. It may include more or fewer components than described above, or a combination of certain components, or different components, for example, it may include input and output devices, network access devices, buses, etc.

[0129] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire user terminal.

[0130] The memory can be used to store computer programs and / or modules. The processor can realize various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as an information collection template display function, a product information release function, etc.); the data storage area can store data created according to the use of the berth status display system (such as product information collection templates corresponding to different product types, product information that different product providers need to release, etc.). In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0131] If the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable medium. Based on this understanding, the present invention implements all or part of the modules / units in the above-mentioned embodiment system, and can also be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable medium, and the computer program can realize the functions of the above-mentioned various system embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. Computer-readable media may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0132] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0133] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A data mining method, characterized in that: The method comprises: Acquire regional logistics information, determine regional purchasing power based on the regional logistics information, and determine acquisition indicators based on the regional purchasing power; Randomly obtain a preset number of file data containing user information uploaded by regional personnel; wherein the preset number is determined by the collection index; Performing content recognition on the file data, and generating target data according to the content recognition result; Inputting the target data into a trained data analysis model to generate an evaluation report; wherein the evaluation report at least includes keywords; The steps of obtaining regional logistics information, determining regional purchasing power according to the regional logistics information, and determining acquisition indicators according to the regional purchasing power include: Obtain the logistics data of any logistics service provider with the region as the destination, and obtain the market share of the logistics service provider; Calculate regional logistics data based on the logistics data and the market share; the regional logistics data includes total value, total weight and transportation cost; Determine regional purchasing power according to the regional logistics data, and determine acquisition indicators according to the regional purchasing power; The step of randomly acquiring a preset number of file data containing user information uploaded by regional personnel includes: Generate a file acquisition request containing rewards based on the statistical feedback ratio and randomly send it to the regional personnel in the area; The personnel in the receiving area upload the file data containing the user information, calculate the number of files, compare the number of files with the reference number, and determine the revised number; wherein the revised number includes a symbol; Determine a correction file according to the correction quantity, and correct the file data according to the correction file; Among them, the reward is related to the file type, and the regional personnel belong to the recipient in the logistics information.

2. The data mining method according to claim 1, characterized in that: The receiving area personnel uploads the file data containing the user information, calculates the number of files, and compares the number of files with the reference number to determine the corrected number, including: Receiving file data containing user information uploaded by personnel in the receiving area, identifying the format of the file data, and determining the unit value according to the format of the file data; Obtaining the quantity of each type of file data, and calculating the number of files according to the quantity and the unit value; Calculating the difference between the number of files and a preset reference number to obtain a corrected number; The types of the file data include text data, image data and audio data.

3. The data mining method according to claim 1, characterized in that: The step of performing content recognition on the file data and generating target data according to the content recognition result comprises: When the file data is text data, traverse the text information in the text file to obtain the separator position; Convert the text information into a string array based on the delimiter position and calculate the total number of strings; Content recognition is performed on the string array according to the total number of strings, and target data is generated according to the content recognition result.

4. The data mining method according to claim 3, characterized in that: The step of performing content recognition on the string array according to the total number of strings and generating target data according to the content recognition result comprises: comparing the total number of strings with a preset hierarchical value and grading the strings according to the comparison result; When the total number of the character strings is below the minimum hierarchical value, extracting keywords from the character strings; When the total number of the character string is below the maximum hierarchical value, performing part-of-speech analysis on the character string to obtain a feature word, and marking the character string as a key sentence according to the feature word; When the total number of the character strings is between the minimum stratification value and the maximum stratification value, the character strings are subjected to part-of-speech analysis to obtain feature words, and the feature words are extracted according to the preset part-of-speech structure to generate feature sentences.

5. The data mining method according to claim 3, characterized in that: The step of performing content recognition on the file data and generating target data according to the content recognition result further includes: When the file data is audio data, decompressing the audio data to generate a pure waveform file; The pure waveform file is silenced and framed to generate multiple segments of speech; Extracting pure acoustic features from the multiple speech segments to generate a feature matrix; The text data is confirmed based on the feature matrix, and content recognition is performed on the text data to obtain target data.

6. The data mining method according to claim 3, characterized in that: The step of performing content recognition on the file data and generating target data according to the content recognition result further includes: When the file data is image data, the image data is input into a trained block model to obtain a sub-image group; Performing content recognition on the sub-image group, and when text data exists in the sub-image, marking the sub-image and extracting corresponding text data; The text data is connected and content recognition is performed, and target data is generated according to the content recognition result.

7. The data mining method according to any one of claims 3 to 6, characterized in that: The step of performing content recognition on the file data and generating target data according to the content recognition result further includes: When the file data is video data, extracting audio data and image data from the video data; The audio data and the image data are converted into text data, and content recognition is performed on the text data to generate target data.

8. The data mining method according to any one of claims 1 to 6, characterized in that: The method further comprises: Establishing a connection channel with a sensitive word library, traversing the target data in units of single words, locating sensitive words and obtaining corresponding sensitive addresses; Acquire a word to be checked based on the sensitive address, establish a connection channel with a sensitive word library, determine whether the word to be checked is a sensitive word based on the sensitive word library, and calculate the number of sensitive words with a sensitive level according to the determination result; The risk level of the target data is determined according to the number of sensitive words containing the sensitivity level, and when the risk level reaches a preset level threshold, the target data is sent to the manual end.

Images