Data analysis method, data analysis device, electronic device and storage medium

Abstract

The present application is applicable to the field of big data technology, and provides a data analysis method, device, electronic device and computer-readable storage medium, wherein the method comprises: obtaining at least one original data stream through a preset data platform; classifying each original data stream according to the data category; cleaning the classified original data stream to obtain a valid data stream; calling a preset analysis tool, extracting target valid data according to the correlation between each valid data stream, and analyzing the target valid data to obtain an analysis map under the dimension to be analyzed. Through the above method, the analysis efficiency and analysis accuracy of data analysis can be improved.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a data analysis method, a data analysis device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Nowadays, due to the changes in the macroeconomic situation and the successive collapse of Internet financial companies, financial cases are on the rise. For financial cases, there are often a large number of parties involved and the public is eager to recover their losses. During the review and prosecution stage, the procuratorate often faces a large number of files and complex accounts, and analyzes the case from them.

[0003] Faced with complex transactions, massive amounts of information and tedious analysis work, traditional analysis graphs are merely used as a data import tool to analyze some or all cases under manual selection, which may lead to relatively scattered analysis results and make it difficult to obtain scenario-based and systematic analysis results. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a data analysis method, a data analysis device, an electronic device and a computer-readable storage medium, which can remove redundant data before analysis and enhance the relevance of the data to be analyzed, thereby improving the efficiency and accuracy of data analysis.

[0005] A first aspect of an embodiment of the present application provides a data analysis method, comprising:

[0006] Obtain at least one original data stream through a preset data platform;

[0007] Classify each raw data stream according to data category;

[0008] Clean the classified original data stream to obtain a valid data stream;

[0009] Call the preset analysis tool, extract the target valid data according to the correlation between each valid data stream, and analyze the above target valid data to obtain the analysis map under the dimension to be analyzed.

[0010] A second aspect of an embodiment of the present application provides a data analysis device, including:

[0011] An acquisition unit, used for acquiring at least one original data stream through a preset data platform;

[0012] A classification unit, used for classifying each original data stream according to data category;

[0013] A cleaning unit, used for cleaning the classified original data stream to obtain a valid data stream;

[0014] The analysis unit is used to call a preset analysis tool, extract target valid data according to the correlation between each valid data stream, and analyze the above target valid data to obtain an analysis map under the dimension to be analyzed.

[0015] A third aspect of an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the electronic device, wherein the processor implements each step of the data analysis method provided by the first solution when executing the computer program.

[0016] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, each step of the data analysis method provided by the first solution is implemented.

[0017] By implementing the data analysis method provided in the embodiment of the present application, first, at least one raw data stream is obtained through a preset data platform, and then each raw data stream is classified according to the data category, and the classified raw data stream is cleaned to obtain a valid data stream, and then the preset analysis tool is called to extract the target valid data according to the correlation between each valid data stream, and the target valid data is analyzed to obtain an analysis map under the dimension to be analyzed. In the above process, the raw data contained in the raw data stream is summarized and sorted by the classification operation of the raw data, so that the originally disordered raw data can be ordered; by the cleaning operation of the raw data, useless data can be screened out, so that only valid data meaningful to subsequent analysis operations is retained after cleaning; and before data analysis, the target valid data is extracted as the data to be analyzed based on the correlation between the valid data streams, which can remove data redundancy, thereby improving data analysis efficiency and obtaining scenario-based and structured analysis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0019] Figure 1 is a flow chart for implementing the data analysis method provided in the embodiment of the present application;

[0020] Figure 2 is a flowchart for implementing step 102 in the data analysis method provided in an embodiment of the present application;

[0021] Figure 3 is a flowchart for implementing step 104 in the data analysis method provided in an embodiment of the present application;

[0022] Figure 4 is another implementation flow chart of step 104 in the data analysis method provided in an embodiment of the present application;

[0023] Figure 5 It is an example diagram of a fund transaction relationship network diagram in the data analysis method provided in the embodiment of the present application;

[0024] Figure 6 is a structural block diagram of a data analysis device provided in an embodiment of the present application;

[0025] Figure 7 It is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0027] The data analysis method involved in the embodiments of the present application can be applied to electronic devices such as servers, desktop computers, mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0028] See also Figure 1 , Figure 1 FIG. 1 shows a flow chart of a data analysis method according to an embodiment of the present application. Figure 1 As shown, the data analysis method provided in this embodiment may include:

[0029] Step 101: Obtain at least one original data stream through a preset data platform.

[0030] In this embodiment, the electronic device may be integrated with a data analysis system, and the data analysis system may be connected to a preset data platform. The data platform refers to a platform with storage, computing and presentation as the purpose, which can provide services such as data access, data processing and data storage for certain specific data categories. Exemplarily, each data platform has set up its own access interface; after that, the electronic device can access the data in each data platform through the access interface to obtain at least one original data stream.

[0031] In some embodiments, the number of the preset data platforms may be one, two or more, and the number of data platforms is not limited here. Based on this, the data analysis system may also adopt a distributed framework to improve the concurrency of data analysis.

[0032] As an example only, the original data stream in this embodiment can be obtained based on the case to be analyzed in the court, such as a financial case, and is not limited here. Different data platforms can import different original data streams based on the same case to be analyzed. For example, data platform 1 can import the bank flow data stream of financial case 1; data platform 2 can import the data stream of the persons involved in financial case 1. The original data streams that can be imported by different data platforms are not limited here.

[0033] In some embodiments, the electronic device may summarize and sort the original data streams obtained after importing the original data streams through various data platforms, for example, integrating two or more original data streams belonging to the same financial case to form a new original data stream.

[0034] Step 102: Classify the above-mentioned original data streams according to data categories.

[0035] In this embodiment, the data contained in the original data stream is the original data. In an original data stream, multiple original data belonging to different categories may be included. As an example only, for the bank flow data stream obtained for financial cases, it may contain various original data such as the fund outflow party, the fund inflow party, the fund amount and the fund flow time; among them, the fund outflow party and the fund inflow party both belong to the category of the subject of the institution involved in the case, the fund amount belongs to the transaction amount category, and the fund flow time belongs to the transaction time category. Since each original data stream may come from different data platforms, and the data categories of the original data contained in different original data streams are different, the original data in the original data stream is relatively disordered; based on this, in this embodiment, the original data contained in each original data stream can be classified, so that the originally disordered original data can be ordered, and the data category to which each original data belongs is obtained, so as to realize the classification of the original data stream.

[0036] In some embodiments, the data analysis system has been pre-divided into several data categories. Exemplarily, the above data categories are usually divided based on content words. For example, the set data categories can include two categories: digital category and non-digital category; among them, the digital category can be further divided into: age category of persons involved in the case, transaction amount category and transaction time category, etc.; the non-digital category can be further divided into: subject category of persons involved in the case, subject category of institutions involved in the case and transaction area category, etc.; the division method of data categories is not limited here. Based on this, please refer to Figure 2 , the above step 102 may specifically include:

[0037] Step 1021, extracting the content words contained in each original data stream respectively to obtain the original data contained in each original data stream;

[0038] Among them, considering that data categories are usually classified based on content words, in order to improve the efficiency of data analysis, only content words contained in each original data stream are extracted here, and the extracted content words are the original data contained in each original data stream.

[0039] Step 1022: for each original data stream, each original data contained in the original data stream is matched with at least one preset data category;

[0040] Among them, each data category has its corresponding format and / or rules. For example, for the transaction time category, it requires that the corresponding data must contain date information, and the date information has its inherent format, such as day / month / year, or year / month / day, or month / day / year, etc.; and the data corresponding to the transaction time category may also contain time information, and the time information also has its inherent format, such as hour: minute: second. Based on this, for any original data stream, the original data stream can be matched with the format and / or rules associated with any data category. If the match is successful, it is considered that the original data stream belongs to the data category; otherwise, the original data stream continues to be matched with the format and / or rules associated with the next data category until the match is successful, so as to determine the data category to which each original data belongs.

[0041] Step 1023, based on the matching result, the data category to which each original data belongs is determined to classify the original data stream.

[0042] Although the original data of each data category can be determined according to the matching results, each original data is still associated with the original data stream from which it comes. For example, each classified original data can be marked, specifically, the source location of each original data is marked, and the source location is used to indicate the original data stream to which the original data belongs.

[0043] In some embodiments, considering that the purpose of analyzing financial cases is to capture the criminal suspects of financial cases, therefore, under the category of the subjects involved in the case, the key original data (that is, the key subjects involved in the case) can be determined through feature extraction operations, and the key original data can be marked; the feature extraction operation can be: for each original data (that is, the subject involved in the case), according to the original data stream to which the above original data belongs, the financial transactions between the above original data and a third party within a preset time period are detected; if the financial transactions with the third party exceed the preset financial threshold, the above original data are determined as key original data. For example, under the subject category of persons involved in the case, there are three subjects A, B and C (original data). Through the bank transaction data flow associated with A, it can be known that the financial transactions between A and the third party within half a year are X1, and through the bank transaction data flow associated with B, it can be known that the financial transactions between B and the third party within half a year are X2. Through the bank transaction data flow associated with C, it can be known that the financial transactions between C and the third party within half a year are X3, among which only X2 exceeds the preset financial threshold; therefore, at this time, B can be determined as the key subject involved in the case (key original data), and B and all original data flows associated with B can be marked by highlighting so that users can grasp the key points when reviewing.

[0044] Step 103, cleaning the classified original data stream to obtain a valid data stream.

[0045] In the embodiments of the present application, it is considered that the original data is often entered into the data platform manually or by optical character recognition (OCR), and these two methods cannot guarantee 100% input accuracy, which may result in invalid data in the original data. Since the data format and data rules to be followed of the original data under the same data category are often the same, the above-mentioned classified original data stream can be cleaned, and the cleaned data stream obtained is the valid data stream.

[0046] In some embodiments, the above cleaning operation is mainly performed on the original data belonging to the data category of the digital class. Based on this, the data category belonging to the digital class can be first screened out from at least one preset data category as the data category to be cleaned, and then for each data category to be cleaned, the original data under the above data category to be cleaned is cleaned to obtain valid data under the above data category to be cleaned. The above cleaning operation includes but is not limited to a blank data filling operation, a noise data removal operation and / or an illegal data removal operation.

[0047] In one application scenario, the cleaning operation includes a blank data filling operation; then the step 103 is specifically performed as follows: for each data category to be cleaned, detect whether there is blank data under the data category to be cleaned; if there is blank data, calculate the mean of the original data under the data category to be cleaned, and fill the blank data with the mean of the original data. For example, under the age category of the person involved in the case, there is a missing age data; then the electronic device can calculate the mean of the original data under the age category of the person involved in the case (that is, the mean age of the person involved in the case), and fill the mean into the missing age data.

[0048] In another application scenario, the above cleaning operation includes a noise data removal operation; then the above step 103 is specifically manifested as follows: for each data category to be cleaned, a corresponding noise data interval has been pre-set in the data analysis system; then when a cleaning operation is required, it is detected whether there is data (i.e., noise data) belonging to the corresponding noise data interval under the above data category to be cleaned; if there is noise data, the noise data is removed. For example, under the age category of the persons involved in the case, considering that there are fewer minors and octogenarians who commit financial cases in real life, the data analysis system can set two noise data intervals, which are less than 16 and greater than 80 (in years old); when the original data under the age category of the persons involved in the case is cleaned, if it is detected that the original data is 8, then the original data is abnormal data belonging to the noise data interval of less than 16, and the original data can be removed at this time.

[0049] In another application scenario, the above cleaning operation includes an illegal data removal operation; then the above step 103 is specifically performed as follows: for each data category to be cleaned, a corresponding illegal data interval has been pre-set in the data analysis system; then when a cleaning operation is required, it is detected whether there is data belonging to the corresponding illegal data interval (i.e., illegal data) under the above data category to be cleaned; if there is illegal data, the above illegal data is removed. For example, under the transaction time category, it may happen that due to an abnormality in the operating system or a virus attack, the input original data (i.e., transaction time) is "x month y day, 1900", which is obviously an illegal time. When performing a cleaning operation on the original data under the transaction time category, the illegal data "x month y day, 1900" needs to be removed to ensure that the remaining original data are all legal data.

[0050] In some embodiments, considering that both noise data and illegal data are data that need to be eliminated, therefore, for each category of data to be cleaned, the electronic device can take the union of the noise data interval and the illegal data interval to obtain an abnormal data interval, so that the original data in the abnormal data interval corresponding to each category of data to be cleaned can be eliminated; that is, the noise data and illegal data are collectively referred to as abnormal data, and the noise data removal operation and / or illegal data removal are combined into one abnormal data removal operation.

[0051] Step 104, calling a preset analysis tool, extracting target valid data according to the association relationship between each valid data stream, and analyzing the target valid data to obtain an analysis graph under the dimension to be analyzed;

[0052] In this embodiment, several analysis dimensions have been pre-configured in the data analysis system, wherein each analysis dimension is associated with at least one data category. After determining the analysis dimension of concern for this analysis operation, that is, the dimension to be analyzed, the preset analysis tool can be called based on the dimension to be analyzed to extract the target valid data, and analyze the above target valid data to obtain the analysis map under the dimension to be analyzed. It should be noted that the number of the above dimensions to be analyzed is not limited here. Exemplarily, the above analysis dimensions may include: account analysis dimension, multi-dimensional gang analysis dimension and capital penetration analysis dimension, etc., which are not limited here. In addition, each analysis dimension may also include multiple sub-analysis dimensions. For example, for the account analysis dimension, it may include: the sub-analysis dimension of the number of funds in and out, the sub-analysis dimension of the amount of funds in and out, and the sub-analysis dimension of the frequency of abnormal transactions, etc., which are not limited here.

[0053] In this embodiment, the analysis tool may first determine the expression form of the analysis graph to be output for the dimension to be analyzed, and draw the corresponding analysis graph based on the expression form to obtain a visualization result. Exemplarily, there are multiple expression forms of the above analysis graph. For example, for the account analysis dimension, the analysis graph of the sub-analysis dimension of the number of funds in and out can be expressed as a pie chart, the sub-analysis dimension of the amount of funds in and out can be expressed as a bar chart, and the sub-analysis dimension of the frequency of abnormal transactions can be expressed as a bar chart and / or a curve chart. The expression form of the analysis graph adopted by each analysis dimension and its sub-analysis dimension is not limited here.

[0054] In some embodiments, the analysis tool can also determine the display scale of each analysis map based on the report template corresponding to the dimension to be analyzed; then draw and generate graphic elements based on the corresponding display scale and the corresponding expression form, and fill the graphic elements with text based on the target valid data to obtain the analysis map.

[0055] In some embodiments, see Figure 3, the analysis tool can obtain the correlation between each valid data stream through a preset cluster analysis model, and extract the corresponding target valid data to reduce data redundancy during data analysis. The above-mentioned step of calling the preset analysis tool to extract the target valid data according to the correlation between each valid data stream can specifically include:

[0056] Step 1041, vectorize the valid data stream to obtain a valid data stream vector.

[0057] The electronic device can pre-configure multiple vector dimensions, each vector dimension is used to indicate a data category; through the configured multiple vector dimensions, each valid data stream is vectorized to obtain a vectorized expression of the valid data stream, that is, a valid data stream vector. As an example only, the vector dimension may include the subject category A of the institution involved in the case, the transaction amount category B, the transaction time category C, etc.; considering that there may be multiple items of original data contained in a data stream, therefore, in the valid data stream vector generated based on the valid data stream, there may be multiple elements under each dimension.

[0058] Step 1042: construct a cluster analysis model based on a preset clustering algorithm, and input the above-mentioned valid data flow vector into the above-mentioned cluster analysis model to obtain at least one group, each group including at least one valid data flow vector.

[0059] In an embodiment of the present application, each valid data flow vector obtained can be put into a vector space, and each valid data flow vector can be clustered. Specifically, the electronic device can pre-construct a clustering analysis model based on a given clustering algorithm (such as a K-means algorithm or other algorithms), and implement clustering operations through the clustering analysis model. The clustering result obtained is at least one group, and each group can be regarded as a cluster. In this way, there will be at least one valid data flow vector in each group. It can be considered that the valid data flow vectors in the same group have a certain correlation; that is, the valid data streams corresponding to the valid data flow vectors in the same group have a certain correlation.

[0060] Step 1043: extract target valid data according to the at least one group.

[0061] In an embodiment of the present application, some groups can be selected from at least one group obtained by clustering as target groups, and the valid data stream associated with the valid data stream vector in the target group can be determined as target valid data. Specifically, through the clustering algorithm, mutually exclusive groups (i.e., groups with a correlation lower than a preset lower limit of correlation) in multiple groups can be determined, recorded as mutually exclusive groups; and associated groups (i.e., groups with a correlation higher than a preset upper limit of correlation) in multiple groups can be recorded as associated groups. According to the determined associated groups and mutually exclusive groups, the target group can be determined in the above at least one group. For example, assuming that groups 1, 2, 3, 4, and 5 are currently obtained, among which group 1 and group 2 are associated groups; group 4 and group 5 are associated groups; group 2 and group 5 are mutually exclusive groups; then the electronic device can determine group 1, group 2 and / or group 3 as target groups; it can also determine group 3, group 4 and / or group 5 as target groups; and groups 1, 2 and groups 4, 5 cannot be determined as target groups at the same time. That is, the target group is determined according to the mutually exclusive and associated relationships that have been obtained, and mutually exclusive groups cannot be determined as target groups at the same time, and it is recommended to determine the associated groups as target groups at the same time. After determining the target group, the electronic device can determine the valid data under the valid data flow associated with the valid data flow vector in the above target group as the target valid data, that is, the data to be analyzed. It can be understood that the correlation between two groups can be calculated based on the distance between the average vectors of the two groups.

[0062] In some embodiments, in order to achieve targeted data analysis, the electronic device may further filter the data to be analyzed based on the data category associated with the dimension to be analyzed. For example, assuming that the valid data stream associated with the valid data stream vector in the above target group is data stream 1 and 2, the electronic device can extract the valid data belonging to the target data category (that is, the data category associated with the dimension to be analyzed) in data stream 1 and 2 as the target valid data. That is, the electronic device can further filter the valid data stream associated with the valid data stream vector in the above target group, and only retain the valid data belonging to the target data category in the valid data stream as the target valid data.

[0063] In some embodiments, see Figure 4 For financial cases, if the dimension to be analyzed is the capital penetration analysis dimension, the above step 104 may specifically include:

[0064] Step 1044, based on the above target valid data, generate a fund transaction relationship network diagram.

[0065] The target data categories related to the fund penetration analysis dimension include but are not limited to the categories of persons involved in the case, transaction amounts, and institutions involved in the case. Through these target data categories and the target valid data under them, the lending direction can be analyzed; thus, the fund transaction relationship network can be generated based on the subjects of persons involved in the case (that is, the target valid data under the subjects of persons involved in the case), transaction amounts (target valid data under the transaction amount category), and lending directions (represented based on the target valid data under the institution involved in the case category).

[0066] In the fund transaction relationship network diagram, the nodes are persons or institutions involved in the case; the nodes are connected based on the lending direction to indicate the flow of funds. Exemplarily, the connection between the nodes can be marked based on the transaction amount between the nodes. For example, if the transaction amount between two nodes is in the preset first amount interval, the connection between the two nodes is marked as green; if the transaction amount between the two nodes is in the preset second amount interval, the connection between the two nodes is marked as yellow; if the transaction amount between the two nodes is in the preset third amount interval, the connection between the two nodes is marked as red. Among them, the maximum value of the first amount interval is less than or equal to the minimum value of the second amount interval; the maximum value of the second amount interval is less than or equal to the minimum value of the third amount interval. Through the above connection marking operation, the flow of abnormal funds can be quickly found. Alternatively, the connection between the nodes can also be marked based on the lending direction. For example, after selecting the target node, the funds flowing into the target node are marked as blue, and the funds flowing out of the target node are marked as red, so as to quickly know the fund inflow / outflow status of a single node.

[0067] Step 1045, monitoring whether a user instruction input based on the above fund transaction relationship network diagram is received.

[0068] After generating the fund transaction relationship network diagram, the electronic device may display the fund transaction relationship network diagram on its screen for the user to review. The user may input user instructions based on the fund transaction relationship network diagram to implement various operations on the fund transaction relationship network diagram. As an example only, the above user instructions may include: edit instructions and view instructions, wherein the view instructions include click view instructions and hover view instructions.

[0069] Step 1046: If the above user instruction is detected, the above fund transaction relationship network diagram is processed accordingly under the instruction of the above user instruction.

[0070] When the electronic device receives the user's instruction, it can process the fund transaction relationship network diagram based on the user's instruction. Here, the user operates through a computer as an example to explain and illustrate the above step 1053:

[0071] Regarding viewing instructions: the user can input viewing instructions to the electronic device by placing a cursor or clicking the left button of the mouse.

[0072] If the electronic device receives a floating view instruction input based on the fund transaction relationship network diagram and the cursor placement operation, it determines the placement position of the cursor; if the placement position is on a node, the electronic device can display the account details contained in the node in the form of a floating box; if the placement position is on the connection line between nodes, the electronic device can display the total transaction amount associated with the nodes in the form of a floating box.

[0073] If the electronic device receives a click-to-view instruction input based on the fund transaction relationship network diagram and the left-button click operation of the mouse, it determines the click position of the left-button click operation of the mouse; if the click position is a node, the electronic device can select the node and highlight the name of the selected node; if the view instruction based on the click position is received again, the name of the node is hidden.

[0074] Regarding editing instructions: Editing instructions include editing instructions for a single node and editing instructions for more than two nodes.

[0075] For the editing instructions of a single node, the user can first select the node by left-clicking the mouse, and then right-clicking the selected node to call out the function menu, which includes an "Edit" option, and clicks the edit option to edit the relevant information of the node. The relevant information includes: node name, node color, node display size and / or node icon. Optionally, the "Edit" option can also call out the edit panel, which displays an ICON to represent the node type; as an example only, the data analysis system has been pre-set with multiple node types, for example, the node types that can be edited by the counterparty include: real estate, ordinary individuals, high-end consumption, automotive services, general consumption, catering, hotels, travel, stocks, futures, trusts, financial products, consumer finance, currency, securities, banks, insurance, capital markets, derivative securities, investment and financial management, funds (private placement, public offering), international balance of payments, fiscal management, trade finance, real estate finance, foreign exchange management and risk management, etc., which are not limited here.

[0076] For editing instructions of more than two nodes, the user can select the edited node by pressing the control button (Control, Ctrl) on the keyboard and clicking on two or more nodes with the left mouse button at the same time; or, directly select a node area by sliding the left mouse button. The electronic device can highlight the selected edited node. The user calls out the function menu by clicking the right mouse button again, and the function menu includes a "Merge" option. The user presses the "Merge" option with the left mouse button, and the fund transaction relationship network diagram will merge the edited options. The electronic device can choose to render the merged nodes with a blur effect.

[0077] In some embodiments, Figure 5 As shown, after completing the merging process of the edited nodes, the electronic device can also display the nodes before merging in a floating window in a designated area of ​​the screen (eg, the lower right corner area) to achieve comparison between the nodes before and after merging.

[0078] In some embodiments, users can also manually annotate the fund transaction relationship network diagram with color blocks and / or features, and the data analysis system can pass the parameters of the annotated nodes to the background of the data system, so that the background can perform machine learning of similar data to form learning data of the training set, which can be used as a data set to provide more accurate clustering and interface intelligent prompts for subsequent data analysis operations. The background can be a Hadoop platform.

[0079] In some embodiments, after step 104, in order to achieve structured output of data analysis results, the data analysis method may further include:

[0080] Output the above analysis graph to the report template corresponding to the above dimension to be analyzed to obtain an analysis report;

[0081] In this embodiment, the electronic device is pre-set with a corresponding report template for each possible analysis dimension, wherein the report template is set with content to be filled (i.e., blank content). By filling in the content to be filled in the report template based on the previously obtained analysis graph, a complete analysis report can be obtained. As an example only, the report template can be " ① Case, subject of case ② wait ③ People, accounts involved ④ Account, Transaction Period ⑤ to ⑤ , cumulative transaction RMB ⑥ Yuan, involving opponents ⑦ A natural person, ⑧Enterprises. ", the underlined parts ①-⑧ are all to be filled in. The target data can be obtained by querying the analysis graphs output under each dimension to be analyzed (and its sub-analysis dimension), and the target data is filled in the corresponding position of the content to be filled in. Through the report template, the analysis results to be displayed under the corresponding analysis dimension are connected in a structured language, which is convenient for users to check the analysis results.

[0082] In some embodiments, the data analysis method further includes:

[0083] The valid data flow, the analysis graph and / or the analysis report are uploaded to the blockchain.

[0084] Among them, in order to ensure the security of data and fairness and transparency to users, each valid data stream, the above-mentioned analysis map and / or the above-mentioned analysis report can be uploaded to the blockchain for evidence storage. Users can then download and obtain valid data streams, the above-mentioned analysis maps and / or the above-mentioned analysis reports from the blockchain through their respective devices to verify whether these data have been tampered with. The blockchain referred to in this embodiment is a new application model that uses computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism and encryption algorithm. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods, each of which contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. The blockchain may include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0085] As can be seen from the above, a data analysis method provided by this embodiment obtains at least one raw data stream through a preset data platform, and then classifies the above raw data streams according to the data category, and cleans the above raw data streams after classification to obtain a valid data stream, and then calls the preset analysis tool to extract the target valid data according to the correlation between the valid data streams, and analyzes the above target valid data to obtain the analysis map under the dimension to be analyzed. In the above process, the raw data contained in the raw data stream is summarized and sorted by the classification operation of the raw data, so that the originally disordered raw data can be ordered; by cleaning the raw data, useless data can be screened out, so that only valid data meaningful to subsequent analysis operations is retained after cleaning; and before data analysis, the target valid data is extracted as the data to be analyzed based on the correlation between the valid data streams, which can remove data redundancy and obtain scenario-based and structured analysis results; finally, the analysis results are output in the form of a report, which can provide users with more organized and targeted analysis results.

[0086] See also Figure 6 , Figure 61 is a structural block diagram of a data analysis device provided in an embodiment of the present application. In this embodiment, the electronic device includes various units for executing various steps in the above-mentioned data analysis method embodiment. For details, please refer to the relevant description in the embodiment corresponding to the above-mentioned data analysis method. For ease of explanation, only the parts related to this embodiment are shown.

[0087] See also Figure 6 , the data analysis device 6 comprises:

[0088] The acquisition unit 601 is used to acquire at least one original data stream through a preset data platform;

[0089] The classification unit 602 is used to classify each original data stream according to the data category;

[0090] A cleaning unit 603 is used to clean the classified original data stream to obtain valid data;

[0091] The analysis unit 604 is used to call a preset analysis tool, extract target valid data according to the association relationship between each valid data, and analyze the above target valid data to obtain an analysis map under the dimension to be analyzed.

[0092] As an embodiment of the present application, the classification unit 602 includes:

[0093] The content word extraction subunit is used to extract the content words contained in each original data stream respectively to obtain the original data contained in each original data stream;

[0094] A data matching subunit, for matching each original data contained in the original data stream with at least one preset data category;

[0095] The category determination subunit is used to determine the data category to which each original data belongs according to the matching result, so as to classify the original data stream.

[0096] As an embodiment of the present application, the analysis unit 604 includes:

[0097] A vectorization subunit, used for performing vectorization processing on the above valid data to obtain a valid data vector;

[0098] A clustering subunit, configured to construct a clustering analysis model based on a preset clustering algorithm, and input the valid data vector into the clustering analysis model to obtain at least one group, each group including at least one valid data vector;

[0099] The extraction subunit is used to extract target valid data according to the at least one group.

[0100] As an embodiment of the present application, the extraction subunit includes:

[0101] A group relationship determination subunit, used to determine mutually exclusive groups and associated groups in the at least one group according to the cluster analysis model;

[0102] a target group determination subunit, configured to determine a target group in the at least one group based on the mutually exclusive group and the associated group;

[0103] The target valid data determination subunit is used to determine the valid data associated with the valid data vector in the target group as the target valid data.

[0104] As an embodiment of the present application, the cleaning unit 603 includes:

[0105] The data category screening subunit is used to screen out data categories belonging to the digital category from at least one preset data category as the data category to be cleaned;

[0106] The original data cleaning subunit is used to clean the original data of each data category to be cleaned, so as to obtain valid data of the data category to be cleaned.

[0107] As an embodiment of the present application, the above-mentioned cleaning operation includes: a blank data filling operation, a noise data removal operation and / or an illegal data removal operation.

[0108] As an embodiment of the present application, if the dimension to be analyzed is a capital penetration analysis dimension, the analysis unit 604 includes:

[0109] A network diagram generation subunit is used to generate a fund transaction relationship network diagram based on the above target valid data;

[0110] An instruction monitoring subunit is used to monitor whether a user instruction input based on the above fund transaction relationship network diagram is received;

[0111] The network diagram processing subunit is used to process the above-mentioned fund transaction relationship network diagram under the instruction of the above-mentioned user instructions if the above-mentioned user instructions are detected.

[0112] As an embodiment of the present application, the user instruction includes an editing instruction and a floating viewing instruction, and the network diagram processing subunit includes:

[0113] A network diagram editing subunit, for determining the node to be edited in the fund transaction relationship network diagram under the instruction of the editing instruction if an editing instruction input based on the fund transaction relationship network diagram is received, and performing node selection and / or node merging operations on the node to be edited in the fund transaction relationship network diagram;

[0114] The network diagram viewing sub-unit is used to display various target valid data related to the node pointed to by the above-mentioned floating view instruction in the form of a floating box under the instruction of the above-mentioned floating view instruction if it is connected to the floating view instruction input based on the above-mentioned fund transaction relationship network diagram.

[0115] As an embodiment of the present application, the data analysis device 6 further includes:

[0116] The uploading unit is used to upload the above-mentioned valid data and / or the above-mentioned analysis map to the blockchain.

[0117] In the embodiment of the present application, the data analysis device obtains at least one raw data stream through a preset data platform, and then classifies each raw data stream according to the data category, and cleans the classified raw data stream to obtain a valid data stream, and then calls a preset analysis tool to extract the target valid data according to the correlation between each valid data stream, and analyzes the target valid data to obtain an analysis map under the dimension to be analyzed. In the above process, the raw data contained in the raw data stream is summarized and sorted by the classification operation of the raw data, so that the originally disordered raw data can be ordered; by cleaning the raw data, useless data can be screened out, so that only valid data meaningful to subsequent analysis operations is retained after cleaning; and when analyzing the data, the target valid data is extracted as the data to be analyzed based on the correlation between the valid data streams, which can remove data redundancy and obtain scenario-based and structured analysis results; finally, the analysis results are output in the form of a report, which can provide users with more organized and targeted analysis results.

[0118] It should be noted that the information interaction and execution process between the above-mentioned units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0119] Figure 7 is a structural block diagram of an electronic device provided by another embodiment of the present application. Figure 7As shown, the electronic device 7 of this embodiment includes: a processor 71, a memory 72, and a computer program 73 stored in the memory 72 and executable on the processor 71, such as a program of the data analysis method. When the processor 71 executes the computer program 73, the steps in each embodiment of the above-mentioned data analysis method are implemented, such as Figure 1 Alternatively, the processor 71 executes the computer program 73 to implement the above Figure 5 The functions of each unit in the corresponding embodiment are, for example, Figure 5 For details on the functions of the units 501 to 504 shown, please refer to Figure 5 The relevant descriptions in the corresponding embodiments are not repeated here.

[0120] Exemplarily, the computer program 73 may be divided into one or more units, and the one or more units are stored in the memory 72 and executed by the processor 71 to complete the present application. The one or more units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 73 in the electronic device 70. For example, the computer program 73 may be divided into an acquisition unit, a classification unit, a cleaning unit, and an analysis unit, and the specific functions of each unit are as described above.

[0121] The electronic device may include, but is not limited to, a processor 71 and a memory 72. Those skilled in the art will appreciate that Figure 7 It is only an example of the electronic device 7 and does not constitute a limitation of the electronic device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the above-mentioned electronic device may also include input and output devices, network access devices, buses, etc.

[0122] The processor 71 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0123] The memory 72 may be an internal storage unit of the electronic device 7, such as a hard disk or memory of the electronic device 7. The memory 72 may also be an external storage device of the electronic device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 7. Further, the memory 72 may include both an internal storage unit and an external storage device of the electronic device 7. The memory 72 is used to store the computer program and other programs and data required by the electronic device. The memory 72 may also be used to temporarily store data that has been output or is to be output.

[0124] In the embodiment of the present application, when the processor 70 executes the computer program 72 to implement the steps in any of the data analysis method embodiments, the data analysis device obtains at least one raw data stream through a preset data platform, and then classifies each raw data stream according to the data category, and cleans the classified raw data stream to obtain a valid data stream, and then calls a preset analysis tool to extract the target valid data according to the association between each valid data stream, and analyzes the target valid data to obtain an analysis map under the dimension to be analyzed. In the above process, by classifying the raw data, the raw data contained in the raw data stream is summarized and sorted, so that the originally disordered raw data can be ordered; by cleaning the raw data, useless data can be screened out, so that only valid data meaningful to subsequent analysis operations is retained after cleaning; and when analyzing the data, based on the association between the valid data streams, the target valid data is extracted as the data to be analyzed, which can remove data redundancy and obtain scenario-based and structured analysis results; finally, the analysis results are output in the form of a report, which can make the user's review of the analysis results more organized and targeted.

[0125] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned data analysis method embodiments can be implemented.

[0126] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned data analysis method embodiments when executing the computer program product.

[0127] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A data analysis method, characterized in that: include: Obtaining at least one original data stream based on the financial case to be analyzed through a preset data platform, including: importing different original data streams based on the same case to be analyzed through different data platforms, and integrating two or more original data streams belonging to the same case to be analyzed into a new original data stream; Classify each raw data stream according to data category; Cleaning the classified original data stream to obtain a valid data stream; Calling a preset analysis tool, extracting target valid data according to the correlation between each valid data stream, and analyzing the target valid data to obtain an analysis graph under the dimension to be analyzed; Outputting the analysis graph to a report template corresponding to the dimension to be analyzed to obtain an analysis report; The classifying of each original data stream according to the data category includes: Extracting the content words contained in each original data stream respectively to obtain the original data contained in each original data stream; For each original data stream, each original data contained in the original data stream is matched with at least one preset data category; According to the matching results, the data category to which each original data belongs is determined to classify the original data stream; The calling of a preset analysis tool to extract target valid data according to the correlation between each valid data stream includes: Performing vectorization processing on the valid data stream to obtain a valid data stream vector; Building a cluster analysis model based on a preset clustering algorithm, and inputting the valid data flow vector into the cluster analysis model to obtain at least one group, each group including at least one valid data flow vector; Determine mutually exclusive groups and associated groups in the at least one group according to the cluster analysis model; Determining a target group in the at least one group based on the mutually exclusive groups and the associated groups; The valid data flow associated with the valid data flow vector in the target group is determined as the target valid data.

2. The data analysis method according to claim 1, characterized in that: If the dimension to be analyzed is a capital penetration analysis dimension, the target valid data is analyzed to obtain an analysis graph under the dimension to be analyzed, including: Based on the target valid data, generate a fund transaction relationship network diagram; Monitoring whether a user instruction input based on the fund transaction relationship network diagram is received; If the user instruction is detected, the fund transaction relationship network diagram is processed under the instruction of the user instruction.

3. The data analysis method according to claim 2, characterized in that: The user instruction includes an editing instruction and a suspended viewing instruction. If the user instruction is detected, the fund transaction relationship network diagram is processed under the instruction of the user instruction, including: If an editing instruction input based on the fund transaction relationship network diagram is accessed, then under the instruction of the editing instruction, a node to be edited in the fund transaction relationship network diagram is determined, and a node selection and / or node merging operation is performed on the node to be edited in the fund transaction relationship network diagram; If a floating view instruction input based on the fund transaction relationship network diagram is accessed, then under the instruction of the floating view instruction, each target valid data related to the node pointed to by the floating view instruction is displayed in the form of a floating box.

4. The data analysis method according to any one of claims 1 to 3, characterized in that: After analyzing the target valid data to obtain an analysis graph under the dimension to be analyzed, the data analysis method further includes: The valid data stream and / or the analysis graph are uploaded to the blockchain.

5. A data analysis device, characterized in that: include: An acquisition unit is used to acquire at least one original data stream based on the financial case to be analyzed through a preset data platform, including: importing different original data streams based on the same case to be analyzed through different data platforms, and integrating two or more original data streams belonging to the same case to be analyzed into a new original data stream; A classification unit, used for classifying each original data stream according to data category; A cleaning unit, used for cleaning the classified original data stream to obtain a valid data stream; An analysis unit is used to call a preset analysis tool, extract target valid data according to the association relationship between each valid data stream, and analyze the target valid data to obtain an analysis graph under the dimension to be analyzed; The data analysis device is further used to output the analysis graph to a report template corresponding to the dimension to be analyzed to obtain an analysis report; Wherein, the classification unit includes: The content word extraction subunit is used to extract the content words contained in each original data stream respectively to obtain the original data contained in each original data stream; a data matching subunit, configured to match, for each original data stream, each original data contained in the original data stream with at least one preset data category; The category determination subunit is used to determine the data category to which each original data belongs according to the matching result, so as to classify the original data stream. Wherein, the analysis unit comprises: A vectorization subunit, used for performing vectorization processing on the valid data to obtain a valid data vector; A clustering subunit, configured to construct a clustering analysis model based on a preset clustering algorithm, and input the valid data vector into the clustering analysis model to obtain at least one group, each group including at least one valid data vector; an extraction subunit, configured to extract target valid data according to the at least one group; Wherein, the extraction subunit comprises: a group relationship determination subunit, configured to determine mutually exclusive groups and associated groups in the at least one group according to the cluster analysis model; a target group determination subunit, configured to determine a target group in the at least one group based on the mutually exclusive group and the associated group; The target valid data determination subunit is used to determine the valid data associated with the valid data vector in the target group as the target valid data.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images