Data analysis methods for market supervision data sharing platform

Abstract

The present invention relates to the field of data analysis, and specifically to a data analysis system and method for a market supervision data sharing platform, the system comprising a data management module, a data sharing module, and a node data optimization module; the data management module is used to divide the data held by each node according to a unified market data division standard, and to perform type feature annotation, and also to divide the data required by the node when the user accesses it into data demand elements with a single type feature; the data sharing module calculates the transmission loss of each data demand element in obtaining data from a node holding data of the type feature, and allocates a data acquisition node to each data demand element according to the transmission loss data; the node data optimization module is used to count the number of times a single type feature data is obtained at a holding node and calculate the access ratio, and to periodically optimize the data structure held by the node through an access ratio threshold.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and specifically to a data analysis method for a market supervision data sharing platform. Background Art

[0002] With the continuous development of big data technology in today's society, data has become an important basis for organizational and personal decision-making; and the reasonable analysis and processing of shared data in the data market plays an important role in promoting the healthy development of the market. However, there are some shortcomings in the data processing methods currently used in data sharing platforms:

[0003] Current data sharing methods include data interface sharing, data platform sharing, and data cloud services. Among them, data interface sharing connects data from different systems through interfaces to improve fluidity. However, in actual use, there are inconsistencies in data standards between systems, and different data systems will become an obstacle to data sharing that cannot be ignored when in use. Data platform sharing integrates and summarizes data by building a unified data platform. However, as the scale of data continues to expand, the pressure on data transmission on the unified platform will increase. Data cloud services use cloud computing technology to share data. When in use, due to the redundancy and duplication of data between different platforms, the amount of data in the cloud is huge and complex. In a complex data environment, the abuse of data will seriously affect the data security of users. Summary of the invention

[0004] The purpose of the present invention is to provide a data analysis method for a market supervision data sharing platform to solve the problems raised in the above-mentioned background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] The data analysis method used for the market supervision data sharing platform includes the following analysis steps:

[0007] Step S100: Build a data sharing platform and publish a unified data classification standard. All nodes connected to the platform will label the type characteristics of the data they hold according to the standard;

[0008] Step S200: The user logs in to the data sharing platform using the login information. When using the node function, the login information is combined with the node identifier and encrypted as an access token to access each node;

[0009] Step S300: When the user uses the node function, the node divides the required data into data demand elements, analyzes the type characteristics of each data demand element by analyzing the data sharing platform, and then selects the data acquisition node for each data demand element according to the data transmission loss between nodes;

[0010] Step S400: The data acquisition node extracts type feature data from the held data according to the data demand element for data sharing, and the demand node obtains the data type feature of the verified transmission and combines the data according to the segmentation method of the data demand element;

[0011] Step S500: Analyze the number of times each type of feature data is acquired in each holding node, calculate the access ratio of each type of feature data in the holding node, and optimize the data holding structure of all nodes.

[0012] In the above technical solution, step S100 includes the following contents: each node divides the held data into data elements with a single type feature according to the market unified data division standard published by the sharing platform and labels each data element with a type feature according to the data division standard;

[0013] Compare the data elements held by each node with different types of features, count the nodes holding each type of feature data, and treat the data elements with the same type of features held by the nodes as the shared data between the nodes, and treat the data elements held by the nodes individually as the unique data of the current node;

[0014] By dividing the data held by different nodes according to the same standards and labeling the type characteristics of the divided data, unified standard support is provided for subsequent data sharing between nodes. The use of unified market standards ensures the normalization and standardization of data transmission and sharing among multiple nodes connected to the data sharing platform.

[0015] In the above technical solution, step S200 includes the following contents: the user uses the login information to log in to the shared data platform. When the user accesses the node to use the node function, the shared data platform encrypts the node identifier and the login information to obtain an access token; when the node obtains the access token, the access token is decrypted and the identifier is verified. When the node identifier is consistent with the current node, data is transmitted; each node is accessed using an access token with a node identifier, and an access token is allocated to the user accessing the node, ensuring the security and convenience of the user's access when using the node function, and avoiding illegal access to intercept data during transmission between nodes.

[0016] In the above technical solution, step S300 includes the following contents: when a user uses a node function, the data required by the node accessed by the user is annotated with type characteristics according to the type characteristic annotation method in step S100, and divided into data demand elements with a single type characteristic, and the type characteristics of each data demand element are sent to the data sharing platform; the data sharing platform obtains the type characteristics of each data demand element of the node, and performs data acquisition node matching for each demand element therein; data interactive access is performed according to the node data type characteristics, so as to reduce the transmission pressure caused by traversing redundant calculations node by node;

[0017] Extract the characteristics of any data type, query all nodes holding the current type of characteristic data, query the nodes containing the current user data among all nodes holding the current type of characteristic data, calculate and analyze the transmission loss of obtaining the current type of characteristic data from each node, according to the formula:

[0018]

[0019] Where j is the node number holding the current type feature data, i is the transmission loss evaluation parameter number, L j is the transmission loss of obtaining the current type feature data from the holding node j, α i is the evaluation weight for the transmission loss evaluation parameter i, The parameter value of the transmission loss evaluation parameter i for node j, ε j The inherent loss of data transmission for holding node j;

[0020] Calculate the transmission loss of all data demand units to obtain data from each holding node, and make initial data acquisition node allocation for each data demand unit. When an allocated data acquisition node is in a data transmission state, analyze the node's estimated transmission time, correct the data transmission loss from the current node, and reallocate the data acquisition node for the current data demand unit; evaluate the source node from which the node obtains data based on the actual data transmission loss, and dynamically update the transmission loss data based on the real-time transmission status to ensure the transmission efficiency and information timeliness when the node obtains data.

[0021] In the above technical solution, step S400 includes the following contents: the data acquisition node extracts single-type characteristic data from the held data according to the type characteristics of the data demand element for data sharing and transmission;

[0022] After the data demand node accessed by the user obtains the single type of characteristic data shared by each data acquisition node, the single type of characteristic data is combined according to the data demand element segmentation method in step S300 to obtain the data required by the node.

[0023] In the above technical solution, step S500 includes the following contents: setting a data holding structure optimization cycle, analyzing the historical data of data sharing between nodes through the data sharing platform, counting the number of times each single type of feature data is obtained in each holding node within the period, calculating the access ratio of the single type of feature data in the holding node, and setting the minimum threshold and the maximum threshold;

[0024] When the access ratio of a single type of feature data in the holding node is less than the minimum threshold, the holding node removes the current type of feature data from the held data; when the access ratio of a single type of feature data in the holding node is greater than the maximum threshold, all other holding nodes remove the current type of feature data from the held data and share the current type of feature data with all non-holding nodes;

[0025] By setting the minimum threshold of access ratio, we can identify the nodes whose type feature data is accessed less frequently among the nodes holding them, and determine that the data utilization rate of the nodes is low. We can remove the current type feature data part from the node holding data, reduce the redundant holding data in the node data holding structure, and also reduce the computing cost of the platform when allocating data acquisition nodes.

[0026] By setting the maximum threshold of the access ratio, when the proportion of the number of times the type feature data is accessed in the holding node is greater than the maximum threshold, it can be determined that the current node has undertaken the main shared transmission task of the current type feature data, and the current type feature data is removed from the remaining holding nodes to reduce the redundant holding data of the node, and the type feature data is not currently held by the node, and is evaluated again in the next cycle, thereby allocating the data sharing pressure of the node and ensuring the stability and timeliness of the node data transmission;

[0027] Analyze and calculate the usage correlation of each type of feature data. By counting the characteristic data type of each node when obtaining data from each node through the data sharing platform, calculate the usage correlation of each type of feature data within the cycle. According to the formula:

[0028]

[0029] Among them, x and y are different types of feature data, R x,y is the usage correlation of data x and y, n x,y is the number of times data x and y are used together, n x ,n y are the number of times data x and y are used respectively;

[0030] Calculate the usage correlation of each type of feature data separately, and set the maximum and minimum thresholds of the usage correlation; when the usage correlation of two types of feature data is greater than the maximum threshold of the usage correlation, share the two types of data with the nodes with the highest access ratio in their respective holding nodes; when the usage correlation of two types of feature data is less than the minimum threshold of the usage correlation, and there are common holding nodes for the two types of feature data, remove the data with the smaller access ratio in all the common holding nodes.

[0031] By calculating and analyzing the usage correlation of different types of feature data, the data can be reasonably summarized and held, the efficiency of data transmission and utilization can be improved, the user node access experience can be further improved, and the sharing coupling between data at different nodes can be deepened, thereby realizing dynamic data sharing based on consistent market standards.

[0032] A data analysis system for a market supervision data sharing platform using the data analysis method for a market supervision data sharing platform in the above solution, the system comprising a data management module, a data sharing module, and a node data optimization module;

[0033] The data management module is used to divide the data held by each node according to the unified data division standard of the market, and to mark the type characteristics. It also divides the data required by the node when the user accesses it into data demand elements with a single type of characteristics. The data sharing module calculates the transmission loss of each data demand element to obtain data from the node holding the type characteristic data, and allocates data acquisition nodes to each data demand element according to the transmission loss data. The node data optimization module is used to count the number of times a single type of characteristic data is obtained at the holding node and calculate the access ratio, and periodically optimize the data structure held by the node through the access ratio threshold judgment.

[0034] The output end of the data management module is connected to the input end of the data sharing module and the node data optimization module; the output end of the data sharing module is connected to the input end of the node data optimization module.

[0035] In the above technical solution, the data management module includes a data division unit and a type feature labeling unit;

[0036] The data partitioning unit divides the data held by each node into data elements according to the unified data partitioning standard of the market, and is also used to divide the data required by the node when the user accesses it into data demand elements with a single type feature; the type feature labeling unit is used to label the divided data elements and data demand elements with type features;

[0037] The output end of the data division unit is connected to the input end of the type feature labeling unit; the output end of the type feature labeling unit is connected to the input end of the data sharing module and the node data optimization module.

[0038] In the above technical solution, the data sharing module includes a transmission loss analysis unit and a data acquisition node allocation unit;

[0039] The transmission loss calculation unit is used to calculate the transmission loss when the data demand unit obtains the type characteristic data from the type characteristic data holding node; the data acquisition node allocation unit is used to allocate the data acquisition node to each data demand unit according to the transmission loss data;

[0040] The output end of the transmission loss analysis unit is connected to the input end of the data acquisition node allocation unit; the output end of the data acquisition node allocation unit is connected to the input end of the node data optimization module.

[0041] In the above technical solution, the node data optimization module includes a data sharing monitoring unit and a data holding structure optimization unit;

[0042] The data sharing monitoring unit is used to detect the historical data of data sharing transmission between nodes, count the number of times a single type of feature data is obtained in the holding node and calculate the access ratio; the data holding structure optimization unit judges the node access ratio by setting the access ratio threshold range, and then periodically optimizes the holding data structure;

[0043] The input end of the data sharing monitoring unit is connected to the output end of the data management module and the data sharing module; the output end of the data sharing monitoring unit is connected to the input end of the data holding structure optimization unit.

[0044] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0045] In the present invention, user access is performed through a data sharing platform to achieve single-point access rights between multiple nodes, thereby ensuring convenient user access. In actual data transmission, the data sharing platform does not participate in actual data transmission, but only assumes the responsibility of user access and data acquisition node allocation, thereby avoiding the stability and fluency of the platform caused by a large amount of data transmission. When a user uses a node function, the node is accessed through an access token with a node identifier, and the data transmitted between nodes cannot be directly obtained, thereby avoiding the shared data of the node network being intercepted during transmission. The transmission loss of each data acquisition node when acquiring type feature data is dynamically updated to ensure the efficiency of data transmission and reduce redundant calculations. By analyzing the accessed data of the type feature data in the holding node, an access frequency threshold is set, and the node data holding authority of the type feature data holding node with a low access frequency is removed, the redundant holding data in the node data holding structure is reduced, the data holding structure is optimized, and the computing cost of the platform when allocating data acquisition nodes is reduced. The type feature data of the holding node with a high access frequency is shared and held, and the data sharing pressure of the node is shared. Data sharing is further deepened according to the use correlation between data, the rationality of the data holding structure is maintained, and the stability and timeliness of node data transmission are ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0047] Figure 1It is a flow chart of the data analysis method used in the market supervision data sharing platform of the present invention;

[0048] Figure 2 It is a structural connection diagram of the data analysis system used in the market supervision data sharing platform of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] See also Figure 1-Figure 2 , the present invention provides a technical solution:

[0051] The data analysis method used for the market supervision data sharing platform includes the following analysis steps:

[0052] Step S100: Build a data sharing platform and publish a unified data classification standard, in which all data roots in the market and all nodes connected to the platform will label the data they hold with type characteristics according to the standard;

[0053] Step S200: The user logs in to the data sharing platform using the login information. When using the node function, the login information is combined with the node identifier and encrypted as an access token to access each node;

[0054] Step S300: When the user uses the node function, the node divides the required data into data demand elements, analyzes the type characteristics of each data demand element by analyzing the data sharing platform, and then selects the data acquisition node for each data demand element according to the data transmission loss between nodes;

[0055] Step S400: The data acquisition node extracts type feature data from the held data according to the data demand element for data sharing, and the demand node obtains the data type feature of the verified transmission and combines the data according to the segmentation method of the data demand element;

[0056] Step S500: Analyze the number of times each type of feature data is acquired in each holding node, calculate the access ratio of each type of feature data in the holding node, analyze and calculate the usage association of each type of feature data, and optimize the data holding structure of all nodes.

[0057] Step S100 includes the following contents: each node divides the held data into data elements of a single type feature according to the market unified data division standard published by the sharing platform and labels each data element with a type feature according to the data division standard;

[0058] Compare the data elements held by each node with different types of features, count the nodes holding each type of feature data, and treat the data elements with the same type of features held by the nodes as the shared data between the nodes, and treat the data elements held by the nodes individually as the unique data of the current node;

[0059] Use the data division standard uniformly recognized by the market to divide the data, remove the original data used for user login, and divide the remaining data into data elements. The divided data elements only contain one type of characteristic information; for node-specific data, it can be regarded as shared data held by only a single node;

[0060] In specific implementation, each application node can be divided according to the data required by the business function; for example, to realize the user travel recommendation function, it is necessary to obtain the user's current geographical location data, the geographical location data of surrounding scenic spots, the user's scenic spot browsing data, the user's travel mode browsing data, etc.

[0061] Step S200 includes the following contents: the user uses the login information to log in to the shared data platform. When the user accesses the node to use the node function, the shared data platform encrypts the node identifier and the login information to obtain an access token;

[0062] When the node obtains the access token, it decrypts the access token and performs identity verification. When the node identity is consistent with the current node, data transmission is performed;

[0063] In specific implementation, when a user accesses a node through a data sharing platform, a node identifier is preset between the platform and each node, and combined encryption can be performed based on the access time, node identifier, and user's platform login information to avoid disguised access caused by the leakage of access tokens after fixed data encryption, which may cause data security issues.

[0064] Step S300 includes the following contents: when a user uses a node function, the data required by the node accessed by the user is annotated with type characteristics according to the type characteristic annotation method described in step S100, and divided into data demand elements with a single type characteristic, and the type characteristics of each data demand element are sent to the data sharing platform; the data sharing platform obtains the type characteristics of each data demand element of the node, and performs data acquisition node matching on each demand element;

[0065] In specific implementation, each node obtains the data demand meta-type features segmented from the demand data through the data sharing platform. For example, when a user browses products in application A, application A publishes data demand to the data sharing platform for using the product recommendation function, including the data of products browsed by users recently, the data of popular products recently, and the data of products in the shopping carts of users on various platforms recently. The data sharing platform publishes data demand meta-data to the nodes holding the relevant data and calculates the data transmission loss.

[0066] Extract the characteristics of any data type, query all nodes holding the current type of characteristic data, query the nodes containing the current user data among all nodes holding the current type of characteristic data, calculate and analyze the transmission loss of obtaining the current type of characteristic data from each node, according to the formula:

[0067]

[0068] Where j is the node number holding the current type feature data, i is the transmission loss evaluation parameter number, L j is the transmission loss of obtaining the current type feature data from the holding node j, α i is the evaluation weight for the transmission loss evaluation parameter i, The parameter value of the transmission loss evaluation parameter i for node j, ε j The inherent loss of data transmission for holding node j;

[0069] When nodes experience data transmission loss, a distributed computing method is used. Each node calculates the transmission loss of the data demand metadata by itself, and sends the transmission loss data to the data sharing platform for comparison through edge computing, and then allocates each data demand metadata to reduce the computing pressure of the platform when processing multi-point data demands;

[0070] When actually calculating the data transmission loss between nodes, transmission loss assessment parameters such as predicted transmission time and data update time can be selected; among them, the predicted transmission time can be calculated based on the data transmission speed, transmission delay, and data packet size between application servers, and the data update time indicates the time when the node obtains the characteristic data of this type. For example, if a user adds an item to the shopping cart in application P a week ago and adds an item to the shopping cart in application Q an hour ago, the data update time of application P and application Q can be set to be one week ago and one hour ago respectively. The timeliness of the data is quantified according to the node data update time, and the data update time is introduced as a consideration standard in the data transmission loss assessment;

[0071] Calculate the transmission loss of all data demand elements in obtaining data from each holding node, initialize the data acquisition node allocation for each data demand element, and when an allocated data acquisition node is in a data transmission state, analyze the node's estimated transmission time, correct the data transmission loss from the current node, and reallocate the data acquisition node for the current data demand element.

[0072] Step S400 includes the following contents: the data acquisition node extracts single-type characteristic data from the held data according to the type characteristics of the data demand element for data sharing and transmission;

[0073] After the data demand node accessed by the user obtains the single type of feature data shared by each data acquisition node, the single type of feature data is combined according to the data demand element segmentation method in step S300 to obtain the data required by the node.

[0074] In step S500, a data holding structure optimization cycle is set, and the historical data of data sharing between nodes is analyzed through the data sharing platform. The number of times each single type of feature data is obtained in each holding node within the period is counted, the access ratio of the single type of feature data in the holding node is calculated, and the minimum threshold and maximum threshold are set;

[0075] When the access ratio of a single type of feature data in the holding node is less than the minimum threshold, the holding node removes the current type of feature data from the held data; when the access ratio of a single type of feature data in the holding node is greater than the maximum threshold, all other holding nodes remove the current type of feature data from the held data and share the current type of feature data with all non-holding nodes;

[0076] Set the minimum threshold of access ratio, identify the nodes whose type feature data is accessed less frequently among the nodes holding it, determine the nodes with low data utilization, remove the current type feature data part in the node holding data, reduce the redundant holding data in the node data holding structure, improve the utilization rate of the type feature data held by each node, and reduce the computing cost of the platform when allocating data acquisition nodes;

[0077] Set the maximum threshold of access ratio. When the number of times that type feature data is accessed in the holding node is greater than the maximum threshold, it can be determined that the current node has undertaken the main shared transmission task of the current type feature data. Then the current type feature data is removed from the remaining holding nodes to reduce the redundant holding data of the node, and the node that does not currently hold the type feature data is re-evaluated in the next cycle to share the data sharing pressure of the node and ensure the stability and timeliness of node data transmission. Through periodic optimization, multiple rounds of screening are performed on the nodes holding type feature data to improve the scalability of the data sharing platform and the universality of the data analysis method in the context of dynamically changing data structures.

[0078] Analyze and calculate the usage correlation of each type of feature data. By counting the characteristic data type of each node when obtaining data from each node through the data sharing platform, calculate the usage correlation of each type of feature data within the cycle. According to the formula:

[0079]

[0080] Among them, x and y are different types of feature data, R x,y is the usage correlation of data x and y, n x,n is the number of times data x and y are used together, nx ,n y are the number of times data x and y are used respectively;

[0081] Calculate the usage correlation of each type of feature data respectively, and set the maximum and minimum thresholds of the usage correlation; when the usage correlation of two types of feature data is greater than the maximum threshold of the usage correlation, share the two types of data with the nodes with the highest access ratio in their respective holding nodes; when the usage correlation of two types of feature data is less than the minimum threshold of the usage correlation, and there are common holding nodes for the two types of feature data, remove the data with the smaller access ratio in the two types of feature data from all the common holding nodes;

[0082] In specific implementation, when application M and application N each hold the user's height, weight data and clothing transaction data, when the user uses the clothing recommendation function in application K, the height, weight data and clothing transaction data of the user using application M and application N are obtained at the same time, then the number of times the user uses the clothing recommendation function is counted into the number of times the height, weight data and clothing transaction data are used together, and then the usage correlation between the two feature types of data is calculated by counting the number of times the two feature types of data are used respectively; when the usage correlation of the two feature types of data is greater than the set maximum usage correlation threshold, the user data of each other is shared with application M and application N, and recorded as the holding node of the two feature types of data; if there is a common holding node of the two feature types of data, and the usage correlation of the two feature types of data is less than the set minimum usage correlation threshold, the access ratio of the two types of feature data in the common holding node is calculated, and the feature type data with a smaller access ratio is removed.

[0083] The data analysis system for the market supervision data sharing platform includes a data management module, a data sharing module, and a node data optimization module; the output end of the data management module is connected to the input end of the data sharing module and the node data optimization module; the output end of the data sharing module is connected to the input end of the node data optimization module;

[0084] The data management module is used to divide the data held by each node according to the unified data division standard of the market, and to mark the type characteristics. It also divides the data required by the node when the user accesses it into data demand elements with a single type of characteristics. The data sharing module calculates the transmission loss of each data demand element to obtain data from the node holding the type characteristic data, and allocates data acquisition nodes to each data demand element according to the transmission loss data. The node data optimization module is used to count the number of times a single type of characteristic data is obtained at the holding node and calculate the access ratio, and periodically optimize the data structure held by the node through the access ratio threshold judgment.

[0085] The data management module includes a data division unit and a type feature labeling unit; the output end of the data division unit is connected to the input end of the type feature labeling unit; the output end of the type feature labeling unit is connected to the input end of the data sharing module and the node data optimization module;

[0086] The data partitioning unit divides the data held by each node into data elements according to the unified data partitioning standard of the market, and is also used to divide the data required by the node when the user accesses it into data demand elements with a single type feature; the type feature labeling unit is used to label the divided data elements and data demand elements with type features; the data management module promotes a unified data partitioning standard to all nodes connected to the data sharing platform, thereby improving the versatility of data and reducing the resistance to data sharing;

[0087] The data sharing module includes a transmission loss analysis unit and a data acquisition node allocation unit; the output end of the transmission loss analysis unit is connected to the input end of the data acquisition node allocation unit; the output end of the data acquisition node allocation unit is connected to the input end of the node data optimization module;

[0088] The transmission loss calculation unit is used to calculate the transmission loss when the data demand unit obtains the type characteristic data from the type characteristic data holding node; the data acquisition node allocation unit is used to allocate the data acquisition node to each data demand unit according to the transmission loss data;

[0089] The node data optimization module includes a data sharing monitoring unit and a data holding structure optimization unit; the input end of the data sharing monitoring unit is connected to the data management module and the output end of the data sharing module; the output end of the data sharing monitoring unit is connected to the input end of the data holding structure optimization unit;

[0090] The data sharing monitoring unit is used to detect the historical data of data sharing transmission between nodes, count the number of times a single type of feature data is obtained in the holding node and calculate the access ratio; the data holding structure optimization unit judges the node access ratio by setting the access ratio threshold range, and then periodically optimizes the holding data structure;

[0091] The data sharing monitoring unit monitors the application server gateway data of each node, counts the historical records of data sharing between nodes in each cycle, and statistically analyzes the historical data of type feature data being accessed in the holding node as the basis for optimizing the node data holding structure.

[0092] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0093] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical type features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. Data analysis methods for market supervision data sharing platforms, the type of which is characterized by: The method comprises the following analysis steps: Step S100: Build a data sharing platform and publish a unified data classification standard, in which all data roots in the market and all nodes connected to the platform will label the data they hold with type characteristics according to the standard; Step S200: The user logs in to the data sharing platform using the login information. When using the node function, the login information is combined with the node identifier and encrypted as an access token to access each node; Step S300: When the user uses the node function, the node divides the required data into data demand elements, analyzes the type characteristics of each data demand element by analyzing the data sharing platform, and then selects the data acquisition node for each data demand element according to the data transmission loss between nodes; When the user uses the node function, the data required by the node accessed by the user is annotated with type characteristics according to the data classification standard described in step S100, and divided into data demand elements with a single type characteristic, and the type characteristics of each data demand element are sent to the data sharing platform; the data sharing platform obtains the type characteristics of each data demand element of the node, and performs data acquisition node matching for each demand element; Extract the characteristics of any data type, query all nodes holding the current type of characteristic data, query the nodes containing the current user data among all nodes holding the current type of characteristic data, calculate and analyze the transmission loss of obtaining the current type of characteristic data from each node, according to the formula: Where j is the node number holding the current type feature data, i is the transmission loss evaluation parameter number, L j is the transmission loss of obtaining the current type feature data from the holding node j, α i The evaluation weight for the transmission loss evaluation parameter i, The parameter value of the transmission loss evaluation parameter i for node j, ε j The inherent loss of data transmission for holding node j; Calculate the transmission loss of all data demand elements to obtain data from each holding node, and perform initial data acquisition node allocation for each data demand element. When there is an allocated data acquisition node in the data transmission state, analyze the node's estimated transmission time, correct the data transmission loss from the current node, and reallocate the data acquisition node of the current data demand element; Step S400: The data acquisition node extracts type feature data from the held data according to the data demand element for data sharing, and the demand node obtains the data type feature of the verified transmission and combines the data according to the segmentation method of the data demand element; Step S500: Analyze the number of times each type of feature data is obtained in each holding node, calculate the access ratio of each type of feature data in the holding node, analyze and calculate the usage correlation of each type of feature data, and optimize the data holding structure of all nodes; Set the data holding structure optimization cycle, analyze the historical data of data sharing between nodes through the data sharing platform, count the number of times each single type of feature data is obtained in each holding node within the period, calculate the access ratio of each type of feature data in the holding node, and set the minimum and maximum thresholds; When the access ratio of a single type of feature data in the holding node is less than the minimum threshold, the holding node removes the current type of feature data from the held data; when the access ratio of a single type of feature data in the holding node is greater than the maximum threshold, all other holding nodes remove the current type of feature data from the held data and share the current type of feature data with all non-holding nodes; Analyze and calculate the usage correlation of each type of feature data. By counting the characteristic data type of each node when obtaining data from each node through the data sharing platform, calculate the usage correlation of each type of feature data within the cycle. According to the formula: Among them, x and y are different types of feature data, R x,y is the usage correlation of data x and y, n x,y is the number of times data x and y are used together, n x ,n y are the number of times data x and y are used respectively; Calculate the usage correlation of each type of feature data separately, and set the maximum and minimum thresholds of the usage correlation; when the usage correlation of two types of feature data is greater than the maximum threshold of the usage correlation, share the two types of data with the nodes with the highest access ratio in their respective holding nodes; when the usage correlation of two types of feature data is less than the minimum threshold of the usage correlation, and there are common holding nodes for the two types of feature data, remove the data with the smaller access ratio in all the common holding nodes.

2. The data analysis method for a market supervision data sharing platform according to claim 1 is characterized in that: Step S100 includes the following contents: each node divides the held data into data elements with a single type feature according to the market unified data division standard published by the sharing platform and labels each data element with a type feature; The data elements held by each node are compared in terms of type characteristics, and the nodes holding each type of characteristic data are counted. The data elements with the same type of characteristics held by the nodes are regarded as the shared data between the nodes, and the data elements held by the nodes individually are regarded as the unique data of the current node.

3. The data analysis method for a market supervision data sharing platform according to claim 1 is characterized in that: Step S200 includes the following contents: the user uses the login information to log in to the shared data platform. When the user accesses the node to use the node function, the shared data platform encrypts the node identifier and the login information to obtain an access token; When the node obtains the access token, it decrypts the access token and performs identity verification. When the node identity is consistent with the current node, data transmission is performed.

4. The data analysis method for a market supervision data sharing platform according to claim 1 is characterized in that: Step S400 includes the following contents: the data acquisition node extracts single-type characteristic data from the held data according to the type characteristics of the data demand element for data sharing and transmission; After the data demand node accessed by the user obtains the single type of feature data shared by each data acquisition node, the single type of feature data is combined according to the data demand element segmentation method in step S300 to obtain the data required by the node.

5. A data analysis system for a market supervision data sharing platform using the data analysis method for a market supervision data sharing platform as described in any one of claims 1 to 4, wherein the system comprises a data management module, a data sharing module, and a node data optimization module; The data management module is used to divide the data held by each node according to the unified data division standard of the market, and to perform type feature annotation, and also divide the data required by the node when the user accesses it into data demand elements with a single type feature; the data sharing module calculates the transmission loss of each data demand element to obtain data from the node holding the type feature data, and allocates a data acquisition node to each data demand element according to the transmission loss data; the node data optimization module is used to count the number of times a single type feature data is obtained at the holding node and calculate the access ratio, and periodically optimize the data structure held by the node through the access ratio threshold judgment; The output end of the data management module is connected to the data sharing module and the input end of the node data optimization module; the output end of the data sharing module is connected to the input end of the node data optimization module.

6. The data analysis system for the market supervision data sharing platform according to claim 5, wherein the data management module comprises a data partitioning unit and a type feature labeling unit; The data partitioning unit partitions the data held by each node into data elements according to the unified market data partitioning standard, and is also used to partition the data required by the node when the user accesses it into data demand elements with a single type feature; the type feature labeling unit is used to label the partitioned data elements and data demand elements with type features; The output end of the data division unit is connected to the input end of the type feature labeling unit; the output end of the type feature labeling unit is connected to the input end of the data sharing module and the node data optimization module.

7. The data analysis system for market supervision data sharing platform according to claim 5, wherein the data sharing module comprises a transmission loss analysis unit and a data acquisition node allocation unit; The transmission loss calculation unit is used to calculate the transmission loss when the data demand unit obtains the type characteristic data from the type characteristic data holding node; the data acquisition node allocation unit is used to allocate the data acquisition node to each data demand unit according to the transmission loss data; The output end of the transmission loss analysis unit is connected to the input end of the data acquisition node allocation unit; the output end of the data acquisition node allocation unit is connected to the input end of the node data optimization module.

8. The data analysis system for market supervision data sharing platform according to claim 5, wherein the node data optimization module comprises a data sharing monitoring unit and a data holding structure optimization unit; The data sharing monitoring unit is used to detect the historical data of data sharing transmission between nodes, count the number of times a single type of characteristic data is obtained in the holding node and calculate the access ratio; the data holding structure optimization unit determines the node access ratio by setting the access ratio threshold range, and then periodically optimizes the holding data structure; The input end of the data sharing monitoring unit is connected to the output end of the data management module and the data sharing module; the output end of the data sharing monitoring unit is connected to the input end of the data holding structure optimization unit.

Images