Big Data-Based Visual Data Information Acquisition System and Its Medium

Abstract

The present invention discloses a visualization data information acquisition system based on big data and its medium, specifically relating to the field of e-commerce, including a data encryption acquisition module, a data cleaning and integration module, a data analysis module, a real-time data monitoring module, a visualization template customization module, and a dynamic data interaction module. The visualization data information acquisition system based on big data and its medium encrypts the data of the target platform through the data encryption acquisition module to reduce the potential risks of data acquisition; performs real-time data processing operations through the real-time data monitoring module and sets data change rules and warning mechanisms to obtain and update data in a timely manner; receives the third key information through the dynamic data interaction module and displays it on the visualization interface in a preset manner, providing users with independent exploration and predictive analysis of dynamic data and increasing the interactivity of the system.

Description

Technical Field

[0001] The present invention relates to the technical field of e-commerce. More specifically, the present invention relates to a visualization data information collection system based on big data and its medium. Background Art

[0002] With the rapid development of information technology and Internet technology, in the technical field of e-commerce, data has become the core asset of enterprise and social development. The collection, processing, and analysis of big data can help enterprises understand market trends, optimize business processes, improve user experience, and ultimately maximize business value.

[0003] The data information collection system greatly improves the comprehensibility and memorability of data by converting complex data sets into intuitive charts and images; in e-commerce, comprehensively collecting user behavior data and clearly presenting the market demand distribution through visualization means can improve marketing effects and user experience.

[0004] However, in actual use, there are still some drawbacks. For example, there are potential safety hazards in data collection. In the process of large-scale data collection, it is easy to face the risk of data leakage, threatening personal privacy and enterprise business secrets; in scenarios with high real-time requirements, data cannot be obtained and updated in a timely manner, resulting in delayed decision-making and missing the best response opportunity; the interactivity of the visualization interface is poor, and it is difficult to independently explore and predictively analyze dynamic data, restricting the performance of groups other than professional data analysts. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides a visualization data information collection system based on big data and its medium, through the following solutions to solve the problems raised in the above background art.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A visualization data information collection system based on big data, characterized in that it includes a system operation database, a system central processing module, and a user information terminal, and further includes:

[0008] A data encryption collection module: used to respond to the incoming data of the visualization data information collection system, and obtain first encrypted information through encryption processing, and transmit it to the data cleaning and integration module and the real-time data monitoring module;

[0009] A data cleaning and integration module: used to perform cleaning and integration operations on the first encrypted information transmitted by the data encryption collection module. The cleaning and integration operations are used to obtain the first key features corresponding to the first encrypted information and transmit them to the data analysis module;

[0010] Data analysis module: It is used to obtain an intelligent analysis model, and based on the first key features transmitted by the data cleaning and integration module through the intelligent analysis model, obtain the first key information and transmit it to the real-time data monitoring module and the visualization template customization module;

[0011] Real-time data monitoring module: It is used to perform real-time data processing operations on the first encrypted information in the data encryption acquisition module and the first key information in the data analysis module through the real-time data monitoring channel. The real-time data processing operation is used to obtain the second key information corresponding to the data changes of the first encrypted information and the first key information, and transmit it to the visualization template customization module;

[0012] Visualization template customization module: It is used to obtain the associated feature information of the first key information in the data analysis module and the second key information in the real-time data monitoring module, formulate a visualization template according to the associated feature information, construct the third key information through the visualization template, and transmit it to the dynamic data interaction module;

[0013] Dynamic data interaction module: It is used to obtain the dynamic interaction data information corresponding to the second key information according to the intelligent analysis model through the third key information in the visualization template customization module, so as to display the dynamic interaction data information corresponding to the second key information on the visualization interface in a preset manner;

[0014] The system operation database includes all data texts of the visualization data information acquisition system, and collects the information texts output by each module in real time. The system central processing module is used to control the information text instructions output by each module in the system, and the user information terminal is an information output device for receiving the visualization data information acquisition system.

[0015] Preferably, the data encryption acquisition module includes an application platform identification unit, a data information acquisition unit, an encryption processing unit, and a first encrypted information output unit;

[0016] The application platform identification unit is used to identify the target platform of the application visualization data information acquisition system, and determine the service type, operating system environment, and data transmission protocol of the target platform;

[0017] The data information acquisition unit generates a data acquisition request corresponding to the target platform based on the service type of the target platform determined by the application platform identification unit, and acquires the target platform data information;

[0018] The encryption processing unit is used to encrypt the target platform data information to generate the first encrypted information, and add a response mark to the first encrypted information;

[0019] The first encrypted information output unit establishes an encrypted transmission channel for transmission based on the response mark of the first encrypted information.

[0020] Preferably, the data encryption acquisition module obtains the first encryption information, specifically including:

[0021] The application platform recognition unit detects the target platform features through network detection technology, determines the protocol header information of the target platform by scanning specific ports, and extracts the feature fingerprint information of the operating system of the target platform to judge the operating system environment.

[0022] Establish a target platform feature library, store and classify the common feature information of different types of platforms, and during the recognition process of the visual data information acquisition system, compare and match the detection results of the target platform with the target platform feature library.

[0023] Preferably, the data encryption acquisition module obtains the first encryption information, specifically including:

[0024] The data acquisition request is based on a preset acquisition rule corresponding to the target platform. The preset acquisition rule includes the type of data to be acquired, the data generation time, and the data format.

[0025] In the preset system operation database, obtain the data acquisition request corresponding to the service type of the target platform. The preset system operation database is used to save the corresponding relationship between the service type of the target platform and the data acquisition request.

[0026] Preferably, the data encryption acquisition module obtains the first encryption information, specifically including:

[0027] The encryption processing unit encrypts the target platform data information collected by the data information acquisition unit according to the data sensitivity of the target platform and the system security policy.

[0028] Add a response mark, and the response mark is used to distinguish the real-time requirements of the target platform data information.

[0029] Generate the first encryption information, which includes user identity information, user account information, transaction orders, transaction logistics, browsing behaviors, and search behaviors.

[0030] Preferably, the data analysis module obtains the first key information, specifically including:

[0031] According to the first key feature, classify the data transmitted by the data cleaning and integration module to obtain the analysis theme corresponding to the first key feature. The analysis theme is the target business area corresponding to the visual data information.

[0032] In the preset analysis database, obtain the intelligent analysis model corresponding to the analysis theme. The preset analysis database is used to save the corresponding relationship between the analysis theme and the intelligent analysis model.

[0033] Preferably, for the real-time data monitoring module, the second key information includes the associated intensity value corresponding to the target platform, the change range, the change trend, the fluctuation period, the reason for trend deviation, and the impact range assessment report.

[0034] Preferably, for the dynamic data interaction module, the dynamic interaction data information includes the specific value and change trend of the association rule, the association detail information, the real-time data fluctuation value, the dynamic adjustment range, and the interaction report.

[0035] The present invention further provides a computer-readable medium, on which a computer program is stored, and the program executes the visualization data information acquisition system based on big data as described in any one of the above.

[0036] The technical effects and advantages of the present invention:

[0037] The present invention encrypts the data of the target platform through the data encryption acquisition module to reduce the hidden dangers of data acquisition.

[0038] The present invention performs real-time data processing operations through the real-time data monitoring module and sets data change rules and warning mechanisms to obtain and update data in a timely manner.

[0039] The present invention receives the third key information through the dynamic data interaction module and displays it on the visualization interface in a preset manner, providing users with independent exploration and predictive analysis of dynamic data and increasing the interactivity of the system. Brief Description of the Drawings

[0040] Figure 1 It is the system flow chart of the present invention.

[0041] Figure 2 It is the system step diagram of the present invention.

[0042] Figure 3 It is the system structure schematic diagram of the present invention. Detailed Embodiments

[0043] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0044] The terms used in the following embodiments of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. As used in the specification of this application, the singular forms "a", "an", "the", and "said" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0045] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of this application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0046] As attached Figure 1 The visualized data information acquisition system based on big data shown in the figure includes a system operation database, a system central processing module, and a user information terminal, and further includes a data encryption acquisition module, a data cleaning and integration module, a data analysis module, a real-time data monitoring module, a visualized template customization module, and a dynamic data interaction module.

[0047] The system operation database includes all data texts of the visualized data information acquisition system and collects the information texts output by each module in real time. The system central processing module is used to control the information text instructions output by each module in the system. The user information terminal is an information output device that receives the visualized data information acquisition system.

[0048] The data encryption acquisition module is used to respond to the incoming data of the visualized data information acquisition system, and through encryption processing, to obtain the first encrypted information and transmit it to the data cleaning and integration module and the real-time data monitoring module.

[0049] In a possible implementation manner, the data encryption acquisition module includes an application platform identification unit, a data information acquisition unit, an encryption processing unit, and a first encrypted information output unit.

[0050] It should be noted that the application platform identification unit is used to identify the target platform that applies the visualized data information acquisition system, determine the service type, operating system environment, and data transmission protocol of the target platform. Among them, the service type of the target platform includes but is not limited to e-commerce trading platforms, user behavior analysis platforms, third-party payment platforms, etc.

[0051] In this embodiment, the target platform is an e-commerce platform built based on the Linux system as an example, and the target platform supports the HTTPS protocol for data transmission.

[0052] Specifically, the visual data information acquisition system detects the target platform characteristics through network detection technology, determines the protocol header information of the target platform by scanning specific ports, and extracts the characteristic fingerprint information of the operating system of the target platform to determine the operating system environment; establishes a target platform characteristic library, stores and classifies the common characteristic information of different types of platforms, and during the recognition process, the visual data information acquisition system compares and matches the detection results of the target platform with the target platform characteristic library; at the same time, when the target platform characteristic library fails to match the characteristic information of the target platform, the visual data information acquisition system automatically updates the target platform characteristic library.

[0053] Based on the service type of the target platform determined by the application platform recognition unit, the data information acquisition unit generates a data acquisition request corresponding to the target platform. The data acquisition request is based on a preset acquisition rule corresponding to the target platform. The preset acquisition rule includes but is not limited to the type of data to be acquired, the data generation time, the data format, etc., and acquires the data information of the target platform; in the preset system operation database, the data acquisition request corresponding to the service type of the target platform is obtained. The preset system operation database is used to save the corresponding relationship between the service type of the target platform and the data acquisition request.

[0054] Specifically, the preset acquisition rule dynamically adjusts the type, time range, and format of the acquired data according to the overall operating state and data analysis requirements of the target platform, and associates and stores the dynamically adjusted acquisition rule with the service type of the target platform in the preset system operation database; multi-threaded and asynchronous acquisition technologies are adopted.

[0055] In this embodiment, when the visual data information acquisition system detects that the target platform is in the peak transaction period, it increases the acquisition frequency and data type of the promotion data and reduces the frequency of the data acquisition request. Among them, the promotion data includes but is not limited to discount information, the sales volume of goods participating in the promotion, the activity time, etc.

[0056] In this embodiment, for the target platform, multiple acquisition threads are started simultaneously, each responsible for the acquisition tasks of different types of data and different data regions; when a user places an order on the target platform, based on the acquisition rule corresponding to the preset system operation database, the data acquisition operation is automatically triggered.

[0057] The encryption processing unit is used to encrypt the data information of the target platform to generate the first encrypted information and add a response mark to the first encrypted information.

[0058] In a possible implementation manner, obtaining the first encrypted information includes: encrypting the target platform data information collected by the data information acquisition unit according to the data sensitivity of the target platform and the system security policy, and the encryption process dynamically selects the corresponding encryption algorithm in the preset system database. The encryption algorithms include but are not limited to AES, RSA, ECC, Blowfish, etc.; adding a response mark, where the response mark is used to distinguish the real-time requirements of the target platform data information; generating the first encrypted information, and the first encrypted information includes user identity information, user account information, transaction orders, transaction logistics, browsing behaviors, and search behaviors.

[0059] Specifically, the first encrypted information is generated for the target platform data information through the AES algorithm, and the specific steps are as follows:

[0060] A1: Group the data length of the target platform data information. If the data length is not an integer multiple of the grouping length, padding is performed.

[0061] In this embodiment, assume that the grouping length of the target platform data information is 128 bits. When the data length is 200 bits, 56 bits are supplemented according to the PKCS#7 rule, and the value of each byte of the supplemented 56-bit data is 56.

[0062] A2: Perform a byte substitution operation on each grouped data. The byte substitution operation is to map the bytes of each grouped data to another byte input matrix based on a predefined matrix.

[0063] In this embodiment, assume that the input data byte is b. When b = 0x2A, the replacement byte is 0xC5.

[0064] A3: Perform a row shift operation on the matrix, and cyclically shift the rows of the data grouping matrix to the left.

[0065] In this embodiment, the data elements of the data grouping matrix are denoted as a ij , where i represents the row index of the data grouping matrix, j represents the column index of the data grouping matrix, and the data grouping matrix is specifically represented as:

[0066] ;

[0067] Perform a row shift operation on the data grouping matrix, and the shifted data elements are denoted as a' ij , and the row shift operation is specifically represented as:

[0068] a' ij =a (i+si)%4，j ,

[0069] Among them, si represents the displacement of the i-th row, s1 = 0, s2 = 1, s3 = 2, s4 = 3; that is, the first row remains unchanged, the second row is shifted left by 1 byte, the third row is shifted left by 2 bytes, and the fourth row is shifted left by 3 bytes;

[0070] After the row shift operation, the corresponding data grouping matrix is specifically represented as:

[0071] ;

[0072] A4: Perform a column mixing operation on the matrix;

[0073] In this embodiment, let the column vector be [a0, a1, a2, a3] T = [0x1A, 0X2B, 0X3C, 0X4D] T , and the column vector after column mixing is denoted as [b0, b1, b2, b3] T , and is specifically represented as:

[0074] ,

[0075] Then [b0, b1, b2, b3] T = [0x06, 0xB9, 0x9C, 0xE0];

[0076] A5: Repeat steps A2 - A4, and perform encryption according to a preset number of rounds to generate the first encrypted information;

[0077] Specifically, when the data grouping is 128 bits, the preset number of rounds is 10 rounds; when the data grouping is 192 bits, the preset number of rounds is 12 rounds; when the data grouping is 256 bits, the preset number of rounds is 14 rounds.

[0078] It should be noted that the user identity information in the first encrypted information includes the user's name, phone number, and email address; the user account information includes the username, login password, account balance, and integral information of the user on the target platform; the transaction order includes the order number, order generation time, order status, as well as the product details and price; the transaction logistics includes the logistics order number, logistics company name, shipping address, receiving address, and logistics status; the browsing behavior includes the URL, browsing time, browsing order, and page stay time of the product pages browsed by the user on the target platform; the search behavior includes the search keywords, search time, and search result click situation of the user on the target platform.

[0079] The first encrypted information output unit establishes an encrypted transmission channel based on the response mark of the first encrypted information for transmission.

[0080] Specifically, the first encrypted information uses multiplexing technology to transmit multiple data flows through the encrypted transmission channel;

[0081] In this embodiment, the response tags are respectively denoted as A and B. A indicates that the first encrypted information needs to be displayed in real time on the visual data information acquisition system, and B indicates that the first encrypted information does not need to be displayed in real time on the visual data information acquisition system; when the response tag is A, the first encrypted information is output to the data cleaning and integration module; when the response tag is B, the first encrypted information is immediately output to the real-time data monitoring module as soon as it is generated.

[0082] The data cleaning and integration module is used to perform cleaning and integration operations on the first encrypted information transmitted by the data encryption acquisition module. The cleaning and integration operations are used to obtain the first key features corresponding to the first encrypted information and transmit them to the data analysis module.

[0083] Specifically, after the visual data information acquisition system obtains the first encrypted information transmitted by the data encryption acquisition module, through the cleaning and integration operations, a plurality of keywords corresponding to the first encrypted information are obtained, that is, the first key features. The cleaning and integration operations are as follows: Decryption operation: Based on the response tag of the first encrypted information, the corresponding decryption key is obtained. In this embodiment, the encryption algorithm is AES, so the corresponding AES decryption key is used for decryption. Handling null values: Scan each data field in the decrypted first encrypted information to detect null values. In this embodiment, when the order amount is null, it is marked as incorrect data and traced back to the data source for correction. Detecting and handling outliers: Use statistical analysis rules to detect outliers. In this embodiment, for the commodity price data of the target platform, the normal range is determined by calculating the mean and standard deviation of the commodity prices. If the price is significantly higher than the average price range of similar commodities, it is marked as an outlier. Data association and merging: Based on the internal logical relationship of the data, the first encrypted information from different data sources is associated. In this embodiment, the user identity information on the e-commerce platform is associated with the transaction order through the user account information. Feature extraction: Based on user requirements, features are extracted to obtain the first key features. In this embodiment, for user behavior, the features of user behavior are extracted through the principal component analysis algorithm.

[0084] The data analysis module is used to obtain an intelligent analysis model, and based on the intelligent analysis model, obtain the first key information according to the first key features transmitted by the data cleaning and integration module, and transmit it to the real-time data monitoring module and the visual template customization module.

[0085] Specifically, the intelligent analysis model is a pre-constructed learning model. By inputting the first key features into the intelligent analysis model, the intelligent analysis model obtains the first key information according to the first key features and sends the first key information to the visual template customization module.

[0086] Specifically, the first key information includes cross-category association rules, user behavior association rules, data fluctuation time, data change trend, and data change cycle.

[0087] In this embodiment, if a user who purchases a specific product on the target platform also purchases a supporting product, this information is listed in a list and stored as association rule information; the sales data of the target platform fluctuates over time; for the corresponding key trend of the target platform, the sales volume shows an upward trend.

[0088] In a possible implementation manner, first key information is obtained, including: classifying the data transmitted by the data cleaning and integration module according to the first key feature to obtain an analysis theme corresponding to the first key feature, where the analysis theme is the target business area corresponding to the visual data information, and the target business area includes but is not limited to the e-commerce field, the financial field, the industrial manufacturing field, etc.; in a preset analysis database, an intelligent analysis model corresponding to the analysis theme is obtained, and the preset analysis database is used to save the corresponding relationship between the analysis theme and the intelligent analysis model.

[0089] In this embodiment, a neural network model is used for predicting user purchase behavior; if product classification management is performed, a decision tree model is used.

[0090] It should be noted that after receiving the first key feature, the intelligent analysis model analyzes and performs pattern recognition on the first key information according to the algorithm structure inside the intelligent analysis model and the parameter system determined by pre-training; when a neural network model is used for predicting user purchase behavior, the intelligent analysis model performs collaborative operations through multiple layers of neurons, and based on the first key features such as the user's historical purchase frequency, the category and duration of recently browsed products, the past consumption amount, and the geographical location, increases the weight of the product categories that the user frequently browses and browses within a fixed time period, and combines the historical purchase frequency and consumption amount to calculate the purchase probability value of the user for each product; when a decision tree model is used for product classification management, the intelligent analysis model makes classification judgments along the branch structure of the decision tree according to the first key feature corresponding to the product; in this embodiment, the intelligent analysis model determines the product category and classification basis information, that is, the first key information, based on the product function, the main material, and the price range.

[0091] The real-time data monitoring module is used to perform real-time data processing operations on the first encrypted information in the data encryption acquisition module and the first key information in the data analysis module through the real-time data monitoring channel. The real-time data processing operation is used to obtain the second key information corresponding to the data changes of the first encrypted information and the first key information, and transmit it to the visualization template customization module.

[0092] It should be noted that the second key information includes the data association relationship corresponding to the target platform, the association strength value, the change amplitude, the change trend, the fluctuation amplitude, the fluctuation period, the trend turning point, the reason for trend deviation, and the impact range assessment report.

[0093] Specifically, upon receiving the first encrypted information from the data encryption collection module, a decryption operation is performed using the encryption algorithm specified by the data encryption collection module; data change rules are set, and when an abnormal situation occurs in the first encrypted information, an early warning mechanism is triggered, which includes a detailed description of data changes, an early warning level, and an impact assessment report; the second key information is generated, which is the key features comprehensively analyzed based on the data change situation of the first encrypted information and business requirements.

[0094] Specifically, for the first key information received from the data analysis module, the decrypted first encrypted information and the first key information are compared and analyzed. For numerical data, matching is performed according to data fields, and the numerical change amount, numerical change rate, and trend smoothness are calculated using the current data value and historical data; for text data, stop words, punctuation marks, and extra spaces are removed, and natural language processing techniques are used to calculate text similarity.

[0095] The visualization template customization module is used to obtain the associated feature information of the first key information in the data analysis module and the second key information in the real-time data monitoring module, and formulate a visualization template based on the associated feature information, so as to construct the third key information through the visualization template and transmit it to the dynamic data interaction module.

[0096] In a possible implementation manner, constructing the third key information includes: constructing the third key information through a visualization template according to the associated feature information of the first key information in the data analysis module and the second key information in the real-time data monitoring module, where the visualization template is the visualization style of the third key information through the user display interface; in the preset system operation database, the corresponding relationship between the visualization template and the third key information is stored.

[0097] Specifically, the visualization template includes but is not limited to a bar chart template, a line chart template, a pie chart template, a map template, a scatter plot template, etc.; based on the bar chart template, the third key information includes the class represented by each column, the value corresponding to the height of the column, and the comparison relationship between columns; based on the line chart template, the third key information includes the horizontal and vertical coordinate values, slope of the data, and the intersection points and distance relationships between different lines; based on the pie chart template, the third key information includes the categories represented by each sector area, the area proportion of the sector, and the relative size relationship between parts; based on the map template, the third key information includes geographical area division, the data value corresponding to each area, the data difference value between areas, and the spatial distribution; based on the scatter plot template, the third key information includes the coordinate values of each scatter point, the distribution pattern of the scatter points, the distance relationship between the scatter points, and the clustering situation.

[0098] The dynamic data interaction module is used to customize the third key information in the visualization template according to the intelligent analysis model, obtain the dynamic interaction data information corresponding to the second key information, and display the dynamic interaction data information corresponding to the second key information on the visualization interface in a preset manner.

[0099] Specifically, the dynamic data interaction module receives the third key information using an encrypted communication protocol, extracts the visualization chart data, and according to the intelligent analysis model, through the third key information, obtains the dynamic interaction data information corresponding to the second key information, predicts the user's behavior actions and data trend directions, and extracts the key turning points and change rates as the dynamic interaction data information. The dynamic interaction data information includes the specific values and change trends of the association rules, association detail information, real-time data fluctuation values, dynamic adjustment ranges, and interaction reports.

[0100] As attached Figure 2 The visualization data information acquisition method based on big data shown in the figure includes: S1: encrypt and collect data, S2: clean and integrate data, S3: analyze data information, S4: real-time monitor data, S5: formulate a visualization template, and S6: dynamic interaction data information.

[0101] S1: Encrypt and collect data: In response to the incoming data of the visualization data information acquisition system, through encryption processing, obtain the first encrypted information;

[0102] S2: Clean and integrate data: Perform a cleaning and integration operation on the first encrypted information. The cleaning and integration operation is used to obtain the first key feature corresponding to the first encrypted information;

[0103] S3: Analyze data information: Obtain the intelligent analysis model, and according to the intelligent analysis model, through the first key feature, obtain the first key information;

[0104] S4: Real-time monitor data: Perform real-time data processing operations on the first encrypted information and the first key information through the real-time data monitoring channel. The real-time data processing operation is used to obtain the second key information corresponding to the data changes of the first encrypted information and the first key information;

[0105] S5: Formulate a visualization template: Obtain the association feature information of the first key information and the second key information, and formulate a visualization template according to the association feature information to construct the third key information through the visualization template;

[0106] S6: Dynamic interaction data information: According to the intelligent analysis model, through the third key information, obtain the dynamic interaction data information corresponding to the second key information, and display the dynamic interaction data information corresponding to the second key information on the visualization interface in a preset manner.

[0107] This embodiment provides a computer-readable medium with a computer program stored thereon, and the program executes a visualization data information acquisition system based on big data as shown in the appendix Figure 1 of the attached drawings.

[0108] Secondly: In the attached drawings of the disclosed embodiments of the present invention, only the structures related to the disclosed embodiments are involved. For other structures, reference can be made to the general design. Without conflict, the same embodiment and different embodiments of the present invention can be combined with each other;

[0109] Finally: The above are only the preferred embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A visualization data information collection system based on big data, comprising a system operation database, a system central processing module and a user information terminal, characterized in that It further includes: Data Encryption and Acquisition Module: Used to respond to the incoming data of the Visual Data Information Acquisition System, perform encryption processing to obtain the first encrypted information, and transmit it to the Data Cleaning and Integration Module and the Real-Time Data Monitoring Module; Data Cleaning and Integration Module: Used to perform cleaning and integration operations on the first encrypted information transmitted by the Data Encryption and Acquisition Module. The cleaning and integration operations are used to obtain the first key features corresponding to the first encrypted information and transmit them to the Data Analysis Module; Data Analysis Module: Used to obtain an intelligent analysis model, perform classification operations based on the first key features transmitted by the Data Cleaning and Integration Module, determine the analysis theme, and obtain the intelligent analysis model corresponding to the analysis theme from the preset analysis database; According to the obtained intelligent analysis model and the first key features, obtain the first key information and transmit it to the Real-Time Data Monitoring Module and the Visualization Template Customization Module; Among them, the analysis theme is the target business area corresponding to the visual data information, the preset analysis database stores the correspondence between the analysis theme and the intelligent analysis model, and the intelligent analysis model analyzes and performs pattern recognition on the first key features according to its internal algorithm structure and pre-trained parameter system to obtain the first key information; Real-Time Data Monitoring Module: Used to perform real-time data processing operations on the first encrypted information in the Data Encryption and Acquisition Module and the first key information in the Data Analysis Module through the real-time data monitoring channel. The specific steps are as follows: SS1. Perform decryption operations on the first encrypted information; SS2. Set data change rules to trigger an early warning mechanism when the first encrypted information appears abnormal; SS3. Compare and analyze the decrypted first encrypted information with the first key information. For numerical data, calculate the numerical change amount, numerical change rate, and trend smoothness. For text data, calculate the text similarity after removing stop words, punctuation marks, and extra spaces; SS4. Based on the data change situation of the first encrypted information and business requirements, obtain the second key information corresponding to the data change between the first encrypted information and the first key information and transmit it to the Visualization Template Customization Module; Visualization Template Customization Module: Used to obtain the associated feature information of the first key information in the Data Analysis Module and the second key information in the Real-Time Data Monitoring Module, formulate a visualization template according to the associated feature information to construct the third key information through the visualization template, and transmit it to the Dynamic Data Interaction Module; Dynamic Data Interaction Module: Used to obtain the dynamic interaction data information corresponding to the second key information according to the intelligent analysis model through the third key information in the Visualization Template Customization Module, and display the dynamic interaction data information corresponding to the second key information on the visualization interface in a preset manner; The system operation database includes all data texts of the Visual Data Information Acquisition System and collects the information texts output by each module in real time. The system central processing module is used to control the information text instructions output by each module in the system, and the user information terminal is an information output device that receives the information of the Visual Data Information Acquisition System.

2. The visualization data information acquisition system based on big data according to claim 1, wherein: The data encryption and collection module includes an application platform identification unit, a data information collection unit, an encryption processing unit, and a first encrypted information output unit; The application platform identification unit is used to identify the target platform of the application visualization data information collection system, and determine the service type, operating system environment, and data transmission protocol of the target platform; Based on the service type of the target platform determined by the application platform identification unit, the data information collection unit generates a data collection request corresponding to the target platform, and collects and obtains the target platform data information; The encryption processing unit is used to encrypt the target platform data information to generate the first encrypted information, and add a response tag to the first encrypted information; The first encrypted information output unit establishes an encrypted transmission channel for transmission based on the response tag of the first encrypted information.

3. The visualization data information acquisition system based on big data according to claim 2, wherein: The data encryption and collection module obtains the first encrypted information, specifically including: The application platform identification unit detects the target platform characteristics through network detection technology, determines the protocol header information of the target platform by scanning specific ports, and extracts the characteristic fingerprint information of the target platform operating system to judge the operating system environment; Establish a target platform feature library, store and classify the common feature information of different types of platforms, and during the identification process of the visualization data information collection system, compare and match the detection results of the target platform with the target platform feature library.

4. The visualization data information acquisition system based on big data according to claim 2, wherein: The data encryption and collection module obtains the first encrypted information, specifically including: The data collection request is based on a preset collection rule corresponding to the target platform. The preset collection rule includes the type of data to be collected, the data generation time, and the data format, and this collection rule will be dynamically adjusted according to the overall operating state and data analysis requirements of the target platform. The adjusted collection rule is associated with the service type of the target platform and stored in the preset system operation database; In the preset system operation database, obtain the data collection request corresponding to the service type of the target platform. The preset system operation database is used to save the corresponding relationship between the service type of the target platform and the data collection request.

5. The visualization data information acquisition system based on big data according to claim 2, wherein: The data encryption and collection module obtains the first encrypted information, specifically including: The encryption processing unit encrypts the target platform data information collected by the data information collection unit according to the data sensitivity of the target platform and the system security policy; Add a response tag, and the response tag is used to distinguish the real-time requirements of the target platform data information; Generate the first encrypted information, and the first encrypted information includes user identity information, user account information, transaction orders, transaction logistics, browsing behavior, and search behavior.

6. The visualization data information acquisition system based on big data according to claim 1, wherein: The data analysis module obtains the first key information, specifically including: According to the first key feature, classify the data transmitted by the data cleaning and integration module to obtain the analysis theme corresponding to the first key feature. The analysis theme is the target business field corresponding to the visualization data information; In the preset analysis database, obtain the intelligent analysis model corresponding to the analysis theme. The preset analysis database is used to save the corresponding relationship between the analysis theme and the intelligent analysis model.

7. The visualization data information acquisition system based on big data according to claim 1, characterized in that: For the real-time data monitoring module, the second key information includes the corresponding correlation intensity value of the target platform, the change amplitude, the change trend, the fluctuation period, the reason for trend deviation, and the impact range assessment report.

8. The visualization data information acquisition system based on big data according to claim 1, characterized in that: For the dynamic data interaction module, the dynamic interaction data information includes the specific value and change trend of the association rule, the association detail information, the real-time data fluctuation value, the dynamic adjustment range, and the interaction report.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it is based on the visualization data information acquisition system according to any one of the above claims 1-8.

Images