Information consultation service platform based on big data analysis and acquisition

Abstract

The invention relates to the technical field of computers, and discloses an information consultation service platform based on big data analysis and acquisition, which comprises a multi-source data acquisition module, a multi-source data cleaning module, a multi-source data acquisition module, a multi-source data calculation module and a multi-source data integration module, the multi-source data acquisition module acquires multi-source data through a network and an offline survey table, and the multi-source data cleaning module performs formatting cleaning on the classified multi-source data. The multi-source data acquisition module is responsible for acquiring information of a structured data unit, an unstructured data unit, a static data unit, a historical data unit and a user privacy data unit, and the multi-source data calculation module calculates a structured data coverage index # imgabs0 #, a real-time data coverage index # imgabs1 #, a user privacy data coverage index # imgabs2 # and a data fluctuation threshold # imgabs3 #. By calculating the indexes and the threshold values, the problems that data capture of a traditional platform is limited and sudden events cannot be dealt with sufficiently are solved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an information consulting service platform based on big data analysis and collection. Background Art

[0002] In the modern society with rapid development of digitization and networking, information consulting service platforms have become important tools for information acquisition, processing and delivery. These platforms effectively manage and utilize large-scale data resources by integrating technical means of data collection, analysis and application, and provide users with timely and accurate consulting services. The construction of such a platform requires not only strong technical support, but also the ability to effectively process large amounts of data. However, although existing technologies can bring convenience to users, they still have some defects, such as data capture restrictions and insufficient response to emergencies, which affect the overall performance and service quality of the platform. Summary of the invention

[0003] 1. Technical issues to be resolved In view of the deficiencies in the prior art, the present invention provides an information consulting service platform based on big data analysis and collection, which has the advantages of unlimited data capture and sufficient response to emergencies, and solves the problems of limited data capture and insufficient response to emergencies.

[0004] (II) Technical solution To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an information consulting service platform based on big data analysis and collection, comprising a multi-source data acquisition module, a multi-source data cleaning module, a multi-source data collection module, a multi-source data calculation module and a multi-source data integration module; The multi-source data acquisition module acquires multi-source data through the network and offline survey forms, and after classification, connects with the multi-source data cleaning module through the network; The multi-source data cleaning module formats and cleans the classified multi-source data, and after classification, connects with the multi-source data acquisition module through the network; The multi-source data acquisition module includes a structured data unit, an unstructured data unit, a static data unit, a historical data unit and a user privacy data unit. The structured data unit collects structured data through an Internet of Things device, and after numbering, it is connected to the multi-source data calculation module through a network. The unstructured data unit collects unstructured data through an Internet of Things device, and after numbering, it is connected to the multi-source data calculation module through a network. The static data unit collects static data through historical data, and after numbering, it is connected to the multi-source data calculation module through a network. The historical data unit collects historical data through an Internet of Things device and a real-time monitoring device, and after numbering, it is connected to the multi-source data calculation module through a network. The user privacy data unit collects user privacy data through an anonymous network and sensitive user information, and the multi-source data acquisition module is connected to the multi-source data calculation module through a network; The multi-source data calculation module includes a data capture unit and a data fluctuation unit. The data capture unit calculates the structured data coverage index based on the structured data and the unstructured data. The data capture unit calculates the real-time data coverage index based on static data and historical data The data capture unit calculates the user private data coverage index based on the user private data The data fluctuation unit calculates the data fluctuation threshold according to structured data, unstructured data, static data, historical data and user privacy data. , the multi-source data calculation module and the multi-source data integration module are connected through a network.

[0005] Preferably, the structured data units are numbered according to the structured data features, and the structured data numbers are , , ,… The unstructured data units are numbered according to the unstructured data characteristics, and the unstructured data numbers are , , ,… .

[0006] Preferably, the data capture unit calculates the structured data coverage index based on the structured data and the unstructured data. , and its calculation formula is: In the formula, represents the structural data coverage index, , , ,… Represents structured data, , , ,… Represents unstructured data.

[0007] Preferably, the static data units are numbered according to the characteristics of the static data, and the static data numbers are , , ,… The historical data units are numbered according to the characteristics of the historical data, and the historical data are numbered as follows: , , ,… .

[0008] Preferably, the data capture unit calculates the real-time data coverage index based on the static data and historical data , and its calculation formula is: In the formula, Indicates the real-time data coverage index, , , ,… Represents static data, , , ,… Represents historical data, Indicates real-time dynamic capture of data.

[0009] Preferably, the user privacy data unit numbers the user private data, anonymous data and open data according to the user privacy data characteristics, and the user private data, anonymous data and open data are numbered as follows: , , .

[0010] Preferably, the data capture unit calculates the user privacy data coverage index based on the user privacy data , and its calculation formula is: In the formula, represents the user private data coverage index, , , Indicates user private data, anonymous data and open data, Represents user network input data.

[0011] Preferably, the data fluctuation unit calculates the data fluctuation threshold value based on structured data, unstructured data, static data, historical data and user privacy data. , and its calculation formula is: In the formula, Indicates the data fluctuation threshold, Represents structured data and unstructured data and, Indicates real-time dynamic capture of data. Represents user private data. 、 、 Respectively , , The weight in the standard value of data fluctuation.

[0012] Preferably, the multi-source data integration module is based on the structure data coverage index , Real-time data coverage index Index of coverage of user private data To formulate the collection channels of platform data, the data objects that need to be collected include: structured data units, unstructured data units, static data units, historical data units and user privacy data units.

[0013] Preferably, the multi-source data integration module is based on the data fluctuation threshold Re-examine the integrated data, and if the review fails, continue to adjust the plan until it passes.

[0014] Compared with the prior art, the present invention provides an information consulting service platform based on big data analysis and collection, which has the following beneficial effects: 1. In the present invention, structured data is expressed by a two-dimensional table structure, while unstructured data is expressed by text, images, audio and video. The structured data coverage index is calculated based on the structured data and the unstructured data. It can evaluate the proportion of structured data in the total data of the information consulting service platform, understand the degree of structuring of the data in the platform, and ensure the unlimited data capture based on the acquisition method of structured and unstructured data.

[0015] 2. The present invention combines structured data, unstructured data, static data, historical data and user privacy data to calculate the data fluctuation threshold It can comprehensively reflect the overall fluctuation of platform data, and introduce real-time dynamic capture data through static data and historical data, which can ensure that the calculation of data fluctuation thresholds can instantly reflect the latest data changes, thereby enhancing real-time response capabilities and solving the problem of insufficient response to sudden events on the platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the structure of the present invention; DETAILED DESCRIPTION The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] See also Figure 1 , an information consulting service platform based on big data analysis and collection, including a multi-source data acquisition module, a multi-source data cleaning module, a multi-source data collection module, a multi-source data calculation module and a multi-source data integration module; The multi-source data acquisition module acquires multi-source data through the Internet and offline survey forms, and after classification, connects with the multi-source data cleaning module through the Internet; The multi-source data cleaning module formats and cleans the classified multi-source data, and after classification, connects with the multi-source data acquisition module through the network; The multi-source data acquisition module includes a structured data unit, an unstructured data unit, a static data unit, a historical data unit and a user privacy data unit. The structured data unit collects structured data through an Internet of Things device, and after numbering, it is connected to the multi-source data calculation module through a network. The unstructured data unit collects unstructured data through an Internet of Things device, and after numbering, it is connected to the multi-source data calculation module through a network. The static data unit collects static data through historical data, and after numbering, it is connected to the multi-source data calculation module through a network. The historical data unit collects historical data through an Internet of Things device and a real-time monitoring device, and after numbering, it is connected to the multi-source data calculation module through a network. The user privacy data unit collects user privacy data through an anonymous network and sensitive user information, and the multi-source data acquisition module is connected to the multi-source data calculation module through a network. The multi-source data calculation module includes a data capture unit and a data fluctuation unit. The data capture unit calculates the structured data coverage index based on the structured data and unstructured data. The data capture unit calculates the real-time data coverage index based on static data and historical data. , the data capture unit calculates the user's private data coverage index based on the user's private data The data fluctuation unit calculates the data fluctuation threshold based on structured data, unstructured data, static data, historical data and user privacy data. ,The multi-source data computing module and the multi-source data integration module are connected through a network.

[0018] The structured data units are numbered according to their structured data characteristics. The structured data numbers are , , ,… , unstructured data units are numbered according to their characteristics. The unstructured data are numbered as , , ,… .

[0019] The data capture unit calculates the structured data coverage index based on the structured data and unstructured data , and its calculation formula is: In the formula, represents the structural data coverage index, , , ,… Represents structured data, , , ,… Represents unstructured data.

[0020] The advantages are: the structured data in the platform is expressed through a two-dimensional table structure, while the unstructured data is expressed using text, images, audio and video. The structured data coverage index is calculated based on the structured data and unstructured data. It can evaluate the proportion of structured data in the total data of the information consulting service platform, understand the degree of structuring of the data in the platform, and ensure the unlimited data capture based on the acquisition method of structured and unstructured data.

[0021] The static data units are numbered according to their characteristics. The static data numbers are , , ,… , the historical data units are numbered according to the characteristics of the historical data, and the historical data numbers are , , ,… .

[0022] The data capture unit calculates the real-time data coverage index based on static data and historical data , and its calculation formula is: In the formula, Indicates the real-time data coverage index, , , ,… Represents static data, , , ,… Represents historical data, Indicates real-time dynamic capture of data.

[0023] The advantages are: historical data includes static data and real-time dynamic capture data, and the real-time data coverage index is calculated , and based on the static data unit and historical data unit, the data collected by the device is monitored in real time, and abnormal information obtained by the platform can be discovered in time.

[0024] The user privacy data unit numbers the user private data, anonymous data and open data according to the characteristics of the user privacy data. The user private data, anonymous data and open data are numbered as follows: , , .

[0025] The data capture unit calculates the user's private data coverage index based on the user's private data , and its calculation formula is: In the formula, represents the user private data coverage index, , , Indicates user private data, anonymous data and open data, Represents user network input data.

[0026] The advantage is: by calculating the user's private data coverage index , can determine the privacy scope of big data users in the platform, and can cover the index of user private data The value of the platform can be used to supplement the unprotected privacy areas in real time to prevent the leakage of user privacy data, thereby gaining the trust of platform users.

[0027] The data fluctuation unit calculates the data fluctuation threshold based on structured data, unstructured data, static data, historical data and user privacy data. , and its calculation formula is: In the formula, Indicates the data fluctuation threshold, Represents structured data and unstructured data and, Indicates real-time dynamic capture of data. Represents user private data. 、 、 Respectively , , The weight in the standard value of data fluctuation.

[0028] The advantage is that the data fluctuation threshold is calculated by combining structured data, unstructured data, static data, historical data and user privacy data. It can comprehensively reflect the overall fluctuation of platform data, and introduce real-time dynamic capture data through static data and historical data, which can ensure that the calculation of data fluctuation thresholds can instantly reflect the latest data changes, thereby enhancing real-time response capabilities and solving the problem of insufficient response to sudden events on the platform.

[0029] The multi-source data integration module covers the structure data index , Real-time data coverage index Index of coverage of user private data To formulate the data collection channels for the platform, the data objects that need to be collected include: structured data units, unstructured data units, static data units, historical data units and user privacy data units. The data capture capability of the platform is enhanced through the collection method of multi-source data fusion.

[0030] The multi-source data integration module is based on the data fluctuation threshold Re-examine the integrated data. If the review fails, continue to adjust the plan until it passes. It can monitor the data fluctuation range of the platform in real time to reversely adjust the parameters of multi-data collection.

[0031] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An information consulting service platform based on big data analysis and collection, characterized in that: It includes a multi-source data acquisition module, a multi-source data cleaning module, a multi-source data collection module, a multi-source data calculation module and a multi-source data integration module; The multi-source data acquisition module acquires multi-source data through the network and offline survey forms, and after classification, connects with the multi-source data cleaning module through the network; The multi-source data cleaning module formats and cleans the classified multi-source data, and after classification, connects with the multi-source data acquisition module through the network; The multi-source data acquisition module includes a structured data unit, an unstructured data unit, a static data unit, a historical data unit and a user privacy data unit. The structured data unit collects structured data through an Internet of Things device, and after numbering, it is connected to the multi-source data calculation module through a network. The unstructured data unit collects unstructured data through an Internet of Things device, and after numbering, it is connected to the multi-source data calculation module through a network. The static data unit collects static data through historical data, and after numbering, it is connected to the multi-source data calculation module through a network. The historical data unit collects historical data through an Internet of Things device and a real-time monitoring device, and after numbering, it is connected to the multi-source data calculation module through a network. The user privacy data unit collects user privacy data through an anonymous network and sensitive user information, and the multi-source data acquisition module is connected to the multi-source data calculation module through a network; The multi-source data calculation module includes a data capture unit and a data fluctuation unit. The data capture unit calculates the structured data coverage index based on the structured data and the unstructured data. The data capture unit calculates the real-time data coverage index based on static data and historical data The data capture unit calculates the user private data coverage index based on the user private data The data fluctuation unit calculates the data fluctuation threshold according to structured data, unstructured data, static data, historical data and user privacy data. , the multi-source data calculation module and the multi-source data integration module are connected through a network.

2. The information consulting service platform based on big data analysis and collection according to claim 1, characterized in that: The structured data units are numbered according to the structured data features, and the structured data numbers are , , ,… The unstructured data units are numbered according to the unstructured data characteristics, and the unstructured data numbers are , , ,… .

3. The information consulting service platform based on big data analysis and collection according to claim 2 is characterized by: The data capture unit calculates the structured data coverage index based on the structured data and the unstructured data , and its calculation formula is: In the formula, represents the structural data coverage index, , , ,… Represents structured data, , , ,… Represents unstructured data.

4. The information consulting service platform based on big data analysis and collection according to claim 1 is characterized by: The static data units are numbered according to the characteristics of the static data, and the static data numbers are , , ,… The historical data units are numbered according to the characteristics of the historical data, and the historical data are numbered as follows: , , ,… .

5. The information consulting service platform based on big data analysis and collection according to claim 4 is characterized by: The data capture unit calculates the real-time data coverage index based on static data and historical data , and its calculation formula is: In the formula, Indicates the real-time data coverage index, , , ,… Represents static data, , , ,… Represents historical data, Indicates real-time dynamic capture of data.

6. The information consulting service platform based on big data analysis and collection according to claim 1, characterized in that: The user privacy data unit numbers the user private data, anonymous data and open data according to the user privacy data characteristics. The user private data, anonymous data and open data are numbered as follows: , , .

7. The information consulting service platform based on big data analysis and collection according to claim 6 is characterized by: The data capture unit calculates the user private data coverage index based on the user private data , and its calculation formula is: In the formula, represents the user private data coverage index, , , Indicates user private data, anonymous data and open data, Represents user network input data.

8. The information consulting service platform based on big data analysis and collection according to claim 6 is characterized by: The data fluctuation unit calculates the data fluctuation threshold according to structured data, unstructured data, static data, historical data and user privacy data. , and its calculation formula is: In the formula, Indicates the data fluctuation threshold, Represents structured data and unstructured data and, Indicates real-time dynamic capture of data. Represents user private data. 、 、 Respectively , , The weight in the standard value of data fluctuation.

9. The information consulting service platform based on big data analysis and collection according to claim 8, characterized in that: The multi-source data integration module is based on the structure data coverage index , Real-time data coverage index Index of coverage of user private data To formulate the collection channels of platform data, the data objects that need to be collected include: structured data units, unstructured data units, static data units, historical data units and user privacy data units.

10. The information consulting service platform based on big data analysis and collection according to claim 9, characterized in that: The multi-source data integration module is based on the data fluctuation threshold Re-examine the integrated data, and if the review fails, continue to adjust the plan until it passes.

Images