Data analysis method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a data analysis method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: generating a current data acquisition strategy according to configuration information and a preset data acquisition frequency by obtaining the configuration information such as an input target identifier of a to-be-tracked target, a target theme label issued by the to-be-tracked target, position information of the to-be-tracked target, time information corresponding to the target theme label and the like; and obtaining initial social data of the to-be-tracked target from the plurality of social media platforms based on the current data acquisition strategy, then preprocessing the initial social data to obtain target social data, and finally determining a data analysis result based on the target social data and the data analysis model. Therefore, the social data obtained from a plurality of social media platforms can be processed and analyzed in a unified manner, and the accuracy and efficiency of data analysis can be improved by preprocessing the initial social data and utilizing the data analysis model.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data analysis method, device, computer equipment, storage medium and program product. Background Art

[0002] With the rapid development of Internet technology, social media has become an indispensable part of people's daily life and work. These social media platforms generate a large amount of data during operation, covering user information, activity information, post information, comment information and other aspects. These data are of great reference value for understanding user needs, analyzing industry trends, and formulating marketing strategies. Therefore, how to analyze the collected social media data has become an urgent problem to be solved in this field. Summary of the invention

[0003] Based on this, it is necessary to provide a data analysis method, device, computer equipment, storage medium and program product that can analyze the collected social media data in response to the above technical problems.

[0004] In a first aspect, the present application provides a data analysis method. The method comprises:

[0005] Acquire input configuration information of the target to be tracked; the configuration information at least includes a target identifier of the target to be tracked, a target subject tag published by the target to be tracked, location information of the target to be tracked, and time information corresponding to the target subject tag;

[0006] Generate a current data collection strategy according to the configuration information and a preset data collection frequency, and acquire initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy;

[0007] Preprocessing the initial social data to obtain target social data;

[0008] A data analysis result is determined based on the target social data and a data analysis model; the data analysis model is obtained by training an initial data analysis model based on historical social data.

[0009] In one embodiment, the method further comprises:

[0010] According to the data analysis results, the current data collection strategy is adjusted to obtain a new data collection strategy, and the new data collection strategy is used as the current data collection strategy. The step of obtaining the initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy is returned to obtain the data analysis results corresponding to the new data collection strategy.

[0011] In one embodiment, the initial social data of the target to be tracked is obtained from multiple social media platforms based on the current data collection strategy, including:

[0012] Acquire encrypted social data from the plurality of social media platforms based on a current data collection strategy;

[0013] The initial social data is determined according to the encrypted social data.

[0014] In one embodiment, the method further comprises:

[0015] Obtain historical encrypted social data from the multiple social media platforms through edge computing nodes;

[0016] The initial data analysis model is trained by the edge computing node based on the historical encrypted social data to obtain the data analysis model.

[0017] In one embodiment, the method further comprises:

[0018] Obtain local historical encrypted social data of the federated learning participant through each federated learning participant; the historical encrypted social data is used by the federated learning participant to train the local sub-initial data analysis model of the federated learning participant to obtain a sub-data analysis model;

[0019] Receive the sub-data analysis models sent by each of the federated learning participants, and determine the data analysis model based on each of the sub-data analysis models.

[0020] In one embodiment, the method further comprises:

[0021] Acquiring monitoring data during the data processing process; the data processing process includes a process of acquiring the configuration information, a process of generating the data collection strategy, a process of training the data analysis model, a process of preprocessing the initial social data, and a process of analyzing the data analysis results;

[0022] When it is determined that the monitoring data is abnormal, an alarm message is generated; the alarm message is used to indicate that the monitoring data is abnormal.

[0023] In a second aspect, the present application also provides a data analysis device. The device comprises:

[0024] A first acquisition module is used to acquire input configuration information of the target to be tracked; the configuration information at least includes a target identifier of the target to be tracked, a target subject tag published by the target to be tracked, location information of the target to be tracked, and time information corresponding to the target subject tag;

[0025] A second acquisition module is used to generate a current data collection strategy according to the configuration information and a preset data collection frequency, and to acquire initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy;

[0026] A first determination module, configured to pre-process the initial social data to obtain target social data;

[0027] The second determination module is used to determine the data analysis result based on the target social data and a data analysis model; the data analysis model is obtained by training an initial data analysis model based on historical social data.

[0028] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0029] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above methods when executed by a processor.

[0030] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of any of the above methods when executed by a processor.

[0031] The above-mentioned data analysis method, device, computer equipment, storage medium and program product obtain the input configuration information such as the target identifier of the target to be tracked, the target topic tag published by the target to be tracked, the location information of the target to be tracked, the time information corresponding to the target topic tag, etc., generate the current data collection strategy according to the configuration information and the preset data collection frequency, and obtain the initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy, and then pre-process the initial social data to obtain the target social data, and finally determine the data analysis results based on the target social data and the data analysis model, so that the social data obtained from multiple social media platforms can be uniformly processed and analyzed, and the pre-processing of the initial social data and the use of the data analysis model can also improve the accuracy and efficiency of the data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is an internal structure diagram of a computer device provided in an embodiment of the present application;

[0033] Figure 2 It is a flow chart of a data analysis method provided in an embodiment of the present application;

[0034] Figure 3is a flowchart of a method for determining initial social data provided by an embodiment of the present application;

[0035] Figure 4 It is a flowchart of a method for determining a data analysis model provided in an embodiment of the present application;

[0036] Figure 5 It is a flowchart of another method for determining a data analysis model provided in an embodiment of the present application;

[0037] Figure 6 It is a flowchart of a method for generating alarm information provided by an embodiment of the present application;

[0038] Figure 7 It is a flowchart of a method for automatic collection and management of social data provided in an embodiment of the present application;

[0039] Figure 8 It is a structural block diagram of a data analysis device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0041] With the rapid development of Internet technology, social media has become an indispensable part of people's daily life and work. These social media platforms generate a large amount of data during operation, covering user information, activity information, post information, comment information and other aspects. These data are of great reference value for understanding user needs, analyzing industry trends, and formulating marketing strategies. Therefore, how to analyze the collected social media data has become an urgent problem to be solved in this field.

[0042] The data analysis method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Figure 1 is an internal structure diagram of a computer device provided in an embodiment of the present application. The computer device may be a server, and its internal structure diagram may be as follows Figure 1As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data analysis method is implemented.

[0043] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0044] In one embodiment, Figure 2 As shown, Figure 2 is a flow chart of a data analysis method provided in an embodiment of the present application, which can be applied to Figure 1 The computer device comprises the following steps:

[0045] S201, obtaining input configuration information of a target to be tracked.

[0046] The configuration information at least includes a target identifier of the target to be tracked, a target topic tag published by the target to be tracked, location information of the target to be tracked, and time information corresponding to the target topic tag.

[0047] In one embodiment, the user can input configuration information such as keywords of the target to be tracked, target identifier, target topic tags published by the target to be tracked, location information of the target to be tracked, and time information corresponding to the target topic tags through a graphical interface or command line. Among them, the target identifier can be a unique identifier of the target to be tracked, such as an identity number, account number, name, etc.; the target topic tags published by the target to be tracked can be topic tags or topics carried by social content published by the target to be tracked; the time information corresponding to the target topic tags can be the moment when the target to be tracked publishes the target topic tags, or the time information corresponding to the content related to the target topic tags in the social platform.

[0048] S202, generating a current data collection strategy according to the configuration information and the preset data collection frequency, and acquiring initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy.

[0049] Optionally, the acquired configuration information can be used as a screening condition for data collection, and the data collection frequency can be determined, and then the current data collection strategy can be generated based on the configuration information and the data collection frequency, so that the initial social data corresponding to the configuration information of the target to be tracked can be obtained from multiple social media platforms according to the current data collection strategy and the data collection frequency.

[0050] It should be noted that the target to be tracked may be one or more. In the case of multiple targets to be tracked, different weights may be assigned to the targets to be tracked according to their importance, so as to optimize resource allocation according to the weights.

[0051] In one embodiment, the social data publicly released on each social media platform can be accessed and obtained through the application programming interface (API) provided by each social media platform. In addition, user authorization can also be obtained based on standard authentication protocols such as Open Authorization 2.0 (OAuth2.0).

[0052] Optionally, a preset time period may be set to check changes in the API versions of the social media platforms at preset time intervals, so that the internal call logic can be adjusted in a timely manner when the API version changes.

[0053] In the embodiment of the present application, during the transmission of social data, the social data of each social media platform can be encrypted based on the SSL / TLS (Secure Sockets Layer / Transport Layer Security) encryption protocol to obtain encrypted social data. Then, the encrypted social data of each social media platform is obtained, so that the initial social data can be determined based on the encrypted social data, so as to reduce the risk of eavesdropping or tampering of the social data during transmission and improve the security of data transmission.

[0054] Optionally, a multi-copy storage mechanism can be introduced to store the acquired initial social data on multiple nodes to achieve data redundancy, so that when a node fails, services can still be provided through other healthy nodes, thereby improving the reliability of social data analysis. In addition, the initial social data can also be stored in a distributed storage structure, so that the storage capacity can be flexibly adjusted according to the amount of data of the initial social data. When the amount of data is large, new nodes can be added to expand the storage space; when the amount of data is small, the nodes can be reduced accordingly to avoid waste of resources.

[0055] S203: pre-process the initial social data to obtain target social data.

[0056] Optionally, the initial social data may be preprocessed based on the type of the initial social data to obtain target social data.

[0057] For example, if the initial social data is text data, the initial social data can be cleaned and formatted based on natural language processing (NLP) technology and multimedia recognition technology to remove noise information in the initial social data. The processed initial social data can then be segmented, stop words removed, etc. to obtain target social data.

[0058] Alternatively, if the initial social data is image data (such as pictures, videos, etc.), the initial social data can be cleaned and formatted based on multimedia recognition technology to remove noise information in the initial social data, and then the feature data or labels of the processed initial social data can be extracted based on the image recognition algorithm to obtain the target social data.

[0059] S204: Determine data analysis results based on the target social data and the data analysis model.

[0060] The data analysis model is obtained by training the initial data analysis model based on historical social data.

[0061] Optionally, the data analysis result may include, for example, at least one of a social trend monitoring result, a hot topic positioning result, and a user emotion prediction result.

[0062] Exemplarily, the time series data of the target social data can be analyzed based on the data analysis model to track the development dynamics of the target theme tag or topic and obtain social trend monitoring results.

[0063] Alternatively, cluster analysis can be performed on the target social data based on the data analysis model to locate current hot topics or events and obtain hot topic location results.

[0064] Alternatively, sentiment analysis can be performed on the target social data based on the data analysis model to predict the qualitative and quantitative information of the target's emotions regarding a specific topic or event, i.e., the user emotion prediction result mentioned above. This enables the user's preferences to be determined based on the user emotion prediction results, and personalized recommendations to be made. When the user emotion prediction results indicate an outburst of negative emotions, timely processing measures can be taken.

[0065] Among them, sentiment analysis may include, for example, the following two methods: 1. Determine the emotional color of the target social data based on a preset sentiment dictionary and grammatical rules; 2. Determine the positive, negative or neutral emotional expression of the target social data based on a pre-trained data analysis model.

[0066] Optionally, a graphical interface may be provided to intuitively display data analysis results in the form of a bar graph, a line graph, a pie chart, etc. The graphical interface may also be used to obtain information such as keywords and filter conditions input by the user, so as to obtain configuration information of the target to be tracked according to the user's input.

[0067] In an embodiment of the present application, by obtaining input configuration information such as a target identifier of a target to be tracked, a target topic tag published by the target to be tracked, location information of the target to be tracked, time information corresponding to the target topic tag, etc., a current data collection strategy is generated according to the configuration information and a preset data collection frequency, and initial social data of the target to be tracked is obtained from multiple social media platforms based on the current data collection strategy, and then the initial social data is preprocessed to obtain target social data, and finally the data analysis result is determined based on the target social data and a data analysis model, thereby enabling unified processing and analysis of the social data obtained from multiple social media platforms, and preprocessing the initial social data and using a data analysis model can also improve the accuracy and efficiency of data analysis.

[0068] Based on the above embodiment, the method further includes the following steps:

[0069] According to the data analysis results, the current data collection strategy is adjusted to obtain a new data collection strategy, the new data collection strategy is used as the current data collection strategy, and the step of obtaining the initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy is returned to obtain the data analysis results corresponding to the new data collection strategy.

[0070] For example, if it is determined based on the data analysis results that the attention paid to a specific topic or event has increased, the current data collection strategy can be adjusted, such as adding targets to be tracked, increasing the data collection frequency, expanding the configuration information, etc., to expand the scope of data collection of the data collection strategy, and then use the new data collection strategy as the current data collection strategy, so that the initial social data of the targets to be tracked can be obtained from multiple social media platforms based on the current data collection strategy, the initial social data can be preprocessed to obtain target social data, and the data analysis results can be determined based on the target social data and the data analysis model.

[0071] It should be noted that the above-mentioned adjustment of the current data collection strategy when the attention to a specific topic or event increases is only a possible implementation method, and is not limited to adjusting the current data collection strategy in this one case. For example, the current data collection strategy can also be adjusted according to any one of the social trend monitoring results, hot topic positioning results and user emotion prediction results.

[0072] Optionally, feedback information from users regarding the data analysis results can be obtained, and then the current data collection strategy can be adjusted based on the feedback information and the data analysis results to obtain a new data collection strategy. The new data collection strategy can be used as the current data collection strategy, and the step of obtaining initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy can be returned to obtain the data analysis results corresponding to the new data collection strategy.

[0073] In an embodiment of the present application, the current data collection strategy is adjusted according to the data analysis results to obtain a new data collection strategy, the new data collection strategy is used as the current data collection strategy, and the step of obtaining initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy is returned to obtain the data analysis results corresponding to the new data collection strategy, so that the data collection strategy can be dynamically updated according to the data analysis results, and the correlation analysis between multimodal social data such as text, images, and audio can be realized, thereby improving the reliability and flexibility of social data collection, and thus improving the accuracy of social data analysis.

[0074] In addition, intelligent recommendation content can be generated based on data analysis results and metadata such as the geographic location and timestamp of the target to be tracked. Therefore, dynamically updating the data collection strategy can also continuously optimize the quality of the intelligent recommendation content.

[0075] Reference Figure 3 , Figure 3 : is a flow chart of a method for determining initial social data provided by an embodiment of the present application. This implementation involves a possible implementation method of how to obtain initial social data of a target to be tracked from multiple social media platforms based on the current data collection strategy. Based on the above embodiment, the above S202 includes the following steps:

[0076] S301, obtaining encrypted social data from multiple social media platforms based on the current data collection strategy.

[0077] In an embodiment of the present application, during the transmission of social data, the social data of each social media platform can be encrypted based on the SSL / TLS (Secure Sockets Layer / Transport Layer Security) encryption protocol to obtain encrypted social data from multiple social media platforms based on the current data collection strategy.

[0078] S302: Determine initial social data according to the encrypted social data.

[0079] Optionally, the encrypted social data may be used as the initial social data. Alternatively, the encrypted social data may be decrypted to obtain the initial social data.

[0080] In an embodiment of the present application, encrypted social data is obtained from multiple social media platforms based on the current data collection strategy, and initial social data is determined based on the encrypted social data, so that the initial social data can be determined based on the encrypted social data to reduce the risk of social data being eavesdropped or tampered with during transmission and improve the security of data transmission.

[0081] Reference Figure 4 , Figure 4 : is a flow chart of a method for determining a data analysis model provided in an embodiment of the present application. Based on the above embodiment, the method further includes the following steps:

[0082] S401, obtain historical encrypted social data from multiple social media platforms through edge computing nodes.

[0083] Optionally, the historical encrypted social data may be directly used as the historical social data, or the historical encrypted social data may be decrypted to obtain the historical social data.

[0084] In one embodiment, a small server or intelligent gateway can be deployed as an edge computing node for a user device. When a user device initiates a task request, the magnitude of the task corresponding to the task request can be determined. If the task corresponding to the task request is less than a preset threshold and is a lightweight task, the task corresponding to the task request can be processed by the edge computing node. Among them, the task request can include, for example, a data analysis model training request, a social data collection request, a social data analysis request, etc.

[0085] Optionally, if the user device initiates a data analysis model training request, and the data analysis model training request is a lightweight task, historical encrypted social data can be obtained from multiple social media platforms through edge computing node processing.

[0086] S402: Train the initial data analysis model based on historical encrypted social data through edge computing nodes to obtain a data analysis model.

[0087] For example, after obtaining historical encrypted social data from multiple social media platforms, the initial data analysis model can be trained based on the historical encrypted social data through the edge computing node to obtain a data analysis model. Then, the small-scale initial social data can be preprocessed and analyzed based on the data analysis model of the edge computing node to obtain data analysis results.

[0088] In an embodiment of the present application, historical encrypted social data is obtained from multiple social media platforms through edge computing nodes, and an initial data analysis model is trained based on the historical encrypted social data through the edge computing nodes to obtain a data analysis model, so that lightweight tasks can be processed through the edge computing nodes, thereby reducing the delay in lightweight task processing and improving the efficiency of lightweight task processing.

[0089] Reference Figure 5 , Figure 5 FIG. 1 is a flow chart of another method for determining a data analysis model provided in an embodiment of the present application. Based on the above embodiment, the method further includes the following steps:

[0090] S501, obtaining local historical encrypted social data of the federated learning participants through each federated learning participant.

[0091] Among them, the historical encrypted social data is used for the federated learning participants to train the local sub-initial data analysis model of the federated learning participants to obtain a sub-data analysis model.

[0092] In one embodiment, the initial data analysis model can be trained using federated learning technology to obtain a data analysis model: each federated learning participant can obtain its own local historical encrypted social data, and then train the sub-initial data analysis model based on the historical encrypted social data locally in the federated learning participant to obtain a sub-data analysis model.

[0093] S502, receiving sub-data analysis models sent by each federated learning participant, and determining a data analysis model based on each sub-data analysis model.

[0094] Exemplarily, the sub-model parameters of the sub-data analysis model sent by each federated learning participant may be received, the target model parameters may be determined according to each sub-model parameter, and then the target model parameters may be used to update the parameters of the initial data analysis model to obtain the data analysis model. The target model parameters may be sent to each federated learning participant so that each federated learning participant updates the parameters of the local sub-data analysis model.

[0095] In an embodiment of the present application, each federated learning participant obtains the local historical encrypted social data of the federated learning participant, receives the sub-data analysis model sent by each federated learning participant, and determines the data analysis model based on each sub-data analysis model, so that the sub-initial data analysis model can be trained locally on each federated learning participant based on its own historical encrypted social data without the need for data sharing, thereby further protecting the privacy of users and improving the security and reliability of social data analysis.

[0096] Reference Figure 6 , Figure 6FIG. 1 is a flow chart of a method for generating alarm information provided by an embodiment of the present application. Based on the above embodiment, the method further includes the following steps:

[0097] S601, obtaining monitoring data in the data processing process.

[0098] The data processing process includes the process of acquiring configuration information, the process of generating data collection strategies, the process of training data analysis models, the process of preprocessing initial social data, and the process of analyzing data analysis results.

[0099] In the embodiment of the present application, the real-time stream data generated in all the above data processing processes can be obtained, and then the obtained real-time stream data can be used as monitoring data.

[0100] Exemplarily, the real-time stream data generated during the data processing process may include: the real-time stream data generated during the acquisition of configuration information in step S201, the real-time stream data generated during the generation of the data collection strategy in step S202, the real-time stream data generated during the preprocessing of the initial social data in step S203, the real-time stream data generated during the analysis of the data analysis results in step S204, and the real-time stream data generated during the training of the data analysis model in steps S401-402 or S501-S502.

[0101] Optionally, a data monitoring tool may be configured at the edge computing node to obtain monitoring data through the edge computing node and monitor the data for monitoring.

[0102] S602, when it is determined that the monitoring data is abnormal, an alarm message is generated; the alarm message is used to indicate that the monitoring data is abnormal.

[0103] Exemplarily, a preset interval and time window can be set. When the monitoring data exceeds the preset interval based on the time window, it is determined that the monitoring data is abnormal. At this time, an alarm message can be generated and displayed through a graphical interface, thereby reminding relevant staff that there is an abnormality in the monitoring data so that the relevant staff can take timely processing measures.

[0104] Alternatively, the monitoring data can also be monitored based on a pre-trained monitoring model. When the monitoring data is determined to be abnormal based on the pre-trained monitoring model, an alarm message is generated and displayed through a graphical interface, thereby reminding relevant staff that there is an abnormality in the monitoring data so that the relevant staff can take timely processing measures.

[0105] Among them, the abnormality of the monitoring data may include, for example, the traffic of the monitoring data exceeds the preset traffic range, the monitoring data contains data packets of unknown sources, etc., which are not specifically limited here.

[0106] In the embodiment of the present application, the monitoring data in the process of acquiring configuration information, generating data collection strategies, training data analysis models, preprocessing initial social data, and analyzing data analysis results is obtained. When it is determined that the monitoring data is abnormal, an alarm message is generated, thereby being able to remind relevant staff members that the monitoring data is abnormal through the alarm message, so that relevant staff members can take timely processing measures, thereby improving the reliability and security of the social data processing process.

[0107] Reference Figure 7 , Figure 7 : is a flow chart of a method for automatic collection and management of social data provided in an embodiment of the present application. The method comprises the following steps:

[0108] S701, obtaining historical encrypted social data from multiple social media platforms, training an initial data analysis model based on the historical encrypted social data, and obtaining a data analysis model.

[0109] S702: Obtain input configuration information of the target to be tracked.

[0110] S703, generating a current data collection strategy according to the configuration information and the preset data collection frequency, and acquiring initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy.

[0111] S704, pre-processing the initial social data based on natural language processing technology and multimedia recognition technology to obtain target social data.

[0112] S705, determining a data analysis result based on the target social data and the data analysis model.

[0113] S706, adjust the current data collection strategy according to the data analysis results to obtain a new data collection strategy, use the new data collection strategy as the current data collection strategy, and return to execute the step of obtaining the initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy to obtain the data analysis results corresponding to the new data collection strategy.

[0114] S707, acquiring monitoring data in the data processing process, and generating alarm information when it is determined that the monitoring data is abnormal; the alarm information is used to indicate that the monitoring data is abnormal.

[0115] For example, assuming that a supermarket needs to understand consumer preferences through social media platforms and optimize product layout and service strategies, it can formulate data collection strategies according to its own needs, and crawl the latest shopping experience sharing, product evaluation and other initial social data from the selected social media platform based on the data collection strategy through the API interface and OAuth2.0 protocol provided by each social media platform. In the process of crawling the initial social data, encrypted social data can be obtained based on SSL / TLS, etc., the initial social data can be determined based on the encrypted social data, and the initial social data can be preprocessed to obtain the target social data. Then, the data analysis results are determined based on the target social data and the data analysis model. After determining the data analysis results, the data collection strategy can be adjusted based on the data analysis results. For example, when a certain type of product is suddenly popular, the attention to topics related to such products is increased; otherwise, unnecessary resource consumption is reduced. As the business expands, the amount of social data generated by the supermarket continues to increase. Therefore, a distributed database architecture can be used to store initial social data and target social data. For some lightweight tasks, limited-time discount notifications of nearby stores can be provided based on the current location of the user's device. The supermarket uses edge computing nodes to accelerate the response speed and bring customers a more immediate service experience. If the edge computing node detects abnormally high traffic or other suspicious activities, it can send alarm information to relevant personnel so that timely action can be taken to prevent potential security threats from affecting normal operations.

[0116] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0117] Based on the same inventive concept, the embodiment of the present application also provides a data analysis device for implementing the data analysis method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data analysis device embodiments provided below can refer to the limitations on the data analysis method above, and will not be repeated here.

[0118] In one embodiment, Figure 8 As shown, Figure 8 800 is a structural block diagram of a data analysis device provided in an embodiment of the present application. The device 800 includes:

[0119] The first acquisition module 801 is used to acquire input configuration information of the target to be tracked; the configuration information at least includes the target identifier of the target to be tracked, the target subject tag published by the target to be tracked, the location information of the target to be tracked, and the time information corresponding to the target subject tag.

[0120] The second acquisition module 802 is used to generate a current data collection strategy according to the configuration information and a preset data collection frequency, and to acquire initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy.

[0121] The first determination module 803 is used to pre-process the initial social data to obtain target social data.

[0122] The second determination module 804 is used to determine the data analysis result based on the target social data and the data analysis model; the data analysis model is obtained by training the initial data analysis model based on the historical social data.

[0123] In one embodiment, the apparatus 800 further includes:

[0124] According to the data analysis results, the current data collection strategy is adjusted to obtain a new data collection strategy, the new data collection strategy is used as the current data collection strategy, and the step of obtaining the initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy is returned to obtain the data analysis results corresponding to the new data collection strategy.

[0125] In one embodiment, the second acquisition module 802 includes:

[0126] An acquisition unit, used for acquiring encrypted social data from multiple social media platforms based on a current data acquisition strategy;

[0127] The determining unit is used to determine the initial social data according to the encrypted social data.

[0128] In one embodiment, the apparatus 800 further includes:

[0129] The third acquisition module is used to obtain historical encrypted social data from multiple social media platforms through edge computing nodes.

[0130] The third determination module is used to train the initial data analysis model based on the historical encrypted social data through the edge computing node to obtain the data analysis model.

[0131] In one embodiment, the apparatus 800 further includes:

[0132] The fourth acquisition module is used to obtain the local historical encrypted social data of the federated learning participants through each federated learning participant; the historical encrypted social data is used for the federated learning participants to train the local sub-initial data analysis model of the federated learning participants to obtain a sub-data analysis model.

[0133] The fourth determination module is used to receive the sub-data analysis models sent by each federated learning participant and determine the data analysis model based on each sub-data analysis model.

[0134] In one embodiment, the apparatus 800 further includes:

[0135] The fifth acquisition module is used to acquire monitoring data in the data processing process; the data processing process includes the acquisition process of configuration information, the generation process of data collection strategy, the training process of data analysis model, the preprocessing process of initial social data, and the analysis process of data analysis results;

[0136] The generation module is used to generate alarm information when it is determined that the monitoring data is abnormal; the alarm information is used to indicate that the monitoring data is abnormal.

[0137] Each module in the above data analysis device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0138] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0139] Acquire input configuration information of the target to be tracked; the configuration information at least includes a target identifier of the target to be tracked, a target topic tag published by the target to be tracked, location information of the target to be tracked, and time information corresponding to the target topic tag;

[0140] Generate a current data collection strategy according to the configuration information and a preset data collection frequency, and obtain initial social data of a target to be tracked from multiple social media platforms based on the current data collection strategy;

[0141] Preprocessing the initial social data to obtain target social data;

[0142] The data analysis results are determined based on the target social data and the data analysis model; the data analysis model is obtained by training the initial data analysis model based on the historical social data.

[0143] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0144] According to the data analysis results, the current data collection strategy is adjusted to obtain a new data collection strategy, the new data collection strategy is used as the current data collection strategy, and the step of obtaining the initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy is returned to obtain the data analysis results corresponding to the new data collection strategy.

[0145] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0146] Obtain encrypted social data from multiple social media platforms based on current data collection strategies;

[0147] Initial social data is determined based on the encrypted social data.

[0148] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0149] Obtain historical encrypted social data from multiple social media platforms through edge computing nodes;

[0150] The initial data analysis model is trained based on historical encrypted social data through edge computing nodes to obtain a data analysis model.

[0151] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0152] Obtain local historical encrypted social data of the federated learning participants through each federated learning participant; the historical encrypted social data is used by the federated learning participants to train the local sub-initial data analysis model of the federated learning participants to obtain a sub-data analysis model;

[0153] Receive the sub-data analysis models sent by each federated learning participant, and determine the data analysis model based on each sub-data analysis model.

[0154] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0155] Acquire monitoring data during the data processing process; the data processing process includes the process of acquiring configuration information, the process of generating data collection strategies, the process of training data analysis models, the process of preprocessing initial social data, and the process of analyzing data analysis results;

[0156] When it is determined that the monitoring data is abnormal, an alarm message is generated; the alarm message is used to indicate that the monitoring data is abnormal.

[0157] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0158] Acquire input configuration information of the target to be tracked; the configuration information at least includes a target identifier of the target to be tracked, a target topic tag published by the target to be tracked, location information of the target to be tracked, and time information corresponding to the target topic tag;

[0159] Generate a current data collection strategy according to the configuration information and a preset data collection frequency, and obtain initial social data of a target to be tracked from multiple social media platforms based on the current data collection strategy;

[0160] Preprocessing the initial social data to obtain target social data;

[0161] The data analysis results are determined based on the target social data and the data analysis model; the data analysis model is obtained by training the initial data analysis model based on the historical social data.

[0162] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0163] According to the data analysis results, the current data collection strategy is adjusted to obtain a new data collection strategy, the new data collection strategy is used as the current data collection strategy, and the step of obtaining the initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy is returned to obtain the data analysis results corresponding to the new data collection strategy.

[0164] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0165] Obtain encrypted social data from multiple social media platforms based on current data collection strategies;

[0166] Initial social data is determined based on the encrypted social data.

[0167] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0168] Obtain historical encrypted social data from multiple social media platforms through edge computing nodes;

[0169] The initial data analysis model is trained based on historical encrypted social data through edge computing nodes to obtain a data analysis model.

[0170] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0171] Obtain local historical encrypted social data of the federated learning participants through each federated learning participant; the historical encrypted social data is used by the federated learning participants to train the local sub-initial data analysis model of the federated learning participants to obtain a sub-data analysis model;

[0172] Receive the sub-data analysis models sent by each federated learning participant, and determine the data analysis model based on each sub-data analysis model.

[0173] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0174] Acquire monitoring data during the data processing process; the data processing process includes the process of acquiring configuration information, the process of generating data collection strategies, the process of training data analysis models, the process of preprocessing initial social data, and the process of analyzing data analysis results;

[0175] When it is determined that the monitoring data is abnormal, an alarm message is generated; the alarm message is used to indicate that the monitoring data is abnormal.

[0176] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0177] Acquire input configuration information of the target to be tracked; the configuration information at least includes a target identifier of the target to be tracked, a target topic tag published by the target to be tracked, location information of the target to be tracked, and time information corresponding to the target topic tag;

[0178] Generate a current data collection strategy according to the configuration information and a preset data collection frequency, and obtain initial social data of a target to be tracked from multiple social media platforms based on the current data collection strategy;

[0179] Preprocessing the initial social data to obtain target social data;

[0180] The data analysis results are determined based on the target social data and the data analysis model; the data analysis model is obtained by training the initial data analysis model based on the historical social data.

[0181] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0182] According to the data analysis results, the current data collection strategy is adjusted to obtain a new data collection strategy, the new data collection strategy is used as the current data collection strategy, and the step of obtaining the initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy is returned to obtain the data analysis results corresponding to the new data collection strategy.

[0183] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0184] Obtain encrypted social data from multiple social media platforms based on current data collection strategies;

[0185] Initial social data is determined based on the encrypted social data.

[0186] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0187] Obtain historical encrypted social data from multiple social media platforms through edge computing nodes;

[0188] The initial data analysis model is trained based on historical encrypted social data through edge computing nodes to obtain a data analysis model.

[0189] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0190] Obtain local historical encrypted social data of the federated learning participants through each federated learning participant; the historical encrypted social data is used by the federated learning participants to train the local sub-initial data analysis model of the federated learning participants to obtain a sub-data analysis model;

[0191] Receive the sub-data analysis models sent by each federated learning participant, and determine the data analysis model based on each sub-data analysis model.

[0192] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0193] Acquire monitoring data during the data processing process; the data processing process includes the process of acquiring configuration information, the process of generating data collection strategies, the process of training data analysis models, the process of preprocessing initial social data, and the process of analyzing data analysis results;

[0194] When it is determined that the monitoring data is abnormal, an alarm message is generated; the alarm message is used to indicate that the monitoring data is abnormal.

[0195] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0196] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0197] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0198] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data analysis method, characterized in that: The method comprises: Acquire input configuration information of the target to be tracked; the configuration information at least includes a target identifier of the target to be tracked, a target subject tag published by the target to be tracked, location information of the target to be tracked, and time information corresponding to the target subject tag; Generate a current data collection strategy according to the configuration information and a preset data collection frequency, and acquire initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy; Preprocessing the initial social data to obtain target social data; A data analysis result is determined based on the target social data and a data analysis model; the data analysis model is obtained by training an initial data analysis model based on historical social data.

2. The method according to claim 1, characterized in that The method further comprises: According to the data analysis results, the current data collection strategy is adjusted to obtain a new data collection strategy, the new data collection strategy is used as the current data collection strategy, and the step of obtaining the initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy is returned to obtain the data analysis results corresponding to the new data collection strategy.

3. The method according to claim 1 or 2, characterized in that: The obtaining of initial social data of the target to be tracked from multiple social media platforms based on the current data collection strategy includes: acquiring encrypted social data from the plurality of social media platforms based on a current data collection strategy; The initial social data is determined according to the encrypted social data.

4. The method according to claim 1 or 2, characterized in that: The method further comprises: Acquire historical encrypted social data from the plurality of social media platforms through edge computing nodes; The initial data analysis model is trained by the edge computing node based on the historical encrypted social data to obtain the data analysis model.

5. The method according to claim 1 or 2, characterized in that: The method further comprises: Obtaining local historical encrypted social data of the federated learning participant through each federated learning participant; the historical encrypted social data is used by the federated learning participant to train the local sub-initial data analysis model of the federated learning participant to obtain a sub-data analysis model; Receive the sub-data analysis models sent by each of the federated learning participants, and determine the data analysis model based on each of the sub-data analysis models.

6. The method according to claim 1 or 2, characterized in that: The method further comprises: Acquiring monitoring data during data processing; the data processing process includes the acquisition process of the configuration information, the generation process of the data collection strategy, the training process of the data analysis model, the preprocessing process of the initial social data, and the analysis process of the data analysis results; When it is determined that the monitoring data is abnormal, an alarm message is generated; the alarm message is used to indicate that the monitoring data is abnormal.

7. A data analysis device, characterized in that: The device comprises: A first acquisition module is used to acquire input configuration information of the target to be tracked; the configuration information at least includes a target identifier of the target to be tracked, a target subject tag published by the target to be tracked, location information of the target to be tracked, and time information corresponding to the target subject tag; A second acquisition module, configured to generate a current data acquisition strategy according to the configuration information and a preset data acquisition frequency, and acquire initial social data of the target to be tracked from multiple social media platforms based on the current data acquisition strategy; A first determination module, configured to pre-process the initial social data to obtain target social data; The second determination module is used to determine the data analysis result based on the target social data and the data analysis model; the data analysis model is obtained by training an initial data analysis model based on historical social data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images