Auxiliary processing system for crowd sensing data

Abstract

The invention discloses a crowd sensing data auxiliary processing system which is managed by a single leader node and receives data collected by member nodes managed by the leader node. The crowd sensing data auxiliary processing system comprises a sensing data preprocessing module, a participant evaluation module and a task release decision module; the perception data preprocessing module is responsible for collecting perception data from task participants and preprocessing the perception data; the participant evaluation module is responsible for comprehensively evaluating each task participant according to the data quality; and the task release decision module is responsible for making a decision on participant selection of the current task according to a historical result of the participant evaluation module. According to the method, rapid verification and clustering analysis of the data collected by the member nodes are realized, the security and accuracy of data processing are improved, and the robustness of the whole crowd sensing system is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of software engineering and the Internet of Things, and in particular relates to a group intelligence perception data auxiliary processing system. Background Art

[0002] With the rapid development of mobile Internet, Internet of Things, big data and cloud computing technologies, crowd sensing has emerged as a new type of intelligent sensing model. Crowd sensing systems use a large number of distributed sensing devices (such as smart phones, sensor nodes, etc.) to collect information about the physical world and achieve perception and understanding of complex environments and large-scale events in a collaborative way. These systems have shown great application potential in environmental monitoring, traffic management, public safety and other fields.

[0003] The leader node plays a core role in the crowd-sensing system, responsible for data aggregation, processing and task allocation. It ensures the accuracy of data and the effectiveness of task execution through efficient data processing capabilities. It is the key to the stable operation of the entire system and the link between the cloud platform and member nodes.

[0004] However, although crowd-sensing systems have attracted more and more attention in the current application context and have begun to play a specific role in certain scenarios, they are still in their infancy and not perfect in many aspects of the software architecture due to limited development time. On the leadership node side, the imperfect software system has problems such as uneven resource allocation and insufficient system robustness, which will affect the performance and reliability of the entire crowd-sensing system. Summary of the invention

[0005] In view of the above problems, the present invention proposes a crowd-sensing data auxiliary processing system, which is used to assist the leader node to efficiently and accurately complete the tasks delivered by the cloud platform, and is committed to solving the problem that the data collected by the crowd-sensing system is redundant and inefficient due to software architecture defects in many cases. The present invention is mainly aimed at the interaction between the leader node and the task group, which is an important part of the working process of the crowd-sensing system.

[0006] A group intelligence perception data auxiliary processing system, the entire data auxiliary processing system is managed by a single leader node, receives data collected from member nodes under the management of the leader node, and combines the results to perform data preprocessing, comprehensive evaluation, decision-making, etc. for the current group.

[0007] The crowd intelligence perception data auxiliary processing system includes perception data preprocessing module, participant evaluation module, and task release decision module;

[0008] (1) The perception data preprocessing module is responsible for collecting perception data from task participants and performing preprocessing;

[0009] This module mainly includes data verification module and cluster analysis module; through detailed analysis of perception data, the participant evaluation module in the auxiliary data processing system can correctly evaluate the participants and realize fair and stable data transaction tasks;

[0010] (2) The participant evaluation module is responsible for comprehensively evaluating each task participant based on data quality; this module mainly includes a participant collaboration analysis module and a participant cost-effectiveness analysis module; wherein the participant collaboration analysis module analyzes the contribution of the participants in the collaboration based on the Shapley value calculation results to reflect the collaboration between the participants, thereby achieving a three-dimensional and diversified evaluation of the task participants;

[0011] (3) The task release decision module is responsible for making decisions on the selection of participants for the current task based on the historical results of the participant evaluation module;

[0012] This module selects appropriate participants to participate in subsequent crowd perception tasks according to the characteristics of participants analyzed by the data auxiliary processing system, thereby improving the data collection efficiency of the perception platform. The crowd perception data auxiliary processing system adds a check header to the terminal device software of the task participants; when the data is wirelessly transmitted to the crowd perception data auxiliary processing system of the leader node, the data verification module will first parse and verify the information in the check header to ensure the correctness of the data before handing it over to the subsequent cluster analysis module for preliminary perception data analysis; if the data verification fails, the data auxiliary processing system will send a request to the member node, requiring the member node to resend the perception data, thereby ensuring the correctness of the analyzed data and the robustness of the system; the cluster analysis module will wait for all the perception data in the group to pass the data verification, or all the data collectors who have not passed the data verification give up sending data, and then start cluster analysis of all data; the cluster analysis module will automatically calculate the data center point as the reference data standard and delete all outlier data, thereby improving the robustness of the system.

[0013] The data verification module verifies all data input into the entire crowd-sensing data auxiliary processing system one by one, and obtains the calculation result after decoding and calculation of the digital error correction code; if the data verification is passed, the data is input into the subsequent cluster analysis module for further processing; if the data verification is not passed, a request is sent to the collector of the data, requiring the member node to resend the sensing data until the data verification is passed;

[0014] The cluster analysis module will perform cluster analysis on the data through an internally integrated clustering algorithm; the basic operation of this module is to automatically select a clustering method, calculate the data center point, delete outlier data, and evaluate the quality of all data based on the distance between each data and the data center point.

[0015] The participant evaluation module includes the participant collaboration analysis module and the participant cost-effectiveness analysis module;

[0016] The crowd-sensing data auxiliary system stores the relevant data of the member nodes completing the sensing task, including all the sensing data processed by the sensing data preprocessing module, the proportion of valid data transmitted by each member node, the data quality evaluation results of the clustering analysis module, and the bids of the participants obtained from the member nodes;

[0017] When all member nodes complete the perception task, the participant evaluation module will extract the valid data percentage, data quality evaluation results and participant bids of the node stored in the crowd intelligence perception auxiliary system;

[0018] First, the participant collaboration analysis module combines the data extracted by the participant evaluation module to calculate the Shapley value and analyze the contribution of the member node based on it;

[0019] Then the participant cost-effectiveness analysis module combines the analysis results of the participant collaboration analysis module, the data quality evaluation results and the participant bids, and calculates the cost-effectiveness of the node's task through a cost-effectiveness evaluation algorithm;

[0020] The crowd-sensing data-assisted system sends the contribution and cost-performance calculation results back to the member nodes, which store the historical evaluation results themselves.

[0021] The participant collaboration analysis module calculates the Shapley value and analyzes the contribution of the participants to the entire task group according to the Shapley value, thereby showing the collaboration between the participants;

[0022] The participant cost-effectiveness analysis module calculates the cost-effectiveness value according to the cost-effectiveness evaluation algorithm by extracting the analysis results of the participant collaboration analysis module, the data quality evaluation results and the participant bids, and provides a historical cost-effectiveness basis for subsequent task release decisions; wherein, the analysis results of the participant collaboration analysis module and the data quality evaluation results serve as the basis for the "quality" in the cost-effectiveness, and the participant bids serve as the basis for the "price" in the cost-effectiveness. The crowd intelligence perception data auxiliary processing system includes a task release decision module;

[0023] After the leader node receives the task information from the cloud platform, the crowd sensing data auxiliary processing system will apply to the group members for the historical evaluation results stored in its own storage. The task release decision module will calculate the task matching degree of the current group members through the task matching algorithm, and then decide which members will participate in the crowd sensing task, thereby improving the work efficiency of the system.

[0024] The working steps of the crowd intelligence perception data auxiliary processing system are as follows:

[0025] Step 1. The cloud platform selects the leader node and the corresponding task group;

[0026] Step 2. After the terminal is selected as the leader node, the crowd-sensing data auxiliary processing system is activated;

[0027] Step 3. The task release decision module selects appropriate task participants based on the task requirements and historical evaluation results of group members;

[0028] Step 4. During the sensing task, the sensing data preprocessing module receives data from the member nodes and processes it accordingly;

[0029] Step 5. The processed data is sent to the cloud platform by the leader node;

[0030] Step 6. After the perception data collection is completed, the participant evaluation module conducts a comprehensive evaluation on each member node, and the evaluation results are sent to the member nodes for storage, waiting for the application of the task release decision module at the next task.

[0031] Beneficial effects:

[0032] The present invention has the following advantages:

[0033] 1. The perception data preprocessing module implemented in the present invention realizes the rapid verification and cluster analysis of the data collected by the member nodes, improves the security and accuracy of data processing, introduces an error retransmission mechanism in the data verification link, enhances the system's ability to handle abnormal data, and improves the robustness of the entire group intelligence perception system.

[0034] 2. The participant evaluation module implemented by the present invention comprehensively considers the two dimensions of collaboration and cost-effectiveness, and conducts a diversified evaluation of the contribution of the participants through methods such as the Shapley value, thereby increasing the fairness and three-dimensionality of the evaluation.

[0035] 3. The task release decision module implemented by the present invention intelligently selects task participants according to historical evaluation results and task matching algorithms, optimizes resource allocation, and improves the efficiency and quality of task execution.

[0036] 4. The present invention has a clear and distinct software architecture with clear division of labor and a flexible, adjustable and upgradeable software algorithm, has good scalability, is easy to upgrade and maintain, and can be expanded or tailored according to actual application requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a block diagram of the crowd-sensing data auxiliary processing system;

[0038] Figure 2It is a flow chart of the application method of the crowd intelligence perception data auxiliary processing system;

[0039] Figure 3 is an efficiency comparison curve between the method of the present invention and the conventional method;

[0040] Figure 4 is a regret comparison curve between the method of the present invention and the conventional method;

[0041] Figure 5 It is a schematic diagram showing the performance of the method of the present invention in terms of participant selection;

[0042] Figure 6 It is a schematic diagram of the performance of the method of the present invention in terms of participant evaluation. DETAILED DESCRIPTION

[0043] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following embodiments, and all technologies realized based on the content of the present invention belong to the scope of the present invention.

[0044] The main content to be achieved by the present invention is to realize an efficient and secure software architecture suitable for data processing of leader nodes in a crowd-sensing system. It is characterized in that the software architecture is a crowd-sensing data auxiliary processing system, which consists of a perception data preprocessing module, a participant evaluation module, and a task release decision module, and is managed by a single leader node, and preprocesses, comprehensively evaluates and makes decisions on the member node data under the management of the leader node, thereby helping to realize an efficient and secure crowd-sensing system.

[0045] Embodiment 1: A system for assisting processing of group intelligence perception data, characterized in that the software architecture consists of a perception data preprocessing module, a participant evaluation module, and a task release decision module. When the terminal is designated as the leader node of this task by the cloud platform, this software will be enabled to assist the leader node in data processing, comprehensive analysis, and decision-making. In this process, the perception data preprocessing module is responsible for data preprocessing, the participant evaluation module is responsible for comprehensive analysis, and the task release decision module is responsible for decision-making and task allocation.

[0046] The perception data preprocessing module is characterized in that it includes a data verification module and a cluster analysis module. The group intelligence perception data auxiliary processing system adds a verification header to the terminal device software of the task participants. The data verification module verifies all data input into the entire group intelligence perception data auxiliary processing system one by one, and obtains the calculation result after decoding and calculation of the digital error correction code. If the data verification is passed, the data is input into the subsequent cluster analysis module for further processing. If the data verification is not passed, a request is sent to the collector of the data, requiring the member node to resend the perception data until the data verification is passed. The cluster analysis module will perform cluster analysis on the data through an internally integrated clustering algorithm. The clustering algorithm includes operations such as automatically calculating the data center point as a reference data standard and deleting all outlier data, so as to improve the robustness of the system.

[0047] The participant evaluation module is characterized in that it includes a participant collaboration analysis module and a participant cost-effectiveness analysis module. The participant collaboration analysis module calculates the marginal contribution and weight factor based on the custom characteristic function and the received data quality evaluation results and the proportion of valid data, and then applies the Shapley value calculation formula to obtain the Shapley value, and then analyzes and calculates the participant's contribution to the entire task group. The participant cost-effectiveness analysis module calculates the cost-effectiveness value based on the cost-effectiveness evaluation algorithm by extracting the data quality evaluation results, the participant contribution obtained by the analysis and calculation of the participant collaboration analysis module, and the participant bid. Among them, the data quality evaluation results and the participant contribution are used as the evaluation factors of "quality" in the cost-effectiveness algorithm, and the participant bid is used as the evaluation factor of "price" in the cost-effectiveness algorithm. The contribution degree and the cost-effectiveness value constitute the comprehensive analysis result of the participant evaluation module.

[0048] Specifically, the cost-performance analysis module works as follows: First, the platform will calculate the accuracy of the data submitted by each participant:

[0049]

[0050] in, represents the data accuracy of participant i in task j, N represents the total amount of data collected by the platform, represents the data of participant i in task j, s j Represents the standard deviation of the data. The standard Euclidean distance was used here to evaluate the accuracy of the data of individual participants.

[0051] Then, based on the theory of group collaboration, the platform will use the Shapley value to calculate the contribution of participants who meet the data requirements in terms of data accuracy. The specific calculation formula is as follows:

[0052]

[0053] Among them, θ i represents the current Shapley value of participant i, W represents the total number of workers, and Q represents the data accuracy of the current data. Here, by evaluating the Shapley value of all workers, we can obtain the contribution of the collaborative data collection behavior between workers to the data accuracy.

[0054] Finally, the platform linearly superimposes the data accuracy and Shapley value of each participant, which is then used for subsequent decisions on worker selection. Specifically:

[0055] A i =α·q i +β·θ i

[0056] Among them, A i That is, it is the total score of participant i. α and β are hyperparameters. The platform can dynamically adjust the values ​​of hyperparameters according to actual conditions to cope with different mobile perception tasks.

[0057] The above Shapley value calculation pseudo code is as follows:

[0058]

[0059] The task release decision module is used for the leading node to calculate the task matching degree and select appropriate task participants according to the task requirements and the historical data of the group members. The task release decision module obtains the historical comprehensive evaluation data stored by the member nodes, calculates the task matching degree of the current group members through the task matching algorithm according to the current task requirements and characteristics, and then decides which members will participate in the group intelligence perception task to improve the work efficiency of the system.

[0060] Working steps:

[0061] 1. The cloud platform selects the leader node and the corresponding task group.

[0062] 2. After the terminal is selected as the leader node, the crowd sensing data auxiliary processing system is activated.

[0063] 3. The task release decision module selects appropriate task participants based on task requirements and historical evaluation results of group members.

[0064] 4. During the perception task, the perception data preprocessing module receives data from member nodes and processes it accordingly.

[0065] 5. The processed data is sent to the cloud platform by the leader node.

[0066] 6. After the perception data collection is completed, the participant evaluation module conducts a comprehensive evaluation of each member node, and the evaluation results are sent to the member nodes for storage, waiting for the next task and the application of the task release decision module.

[0067] Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A crowd intelligence perception data auxiliary processing system, characterized in that: The entire data-assisted processing system is managed by a single leader node, which receives data collected from member nodes managed by the leader node; The crowd intelligence perception data auxiliary processing system includes perception data preprocessing module, participant evaluation module, and task release decision module; (1) The perception data preprocessing module is responsible for collecting perception data from task participants and performing preprocessing; This module mainly includes data verification module and cluster analysis module; through detailed analysis of perception data, the participant evaluation module in the auxiliary data processing system can correctly evaluate the participants and realize fair and stable data transaction tasks; (2) The participant evaluation module is responsible for comprehensively evaluating each task participant based on data quality; this module mainly includes a participant collaboration analysis module and a participant cost-effectiveness analysis module; wherein the participant collaboration analysis module analyzes the contribution of the participants in the collaboration based on the Shapley value calculation results to reflect the collaboration between the participants, thereby achieving a three-dimensional and diversified evaluation of the task participants; (3) The task release decision module is responsible for making decisions on the selection of participants for the current task based on the historical results of the participant evaluation module; This module selects appropriate participants to participate in subsequent crowd perception tasks according to the characteristics of the participants analyzed by the data-assisted processing system, thereby improving the data collection efficiency of the perception platform.

2. The crowd intelligence perception data auxiliary processing system according to claim 1 is characterized in that: The crowd-sensing data auxiliary processing system adds a check header to the terminal device software of the task participants; when the data is wirelessly transmitted to the crowd-sensing data auxiliary processing system of the leader node, the data verification module will first parse and verify the information in the check header to ensure the correctness of the data, and then hand it over to the subsequent cluster analysis module for preliminary perception data analysis; if the data verification fails, the data auxiliary processing system will send a request to the member node, requiring the member node to resend the perception data, so as to ensure the correctness of the analyzed data and the robustness of the system; the cluster analysis module will wait for all the perception data in the group to pass the data verification, or all the data collectors who have not passed the data verification give up sending data, and then start cluster analysis of all the data; The data verification module verifies all data input into the entire crowd-sensing data auxiliary processing system one by one, and obtains the calculation result after decoding and calculation of the digital error correction code; if the data verification is passed, the data is input into the subsequent cluster analysis module for further processing; if the data verification is not passed, a request is sent to the collector of the data, requiring the member node to resend the sensing data until the data verification is passed; The cluster analysis module will perform cluster analysis on the data through an internally integrated clustering algorithm; the basic operation of this module is to automatically select a clustering method, calculate the data center point, delete outlier data, and evaluate the quality of all data based on the distance between each data and the data center point.

3. The crowd intelligence perception data auxiliary processing system according to claim 1 is characterized in that: The participant evaluation module includes the participant collaboration analysis module and the participant cost-effectiveness analysis module; The crowd-sensing data auxiliary system stores the relevant data of the member nodes completing the sensing task, including all the sensing data processed by the sensing data preprocessing module, the proportion of valid data transmitted by each member node, the data quality evaluation results of the clustering analysis module, and the bids of the participants obtained from the member nodes; When all member nodes complete the perception task, the participant evaluation module will extract the valid data percentage, data quality evaluation results and participant bids of the node stored in the crowd intelligence perception auxiliary system; First, the participant collaboration analysis module combines the data extracted by the participant evaluation module to calculate the Shapley value and analyze the contribution of the member node based on it; Then the participant cost-effectiveness analysis module combines the analysis results of the participant collaboration analysis module, the data quality evaluation results and the participant bids, and calculates the cost-effectiveness of the node's task through a cost-effectiveness evaluation algorithm; The crowd-sensing data-assisted system sends the contribution and cost-performance calculation results back to the member nodes, which store the historical evaluation results themselves. The participant collaboration analysis module calculates the Shapley value and analyzes the contribution of the participants to the entire task group according to the Shapley value, thereby showing the collaboration between the participants; The participant cost-effectiveness analysis module calculates the cost-effectiveness value according to the cost-effectiveness evaluation algorithm by extracting the analysis results of the participant collaborative analysis module, the data quality evaluation results and the participant bids, thereby providing a historical cost-effectiveness basis for subsequent task release decisions.

4. The crowd intelligence perception data auxiliary processing system according to any one of claims 1 to 3, characterized in that: The crowd-sensing data auxiliary processing system includes a task release decision module; After the leader node receives the task information from the cloud platform, the crowd-sensing data auxiliary processing system will request the historical evaluation results stored by the group members. The task release decision module will make decisions based on the historical evaluation results of the group members, including the historical contribution and cost-effectiveness of the participants in performing the task. The task release decision module will calculate the task matching degree of the current group members through the task matching algorithm, and then decide which members will participate in the group intelligence perception task, thereby improving the work efficiency of the system.

5. The crowd intelligence perception data auxiliary processing system according to claim 4 is characterized in that: The working steps of the crowd intelligence perception data auxiliary processing system are as follows: Step 1. The cloud platform selects the leader node and the corresponding task group; Step 2. After the terminal is selected as the leader node, the crowd-sensing data auxiliary processing system is activated; Step 3. The task release decision module selects appropriate task participants based on the task requirements and historical evaluation results of group members; Step 4. During the sensing task, the sensing data preprocessing module receives data from the member nodes and processes it accordingly; Step 5. The processed data is sent to the cloud platform by the leader node; Step 6. After the perception data collection is completed, the participant evaluation module conducts a comprehensive evaluation on each member node, and the evaluation results are sent to the member nodes for storage, waiting for the application of the task release decision module at the next task.

Images