A data verification method and system based on federal semi-supervised learning, a terminal and a storage medium

Abstract

This invention relates to the field of data processing technology and discloses a data verification method, system, terminal, and storage medium based on federated semi-supervised learning. The method includes: acquiring timestamps, spatial coordinates, and raw data; performing cluster analysis on the timestamps, spatial coordinates, and raw data to obtain multiple spatiotemporal clusters; acquiring distribution information of abnormal data points; calculating authentication confidence scores by using a look-forward algorithm on all spatiotemporal clusters and abnormal data point distribution information; labeling the authentication confidence scores to obtain pseudo-labels; acquiring original features; inputting the pseudo-labels and original features into a global neural network for federated semi-supervised learning training to obtain a global data verification model; inputting the data to be tested into the global data verification model for verification to obtain the verification result. This invention generates pseudo-labels based on spatiotemporal clustering and a look-forward algorithm, and uses federated semi-supervised learning to train the global model, achieving efficient data verification.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data verification method, system, terminal, and computer-readable storage medium based on federated semi-supervised learning. Background Technology

[0002] Blockchain-based data storage technology: To prevent data tampering, existing technologies often combine blockchain technology to store data, leveraging its immutability to ensure data integrity. Traditional centralized machine learning verification technology: Existing solutions attempt to use AI algorithms to detect anomalies in collected data to verify its authenticity.

[0003] Insufficient Data Authenticity Verification: While existing IoT (Internet of Things) technologies can automate data collection, they cannot identify deeper-level fraudulent activities such as "data drift" or "human-induced chemical manipulation." Simply relying on blockchain can only guarantee that data uploaded to the chain is not modified, but it cannot solve the problem of "garbage in, garbage out" (i.e., the source data itself is forged). Lack of Spatiotemporal Correlation Analysis: Existing verification algorithms often treat data as independent samples, ignoring the "spatiotemporal correlation" in production events. For example, there is a logical connection between energy consumption and production activities in different factories within the same time period; existing technologies do not utilize this spatiotemporal clustering feature to assist verification. Data Privacy and Silo Issues: In enterprises with multiple factories (multiple clients), using AI for global verification typically requires uploading data from each factory centrally. This not only leads to high communication overhead but also raises the risk of internal data privacy leaks. Existing centralized training models cannot perform joint modeling while protecting the data privacy of each factory (client). Label data scarcity: In the field of original data authenticity, labeling data samples as "real" and "fake" is very difficult and expensive. Existing supervised learning methods rely on a large amount of labeled data and perform poorly in semi-supervised scenarios.

[0004] Existing methods for verifying the authenticity of raw data suffer from several drawbacks. They rely solely on automatic IoT collection and blockchain evidence storage (which fails to identify source data fraud), ignore the spatiotemporal correlations in production events (lacking logical verification across multiple factories), and employ centralized training to protect data privacy (leading to data silos and hindering collaborative modeling). Furthermore, they depend on a large number of manually labeled real samples (which perform poorly in semi-supervised scenarios). Consequently, in cross-regional, multi-factory joint verification scenarios, it is difficult to balance data privacy protection with high-precision source data fraud identification, making this a pressing issue that needs to be addressed.

[0005] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0006] The main objective of this invention is to provide a data verification method, system, terminal, and computer-readable storage medium based on federated semi-supervised learning. This invention aims to solve the problem in the prior art that, in cross-regional multi-factory joint data verification scenarios, it is impossible to identify source data fraud and protect data privacy, which makes it difficult to improve the accuracy of data authenticity identification.

[0007] To achieve the above objectives, the present invention provides a data verification method based on federated semi-supervised learning, which includes the following steps: The timestamps, spatial coordinates, and raw data collected by IoT devices are obtained, and cluster analysis is performed on the timestamps, spatial coordinates, and raw data to obtain multiple spatiotemporal clusters; Obtain the distribution information of abnormal data points, calculate the authentication confidence score by using a forward and backward look algorithm on all the spatiotemporal clusters and the distribution information of the abnormal data points, and mark the authentication confidence score according to a preset threshold range to obtain pseudo-labels; The original features and test data are obtained. The pseudo-labels and the original features are input into a global neural network for federated semi-supervised learning training to obtain a global data validation model. The test data is then input into the global data validation model for validation to obtain the validation result.

[0008] Optionally, the data verification method based on federated semi-supervised learning, wherein obtaining the timestamps, spatial coordinates, and raw data collected by the IoT device, and performing cluster analysis on the timestamps, spatial coordinates, and raw data to obtain multiple spatiotemporal clusters, specifically includes: Acquire timestamps, spatial coordinates, and raw data collected by IoT devices; The spatial coordinates are normalized using a normalization function to obtain a spatial cluster; Time clusters are obtained by performing time period analysis on the timestamps using a time decay function; Based on the DBSCAN algorithm, density clustering is performed on the original data according to the spatial cluster and the temporal cluster to obtain multiple spatiotemporal clusters.

[0009] Optionally, the data verification method based on federated semi-supervised learning, wherein the DBSCAN algorithm is used to perform density clustering on the original data according to the spatial clusters and the temporal clusters to obtain multiple spatiotemporal clusters, specifically includes: The spatial neighborhood radius and minimum number of points are obtained. Based on the DBSCAN algorithm, the original data is allocated according to the spatial neighborhood radius and the minimum number of points to obtain spatiotemporal cluster labels. Density clustering is performed on the original data based on the spatial cluster, the temporal cluster, and the spatiotemporal cluster labels to obtain multiple spatiotemporal clusters.

[0010] Optionally, in the data verification method based on federated semi-supervised learning, the pseudo-labels include abnormal labels and real labels; The process of obtaining abnormal data point distribution information involves calculating the authentication confidence score by using a look-forward algorithm on all spatiotemporal clusters and the abnormal data point distribution information, and then labeling the authentication confidence score according to a preset threshold range to obtain pseudo-labels. Specifically, this includes: Obtain the distribution information of abnormal data points, identify the cluster affiliation identifier and statistical characteristics of all the spatiotemporal clusters and the distribution information of the abnormal data points, and obtain the clustering results; The clustering results are calculated using a look-forward and look-backward algorithm to obtain the authentication confidence score. The authentication confidence score is labeled according to a preset threshold range to obtain anomaly labels and true labels.

[0011] Optionally, the data verification method based on federated semi-supervised learning, wherein obtaining the distribution information of abnormal data points, identifying the cluster affiliation identifiers and statistical characteristics of all the spatiotemporal clusters, and the distribution information of the abnormal data points to obtain clustering results, specifically includes: Obtain information on the distribution of abnormal data points, analyze the cluster affiliation identifiers of all the spatiotemporal clusters, and obtain the production mode and spatiotemporal status; Feature extraction is performed on the statistical features to obtain the center point, target density, and spatiotemporal span; The abnormal data point distribution information is identified to obtain the abnormal state; The production mode, the spatiotemporal state, the center point, the target density, the spatiotemporal span, and the abnormal state are aggregated to obtain the clustering result.

[0012] Optionally, the data verification method based on federated semi-supervised learning, wherein the steps of acquiring the original features and the data to be tested, inputting the pseudo-labels and the original features into a global neural network for federated semi-supervised learning training to obtain a global data verification model, and inputting the data to be tested into the global data verification model for verification to obtain a verification result, specifically include: Obtain the original features, and perform weighted calculations on the data volume of the original features to obtain the global neural network model parameters; The client obtains the model gradient and the test data, inputs the pseudo-label and the global neural network model parameters into the global neural network for federated semi-supervised learning training to obtain an initial data validation global model, iteratively updates the initial data validation global model according to the model gradient to obtain a data validation global model, inputs the test data into the data validation global model for validation, and obtains the validation result.

[0013] Optionally, the data verification method based on federated semi-supervised learning, wherein obtaining the original features and performing weighted calculations on the data volume of the original features to obtain global neural network model parameters specifically includes: Obtain the original features and the client's local model parameters. Then, weight the data volume of the original features and the local model parameters using a federated averaging algorithm to obtain the global neural network model parameters. ; in, Represents the parameters of the global neural network model. Indicates the amount of data. Indicates the total number of clients. This represents the local model parameters.

[0014] Furthermore, to achieve the above objectives, the present invention also provides a data verification system based on federated semi-supervised learning, wherein the data verification system based on federated semi-supervised learning: The data clustering analysis module is used to acquire the timestamps, spatial coordinates, and raw data collected by IoT devices, and to perform clustering analysis on the timestamps, spatial coordinates, and raw data to obtain multiple spatiotemporal clusters; The data tagging module is used to obtain the distribution information of abnormal data points, calculate the authentication confidence score by using a forward and backward look algorithm on all the spatiotemporal clusters and the distribution information of the abnormal data points, and tag the authentication confidence score according to a preset threshold range to obtain pseudo-labels; The data verification module is used to acquire the original features and the data to be tested, input the pseudo-labels and the original features into the global neural network for federated semi-supervised learning training to obtain the global data verification model, input the data to be tested into the global data verification model for verification, and obtain the verification result.

[0015] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a data verification program based on federated semi-supervised learning, and the data verification program based on federated semi-supervised learning, when executed by a processor, implements the steps of the data verification method based on federated semi-supervised learning as described above.

[0016] This invention acquires timestamps, spatial coordinates, and raw data collected by IoT devices. Cluster analysis is performed on the timestamps, spatial coordinates, and raw data to obtain multiple spatiotemporal clusters. Abnormal data point distribution information is acquired, and an authentication confidence score is calculated using a look-forward algorithm for all spatiotemporal clusters and abnormal data point distribution information. The authentication confidence score is then labeled according to a preset threshold range to obtain pseudo-labels. Original features and test data are acquired, and the pseudo-labels and original features are input into a global neural network for federated semi-supervised learning training to obtain a global data verification model. The test data is then input into the global data verification model for verification to obtain the verification result. This invention generates pseudo-labels based on spatiotemporal clustering and a look-forward algorithm, and uses federated semi-supervised learning to train a global model, achieving efficient data verification. Attached Figure Description

[0017] Figure 1 This is a flowchart of a preferred embodiment of the data verification method based on federated semi-supervised learning of the present invention; Figure 2 This is a schematic diagram of global model training in a preferred embodiment of the data verification method based on federated semi-supervised learning of the present invention; Figure 3 This is a structural diagram of a preferred embodiment of the data verification system based on federated semi-supervised learning of the present invention; Figure 4 This is a structural diagram of a preferred embodiment of the terminal of the device of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] Existing methods for verifying the authenticity of raw data suffer from several drawbacks. They rely solely on automatic IoT data collection and blockchain storage (which fails to identify source data fraud), ignore the spatiotemporal correlations in production events (lacking logical verification across multiple factories), and employ centralized training to protect data privacy (leading to data silos and hindering collaborative modeling). Furthermore, they depend on a large number of manually labeled real samples (which perform poorly in semi-supervised scenarios). Consequently, in cross-regional, multi-factory joint verification scenarios, it is difficult to balance data privacy protection with high-precision source data fraud identification. Therefore, a data verification method based on federated semi-supervised learning is needed. This method generates pseudo-labels based on spatiotemporal clustering and look-forward / look-back algorithms, and trains a global model using federated semi-supervised learning to achieve efficient data verification.

[0020] The preferred embodiment of the data verification method based on federated semi-supervised learning described in this invention, such as... Figure 1 and Figure 2 As shown, the data verification method based on federated semi-supervised learning includes the following steps: Step S10: Obtain the timestamp, spatial coordinates, and raw data collected by the IoT device, and perform cluster analysis on the timestamp, spatial coordinates, and raw data to obtain multiple spatiotemporal clusters.

[0021] Step S10 includes: Step S11: Obtain the timestamp, spatial coordinates, and raw data collected by the IoT device; Step S12: Normalize the spatial coordinates using a normalization function to obtain a spatial cluster; Step S13: Perform time period analysis on the timestamps using a time decay function to obtain time clusters; Step S14: Based on the DBSCAN algorithm, perform density clustering on the original data according to the spatial cluster and the temporal cluster to obtain multiple spatiotemporal clusters.

[0022] Specifically, the timestamps, spatial coordinates, and raw data (dataset D, containing raw data such as energy consumption, water consumption, sewage discharge, and employee benefits) collected by IoT devices are obtained. The spatial coordinates are normalized using a normalization function to obtain spatial clusters (combined with normalized spatial distances (Haversine distance)). The timestamps are then analyzed for time periods using a time decay function to obtain time clusters. Based on the DBSCAN algorithm (Density-Based Spatial Clustering of Applications with Noise), the raw data is density-clustered according to the spatial clusters and time clusters to obtain multiple spatiotemporal clusters.

[0023] In this embodiment, the Haversine formula is used to calculate the spherical distance between different factories. For example, the system will identify that "Factory C" and "Factory D" are far apart and belong to different spatial clusters, while workshops A and B of "Factory D" are close and belong to the same spatial cluster; time periodicity awareness is introduced. For example, if the period P is set to 86400 seconds (24 hours), the system will identify that the production data at "8:00 AM on Monday" and the data at "3:00 AM on Saturday" are not discontinuous in time.

[0024] Step S14 includes: Step S141: Obtain the spatial neighborhood radius and minimum number of points. Based on the DBSCAN algorithm, allocate the original data according to the spatial neighborhood radius and the minimum number of points to obtain spatiotemporal cluster labels. Step S142: Perform density clustering on the original data according to the spatial cluster, the temporal cluster, and the spatiotemporal cluster label to obtain multiple spatiotemporal clusters.

[0025] Specifically, the spatial neighborhood radius and minimum number of points are obtained. Based on the DBSCAN algorithm, the original data is allocated according to the spatial neighborhood radius and the minimum number of points to obtain spatiotemporal cluster labels (density clustering is performed on the data, and a "spatiotemporal cluster label (ID)" is assigned to each piece of original data). The original data is then density clustered according to the spatial cluster, the temporal cluster, and the spatiotemporal cluster label to obtain multiple spatiotemporal clusters (for example, density clustering: the original data is divided into different spatiotemporal clusters by optimizing parameters through k-distance graph analysis and grid search).

[0026] Step S20: Obtain the distribution information of abnormal data points. Calculate the distribution information of all spatiotemporal clusters and abnormal data points using a forward-backward algorithm to obtain an authentication confidence score. Mark the authentication confidence score according to a preset threshold range to obtain a pseudo-label.

[0027] Step S20 includes: Step S21: Obtain abnormal data point distribution information, identify the cluster affiliation identifier and statistical characteristics of all spatiotemporal clusters and the abnormal data point distribution information to obtain clustering results; Step S22: Calculate the clustering results using a look-forward algorithm to obtain the authentication confidence score; Step S23: Mark the authentication confidence score according to the preset threshold range to obtain abnormal labels and true labels.

[0028] Specifically, the distribution information of abnormal data points is obtained. The cluster affiliation identifiers and statistical characteristics of all spatiotemporal clusters, along with the distribution information of the abnormal data points, are identified to obtain cluster IDs. The cluster IDs are then calculated using a look-forward and look-backward algorithm to obtain authentication confidence scores (calculating the authentication confidence score for each data point). These authentication confidence scores are then labeled according to a preset threshold range to obtain abnormal labels and true labels (pseudo-labels are not simply true or false markings; they are automatically generated by the system to guide subsequent model training, and data is automatically labeled as "true labels" or "abnormal labels" according to a preset threshold range. This step transforms unsupervised spatiotemporal analysis into a label source for semi-supervised learning).

[0029] Step S21 includes: Step S211: Obtain the distribution information of abnormal data points, analyze the cluster affiliation identifiers of all the spatiotemporal clusters, and obtain the production mode and spatiotemporal status; Step S212: Extract features from the statistical features to obtain the center point, target density, and spatiotemporal span; Step S213: Identify the distribution information of the abnormal data points to obtain the abnormal state; Step S214: Aggregate the production mode, the spatiotemporal state, the center point, the target density, the spatiotemporal span, and the abnormal state to obtain the clustering result.

[0030] Specifically, the distribution information of abnormal data points is obtained, the cluster affiliation identifiers of all the spatiotemporal clusters are analyzed to obtain the production mode and spatiotemporal state (cluster affiliation identifier, which represents that the data belongs to a certain "production mode" or "spatiotemporal state"). Feature extraction is performed on the statistical features to obtain the center point, target density, and spatiotemporal span (i.e., the center point (average energy consumption / emission value), density (the density of data points), and spatiotemporal span of the cluster). The distribution information of abnormal data points is identified to obtain the abnormal state (abnormal state, which describes whether the cluster ID of the current data point is consistent with the cluster ID of the data point at the previous time and the next time). The production mode, the spatiotemporal state, the center point, the target density, the spatiotemporal span, and the abnormal state are aggregated to obtain the clustering result.

[0031] In this embodiment, the spatiotemporal clustering results include: (1) Cluster affiliation identifier, which represents that the data belongs to a certain "production mode" or "spatiotemporal state". For example, Cluster_A (ID=1): represents "D factory - normal day shift production mode", Cluster_B (ID=2): represents "C factory - normal production mode". (2) Statistical characteristics of the cluster, namely the cluster center point (average energy consumption / emission value), density (density of data points) and spatiotemporal span. For example, the "average energy consumption center" of Cluster_A is 120kW, and the data points are very dense (small variance). (3) Spatiotemporal continuity state, which describes whether the cluster ID of the current data point is consistent with the cluster ID of the data point at the previous time and the next time. For example, abnormal state: 09:00 is Cluster_A, 09:01 suddenly becomes Cluster_B (or noise), 09:02 changes back to Cluster_A to state jump. "Pseudo-labels" are not simply "true" or "false" marks; they are "reference answers" automatically generated by the system to guide subsequent model training.

[0032] Step S30: Obtain the original features and the data to be tested. Input the pseudo-labels and the original features into the global neural network for federated semi-supervised learning training to obtain the global data verification model. Input the data to be tested into the global data verification model for verification to obtain the verification result.

[0033] Step S30 includes: Step S31: Obtain the original features, and perform weighted calculation on the data volume of the original features to obtain the global neural network model parameters; Step S32: Obtain the model gradient and test data from the client, input the pseudo-label and the global neural network model parameters into the global neural network for federated semi-supervised learning training to obtain an initial data validation global model, iteratively update the initial data validation global model according to the model gradient to obtain a data validation global model, input the test data into the data validation global model for validation, and obtain the validation result.

[0034] Specifically, the process involves acquiring original features, performing weighted calculations on the data volume of the original features to obtain global neural network model parameters, acquiring the client's model gradient and the test data, inputting the pseudo-labels and the global neural network model parameters into the global neural network for federated semi-supervised learning training to obtain an initial data validation global model, iteratively updating the initial data validation global model based on the model gradient (the client encrypts and uploads the model gradient to the cloud, and the cloud performs weighted average updates to the global model, iterating until convergence), obtaining a data validation global model, inputting the test data into the data validation global model for validation, and obtaining the validation result.

[0035] Further, the original features and the client's local model parameters are obtained. The global neural network model parameters are then calculated by weighting the data volume of the original features and the local model parameters using a federated averaging algorithm (initializing the global neural network model and employing the FedAvg (federated averaging) algorithm as the aggregation strategy). ; in, Represents the parameters of the global neural network model. Indicates the amount of data. Indicates the total number of clients. This represents the local model parameters.

[0036] In this embodiment, the server does not simply sum the model parameters of all factories and take the average. Instead, it weights the parameters based on the amount of data each factory uses for training. Factories with larger amounts of data have a greater impact on the global model. The client (edge ​​gateway) receives the global model and trains it using local data (including generated pseudo-labels). Consistency regularization and pseudo-labelling techniques are introduced to maintain model robustness even with very little labeled data.

[0037] Furthermore, such as Figure 3 As shown, based on the above-described data verification method based on federated semi-supervised learning, this invention also provides a data verification system based on federated semi-supervised learning, wherein the data verification system based on federated semi-supervised learning includes: The data clustering analysis module 51 is used to obtain the timestamps, spatial coordinates and raw data collected by IoT devices, and to perform clustering analysis on the timestamps, spatial coordinates and raw data to obtain multiple spatiotemporal clusters; Data tagging module 52 is used to obtain abnormal data point distribution information, calculate the authentication confidence score by using a forward and backward look algorithm on all the spatiotemporal clusters and the abnormal data point distribution information, and tag the authentication confidence score according to a preset threshold range to obtain pseudo-labels; The data verification module 53 is used to acquire the original features and the data to be tested, input the pseudo-labels and the original features into the global neural network for federated semi-supervised learning training to obtain a global data verification model, input the data to be tested into the global data verification model for verification, and obtain the verification result.

[0038] Furthermore, such as Figure 4 As shown, based on the above-mentioned data verification method and system based on federated semi-supervised learning, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 4 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0039] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a data verification program 40 based on federated semi-supervised learning, which can be executed by the processor 10 to implement the data verification method based on federated semi-supervised learning in this application.

[0040] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the data verification method based on federated semi-supervised learning.

[0041] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The terminals communicate with each other via a system bus.

[0042] In one embodiment, when the processor 10 executes the data verification program 40 based on federated semi-supervised learning in the memory 20, the following steps are performed: The timestamps, spatial coordinates, and raw data collected by IoT devices are obtained, and cluster analysis is performed on the timestamps, spatial coordinates, and raw data to obtain multiple spatiotemporal clusters; Obtain the distribution information of abnormal data points, calculate the authentication confidence score by using a forward and backward look algorithm on all the spatiotemporal clusters and the distribution information of the abnormal data points, and mark the authentication confidence score according to a preset threshold range to obtain pseudo-labels; The original features and test data are obtained. The pseudo-labels and the original features are input into a global neural network for federated semi-supervised learning training to obtain a global data validation model. The test data is input into the global data validation model for validation to obtain the validation result. Specifically, the process of acquiring timestamps, spatial coordinates, and raw data collected by IoT devices, and performing cluster analysis on the timestamps, spatial coordinates, and raw data to obtain multiple spatiotemporal clusters, includes: Acquire timestamps, spatial coordinates, and raw data collected by IoT devices; The spatial coordinates are normalized using a normalization function to obtain a spatial cluster; Time clusters are obtained by performing time period analysis on the timestamps using a time decay function; Based on the DBSCAN algorithm, density clustering is performed on the original data according to the spatial cluster and the temporal cluster to obtain multiple spatiotemporal clusters.

[0043] Specifically, the DBSCAN algorithm-based density clustering of the original data, based on the spatial and temporal clusters, yields multiple spatiotemporal clusters, including: The spatial neighborhood radius and minimum number of points are obtained. Based on the DBSCAN algorithm, the original data is allocated according to the spatial neighborhood radius and the minimum number of points to obtain spatiotemporal cluster labels. Density clustering is performed on the original data based on the spatial cluster, the temporal cluster, and the spatiotemporal cluster labels to obtain multiple spatiotemporal clusters.

[0044] The pseudo-labels include abnormal labels and real labels; The process of obtaining abnormal data point distribution information involves calculating the authentication confidence score by using a look-forward algorithm on all spatiotemporal clusters and the abnormal data point distribution information, and then labeling the authentication confidence score according to a preset threshold range to obtain pseudo-labels. Specifically, this includes: Obtain the distribution information of abnormal data points, identify the cluster affiliation identifier and statistical characteristics of all the spatiotemporal clusters and the distribution information of the abnormal data points, and obtain the clustering results; The clustering results are calculated using a look-forward and look-backward algorithm to obtain the authentication confidence score. The authentication confidence score is labeled according to a preset threshold range to obtain anomaly labels and true labels.

[0045] The step of obtaining abnormal data point distribution information, identifying the cluster affiliation identifiers and statistical characteristics of all the spatiotemporal clusters, and obtaining clustering results specifically includes: Obtain information on the distribution of abnormal data points, analyze the cluster affiliation identifiers of all the spatiotemporal clusters, and obtain the production mode and spatiotemporal status; Feature extraction is performed on the statistical features to obtain the center point, target density, and spatiotemporal span; The abnormal data point distribution information is identified to obtain the abnormal state; The production mode, the spatiotemporal state, the center point, the target density, the spatiotemporal span, and the abnormal state are aggregated to obtain the clustering result.

[0046] The process of acquiring the original features and the data to be tested, inputting the pseudo-labels and the original features into a global neural network for federated semi-supervised learning training to obtain a global data validation model, and inputting the data to be tested into the global data validation model for validation to obtain the validation result, specifically includes: Obtain the original features, and perform weighted calculations on the data volume of the original features to obtain the global neural network model parameters; The client obtains the model gradient and the test data, inputs the pseudo-label and the global neural network model parameters into the global neural network for federated semi-supervised learning training to obtain an initial data validation global model, iteratively updates the initial data validation global model according to the model gradient to obtain a data validation global model, inputs the test data into the data validation global model for validation, and obtains the validation result.

[0047] Specifically, obtaining the original features and performing weighted calculations on the data volume of the original features to obtain the global neural network model parameters includes: Obtain the original features and the client's local model parameters. Then, weight the data volume of the original features and the local model parameters using a federated averaging algorithm to obtain the global neural network model parameters. ; in, Represents the parameters of the global neural network model. Indicates the amount of data. Indicates the total number of clients. This represents the local model parameters.

[0048] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a data verification program based on federated semi-supervised learning, and the data verification program based on federated semi-supervised learning, when executed by a processor, implements the steps of the data verification method based on federated semi-supervised learning as described above.

[0049] In summary, this invention provides a data verification method, system, terminal, and storage medium based on federated semi-supervised learning. The method includes: acquiring timestamps, spatial coordinates, and raw data collected by IoT devices; performing cluster analysis on the timestamps, spatial coordinates, and raw data to obtain multiple spatiotemporal clusters; acquiring abnormal data point distribution information; calculating authentication confidence scores by using a look-forward algorithm on all spatiotemporal clusters and abnormal data point distribution information; labeling the authentication confidence scores according to a preset threshold range to obtain pseudo-labels; acquiring raw features and test data; inputting the pseudo-labels and raw features into a global neural network for federated semi-supervised learning training to obtain a global data verification model; inputting the test data into the global data verification model for verification to obtain a verification result. This invention generates pseudo-labels based on spatiotemporal clustering and a look-forward algorithm, and uses federated semi-supervised learning to train a global model, achieving efficient data verification.

[0050] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal system that includes that element.

[0051] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0052] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A data verification method based on federated semi-supervised learning, characterized in that, The data verification method based on federated semi-supervised learning includes: The timestamps, spatial coordinates, and raw data collected by IoT devices are obtained, and cluster analysis is performed on the timestamps, spatial coordinates, and raw data to obtain multiple spatiotemporal clusters; Obtain the distribution information of abnormal data points, calculate the authentication confidence score by using a forward and backward look algorithm on all the spatiotemporal clusters and the distribution information of the abnormal data points, and mark the authentication confidence score according to a preset threshold range to obtain pseudo-labels; The original features and test data are obtained. The pseudo-labels and the original features are input into a global neural network for federated semi-supervised learning training to obtain a global data validation model. The test data is then input into the global data validation model for validation to obtain the validation result.

2. The data verification method based on federated semi-supervised learning according to claim 1, characterized in that, The process of acquiring timestamps, spatial coordinates, and raw data collected by IoT devices, and performing cluster analysis on the timestamps, spatial coordinates, and raw data to obtain multiple spatiotemporal clusters, specifically includes: Acquire timestamps, spatial coordinates, and raw data collected by IoT devices; The spatial coordinates are normalized using a normalization function to obtain a spatial cluster; Time clusters are obtained by performing time period analysis on the timestamps using a time decay function; Based on the DBSCAN algorithm, density clustering is performed on the original data according to the spatial cluster and the temporal cluster to obtain multiple spatiotemporal clusters.

3. The data verification method based on federated semi-supervised learning according to claim 2, characterized in that, The DBSCAN algorithm performs density clustering on the original data based on the spatial clusters and the temporal clusters to obtain multiple spatiotemporal clusters, specifically including: The spatial neighborhood radius and minimum number of points are obtained. Based on the DBSCAN algorithm, the original data is allocated according to the spatial neighborhood radius and the minimum number of points to obtain spatiotemporal cluster labels. Density clustering is performed on the original data based on the spatial cluster, the temporal cluster, and the spatiotemporal cluster labels to obtain multiple spatiotemporal clusters.

4. The data verification method based on federated semi-supervised learning according to claim 1, characterized in that, The pseudo-labels include abnormal labels and real labels; The process of obtaining abnormal data point distribution information involves calculating the authentication confidence score by using a look-forward algorithm on all spatiotemporal clusters and the abnormal data point distribution information, and then labeling the authentication confidence score according to a preset threshold range to obtain pseudo-labels. Specifically, this includes: Obtain the distribution information of abnormal data points, identify the cluster affiliation identifier and statistical characteristics of all the spatiotemporal clusters and the distribution information of the abnormal data points, and obtain the clustering results; The clustering results are calculated using a look-forward and look-backward algorithm to obtain the authentication confidence score. The authentication confidence score is labeled according to a preset threshold range to obtain anomaly labels and true labels.

5. The data verification method based on federated semi-supervised learning according to claim 4, characterized in that, The step of obtaining abnormal data point distribution information, identifying the cluster affiliation identifiers and statistical characteristics of all the spatiotemporal clusters, and obtaining clustering results specifically includes: Obtain information on the distribution of abnormal data points, analyze the cluster affiliation identifiers of all the spatiotemporal clusters, and obtain the production mode and spatiotemporal status; Feature extraction is performed on the statistical features to obtain the center point, target density, and spatiotemporal span; The abnormal data point distribution information is identified to obtain the abnormal state; The production mode, the spatiotemporal state, the center point, the target density, the spatiotemporal span, and the abnormal state are aggregated to obtain the clustering result.

6. The data verification method based on federated semi-supervised learning according to claim 1, characterized in that, The process of acquiring original features and test data, inputting the pseudo-labels and original features into a global neural network for federated semi-supervised learning training to obtain a global data validation model, and inputting the test data into the global data validation model for validation to obtain validation results specifically includes: Obtain the original features, and perform weighted calculations on the data volume of the original features to obtain the global neural network model parameters; The client obtains the model gradient and the test data, inputs the pseudo-label and the global neural network model parameters into the global neural network for federated semi-supervised learning training to obtain an initial data validation global model, iteratively updates the initial data validation global model according to the model gradient to obtain a data validation global model, inputs the test data into the data validation global model for validation, and obtains the validation result.

7. The data verification method based on federated semi-supervised learning according to claim 6, characterized in that, The process of obtaining the original features and weighting the data volume of the original features to obtain the global neural network model parameters specifically includes: Obtain the original features and the client's local model parameters. Then, weight the data volume of the original features and the local model parameters using a federated averaging algorithm to obtain the global neural network model parameters. ； in, Represents the parameters of the global neural network model. Indicates the amount of data. Indicates the total number of clients. This represents the local model parameters.

8. A data verification system based on federated semi-supervised learning, characterized in that, The data verification system based on federated semi-supervised learning includes: The data clustering analysis module is used to acquire the timestamps, spatial coordinates, and raw data collected by IoT devices, and to perform clustering analysis on the timestamps, spatial coordinates, and raw data to obtain multiple spatiotemporal clusters; The data tagging module is used to obtain the distribution information of abnormal data points, calculate the authentication confidence score by using a forward and backward look algorithm on all the spatiotemporal clusters and the distribution information of the abnormal data points, and tag the authentication confidence score according to a preset threshold range to obtain pseudo-labels; The data verification module is used to acquire the original features and the data to be tested, input the pseudo-labels and the original features into the global neural network for federated semi-supervised learning training to obtain the global data verification model, input the data to be tested into the global data verification model for verification, and obtain the verification result.

9. A terminal, characterized in that, The terminal includes: a memory, a processor, and a data verification program based on federated semi-supervised learning stored in the memory and executable on the processor. When the data verification program based on federated semi-supervised learning is executed by the processor, it implements the steps of the data verification method based on federated semi-supervised learning as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a data verification program based on federated semi-supervised learning, which, when executed by a processor, implements the steps of the data verification method based on federated semi-supervised learning as described in any one of claims 1-7.

Images