A multi-source heterogeneous device motion data deduplication and fusion method

Abstract

The application relates to a multi-source heterogeneous device motion data deduplication and fusion method, which comprises the following steps: receiving uploaded data and performing same-device and cross-data-source duplication detection to obtain duplicate records; calculating multi-dimensional quality scores of the uploaded data and the duplicate records, and calculating a score difference value; when the score difference value is greater than a replacement threshold value, replacing the duplicate record with the uploaded data; when the absolute value of the score difference value is not greater than the replacement threshold value, aligning time axes of the two pieces of data and performing cross-complementary to complete fusion; and when the score difference value is less than a negative replacement threshold value, supplementing a part of the increment of the missing time period in the compensation data to the original motion data. The application can effectively avoid multi-device duplication storage, guarantee optimal data quality, and ensure the integrity and continuity of motion data records.

Description

Technical Field

[0001] This application relates to the field of motion data fusion technology, and in particular to a method for deduplication and fusion of motion data from multi-source heterogeneous devices. Background Technology

[0002] With the widespread adoption of smart fitness devices, user exercise data collection has expanded from single-device to multi-device parallel processing. When a user exercises on a treadmill, the treadmill hardware collects data such as speed, distance, and current in real time, while a mobile app collects data like heart rate and cadence via Bluetooth. Smart bracelets independently record data such as acceleration and heart rate and upload them after connecting to the internet. Different devices use their own data formats and time reference systems. For example, treadmills use a second-level offset after the start of exercise as their time reference, while smart bracelets use an absolute timestamp. This multi-source, heterogeneous data collection mode, while enriching the dimensions of exercise data, also presents challenges to data deduplication, fusion, and integrity assurance.

[0003] Existing technologies include deduplication schemes for single device types, which identify and intercept duplicate exercise records uploaded by the same device using device identifiers, user identifiers, and time information, thus avoiding duplicate storage of the same device to some extent. However, when the same exercise is recorded and uploaded separately by a treadmill and a mobile app, existing schemes cannot identify such cross-data source duplicate records because the two data sources belong to different device types, resulting in multiple duplicate records of the same exercise stored in the database. Furthermore, existing technologies lack quantitative assessment of data quality when identifying duplicate records, making decisions based solely on simple first-come, first-served or last-come overwrite strategies, causing high-quality data to be overwritten by low-quality data, thus reducing the quality of exercise data recording. At the same time, for compensation data generated in offline transmission by wristbands, Bluetooth disconnection, or weak network environments, existing technologies simply store them as independent records or simply overwrite existing data, causing the high-quality parts of the original data to be overwritten, resulting in incomplete exercise data recording.

[0004] Therefore, it is necessary to improve one or more of the problems existing in the above-mentioned related technical solutions.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this disclosure is to provide a method for deduplication and fusion of motion data from multi-source heterogeneous devices, thereby overcoming, to at least to some extent, one or more problems caused by the limitations and defects of related technologies.

[0007] This application provides a method for deduplication and fusion of motion data from multi-source heterogeneous devices, including: Receive uploaded data from at least one sports device, and perform duplicate detection on the stored sports data based on the uploaded data, both within the same device and across data sources, to obtain duplicate records; Based on the duplicate records and the uploaded data, a quality score for the duplicate records and a quality score for the uploaded data are calculated respectively; the data quality score includes data completeness, data coverage, sensor data richness, and data reliability. Calculate the score difference, and obtain replacement data and first compensation data when the score difference is greater than a preset replacement threshold; obtain the data to be fused when the absolute value of the score difference is not greater than the preset replacement threshold; obtain the second compensation data when the score difference is less than the negative value of the preset replacement threshold; the score difference is the difference between the uploaded data quality score and the duplicate record quality score. When replacement data is obtained, the duplicate records in the stored motion data are replaced with the replacement data; When the data to be fused is obtained, the fine-grained motion sequences of the two data sets to be fused are aligned along the time axis. On the aligned time axis, the field values ​​that are missing in one data set but present in the other data set are cross-filled to generate the complete motion data sequence after fusion. When the second compensation data is obtained, the incomplete motion data in the stored motion data is searched based on the user identifier and device fingerprint, and used as the original motion data. After the original motion data is found, it is compared with the second compensation data, and the part of the missing time period in the second compensation data is added to the original motion data to obtain the updated original motion data.

[0008] The technical solution provided in this application may include the following beneficial effects: This application presents a method for deduplication and fusion of motion data from multi-source heterogeneous devices. Through two levels of duplicate detection—both within the same device and across data sources—it accurately identifies duplicate motion records from different devices and data sources before data is written, effectively solving the problem of duplicate data storage in scenarios involving parallel recording by multiple devices. Furthermore, by constructing a multi-dimensional quality scoring system encompassing data integrity, second-level data coverage, sensor data richness, and anti-fraud reliability, it quantitatively evaluates duplicate records and automatically determines the processing path based on the score difference. This ensures that duplicate records are replaced when the uploaded data is significantly better than existing records, and replaced when the quality is similar but each has its own advantages. The system performs fusion in real-time and only retains compensated data when the quality is significantly inferior to existing records. This ensures optimal overall data quality while avoiding data loss caused by a single decision-making model. Furthermore, for the fusion path, fine-grained motion sequences from different devices are aligned along the time axis and cross-completed, allowing the fused data to leverage the advantages of multi-device data acquisition at every time point. For the compensation path, incomplete original motion records are located and only the missing portions are incrementally written, enabling breakpoint resumption while avoiding overwriting existing high-quality data, thus ensuring the integrity and continuity of motion data records.

[0009] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0011] Figure 1 This diagram illustrates a flowchart of a method for deduplication and fusion of motion data from multi-source heterogeneous devices in an exemplary embodiment of this disclosure. Figure 2 A detailed flowchart of step S100 of the method for deduplication and fusion of motion data from multi-source heterogeneous devices in an exemplary embodiment of this disclosure is shown. Figure 3 A detailed flowchart of step S300 of the method for deduplication and fusion of motion data from multi-source heterogeneous devices in an exemplary embodiment of this disclosure is shown. Figure 4 A detailed flowchart of step S500 of the multi-source heterogeneous device motion data deduplication and fusion method in an exemplary embodiment of this disclosure is shown. Figure 5 A detailed flowchart of step S600 of the method for deduplication and fusion of motion data from multi-source heterogeneous devices in an exemplary embodiment of this disclosure is shown. Detailed Implementation

[0012] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0013] This example implementation first provides a method for deduplication and fusion of motion data from multiple heterogeneous devices. This method can be applied to a terminal device, such as a mobile terminal like a mobile phone, desktop computer, personal digital assistant, laptop, tablet, or smartwatch. (See reference) Figure 1 As shown, the method may include the following steps: Step S100: Receive uploaded data from at least one sports device, and perform duplicate detection on the stored sports data based on the uploaded data for both same device and cross-data source to obtain duplicate records.

[0014] Step S200: Based on the duplicate records and the uploaded data, calculate the duplicate record quality score and the uploaded data quality score respectively; the data quality score includes data completeness, data coverage, sensor data richness and data reliability.

[0015] Step S300: Calculate the score difference, and obtain replacement data and first compensation data when the score difference is greater than a preset replacement threshold, obtain the data to be fused when the absolute value of the score difference is not greater than the preset replacement threshold, and obtain second compensation data when the score difference is less than the negative value of the preset replacement threshold; the score difference is the difference between the uploaded data quality score and the duplicate record quality score.

[0016] Step S400: When replacement data is obtained, the duplicate record in the stored motion data is replaced with the replacement data.

[0017] Step S500: When the data to be fused is obtained, the fine-grained motion sequences of the two data sets to be fused are aligned along the time axis. On the aligned time axis, the field values ​​that are missing in one data set but present in the other data set are cross-filled to generate the fused complete motion data sequence.

[0018] Step S600: When the second compensation data is obtained, the incomplete motion data in the stored motion data is searched based on the user identifier and device fingerprint as the original motion data. After the original motion data is found, it is compared with the second compensation data, and the part of the missing time period in the second compensation data is added to the original motion data to obtain the updated original motion data.

[0019] The aforementioned method effectively solves the problem of duplicate data storage in multi-device parallel recording scenarios by identifying duplicate motion records from different devices and data sources before data writing through a two-level duplicate detection mechanism involving both the same device and different data sources. Furthermore, it quantifies duplicate records through a multi-dimensional quality scoring system and automatically determines the optimal processing path based on the score difference. When the quality of new data is significantly better than existing records, it performs replacement; when the quality is similar but each has its own advantages, it performs fusion; and when the quality is significantly worse than existing records, it is only retained as compensation, avoiding data quality loss caused by a single decision-making mode. At the same time, for compensation data with significantly lower quality than existing records, it locates incomplete original motion records and writes incremental portions of the compensation data that belong to the missing time periods. This achieves breakpoint resume while avoiding overwriting existing data, ensuring the integrity and continuity of motion data recording.

[0020] Below, we will refer to Figures 2 to 5 The steps of the method described above in this example embodiment will be explained in more detail.

[0021] In step S100, uploaded data from at least one motion device is received, and duplicate detection of the stored motion data is performed on the same device and across data sources based on the uploaded data to obtain duplicate records.

[0022] It should be noted that the exercise equipment can include various types such as treadmills, stationary bikes, exercise bikes, rowing machines, and smart bracelets; data sources cover multiple channels including mobile applications, device hardware, semi-hardware devices, and virtual game data. Different devices use different storage structures and time reference systems for the exercise data collected: devices such as treadmills, stationary bikes, rowing machines, and mobile applications use a second-level offset after the start of exercise as the time reference; smart bracelet devices use an absolute timestamp as the time reference. Each device has a unique device fingerprint as an identifier to distinguish different physical devices.

[0023] In one embodiment, such as Figure 2 As shown, step S100 may include the following sub-steps.

[0024] In step S110, uploaded data is received from at least one sports device; the uploaded data includes user identifier, device fingerprint, device type, data source type, sports start time, sports distance, sports type, sports status and modification time.

[0025] It should be noted that the uploaded data carries motion status information, including three states: in progress, finished, and paused. In addition, the uploaded data may also contain Bluetooth connection status flags and network environment status flags, which are used to identify whether the data upload occurred in a Bluetooth interruption or weak network environment. These flags will play a triggering role in the subsequent compensation processing.

[0026] In step S120, based on the device type, and through user identification, device fingerprint, motion start time, motion type and motion status, duplicate detection of the same device is performed.

[0027] It should be noted that duplicate detection of the same device refers to checking whether a record corresponding to the exercise already exists in the database for uploaded data from the same physical device. Different matching strategies are used for different device types: wristband devices, because their offline data transmission timestamp accuracy is only at the minute level, require fuzzy matching using a time window method; devices such as treadmills can use precise matching.

[0028] Furthermore, step S120 may include the following sub-steps.

[0029] In step S121, when the device type in the uploaded data is a wristband, a time window of a preset duration is constructed based on the start time of the exercise. The database is queried for historical records that meet the conditions that the user identifier, device fingerprint, exercise type, and exercise status are all completed. If a record is found, it is determined to be a duplicate of the same device.

[0030] It should be noted that the exercise types of fitness trackers are divided into two main categories: mobile-initiated and hardware-initiated. Mobile-initiated types refer to scenarios where the tracker is connected to a mobile phone and the phone takes the lead in collecting data; hardware-initiated types refer to scenarios where the tracker operates completely independently, stores exercise data offline, and re-transmits it after connecting to the internet. The latter covers a variety of sports such as free training, rowing machines, indoor and outdoor running, indoor and outdoor walking, and indoor and outdoor cycling.

[0031] The wristband uses a preset time window for deduplication, for example, extending two minutes beyond the minute of the start time of the exercise. This is to address the issue that offline data from the wristband only provides time stamps with minute-level precision. The query matching criteria are user ID, device fingerprint, exercise type, and exercise status all being "completed". It should be noted that in existing technologies, the device fingerprint filtering condition is actually ineffective, causing similar exercise data uploaded by different wristband devices within the same time period to be indistinguishable. This application addresses this filtering condition to ensure the accuracy of deduplication.

[0032] In step S122, when the device type in the uploaded data is a treadmill, the database is queried for historical records that meet the conditions of user identifier, treadmill identifier, device fingerprint, and modification time not earlier than the start time of exercise. If a record is found, it is determined to be a duplicate of the same device.

[0033] It should be noted that for deduplication within the same device in the treadmill scenario, the query matching conditions are that the user ID, treadmill number, and device fingerprint all match, and the modification time of the existing record in the database is not earlier than the start time of the exercise of the currently uploaded data. The most recent record that meets the conditions is retrieved during the query.

[0034] In step S130, when no duplicate devices are detected, duplicate detection is performed across data sources using user ID, exercise start time, exercise mileage, data source type, and exercise type to obtain duplicate records.

[0035] It should be noted that cross-data source duplicate detection is a second-level deduplication step performed when the same-device duplicate detection fails. Its purpose is to identify duplicate records resulting from the same exercise being recorded and uploaded separately by different data sources. The core criteria for cross-source duplicate determination include: identical user identifiers, similar exercise start times (device deviation less than a preset time threshold), similar exercise mileage (device deviation less than a preset mileage threshold), different data source types, and the exercise types belonging to the same major exercise category.

[0036] Furthermore, step S130 may include the following sub-steps.

[0037] In step S131, if no duplicate devices are detected, the stored motion data is queried to find historical records that have the same user identifier as the currently uploaded data, have a motion start time deviation of less than the first threshold, a motion mileage deviation of less than the second threshold, different data source types, and belong to the same motion category, and these are used as candidate historical records.

[0038] It should be noted that the exercise start time deviation threshold and exercise mileage deviation threshold can be flexibly configured according to actual business needs. The determination that exercise types belong to the same exercise category means that, for example, indoor running initiated by the wristband hardware and outdoor running belong to the same exercise category, as do running initiated by the mobile application and running initiated by the wristband hardware.

[0039] In step S132, for the candidate historical record, the fine-grained motion sequence of the candidate historical record is obtained from the stored motion data and the velocity sequence is extracted. At the same time, the velocity sequence is extracted from the fine-grained motion sequence of the currently uploaded data. The similarity of the velocity curves of the two is calculated. If the similarity exceeds the third threshold, it is determined to be a cross-source duplicate and the candidate historical record is treated as a duplicate record.

[0040] It should be noted that the calculation of speed curve similarity utilizes second-level motion data stored in the database. Specifically, it can be done by: extracting speed sequences from the second-level data of candidate historical records, and simultaneously extracting speed sequences from the second-level data of the currently uploaded data; aligning the two speed sequences along the time axis and calculating the similarity; if the similarity exceeds a preset threshold, it is further confirmed that the two records are the same motion.

[0041] In step S200, based on the duplicate records and the uploaded data, a duplicate record quality score and an uploaded data quality score are calculated respectively; the data quality score includes data completeness, data coverage, sensor data richness, and data reliability.

[0042] It should be noted that when multiple records of the same movement are identified, the retention, replacement, or fusion strategy is determined by a data quality score. The data quality score is evaluated from the following four dimensions: The first dimension is data completeness, which is evaluated based on the ratio of exercise duration to target duration, or the ratio of exercise mileage to target mileage.

[0043] The second dimension is the second-level data coverage, which is the actual number of second-level data records in the database divided by the duration of the activity, used to assess whether there are any missing data per second.

[0044] The third dimension is the richness of sensor data, which is evaluated based on the types of sensor data carried by different data sources. For example, the device hardware has real-time current and speed data, the mobile application has positioning trajectory and acceleration counter, and the smart bracelet has independent sensor data.

[0045] The fourth dimension is anti-cheating credibility, which is evaluated based on whether the sports data passes the cheating detection and the cheating risk level. Data that fails the cheating detection and has a low risk level is given priority.

[0046] Furthermore, the formula for calculating the quality score is as follows: in, To rate the quality, For data integrity, For second-level data coverage, For sensor data richness, To improve the credibility of anti-cheating measures, As the first weight, As the second weight, As the third weight, As the fourth weight, This is the actual time. For the target time, This is the actual mileage. For the target mileage, This represents the actual number of data entries. Summing the exponential function across all dimensions, This represents the total number of sensor dimensions.

[0047] It should be noted that the cumulative value of sensor data richness in the formula represents the summation of all sensor data dimensions contained in each motion record. The total number of sensor dimensions includes all collectable data dimension types such as heart rate, current, rotational speed, cadence, and positioning trajectory. Each dimension is a binary indicator; that is, if a record has data in that dimension, it is counted as 1, otherwise it is counted as 0.

[0048] Let R be an indicator function, R be a motion record, and k be the index of the sensor dimension. To sum over k, When a record R contains data from the k-th sensor, the percentage of non-empty and non-zero entries is greater than 50%, otherwise it is 0. In other words, in the second-level data, each entry is counted individually: only when the percentage of non-empty and non-zero data points in the entire second-level data of the motion is greater than 50% is the record considered to actually contain this dimension, denoted as 1. This refers to the number of sensor dimensions that are truly effective in this record R, for example, including four types: speed, distance, heart rate, and cadence. =4, .

[0049] In step S300, a score difference is calculated, and replacement data and first compensation data are obtained when the score difference is greater than a preset replacement threshold. Data to be fused is obtained when the absolute value of the score difference is not greater than the preset replacement threshold. Second compensation data is obtained when the score difference is less than the negative value of the preset replacement threshold. The score difference is the difference between the uploaded data quality score and the duplicate record quality score.

[0050] It should be noted that the score difference is the difference between the uploaded data quality score and the duplicate record quality score; based on the comparison result of this score difference and the preset replacement threshold, it enters one of three processing paths: replacement, fusion, or compensation. If the difference is greater than the preset replacement threshold, it means that the quality of the uploaded data is significantly better than the existing records, and the replacement path is entered; If the absolute value of the difference is not greater than the preset replacement threshold, it means that the two records are of similar quality but each has its own advantages, and they enter the fusion path. If the difference is less than a negative value of the preset replacement threshold, it indicates that the quality of the uploaded data is significantly worse than that of the existing records, and the compensation path is entered.

[0051] In one embodiment, such as Figure 3 As shown, step S300 may include the following sub-steps.

[0052] In step S310, the difference between the uploaded data quality score and the duplicate record quality score is taken as the score difference value.

[0053] It should be noted that the score difference is the uploaded data quality score minus the duplicate record quality score.

[0054] In step S320, if the score difference is greater than the preset replacement threshold, the uploaded data quality score is used as the replacement data, and the duplicate record is used as the first compensation data.

[0055] It should be noted that when the score difference is greater than the preset replacement threshold, it means that the uploaded data is superior to the existing duplicate records in all quality dimensions. At this time, the replacement strategy is adopted to overwrite the existing records with the uploaded data, while the existing records are marked as the first compensation data and retained for future reference.

[0056] In step S330, if the absolute value of the score difference is not greater than the preset replacement threshold, the duplicate record and the uploaded data are used as data to be merged.

[0057] It should be noted that when the absolute value of the score difference is not greater than the preset replacement threshold, it indicates that although the two records are of comparable overall quality, they each have advantages in different data dimensions. For example, the device hardware data has physical quantities such as real-time current and rotation speed, but lacks heart rate information; the mobile application data has heart rate information, but lacks physical quantity data from the device hardware. In this case, the data from the two records is complementary and should enter the fusion process.

[0058] In step S340, if the score difference is less than the negative value of the preset replacement threshold, the uploaded data is used as the second compensation data.

[0059] It should be noted that when the score difference is less than the negative value of the preset replacement threshold, it indicates that the quality of the uploaded data is significantly worse than the existing duplicate records. In this case, the existing records are not overwritten, but the uploaded data is marked as the second compensation data and retained as auxiliary reference data, which does not affect the quality of the master data.

[0060] In step S400, when replacement data is obtained, the duplicate record in the stored motion data is replaced with the replacement data.

[0061] It should be noted that the replacement operation is used in scenarios where the quality of the uploaded data is significantly better than that of existing duplicate records. During replacement, the summary information and fine-grained motion data of the uploaded data are used to fully cover all the data corresponding to the existing record, while the existing record is marked as the first compensation data so that historical data can be traced when needed.

[0062] In step S500, when the data to be fused is obtained, the fine-grained motion sequences of the two data to be fused are aligned along the time axis. On the aligned time axis, the field values ​​that are missing in one data and present in the other data are cross-filled to generate the fused complete motion data sequence.

[0063] It should be noted that when two records are of similar quality but each has its own advantageous dimensions, they enter the fusion process. The overall idea of ​​fusion is as follows: first, the fine-grained motion data of the two records are aligned along the time axis, and then the data dimensions missing in one record but present in the other are cross-completed to generate a complete motion data sequence that combines the advantages of both records.

[0064] In one embodiment, such as Figure 4 As shown, step S500 may include the following sub-steps.

[0065] In step S510, for the data to be fused that uses the time reference in the fine-grained motion sequence as the running time offset, the data is aligned according to the running time offset with the motion start time as the reference. For the data to be fused that uses the time reference in the fine-grained motion sequence as the absolute timestamp, the absolute timestamp is directly used for positioning and alignment.

[0066] It should be noted that different types of devices use different time reference systems for fine-grained motion data: data from device hardware and mobile applications uses the motion start time as the reference and a second-level offset as the time reference. When aligning, the motion start time is used as the reference and the second-level offset is used for alignment. Data from wristband devices uses an absolute timestamp as the time reference, and the absolute timestamp is used directly for positioning when aligning.

[0067] Before performing timeline alignment, format alignment needs to be completed first: map the two types of heterogeneous data to a common data model with a granularity of seconds, establish semantic correspondence between fields, such as speed corresponding to speed, heart rate corresponding to heart rate, mileage corresponding to mileage, and cadence corresponding to cadence, and unify the time reference to a second-level offset, which can be done by subtracting the start time of exercise from the absolute timestamp of the wristband data to convert it into an offset.

[0068] In step S520, the aligned data to be fused are generated into the same timeline sequence. The data to be fused is fused by traversing each time point of the timeline sequence and when the motion data of the data to be fused at the same time point is inconsistent, a judgment is made according to the preset data source type priority.

[0069] It should be noted that after aligning the two sets of data to be merged along the timeline, a unified timeline sequence is generated. Each time point in this timeline sequence is traversed, and the data of the two records is compared at the same time point: if one record is missing a certain data dimension at that time point while the other record has that data, it is filled in; if both records have the same data dimension at the same time point but are inconsistent, the data is decided according to a preset data source priority. For example, treadmill data has the highest priority.

[0070] Furthermore, the step of traversing each time point of the timeline sequence includes: If the first data lacks the first motion data at the first time point, but the second data contains the first motion data, then the first motion data value of the second data is copied to the corresponding position of the first data. If the second data lacks second motion data at the second time point, while the first data contains the second motion data, then the second motion data value of the first data is copied in reverse to the corresponding position of the second data.

[0071] It should be noted that the cross-completion rule is as follows: traverse each time point of the unified timeline sequence. If the first data point lacks heart rate at a certain time point while the second data point has heart rate, then copy the heart rate value of the second data point to the corresponding position of the first data point. Conversely, if the second data point lacks real-time current at a certain time point while the first data point has real-time current, then copy the real-time current value of the first data point to the corresponding position of the second data point. This point-by-point traversal and bidirectional completion ensures that the fused data is as complete as possible at each time point.

[0072] In step S530, for the fused data, the motion duration, motion mileage and total number of steps are recalculated based on the complete fine-grained motion sequence to obtain the fused data.

[0073] It should be noted that after fusion, the summary indicators such as exercise duration, exercise distance, and total steps are recalculated based on the complete fine-grained motion sequence, and the record is marked as having undergone fusion processing. The calculation of summary indicators should be based on the complete fused data sequence, rather than simply adding the summary values ​​of the two records, to ensure the accuracy of the statistical results.

[0074] In step S600, when the second compensation data is obtained, incomplete motion data in the stored motion data is searched based on the user identifier and device fingerprint as the original motion data. After the original motion data is found, it is compared with the second compensation data, and the part of the second compensation data belonging to the missing time period is added to the original motion data to obtain updated original motion data.

[0075] It should be noted that the scenario where the score difference is less than a negative value of the preset replacement threshold can trigger the compensation upload process when the uploaded data carries a Bluetooth disconnection flag or a weak network data flag. The core idea of ​​the compensation process is as follows: First, based on the user identifier and device fingerprint in the second compensation data, the most recent record with a status of "in progress" is searched from the stored motion data as the original motion data; then, the second compensation data is compared with the original motion data to identify the missing time period in the original data; finally, only the part of the second compensation data that belongs to the missing time period is written into the original data, without overwriting the existing data.

[0076] Data in the "In Progress" state is a normal product of the exercise process. Its sources include: when the user starts exercising, the first piece of data with the status of "In Progress" is uploaded first, followed by incremental data uploaded periodically, and it is only marked as "Completed" when the exercise ends; in the case of Bluetooth disconnection, the bracelet continues to record independently while the server records the status as "In Progress"; in the case of weak network environment, the device cannot send the exercise end signal in time, causing the server to keep the record in the "In Progress" state; in the case of the bracelet operating completely independently and storing data offline, the recording status when it first connects and uploads is "In Progress".

[0077] In one embodiment, such as Figure 5 As shown, step S600 may include the following sub-steps.

[0078] In step S610, based on the user identifier and device fingerprint in the second compensation data, motion data with an incomplete status is searched from the stored motion data as the original motion data.

[0079] It should be noted that when searching for original activity records, the basic matching criteria are user ID and device fingerprint matching, and the activity status being "in progress". The most recent record that meets the criteria is taken as the original activity data. However, when the compensation data comes from the wristband device and its activity type is the hardware offline data transmission type, a more refined matching process is required because the timestamp accuracy of the wristband's offline data is only at the minute level.

[0080] Furthermore, in step S610, when the device type of the second compensation data is a wristband device and its motion type is hardware-side offline data transfer, it includes: In step S611, the absolute timestamp sequence and the step sequence are extracted from the second compensation data.

[0081] It should be noted that when the device type of the second compensation data is a wristband and its activity type is hardware-based offline data transmission, the absolute timestamp sequence and step count sequence are first extracted from the second compensation data as the basis for subsequent precise matching. The absolute timestamp is used to determine the time range of the activity, and the step count sequence is used to verify the consistency of the data.

[0082] In step S612, the stored motion data is queried to find historical records that match the user identifier, belong to the hardware offline retransmission type, and whose motion start time deviates from the first absolute timestamp of the second compensation data by less than a preset tolerance, as candidate sets.

[0083] It should be noted that historical records in the stored data that meet the following conditions are selected as the candidate set: the user identifier matches, the activity type belongs to the hardware offline retransmission type, and the deviation between the activity start time and the first absolute timestamp of the compensation data is less than the preset tolerance. This tolerance is used to cope with the time deviation caused by clock drift of the wristband device or data upload delay.

[0084] In step S613, for each candidate historical record in the candidate set, a time range is extracted from its fine-grained motion sequence, the overlap rate between the time range and the second compensation data time range is calculated, and candidate historical records with an overlap rate of more than 50% are retained.

[0085] It should be noted that for each candidate historical record in the candidate set, the time range of its fine-grained motion data is extracted, and the overlap rate between this time range and the time range of the compensation data is calculated. Specifically, the intersection time length is divided by the total time length of the compensation data. Candidate historical records with an overlap rate greater than half are retained. If the overlap rate is too low, it indicates that the two records do not match in time and do not belong to the same motion.

[0086] In step S614, within the overlapping time period, the step sequence of the candidate historical record and the step sequence of the second compensation data are aligned by time, the step difference is calculated point by point, and the percentage of time points within the preset tolerance deviation range of the step difference is counted. When the percentage of time points exceeds 80%, it is regarded as the original motion data found; otherwise, it is regarded as the original motion data not found.

[0087] It should be noted that within the overlapping time period, the step count sequence of the candidate historical records and the step count sequence of the compensation data are aligned by time, the step count difference is calculated point by point, and the percentage of time points where the step count difference is within the preset tolerance deviation range is statistically analyzed. As a continuously accumulated quantity during the movement, the trend of step count changes can effectively reflect the consistency of the movement process. When the step count consistency exceeds 80%, the candidate record is determined to be the corresponding original movement data; otherwise, it is considered that no original movement data was found.

[0088] In step S620, a time reference value is extracted from the second compensation data; wherein, if the device type of the second compensation data is a wristband device, the absolute timestamp in its fine-grained motion sequence is extracted, and if the device type of the second compensation data is a treadmill device, the running time offset in its fine-grained motion sequence is extracted.

[0089] It should be noted that after retrieving the original exercise data, time reference values ​​are extracted from the compensated data for subsequent missing segment comparison: if the device type of the compensated data is a fitness tracker, the absolute timestamp is extracted from its fine-grained data; if the device type of the compensated data is a treadmill, mobile application, stationary bike, or rowing machine, the second-level offset is extracted from its fine-grained data. Different time reference values ​​are used for different device types to adapt to their respective data storage formats.

[0090] In step S630, the time reference value is compared point by point with the time range of the fine-grained motion sequence that already exists in the original motion data to identify time points in the original motion data that are not covered by any fine-grained data, thus forming missing time periods.

[0091] It should be noted that the extracted time reference values ​​are compared point-by-point with the time range of the existing fine-grained motion sequences in the original motion data to identify time points in the original motion data that are not covered by any fine-grained data. These time points constitute the missing time periods. This step achieves precise identification of missing segments, pinpointing exactly which seconds of data are missing, rather than simply determining whether the overall data is complete.

[0092] In step S640, a subset of data whose time points fall within the missing time period is selected from the second compensation data and written into the original motion data without overwriting the data already existing in the original motion data.

[0093] It should be noted that a subset of data points falling within the missing time period is selected from the compensated data and written into the original motion data. During the writing process, existing data in the original motion data is not overwritten; only the missing portions are incrementally filled in. This avoids full overwrite that could result in existing high-quality data being covered by low-quality compensated data, while preserving valid information already present in the original data, thus achieving precise fusion of the compensated and original data.

[0094] In step S650, after the writing is completed, the exercise duration, exercise mileage and total number of steps are recalculated based on the complete fine-grained motion sequence to obtain updated original motion data.

[0095] It should be noted that after incremental completion, the summary indicators such as exercise duration, exercise distance, and total steps are recalculated based on the complete fine-grained exercise sequence, and the record is marked as having undergone compensation processing. The calculation of summary indicators should be based on the completed data sequence after completion to ensure that the statistical results are consistent with the actual exercise process.

[0096] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A method for deduplication and fusion of motion data from multi-source heterogeneous devices, characterized in that, include: Receive uploaded data from at least one sports device, and perform duplicate detection on the stored sports data based on the uploaded data, both within the same device and across data sources, to obtain duplicate records; Based on the duplicate records and the uploaded data, a quality score for the duplicate records and a quality score for the uploaded data are calculated respectively; the data quality score includes data completeness, data coverage, sensor data richness, and data reliability. Calculate the score difference, and obtain replacement data and first compensation data when the score difference is greater than a preset replacement threshold; obtain the data to be fused when the absolute value of the score difference is not greater than the preset replacement threshold; obtain the second compensation data when the score difference is less than the negative value of the preset replacement threshold; the score difference is the difference between the uploaded data quality score and the duplicate record quality score. When replacement data is obtained, the duplicate records in the stored motion data are replaced with the replacement data; When the data to be fused is obtained, the fine-grained motion sequences of the two data sets to be fused are aligned along the time axis. On the aligned time axis, the field values ​​that are missing in one data set but present in the other data set are cross-filled to generate the complete motion data sequence after fusion. When the second compensation data is obtained, the incomplete motion data in the stored motion data is searched based on the user identifier and device fingerprint, and used as the original motion data. After the original motion data is found, it is compared with the second compensation data, and the part of the missing time period in the second compensation data is added to the original motion data to obtain the updated original motion data.

2. The method for deduplication and fusion of motion data from multi-source heterogeneous devices according to claim 1, characterized in that, The steps of receiving uploaded data from at least one sports device and performing duplicate detection on the stored sports data based on the uploaded data to obtain duplicate records include: Receive uploaded data from at least one sports device; the uploaded data includes user identifier, device fingerprint, device type, data source type, exercise start time, exercise distance, exercise type, exercise status, and modification time; Based on device type, and through user identifier, device fingerprint, movement start time, movement type and movement status, duplicate detection of the same device is performed; If no duplicate devices are detected, duplicate records are obtained by performing cross-data source duplicate detection based on user ID, exercise start time, exercise mileage, data source type, and exercise type.

3. The method for deduplication and fusion of motion data from multi-source heterogeneous devices according to claim 2, characterized in that, The step of detecting duplicate devices based on device type and using user identifier, device fingerprint, movement start time, movement type, and movement state includes: When the device type in the uploaded data is a wristband, a time window of preset duration is constructed based on the start time of the exercise. The database is queried for historical records that meet the conditions that the user identifier, device fingerprint, exercise type and exercise status are all completed. If a record is found, it is determined to be a duplicate of the same device. When the device type in the uploaded data is a treadmill, the database is queried for historical records that meet the conditions of user identifier, treadmill identifier, device fingerprint, and modification time not earlier than the start time of exercise. If a record is found, it is determined to be a duplicate of the same device.

4. The method for deduplication and fusion of motion data from multi-source heterogeneous devices according to claim 2, characterized in that, The step of performing cross-data source duplicate detection to obtain duplicate records when no duplicate devices are detected includes: When no duplicate devices are detected, the stored exercise data is queried to find historical records that have the same user ID as the currently uploaded data, whose exercise start time deviation is less than the first threshold, whose exercise mileage deviation is less than the second threshold, whose data source type is different, and whose exercise type belongs to the same exercise category, and these are used as candidate historical records. For the candidate historical record, the fine-grained motion sequence and velocity sequence are obtained from the stored motion data. At the same time, the velocity sequence is extracted from the fine-grained motion sequence of the currently uploaded data. The similarity of the velocity curves of the two is calculated. If the similarity exceeds the third threshold, it is determined to be a cross-source duplicate and the candidate historical record is treated as a duplicate record.

5. The method for deduplication and fusion of motion data from multi-source heterogeneous devices according to claim 1, characterized in that, The formula for calculating the quality score is as follows: in, To rate the quality, For data integrity, For second-level data coverage, For sensor data richness, To improve the credibility of anti-cheating measures, As the first weight, As the second weight, As the third weight, As the fourth weight, This is the actual time. For the target time, This is the actual mileage. For the target mileage, This represents the actual number of data entries. Summing the exponential function across all dimensions, This represents the total number of sensor dimensions.

6. The method for deduplication and fusion of motion data from multi-source heterogeneous devices according to claim 1, characterized in that, The steps of calculating the score difference, obtaining replacement data and first compensation data when the score difference is greater than a preset replacement threshold, obtaining data to be fused when the absolute value of the score difference is not greater than the preset replacement threshold, and obtaining second compensation data when the score difference is less than a negative value of the preset replacement threshold include: The difference between the uploaded data quality score and the duplicate record quality score is taken as the score difference value; If the score difference is greater than the preset replacement threshold, the uploaded data quality score will be used as the replacement data, and the duplicate record will be used as the first compensation data. If the absolute value of the score difference is not greater than the preset replacement threshold, the duplicate records and the uploaded data will be used as data to be merged. If the score difference is less than the negative value of the preset replacement threshold, then the uploaded data will be used as the second compensation data.

7. The method for deduplication and fusion of motion data from multi-source heterogeneous devices according to claim 1, characterized in that, The step of aligning the fine-grained motion sequences of the two data sets to be fused along the time axis when the data to be fused is obtained, and then interpolating and filling in the missing field values ​​of one data set while the other data set exists on the aligned time axis to generate a fused complete motion data sequence, includes: For data to be fused that uses the time reference in the fine-grained motion sequence as the running time offset, the motion start time is used as the reference and the running time offset is used for alignment. For data to be fused that uses the time reference in the fine-grained motion sequence as the absolute timestamp, the absolute timestamp is used directly for positioning and alignment. The aligned data to be fused are generated into the same timeline sequence. Each time point of the timeline sequence is traversed, and the data to be fused is fused. When the motion data of the data to be fused at the same time point is inconsistent, a judgment is made according to the priority of the preset data source type. Based on the fused data, the motion duration, motion distance, and total number of steps are recalculated according to the complete fine-grained motion sequence to obtain the fused data.

8. The method for deduplication and fusion of motion data from multi-source heterogeneous devices according to claim 7, characterized in that, The step of traversing each time point of the timeline sequence includes: If the first data lacks the first motion data at the first time point, but the second data contains the first motion data, then the first motion data value of the second data is copied to the corresponding position of the first data. If the second data lacks second motion data at the second time point, while the first data contains the second motion data, then the second motion data value of the first data is copied in reverse to the corresponding position of the second data.

9. The method for deduplication and fusion of motion data from multi-source heterogeneous devices according to claim 2, characterized in that, The step of searching for incomplete motion data in the stored motion data based on the user identifier and device fingerprint when the second compensation data is obtained, using this as the original motion data, and comparing the original motion data with the second compensation data after finding it, and supplementing the missing time period portion of the second compensation data into the original motion data to obtain updated original motion data, includes: Based on the user identifier and device fingerprint in the second compensation data, retrieve the motion data with an incomplete status from the stored motion data as the original motion data; Extract time reference values ​​from the second compensation data; wherein, if the device type of the second compensation data is a wristband device, extract the absolute timestamp in its fine-grained motion sequence; if the device type of the second compensation data is a treadmill device, extract the running time offset in its fine-grained motion sequence. The time reference value is compared point by point with the time range of the fine-grained motion sequence that already exists in the original motion data to identify time points in the original motion data that are not covered by any fine-grained data, thus forming missing time periods. The data subset whose time point falls within the missing time period is selected from the second compensation data and written into the original motion data, without overwriting the data that already exists in the original motion data; After the data is written, the exercise duration, distance, and total steps are recalculated based on the complete fine-grained motion sequence to obtain updated original motion data.

10. The method for deduplication and fusion of motion data from multi-source heterogeneous devices according to claim 9, characterized in that, In the step of searching for incomplete motion data from the stored motion data based on the user identifier and device fingerprint in the second compensation data as the original motion data, when the device type of the second compensation data is a wristband device and its motion type is hardware-side offline data transfer, the following steps are included: Extract the absolute timestamp sequence and step sequence from the second compensation data; The candidate set is selected from the stored motion data that matches the user identifier, the motion type belongs to the hardware offline retransmission type, and the deviation between the motion start time and the first absolute timestamp of the second compensation data is less than the preset tolerance. For each candidate historical record in the candidate set, a time range is extracted from its fine-grained motion sequence, the overlap rate between the time range and the time range of the second compensation data is calculated, and candidate historical records with an overlap rate of more than 50% are retained. Within the overlapping time period, the step sequence of the candidate historical record and the step sequence of the second compensation data are aligned by time, the step difference is calculated point by point, and the percentage of time points within the preset tolerance deviation range of the step difference is counted. When the percentage of time points exceeds 80%, it is regarded as the original motion data found; otherwise, it is regarded as the original motion data not found.

Images