An athletic data lossless compression format for artificial intelligence training and a coding method and system thereof

Abstract

This invention relates to the field of data encoding and compression technology, specifically to a lossless compression format for motion data used in artificial intelligence training, along with its encoding and decoding method and system. It includes the following steps: acquiring raw FIT data files collected by a motion device; parsing the message header and data body using a FIT parser to extract device information, motion session metadata, and time-series records containing timestamps, GPS coordinates, and multi-channel sensor data to generate a structured internal data object; establishing a HAD compression container and sequentially writing the file header and compression metadata area into the HAD compression container; and compressing the timestamps based on the structured internal data object using sampling interval detection. This invention significantly reduces structural redundancy in temporal, spatial, and multi-channel physiological parameters by introducing mechanisms such as timestamp frame encoding, GPS coordinate differential and run-length compression, and sensor data stationary segment detection and hybrid encoding.

Description

Technical Field

[0001] This invention relates to the field of data encoding and compression technology, and more specifically, to a lossless compression format for motion data for artificial intelligence training, as well as its encoding and decoding methods and systems. Background Technology

[0002] With the widespread use of wearable devices such as sports watches, smart bracelets, and GPS trackers in running, cycling, hiking, and mountaineering, these devices generate massive amounts of motion and physiological sensor data daily. This data is generally stored and transmitted using the FIT (Flexible and Interoperable Data Transfer) format. While this format offers good device compatibility and cross-platform readability, its design, originating from early embedded system requirements, has revealed significant efficiency deficiencies in the current context of AI-driven big data training. The FIT format repeatedly stores a 32-bit absolute timestamp in each record, resulting in high redundancy of time information even at a fixed sampling rate of 1Hz. GPS latitude and longitude are saved point-by-point with float32 precision, leading to significant spatial data duplication due to small changes in adjacent coordinates, particularly noticeable in stationary or low-speed states. Physiological parameters such as heart rate, cadence, temperature, and altitude often exhibit stable or slow-changing trends, yet FIT still writes the same values ​​for each sample, without employing differential or run-length encoding compression. Furthermore, the explicit storage of physical quantity units and precision information, often retaining excessive decimal places (e.g., 6 decimal places for temperature), further increases storage and parsing overhead. Furthermore, FIT employs a linear sequential structure, lacking field segmentation, indexing mechanisms, and self-describing metadata. This necessitates byte-by-byte scanning for parsing, hindering rapid field location and parallel reading, severely impacting cloud-based batch processing and AI model data loading efficiency. Particularly in training long sequence models like Transformer, redundant data significantly increases input length, reduces context utilization, and intensifies memory and computational pressure. While general compression algorithms can reduce size to some extent, they cannot eliminate structural redundancy at the semantic level, and complete parsing is still required after decompression. Therefore, this paper proposes a lossless compression format for motion data in AI training, along with its encoding / decoding method and system. Summary of the Invention

[0003] The purpose of this invention is to provide a lossless compression format for motion data for artificial intelligence training, as well as its encoding and decoding method and system, to solve the problems of low storage efficiency, high parsing overhead, and poor AI training performance caused by the existing FIT format in motion data storage, which has temporal redundancy, spatial redundancy, parameter stationary segment redundancy, unit precision redundancy, and insufficient structure.

[0004] To achieve the above objectives, the present invention aims to provide a lossless compression format for motion data in artificial intelligence training and its encoding / decoding method, comprising: S1. Obtain the raw FIT data file collected by the sports equipment, parse its message header and data body through the FIT parser, extract the equipment information, sports session metadata, and time series records containing timestamps, GPS coordinates and multi-channel sensor data, and generate structured internal data objects. S2. Create a HAD compression container and write the file header and compression metadata area into the HAD compression container in sequence. S3. Based on structured internal data objects, timestamps are compressed by sampling interval detection, GPS coordinates are compressed using a GPS coordinate compression method based on motion vector prediction, and sensor data sequences are compressed using a sensor data residual coding method based on piecewise polynomial fitting. S4. The compressed timestamps, GPS coordinates and multi-channel sensor data are sequentially spliced ​​in a predetermined order to generate the final HAD file; S5. Calculate the CRC checksum of the final HAD file for decoding, restoration, and system verification.

[0005] As a further improvement to this technical solution, in step S1, the message header and data body are parsed by the FIT parser to extract device information, motion session metadata, and time series records, generating a structured internal data object, including the following steps: S1.1 Read the raw FIT file binary stream collected by the motion device and decode the first 12 bytes of the file header; S1.2 Detect the high-order flag bit of the original FIT file data packet byte to determine the data packet type of the current original FIT file; S1.3 When a data packet of type defined message is detected, the local message type, byte order identifier, global message number and field definition in the defined message are read, the reading result is stored in the definition mapping table, and a device information object is generated. S1.4 When the data packet type is detected as a data message, the field definition template is obtained from the definition mapping table according to the local message type of the data message, the corresponding number of bytes is read in sequence and the field decoding is completed, and a session metadata object and a time series record object are generated. The data message includes at least a continuous timestamp sequence, a GPS coordinate sequence, and data sequences from each sensor. S1.5. For the scaling and offset existing in the original FIT field, perform unit conversion on the numerical values ​​according to the FIT Profile; S1.6. Encapsulate the parsed device information, session metadata, and time series records into structured internal data objects in a unified format.

[0006] As a further improvement to this technical solution, in step S2, a HAD compression container is established, and the file header and compression metadata area are written sequentially into the HAD compression container, including the following steps: S2.1. Based on structured internal data objects, initialize the HAD compression container building module, define the logical structure and writing order of the HAD compression container, and set the basic organization format of the HAD file. S2.2 Write a file header at the beginning of the HAD compressed container; S2.3. Based on the structured internal data object, generate a meta-information description structure, wherein the meta-information description structure includes at least device information, motion session information, channel definition table, and encoding parameter table; S2.4. The metadata description structure is compressed using the zlib Deflate algorithm to generate a compressed stream; S2.5 Write the compressed stream into the metadata area of ​​the HAD compression container, and update the offset pointer in the HAD compression container to the beginning position of the timestamp data block after writing. S2.6 Write the container index at the end of the HAD compression container, including at least the starting offset, length information and integrity check value of each data block type; The data block types include timestamp data blocks, GPS coordinate blocks, and sensor data blocks.

[0007] As a further improvement to this technical solution, in step S3, timestamps are compressed by sampling interval detection, GPS coordinates are compressed using a GPS coordinate compression method based on motion vector prediction, and sensor data sequences are compressed using a sensor data residual encoding method based on piecewise polynomial fitting, including the following steps: S3.1 Perform sampling interval detection on the timestamps, and adaptively select the compression method based on the sampling interval detection results to generate timestamp compressed blocks; S3.2. GPS coordinates are fixed-point and differentially encoded using a GPS coordinate compression method based on motion vector prediction to generate GPS coordinate compressed blocks. GPS coordinates include latitude and longitude data; S3.3. The sensor data residual coding method based on piecewise polynomial fitting is adopted to sequentially integerize and hybridize the sensor data sequence to generate sensor compressed blocks. S3.4. Concatenate the timestamp compressed block, GPS coordinate compressed block, and sensor compressed block in sequence to form a continuous compressed data area, and update the HAD compressed container index table.

[0008] As a further improvement to this technical solution, in step S3.1, the compression method is adaptively selected based on the sampling interval detection results to generate a timestamp compression block, involving the following specific steps: Calculate the time difference between adjacent records If the time difference of all adjacent records If all values ​​are equal, it is determined to be an equal-interval sampling mode; otherwise, it is determined to be a non-equal-interval sampling mode. Each record contains a timestamp and a sensor measurement value. When using equal-interval sampling mode, frame coding is employed; when using non-equal-interval sampling mode, differential coding is employed, calculating the difference between adjacent timestamps. For the difference The sign bit is processed using ZigZag encoding and compressed using variable-length integer encoding to represent the variable difference in the fewest possible bytes.

[0009] As a further improvement to this technical solution, in step S3.2, the GPS coordinates are fixed-pointed and differentially encoded using a GPS coordinate compression method based on motion vector prediction, including the following steps: S3.21. Perform point-to-point processing on the latitude and longitude data, multiplying each longitude and latitude value by... And round down to obtain a precision of 1. Fixed-point integer representation; S3.22 Record the absolute values ​​of latitude and longitude of the first sampling point as the reference position; S3.23. Based on the motion trajectory of the first 3 to 5 sampling points, calculate the average motion vector to predict the next point; for each sampling point... The difference between the actual coordinates and the predicted coordinates is calculated as the prediction error. S3.24 When the prediction error value exceeds the set threshold When the condition is met, the traditional differential coding mode is used; otherwise, the prediction error coding path is used. S3.25. Perform ZigZag encoding on the prediction error values ​​and then perform variable-length integer encoding for compressed storage. S3.26. If the absolute value of several consecutive prediction error values ​​is less than the set threshold... If the motion device is in a predicted stable state, run-length encoding is triggered, and only the run length and stop flag of the stationary point are recorded.

[0010] As a further improvement to this technical solution, in step S3.3, a sensor data residual encoding method based on piecewise polynomial fitting is adopted to sequentially perform integerization and mixed encoding on the sensor data sequence, including the following steps: S3.31. Based on the unit scaling factor in the compressed metadata, convert the floating-point values ​​of the multi-channel sensor data into integer form; S3.32. Perform stationarity detection on the integer sequence of each sensor channel: calculate the local variance of the sequence. If it exceeds the set threshold m, it is determined to be a non-stationary sequence and enters the fitting residual coding path; otherwise, it is determined to be a stationary sequence and enters the differential coding path. The fitted residual encoding path includes the following steps: For each sensor channel, continuous data points are segmented in real time, and a quadratic polynomial fitting function is used to estimate the trend of each segment, resulting in a set of polynomial parameters. ; Calculate the residual value between each sampling point and the fitted curve, and perform adaptive quantization on the residual value to map the continuous residual value to a finite discrete level; For the polynomial parameter set Differential coding is performed, and entropy coding is applied to the residual values ​​after adaptive quantization. The differential coding path includes the following steps: Establish initial reference values ​​independently for each channel. And calculate the difference between adjacent sampling points. ; If the difference of several consecutive samples over a period of time (h) All values ​​are zero and the number of repetitions exceeds a set threshold. If the channel has entered the steady-state region, run-length encoding is triggered. If the difference If the difference is not equal to 0, incremental differential encoding is triggered. ZigZag encoding is performed on the differential value to process the sign bit, and then variable-length integer encoding is used for compression and storage as a further improvement of this technical solution. In step S4, the final HAD file is generated, which involves the following specific steps: Write the HAD file header and compressed metadata area to identify the file type, version number, and channel definition; write the timestamp compressed block, GPS coordinate compressed block, and sensor compressed block into the data block area in logical order, and insert synchronization identifiers between the data blocks; after all data blocks are assembled, perform cyclic redundancy check on the entire HAD file data segment to obtain the integrity check value of the HAD file, and write the integrity check value and the end marker together to the end of the HAD file to form the final HAD file.

[0011] As a further improvement to this technical solution, in S5, the calculation of the CRC check value of the final HAD file involves the following specific steps: recalculate the integrity check value of the read HAD file and compare it with the tail record value. If the two are consistent, it is determined that the HAD file data is complete and has not been destroyed, and the decoding and restoration process is entered; if they are inconsistent, the exception handling mechanism is triggered.

[0012] On the other hand, the present invention provides a lossless compression format for motion data for artificial intelligence training and its encoding and decoding system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the lossless compression format for motion data for artificial intelligence training and its encoding and decoding method as described above.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. The lossless compression format for motion data in artificial intelligence training, its encoding and decoding method, and system, as disclosed in this invention, significantly reduce structural redundancy in temporal, spatial, and multi-channel physiological parameters by introducing mechanisms such as timestamp frame encoding, GPS coordinate differential and run-length compression, and sensor data stationary segment detection and hybrid encoding. This achieves an overall improvement in compression ratio while ensuring complete data losslessness. Simultaneously, the adoption of an implicit unit system and structured data block design eliminates redundant unit declarations and floating-point precision, reducing storage volume and transmission overhead, avoiding precision drift issues during cross-platform parsing, and improving data consistency and reliability.

[0014] 2. The lossless compression format for motion data used in AI training, along with its encoding / decoding method and system, relates to this invention. It employs a layered container structure of "Header+Meta+Block+CRC," supporting self-describing fields and independent block-level decoding. This significantly simplifies the data reading process for cloud-based batch processing and AI models, reduces I / O bandwidth consumption, improves batch processing efficiency, enhances context window utilization, and significantly alleviates the memory and computational pressure during large-scale model training. Furthermore, this format possesses excellent backward compatibility and scalability; adding new sensor channels requires no modification to the overall structure, only declaration in the metadata, facilitating multi-device fusion and system iteration. The overall solution achieves comprehensive efficiency improvements in storage, transmission, parsing, and training, providing an efficient and reliable data infrastructure for large-scale sports and health data analysis and intelligent algorithm evolution. Attached Figure Description

[0015] Figure 1 This is a flowchart of the overall method of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0017] Example 1: Please refer to Figure 1 As shown, this embodiment provides a lossless compression format for motion data in artificial intelligence training and its encoding / decoding method, including the following steps: S1. Obtain the raw FIT (Flexible and Interoperable Data Transmission) data file collected by the sports equipment, parse its message header and data body through the FIT parser, extract the equipment information, sports session metadata, and time series records containing timestamps, GPS coordinates and multi-channel sensor data, and generate structured internal data objects. In this embodiment, generating a structured internal data object includes the following steps: S1.1 Read the raw FIT file binary stream acquired by the motion device, decode the first 12 bytes of the file header, and extract the following in sequence: Header Length: indicates the length of the file header; Protocol Version and Profile Version: determine the FIT specification version on which the parsing is based; Data Length: defines the number of bytes in the data body; Data Type (".FIT" identifier): verifies the validity of the file; after decoding, perform cyclic redundancy check on the file header to ensure that the file header is complete and error-free; S1.2 In the data body section, the FIT format adopts an alternating structure of definition messages and data messages: the parser first detects the high-order flag bit (Normal Header or Compressed TimestampHeader) of the original FIT file data packet byte to determine the data packet type of the current original FIT file; S1.3 When a data packet of type definition message is detected, the local message type, byte order identifier, global message number, and field definition (including field number, size, and basic type) in the definition message are read. The reading results are stored in the definition map so that the field template can be called and a device information object can be generated when parsing data messages later. If the global message number corresponds to device information (such as File_ID), the parser extracts fields such as device model, manufacturer, serial number and firmware version number to generate a device information object. S1.4 When a data packet of type Data Message is detected, the parser retrieves the field definition template from the Definition Map based on the local message type of the data message, reads the corresponding number of bytes sequentially, and completes field decoding (i.e., reads the number of bytes corresponding to each field definition sequentially according to the field definition template). For data packets using a Compressed Timestamp Header, the parser accumulates and recovers the previous timestamp based on the 5-bit time difference carried in the header, thereby generating a continuous time series. Each parsed record is stored in the form of field name-field value key-value pairs, and a session metadata object and a time series record object are generated. When the global message number corresponds to the Session or Lap type, extract fields such as exercise start time, duration, exercise type, distance, average power, and average heart rate to generate a session metadata object; when the global message number corresponds to the Record type, read field data from consecutive time points to generate a time series record object. The data message includes at least a continuous timestamp sequence, a GPS coordinate sequence (latitude and longitude), and data sequences from various sensors (including at least heart rate, cadence, speed, altitude, and temperature). S1.5. For the scaling and offset in the original FIT fields, the parser performs unit conversion on the values ​​according to the FIT Profile. For example, the speed field is converted to meters per second (m / s) by multiplying the original value by 0.001. At the same time, semantic mapping and unified naming are performed on the field names. For example, heart rate is mapped to HR, cadence is mapped to CAD, speed is mapped to SPD, and power is mapped to PWR to ensure that the data collected by different devices have a consistent semantic expression. S1.6. The parsed device information, session metadata, and time series records are encapsulated into structured internal data objects in a unified format. The structured internal data objects include metadata description items such as device information field (device_info) and session information field (session). The device information field is used to record device attributes such as the model, firmware version, and sampling frequency of the acquisition device. The session information field is used to identify the start and end time, motion type, and data statistics of a single data acquisition process.

[0018] S2. Create a HAD (Lossless Motion Data Compression Format) compression container, and write the file header and compression metadata area into the HAD compression container in sequence; In this embodiment, establishing a HAD compression container includes the following steps: S2.1. Based on structured internal data objects, initialize the HAD compression container building module, define the logical structure and writing order of the HAD compression container, and set the basic organization format of the HAD file: the logical order is as follows: file header, metadata area, time series block, spatial location block, sensor data block and check area (CRC). S2.2 Write a file header at the beginning of the HAD compressed container (the file header defines the identification and parsing rules of the entire container, and the decoding end can automatically identify the file structure and enter the metadata reading stage based on it). The file header should include at least the following fields: Magic identifier: fixed as the characters "HAD", used to identify the file type; Version: the current container version number, used for compatibility judgment by the decoding end; Header Length: the length of the file header in bytes. S2.3. Based on the device information field (device_info) and session information field (session) in the structured internal data object, generate a meta-information description structure. The meta-information description structure includes at least the device information (including device model, firmware version, sampling rate, and sensor type), motion session information (including motion type, start and end time, total number of samples, etc.), channel definition table (recording the name of each channel, unit conversion factor, fixed-point precision, and data type), and encoding parameter table (defining the trigger thresholds for subsequent frame differential, run-length encoding (RLE), and ZigZag encoding). The process of generating the metadata description structure is as follows: First, iterate through the structured internal data objects and extract information such as device model, firmware version, sampling rate, and sensor type from the device information field (device_info), and motion type, start and end time, and total number of samples from the session information field (session). Then, according to the predefined metadata description structure template, map the values ​​of each field to the corresponding positions and generate a channel definition table and an encoding parameter table. The channel definition table records the name, unit conversion factor, fixed-point accuracy, and data type of each sensor channel, while the encoding parameter table defines the trigger thresholds for frame differential, run-length encoding (RLE), and ZigZag encoding. Finally, all information is integrated and encapsulated into a complete metadata description structure. S2.4. The metadata description structure is compressed using the zlib Deflate algorithm (compression level set to the highest level) to generate a compressed stream. Specifically, after serializing the generated metadata description structure into a byte stream, the zlib Deflate algorithm is called to compress the byte stream, and the compression level is set to the highest level to maximize the compression ratio. During the compression process, the algorithm performs block segmentation, Huffman coding, and LZ77 matching on the input byte stream according to the Deflate process, while maintaining the compression state and window buffer. Finally, a continuous compressed stream is output as the storage data of the metadata area of ​​the HAD container, which is used to quickly restore the metadata structure during the decoding stage, realizing efficient and lossless data transmission and storage. S2.5 Write the compressed stream into the metadata area of ​​the HAD compression container. After writing, update the offset pointer inside the HAD compression container to the starting position of the timestamp data block. Here, the timestamp data block refers to the first data block in the HAD container used to store the compressed and encoded timestamp sequence, spatial location (GPS) sequence, and multi-channel sensor data. After writing the compressed metadata, the system updates the offset pointer inside the container to the starting position of this block so that the compressed data can be written sequentially in the future. S2.6. Write a container index (the container index is an index table stored at the end of the HAD compressed container, used to record the type, starting offset, length information and CRC check value of each data block, and also includes the file version number and encoding method identifier, so as to support the decoding end to quickly locate the data block and verify the data integrity) at least includes the starting offset, length information and integrity check value of each data block type, and also records the overall file version number, encoding algorithm identifier and metadata compression method, so that the decoding end can quickly locate the corresponding block and perform integrity verification. After writing, perform a global CRC calculation to generate a check code for the interval of the two consecutive logical parts at the beginning of the HAD file (the interval from Header (file header) to Meta (meta information area)) and append it to the end of the index. At this point, the initialization stage of the HAD container is completed. The data block types include timestamp data blocks, GPS coordinate blocks, and sensor data blocks.

[0019] S3. Based on structured internal data objects, timestamps are compressed by sampling interval detection, GPS coordinates are compressed using a GPS coordinate compression method based on motion vector prediction, and sensor data sequences are compressed using a sensor data residual coding method based on piecewise polynomial fitting. In this embodiment, timestamps are compressed by sampling interval detection, GPS coordinates are compressed using a GPS coordinate compression method based on motion vector prediction, and sensor data sequences are compressed using a sensor data residual encoding method based on piecewise polynomial fitting. The process includes the following steps: S3.1 Perform sampling interval detection on the timestamps, and adaptively select the compression method based on the sampling interval detection results to generate timestamp compressed blocks; The process of adaptively selecting a compression method based on the sampling interval detection results to generate timestamp compressed blocks involves the following specific steps: Calculate the time difference between adjacent records If the time difference of all adjacent records If all values ​​are equal, it is determined to be an equal-interval sampling mode; otherwise, it is determined to be a non-equal-interval sampling mode. Each record contains a timestamp and the corresponding sensor measurements such as GPS location, heart rate, and speed. When using equal-interval sampling mode, frame encoding is employed, and only the first timestamp is recorded. Fixed sampling interval Total number of samples The decoding end can then use the formula To recover the complete sequence, in the formula, This represents the time-order index. In non-equal-interval sampling mode, differential coding is used (differential coding involves calculating the difference between the current value and a reference value (such as the actual value, predicted value, or parameters of the previous sampling point), then using ZigZag coding to convert potentially negative differences into unsigned integers to optimize sign bit storage, and finally using variable-length integer coding (such as Varint) to compress the converted difference, thereby eliminating local redundancy in the time, space, or parameter sequence of the data; this mechanism adaptively triggers based on preset thresholds in different scenarios (such as timestamps, GPS coordinates, sensor data, and polynomial parameters), and dynamically switches with run-length encoding, predictive coding, and other modes to achieve efficient and lossless compression), calculating the difference between adjacent timestamps. (To reflect the changing pattern of time increments), for the difference The sign bit is processed using ZigZag encoding, and compressed for storage using variable-length integer encoding (such as Varint). (Specifically, this is because timestamps may experience rollback or abnormal transitions.) It can be negative, therefore the difference First, ZigZag encoding is performed to map signed integers to unsigned integers, allowing both positive and negative differences to be represented with a shorter bit length. Then, the ZigZag encoded result is encoded using variable-length integer encoding (Varint), with each byte encoded in groups of 7 bits. Only when the high-order flag is set to 1 does it indicate that there are more bytes to follow, thus ensuring that small differences occupy fewer bytes. This achieves efficient and lossless compression of time-series differences by representing variable differences with the minimum number of bytes required.

[0020] S3.2. GPS coordinates are fixed-point and differentially encoded using a GPS coordinate compression method based on motion vector prediction to generate GPS coordinate compressed blocks. GPS coordinates include latitude and longitude data; Furthermore, when the trajectory is not an ideal straight line or there are high-frequency jitters (such as frequent cornering while running in the city or GPS signal drift), the difference values ​​between adjacent points become huge and disordered, leading to a decrease in the efficiency of ZigZag-Varint encoding. The GPS coordinate compression method based on motion vector prediction approximates the real motion state through a more intelligent prediction model, thereby producing smaller and more stable prediction errors and achieving a higher compression ratio. It calculates the average motion vector through a sliding window, capturing the macroscopic direction and speed of the motion, and can more accurately describe the continuity brought about by the inertia of the object. Its core function is to generate a prediction error sequence with smaller values ​​and a more concentrated distribution, which significantly improves the efficiency of subsequent ZigZag and Varint encoding. Especially when dealing with curvilinear motion or uniformly accelerated motion, it can reduce data entropy more effectively than simple difference, thereby achieving a higher compression ratio while ensuring lossless operation, and may provide smoother and denoised trajectory data for AI training. GPS coordinates are digitized and differentially encoded using a GPS coordinate compression method based on motion vector prediction, including the following steps: S3.21. Perform fixed-point conversion on the latitude and longitude data (floating-point values), converting each longitude ( ) and latitude ( The values ​​are multiplied by respectively And round down to obtain a precision of 1. Fixed-point integer representation; S3.22 Record the absolute values ​​of latitude and longitude of the first sampling point. As a reference point, the sampling point here refers to the observation unit composed of time, space and sensor measurements recorded by the equipment at each moment during the continuous sampling process, which is the basic element that constitutes time series data; S3.23. Based on the motion trajectory of the first 3 to 5 sampling points, calculate the average motion vector (including direction and velocity) to predict the next point; for each sampling point... The difference between the actual coordinates and the predicted coordinates is calculated as the prediction error (the coding mode is dynamically selected based on the magnitude of the error), instead of the traditional neighboring point difference. Specifically, first select the coordinates of the first 3 to 5 consecutive sampling points within the sliding window. Calculate the displacement increment and time interval between adjacent points to obtain the instantaneous velocity vector at each step. Then, these velocity vectors are averaged to obtain the average motion vector. Its direction indicates the overall motion direction, and the modulus represents the average velocity. Based on the latest sampling point coordinates... And the average motion vector, predict the coordinates of the next sampling point as Finally, calculate the difference between the actual coordinates and the predicted coordinates. This difference is the coordinate of the prediction error; S3.24 When the prediction error value exceeds the set threshold When the prediction model parameters are not in use, the prediction model parameters are automatically adjusted or the traditional differential coding mode is used (i.e., the difference between adjacent points is directly calculated for consecutive GPS sampling points and the difference is stored using ZigZag and variable-length integer encoding); otherwise, the prediction error coding path is used (i.e., the difference between the actual coordinates and the predicted coordinates of the GPS sampling points (prediction error) is used as the coding object and stored using ZigZag and variable-length integer encoding). Specifically, after predicting the coordinates of each sampling point, the difference between the actual coordinates and the predicted coordinates is calculated. And take its modulus length This is used as the prediction error value; when this error value exceeds a set threshold... When a sudden change in motion trend or a decrease in prediction accuracy occurs, the system determines that the current prediction model is no longer applicable and triggers an adaptive adjustment mechanism: updating the average motion vector window length, re-estimating velocity weights, or directly switching to the traditional differential coding mode to ensure compression stability and reversibility; if the error value does not exceed a set threshold... If the prediction model remains unchanged, only the prediction error is encoded (e.g., ZigZag+Varint) and stored along the prediction error encoding path, thereby ensuring coordinate reconstruction accuracy while maintaining a high compression ratio. S3.25. Perform ZigZag encoding on the prediction error values ​​and then perform variable-length integer encoding for compressed storage. Specifically, after obtaining the prediction error value for each sampling point, it is first converted into fixed-point integer form for lossless integer encoding. Since the error value may be positive or negative, ZigZag encoding is performed on each error component to map the signed integer to an unsigned integer, thus allowing the sign bit to be represented efficiently. Next, the unsigned integers after ZigZag encoding are processed using variable-length integer encoding (Varint): they are stored in groups of 7 bits, and a continuation flag is set in the highest bit of each byte. When the highest bit is 1, it indicates that there are more bytes to follow, and 0 indicates the end. This process allows smaller error values ​​to occupy fewer bytes, thus significantly improving the compression ratio. Finally, the encoded byte sequence is written sequentially into the GPS coordinate compression block to form a compact prediction error compressed stream. S3.26. If the absolute value of several consecutive prediction error values ​​is less than or equal to a set threshold... (Right now If the motion device is in a predicted steady state (i.e., the prediction model can effectively describe the motion pattern), then run-length encoding (RLE) is triggered. (RLE consists of two parts: a run value field, which records the repeated data values; and a run length field, which records the number of times the value appears consecutively). Only the run length of the stationary point and the stop flag are recorded. (The stop flag being set to 1 indicates the end of the current run-length encoded segment. When the decoder encounters this flag during parsing, it should switch back to normal encoding mode (such as prediction error encoding or traditional differential encoding). This ensures that the decoder can correctly identify the boundaries of the run-length encoded block and avoid data parsing errors. For example, in compressed data, the run-length encoded block is represented as: [run length value] + [stop flag = 1], where the stop flag acts as a separator.) This omits repeated coordinate data and further improves compression efficiency. An embedded flag in the compressed data indicates whether traditional differential encoding or prediction encoding mode is currently used, ensuring that the decoder can correctly restore the data.

[0021] S3.3. A sensor data residual coding method based on piecewise polynomial fitting is adopted to sequentially integerize and hybridize the sensor data sequence (including heart rate, cadence, power, speed, temperature, altitude, etc.) to generate sensor compressed blocks. Furthermore, the sensor data residual encoding method based on piecewise polynomial fitting addresses the insufficient compression efficiency of existing RLE / Delta hybrid encoding when dealing with complex, non-stationary sensor signals (such as the rising and falling trends of heart rate, periodic changes in cadence, and nonlinear drift of temperature). RLE is only effective for completely constant values, while Delta encoding only utilizes first-order local correlation. When the data exhibits higher-order dynamics (such as accelerated rises, falls, or waveforms), its difference values ​​remain large and its entropy is high. The sensor data residual encoding method based on piecewise polynomial fitting upgrades from repetitive detection values ​​to extracting and parameterizing the inherent continuous variation function of the data. Through data adaptation, it can more accurately capture the physical laws of smooth changes in sensor signals. By decomposing the original signal into highly compressible "low-dimensional trend parameters" and "low-entropy residuals," it can achieve extremely high compression ratios even without long segments of constant values. This not only significantly reduces storage volume but, more importantly, provides smoothed and trend-extracted normalized data for AI training. After the polynomial fitting parameters and residual entropy encoding are completed, a cross-segment difference and steady-state detection mechanism (i.e., steps S3.35-S3.37) is introduced to achieve unified optimization of intra-segment and inter-segment compression. This maintains the continuity of data trends and further reduces time redundancy, thereby improving the overall lossless compression ratio and decoding accuracy. In S3.34, a set of parameters is generated after each segment fitting compression. Along with the residual sequence, these two parts describe the numerical variation characteristics of the segment. However, there are still reference value jumps between segments (for example, the last sample value of the previous segment may be different from the first sample value of the next segment). Therefore, when entering S3.35: the last sample value of each segment is used as the reference value. Calculate the difference of the first sample in the next segment. This step achieves continuity of data between segments; if the values ​​stabilize after fitting multiple consecutive segments, RLE is triggered; if the values ​​change significantly between segments, differential incremental encoding is performed. A sensor data residual coding method based on piecewise polynomial fitting is adopted to sequentially perform integerization and mixed coding on the sensor data sequence, including the following steps: S3.31. Based on the unit scaling factor in the compressed element information, convert the floating-point values ​​of the multi-channel sensor data into integer form (e.g., speed × 1000, temperature × 10, etc.) to eliminate floating-point storage errors and unify numerical precision. S3.32. Perform stationarity detection on the integer sequence of each sensor channel: calculate the local variance or difference statistic of the sequence. If it exceeds the set threshold, it is determined to be a non-stationary sequence and enters the fitting residual coding path; otherwise, it is determined to be a stationary sequence and enters the differential coding path (the fitting residual coding path is suitable for data with complex changes and trends; the differential coding path is suitable for data with gentle changes and small differences; the encoder automatically selects the path according to the data characteristics and records the path identifier in the compressed block so that the decoding end can correctly restore it). The fitted residual encoding path includes the following steps: The continuous data points of each sensor channel are segmented in real time, with each segment containing 8-16 sampling points. A quadratic polynomial fitting function (i.e., Trend estimation is performed on this data segment to obtain the set of polynomial parameters. and with This represents the global trend curve for that segment, where, The coefficient of the quadratic term, The coefficient of the linear term, For constant terms, These are the fitted values. Let be the independent variable, defined as the normalized time offset of each sampling point within the data segment relative to the beginning of the segment. The specific calculation method is as follows: Let the absolute timestamp of the first sampling point within the segment be . , No. The absolute timestamp of each sampling point is ,but ,in, This serves as the base time unit for the data segment (usually the minimum time interval within the segment or a fixed scaling factor, such as 1 second). Through this normalization process, The value is constrained to a small numerical range (typically 0 to the number of sampling points within the segment), thus avoiding the direct calculation using large numerical absolute timestamps. This process avoids potential numerical overflow or loss of precision, and ensures the numerical stability and reproducibility of the fitting process. The resulting polynomial parameter set is then obtained. Then, the fitted curve can be used to describe the overall trend of the data segment; Calculate the residual value between each sampling point and the fitted curve: Residual value = Actual value (the actual measured data value at the sampling point, i.e., the original output of the device or sensor) - Fitted value (referring to the value obtained by fitting the curve using a quadratic polynomial function (i.e., ...)). (Based on the predicted values ​​calculated from adjacent historical sampling points), the residual values ​​are adaptively quantized to map continuous residual values ​​to a finite discrete level; the quantization step size is dynamically adjusted according to the statistical characteristics of the residuals to ensure that the quantization error is within an acceptable range; Specifically, the quantization step size is first dynamically determined based on the statistical characteristics of the current data sequence. (Calculate the standard deviation of the residuals) This is used to characterize the magnitude of local changes; then, based on empirical coefficients... (e.g., 0.5 to 1.0) Determine the quantization step size This allows for adaptive adjustment of quantization precision; subsequently, each residual value is processed according to the formula... Discretization is performed, mapping continuous residual values ​​to a finite number of integer levels, thereby reducing numerical precision in exchange for storage efficiency. To avoid the accumulation of quantization errors, the system can periodically update the step size parameter. Alternatively, an error feedback mechanism can be introduced to maintain reconstruction accuracy; the final quantized residual sequence Then proceed to the subsequent entropy coding or differential compression stage to achieve efficient and reversible storage of residual information; For the polynomial parameter set Perform differential encoding (that is, subtract the current segment parameter from the corresponding parameter of the previous segment to obtain the difference vector). The residual values ​​are then encoded using ZigZag and variable-length integer encoding for efficient storage, reducing redundancy across segments. Entropy encoding (such as Huffman coding or arithmetic coding) is performed on the adaptively quantized residual values ​​to further compress the uneven symbol frequency portions. Specifically, the frequency of each discrete symbol in the quantized residual sequence is first statistically analyzed to construct a symbol probability distribution model (this model reflects the probability of each symbol appearing in the current context, providing a probability basis for subsequent variable-length encoding); then, based on this distribution, entropy encoding algorithms (such as Huffman coding or arithmetic coding) are used for compression. Symbols that appear frequently are assigned shorter codewords, while low-frequency symbols are assigned longer codewords, thus bringing the overall encoding length close to the lower limit of information entropy. During the encoding process, the probability model can be dynamically updated based on the distribution within the data block to adapt to changes in local features. Ultimately, two compressed data streams are generated: one is a polynomial parameter stream after differential encoding, and the other is a residual symbol stream after entropy encoding (i.e., an entropy-encoded bit stream). These two streams are written together into the sensor compressed block, and a path identifier is added to the data block (for example, using 0x01 to indicate that this data segment uses the "fitted residual encoding" path). This step creates two logically independent but complementary data streams: a trend stream, capturing macroscopic, slowly changing physical laws (such as the temperature rise curve of a device during startup); and a residual stream, capturing microscopic, random fluctuations and noise. This decomposition brings a synergistic effect: after extracting the trend, the numerical range and entropy of the residuals are greatly reduced, resulting in a dramatic improvement in the efficiency of subsequent entropy encoding. Simultaneously, the trend parameters themselves constitute a minimal and highly compressible sequence. The differential coding path includes the following steps: Establish initial reference values ​​independently for each channel. And calculate the difference between adjacent sampling points. ; If the difference of several consecutive samples over a period of time (h) All are zero ( And the number of repetitions exceeds the set threshold. If the channel enters the steady-state region, run-length encoding (RLE) is triggered, and the difference value (zero) and the number of consecutive samples are recorded. If the difference If the difference is not equal to 0, incremental differential encoding is triggered (incremental differential encoding is only performed on non-zero differences and is further compressed in combination with ZigZa+ variable-length integer encoding). ZigZag encoding is performed on the difference value to process the sign bit, and then compressed and stored by variable-length integer encoding (such as Varint). During the encoding process, each channel's data generates an independent tag stream, which is composed of alternating tags and encoded values. This ensures that the multi-channel sensor data remains logically isolated in the compressed file, facilitating rapid separation and channel-by-channel reconstruction at the decoding end.

[0022] S3.4. Concatenate the timestamp compressed block, GPS coordinate compressed block, and sensor compressed block in sequence to form a continuous compressed data area, and update the HAD compressed container index table to record the type identifier, starting offset, length, and CRC check code of each compressed block; finally, output the comprehensive compression result and write it into the HAD compressed container as the HAD file data body.

[0023] S4. The compressed timestamps, GPS coordinates and multi-channel sensor data are sequentially spliced ​​in a predetermined order to generate the final HAD file; In this embodiment, generating the final HAD file involves the following specific steps: writing the HAD file header and compressed metadata area to identify the file type, version number, and channel definition; writing the timestamp compressed block, GPS coordinate compressed block, and sensor compressed block into the data block area in logical order, and inserting synchronization identifiers between data blocks to ensure the correspondence between time and channel; after all data blocks are concatenated, performing cyclic redundancy check (CRC) on the entire HAD file data segment to obtain the integrity check value of the HAD file, and writing the integrity check value and the end marker (which includes the file termination identifier) ​​together to the end of the HAD file to form a structurally complete and stream-parseable final HAD file, providing a highly reliable compressed data carrier for subsequent decoding, restoration, and artificial intelligence training.

[0024] S5. Calculate the CRC check value of the final HAD file for decoding, restoration, and system verification; In this embodiment, calculating the CRC checksum of the final HAD file involves the following specific steps: recalculating the integrity checksum of the read HAD file and comparing it with the tail record value (i.e., the previously stored reference checksum). If the two match, the HAD file data is determined to be complete and undamaged, and the decoding and restoration process begins (the decoding and restoration process includes: parsing the file header and index information to determine the position and type of each data block; sequentially performing differential and ZigZag decoding on the timestamp block; performing prediction error restoration and fixed-point inversion on the spatial coordinate block; performing polynomial fitting inverse solution and residual reconstruction on the sensor data block; and finally, reconstructing the restored timestamp file). The interstamp, spatial coordinates, and multi-channel sensor data are recombined into a complete structured motion data object, enabling reversible recovery from compressed format to the original record. If inconsistencies are found, an exception handling mechanism is triggered (i.e., when file header corruption, verification failure, missing data blocks, or decoding anomalies are detected, corresponding processing measures are triggered according to the exception type, such as rereading, skipping erroneous blocks, rolling back to the previous valid state, or generating an error report, to ensure the stability of the data parsing process and the integrity of the file structure), for system integrity verification and error recovery, thereby ensuring the reliability and traceability of HAD files in artificial intelligence training applications.

[0025] Example 2: This example provides a lossless compression format for motion data for artificial intelligence training and its encoding / decoding system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the lossless compression format for motion data for artificial intelligence training and its encoding / decoding method as described above.

[0026] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A lossless compression format for motion data in artificial intelligence training and its encoding / decoding method, characterized in that, Includes the following steps: S1. Obtain the raw FIT data file collected by the sports equipment, parse its message header and data body through the FIT parser, extract the equipment information, sports session metadata, and time series records containing timestamps, GPS coordinates and multi-channel sensor data, and generate structured internal data objects. S2. Create a HAD compression container and write the file header and compression metadata area into the HAD compression container in sequence. S3. Based on structured internal data objects, timestamps are compressed by sampling interval detection, GPS coordinates are compressed using a GPS coordinate compression method based on motion vector prediction, and sensor data sequences are compressed using a sensor data residual coding method based on piecewise polynomial fitting. S4. The compressed timestamps, GPS coordinates and multi-channel sensor data are sequentially spliced ​​in a predetermined order to generate the final HAD file; S5. Calculate the CRC checksum of the final HAD file for decoding, restoration, and system verification.

2. The lossless compression format for motion data for artificial intelligence training and its encoding / decoding method according to claim 1, characterized in that: In step S1, the message header and data body are parsed by the FIT parser to extract device information, motion session metadata, and time series records, generating a structured internal data object, including the following steps: S1.1 Read the raw FIT file binary stream collected by the motion device and decode the first 12 bytes of the file header; S1.2 Detect the high-order flag bit of the original FIT file data packet byte to determine the data packet type of the current original FIT file; S1.3 When a data packet of type defined message is detected, the local message type, byte order identifier, global message number and field definition in the defined message are read, the reading result is stored in the definition mapping table, and a device information object is generated. S1.4 When the data packet type is detected as a data message, the field definition template is obtained from the definition mapping table according to the local message type of the data message, the corresponding number of bytes is read in sequence and the field decoding is completed, and a session metadata object and a time series record object are generated. The data message includes at least a continuous timestamp sequence, a GPS coordinate sequence, and data sequences from each sensor. S1.

5. For the scaling and offset existing in the original FIT field, perform unit conversion on the numerical values ​​according to the FIT Profile; S1.

6. Encapsulate the parsed device information, session metadata, and time series records into structured internal data objects in a unified format.

3. The lossless compression format for motion data for artificial intelligence training and its encoding / decoding method according to claim 1, characterized in that: In step S2, a HAD compression container is created, and the file header and compression metadata area are written into the HAD compression container sequentially, including the following steps: S2.

1. Based on structured internal data objects, initialize the HAD compression container building module, define the logical structure and writing order of the HAD compression container, and set the basic organization format of the HAD file. S2.2 Write a file header at the beginning of the HAD compressed container; S2.

3. Based on the structured internal data object, generate a meta-information description structure, wherein the meta-information description structure includes at least device information, motion session information, channel definition table, and encoding parameter table; S2.

4. The metadata description structure is compressed using the zlib Deflate algorithm to generate a compressed stream; S2.5 Write the compressed stream into the metadata area of ​​the HAD compression container, and update the offset pointer in the HAD compression container to the beginning position of the timestamp data block after writing. S2.6 Write the container index at the end of the HAD compression container, including at least the starting offset, length information and integrity check value of each data block type; The data block types include timestamp data blocks, GPS coordinate blocks, and sensor data blocks.

4. The lossless compression format for motion data for artificial intelligence training and its encoding / decoding method according to claim 1, characterized in that: In step S3, timestamps are compressed through sampling interval detection, GPS coordinates are compressed using a GPS coordinate compression method based on motion vector prediction, and sensor data sequences are compressed using a sensor data residual encoding method based on piecewise polynomial fitting. This includes the following steps: S3.1 Perform sampling interval detection on the timestamps, and adaptively select the compression method based on the sampling interval detection results to generate timestamp compressed blocks; S3.

2. GPS coordinates are fixed-point and differentially encoded using a GPS coordinate compression method based on motion vector prediction to generate GPS coordinate compressed blocks. GPS coordinates include latitude and longitude data; S3.

3. The sensor data residual coding method based on piecewise polynomial fitting is adopted to sequentially integerize and hybridize the sensor data sequence to generate sensor compressed blocks. S3.

4. Concatenate the timestamp compressed block, GPS coordinate compressed block, and sensor compressed block in sequence to form a continuous compressed data area, and update the HAD compressed container index table.

5. The lossless compression format for motion data for artificial intelligence training and its encoding / decoding method according to claim 4, characterized in that: In step S3.1, the compression method is adaptively selected based on the sampling interval detection results to generate a timestamp compressed block, which involves the following specific steps: Calculate the time difference between adjacent records If the time difference of all adjacent records If all values ​​are equal, it is determined to be an equal-interval sampling mode; otherwise, it is determined to be a non-equal-interval sampling mode. Each record contains a timestamp and a sensor measurement value. When using equal-interval sampling mode, frame coding is employed; when using non-equal-interval sampling mode, differential coding is employed, calculating the difference between adjacent timestamps. For the difference The sign bit is processed using ZigZag encoding and compressed using variable-length integer encoding to represent the variable difference in the fewest possible bytes.

6. The lossless compression format for motion data for artificial intelligence training and its encoding / decoding method according to claim 4, characterized in that: In step S3.2, the GPS coordinates are digitized and differentially encoded using a GPS coordinate compression method based on motion vector prediction, including the following steps: S3.

21. Perform point-to-point processing on the latitude and longitude data, multiplying each longitude and latitude value by... And round down to obtain a precision of 1. Fixed-point integer representation; S3.22 Record the absolute values ​​of latitude and longitude of the first sampling point as the reference position; S3.

23. Based on the motion trajectory of the first 3 to 5 sampling points, calculate the average motion vector to predict the next point; for each sampling point... The difference between the actual coordinates and the predicted coordinates is calculated as the prediction error. S3.24 When the prediction error value exceeds the set threshold When the condition is met, the traditional differential coding mode is used; otherwise, the prediction error coding path is used. S3.

25. Perform ZigZag encoding on the prediction error values ​​and then perform variable-length integer encoding for compressed storage. S3.

26. If the absolute value of several consecutive prediction error values ​​is less than the set threshold... If the motion device is in a predicted stable state, run-length encoding is triggered, and only the run length and stop flag of the stationary point are recorded.

7. The lossless compression format for motion data for artificial intelligence training and its encoding / decoding method according to claim 4, characterized in that: In step S3.3, a sensor data residual encoding method based on piecewise polynomial fitting is used to sequentially perform integerization and mixed encoding on the sensor data sequence, including the following steps: S3.

31. Based on the unit scaling factor in the compressed metadata, convert the floating-point values ​​of the multi-channel sensor data into integer form; S3.

32. Perform stationarity detection on the integer sequence of each sensor channel: calculate the local variance of the sequence. If it exceeds the set threshold m, it is determined to be a non-stationary sequence and enters the fitting residual coding path; otherwise, it is determined to be a stationary sequence and enters the differential coding path. The fitted residual encoding path includes the following steps: For each sensor channel, continuous data points are segmented in real time, and a quadratic polynomial fitting function is used to estimate the trend of each segment, resulting in a set of polynomial parameters. ; Calculate the residual value between each sampling point and the fitted curve, and perform adaptive quantization on the residual value to map the continuous residual value to a finite discrete level; For the polynomial parameter set Differential coding is performed, and entropy coding is applied to the residual values ​​after adaptive quantization. The differential coding path includes the following steps: Establish initial reference values ​​independently for each channel. And calculate the difference between adjacent sampling points. ; If the difference of several consecutive samples over a period of time (h) All values ​​are zero and the number of repetitions exceeds a set threshold. If the channel has entered the steady-state region, run-length encoding is triggered. If the difference If the result is not equal to 0, incremental differential encoding is triggered. ZigZag encoding is performed on the differential value to process the sign bit, and then it is compressed and stored using variable-length integer encoding.

8. The lossless compression format for motion data for artificial intelligence training and its encoding / decoding method according to claim 1, characterized in that: In step S4, generating the final HAD file involves the following specific steps: Write the HAD file header and compressed metadata area to identify the file type, version number, and channel definition; write the timestamp compressed block, GPS coordinate compressed block, and sensor compressed block into the data block area in logical order, and insert synchronization identifiers between the data blocks; after all data blocks are assembled, perform cyclic redundancy check on the entire HAD file data segment to obtain the integrity check value of the HAD file, and write the integrity check value and the end marker together to the end of the HAD file to form the final HAD file.

9. The lossless compression format for motion data for artificial intelligence training and its encoding / decoding method according to claim 1, characterized in that: In step S5, the calculation of the CRC check value of the final HAD file involves the following specific steps: recalculate the integrity check value of the read HAD file and compare it with the tail record value. If the two are consistent, it is determined that the HAD file data is complete and has not been corrupted, and the decoding and restoration process is entered; if they are inconsistent, the exception handling mechanism is triggered.

10. A lossless compression format for motion data for artificial intelligence training and its encoding / decoding system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The processor executes a computer program to implement the lossless compression format for motion data for artificial intelligence training and its encoding / decoding method as described in any one of claims 1-9.

Images