Data management method and system based on OpenHarmony

Abstract

The invention relates to a data management method and a data management system based on OpenHarmony. The method comprises the following steps: constructing a cross-device data storage layer by utilizing a distributed soft bus technology, and carrying out fragmentation storage and redundant backup on target data to obtain a distributed data storage architecture; performing feature extraction to obtain a data classification result and a fine-grained data mark; comprehensively analyzing the network condition, the equipment performance and the user behavior to obtain a data synchronization strategy; executing two-stage protocol submission and vector clock concurrency control on the cross-device operation to obtain a distributed transaction processing result; performing structural feature and repetitive mode analysis to obtain a compression algorithm selection result and a compression parameter; and carrying out compression processing, and carrying out adaptive routing selection and load balancing on the compressed data transmission path to obtain a multi-device data flow management scheme. According to the method, data integrity and availability are guaranteed, data are effectively compressed, and storage space and transmission bandwidth are saved.

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and in particular to a data management method and system based on OpenHarmony. Background Art

[0002] With the rapid development of the Internet of Things and distributed systems, the collaborative work of multiple devices has become a general trend. As an open source distributed operating system, OpenHarmony provides a basic platform for the interconnection of multiple devices. However, in practical applications, cross-device data management faces many challenges. Data is distributed on different devices, and how to efficiently store, synchronize and access these scattered data has become a key issue.

[0003] Secondly, the network conditions and performance differences between devices and the dynamic changes in user behavior make it difficult for traditional static data management strategies to adapt to complex and changing application scenarios. In addition, during the data transmission process, how to ensure data consistency and improve transmission efficiency while considering the limited device resources is also an urgent problem to be solved. Summary of the invention

[0004] The main purpose of the present invention is to provide a data management method and system based on OpenHarmony, so as to effectively compress data while ensuring data integrity and availability, thereby saving storage space and transmission bandwidth.

[0005] To achieve the above object, the present invention provides a data management method based on OpenHarmony, comprising the following steps: Use distributed soft bus technology to build a cross-device data storage layer, perform shard storage and redundant backup of target data, and obtain a distributed data storage architecture; Based on the distributed data storage architecture, the target data is input into a machine learning model for feature extraction and analysis to obtain data classification results and fine-grained data labels; Based on the data classification results and the fine-grained data tags, a comprehensive analysis is performed on the network status, device performance and user behavior to obtain a data synchronization strategy; Based on the data synchronization strategy, a two-phase commit protocol and vector clock concurrency control are performed on cross-device operations to obtain a distributed transaction processing result; Inputting the distributed transaction processing result into a data feature analyzer for structural feature and repetitive pattern analysis to obtain a compression algorithm selection result and compression parameters; Based on the compression algorithm selection result and the compression parameters, the target data in the distributed data storage architecture is compressed, and the compressed data transmission path is adaptively routed and load balanced to obtain a multi-device data flow management solution.

[0006] The present invention also provides a data management system based on OpenHarmony, comprising: A construction module is used to construct a cross-device data storage layer using distributed soft bus technology, perform shard storage and redundant backup of target data, and obtain a distributed data storage architecture; An extraction module, used to input the target data into a machine learning model for feature extraction and analysis based on the distributed data storage architecture, to obtain data classification results and fine-grained data tags; An analysis module, configured to comprehensively analyze network conditions, device performance, and user behavior based on the data classification results and the fine-grained data tags to obtain a data synchronization strategy; An execution module, configured to execute a two-phase commit protocol and vector clock concurrency control on cross-device operations based on the data synchronization strategy to obtain a distributed transaction processing result; A processing module, used for inputting the distributed transaction processing result into a data feature analyzer for structural feature and repetitive pattern analysis to obtain a compression algorithm selection result and compression parameters; A compression module is used to compress the target data in the distributed data storage architecture based on the compression algorithm selection result and the compression parameters, and to perform adaptive routing selection and load balancing on the compressed data transmission path to obtain a multi-device data flow management solution.

[0007] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0008] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned methods are implemented.

[0009] In summary, the technical solution provided by the present invention utilizes distributed soft bus technology to construct a cross-device data storage layer, realizes data sharding storage and redundant backup, and improves data reliability and access efficiency. The machine learning model is used to extract and analyze data features, realize automatic classification and fine-grained labeling of data, and dynamically generate data synchronization strategies by comprehensively analyzing network conditions, device performance, and user behavior, thereby improving the adaptability and efficiency of data synchronization. The two-phase commit protocol and vector clock concurrency control mechanism are introduced to ensure the consistency of distributed transactions and the accuracy of concurrency control. The data feature analyzer is used to analyze structural features and repetitive patterns, realizes intelligent selection and parameter optimization of compression algorithms, and improves data compression efficiency. Through adaptive routing selection and load balancing algorithms, the transmission path of compressed data is optimized, and the management efficiency of multi-device data streams is improved. The overall solution fully considers the distributed characteristics of OpenHarmony, realizes the optimization of the entire process from data storage, classification, synchronization, transaction processing to compressed transmission, and significantly improves the data management performance in a distributed environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a schematic diagram of the steps of a data management method based on OpenHarmony in one embodiment of the present invention; Figure 2 This is a structural block diagram of a data management system based on OpenHarmony in one embodiment of the present invention; Figure 3 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0011] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0013] Reference Figure 1 , this embodiment provides a data management method based on OpenHarmony, comprising the following steps: S1, using distributed soft bus technology to build a cross-device data storage layer, sharding and redundantly backing up the target data to obtain a distributed data storage architecture; Among them, distributed soft bus technology is used to establish cross-device data transmission and communication channels between multiple devices, which serves as the basic communication layer for data management. The target data is divided into data blocks and divided into multiple independent data blocks. These data blocks are flexibly divided according to factors such as data size, type or access frequency. After the division is completed, the data blocks are allocated to different devices according to the preset allocation strategy or the data block distribution scheme obtained by real-time calculation, and the storage resources and computing power of each device are fully utilized to obtain the initial data distribution results. The initial data distribution results are analyzed for redundancy to obtain data redundancy requirements to determine whether the distribution of each data block between different devices can meet the requirements of data security and availability. According to the analysis results, determine which data blocks need to be redundantly backed up, and generate specific data redundancy requirements. Based on the data redundancy requirements, multiple data blocks are copied to obtain redundant data blocks. The number and location of redundant data blocks should be optimized and allocated according to factors such as the failure rate, storage capacity and network bandwidth of the device to avoid the impact of single point failure on data storage. Redundant data blocks are allocated to other devices different from the devices where the original data blocks are located to form a decentralized redundant backup structure, which can still ensure the integrity and availability of data when some devices fail. Based on the initial data distribution results and redundant backup results, a distributed index structure is constructed. A global data index is formed by uniformly indexing and managing the location, access path, and other information of each data block and its redundant backup. The global data index effectively manages and locates data blocks distributed on various devices, making data access and synchronization more efficient and accurate. The constructed global data index is synchronized to all devices participating in data storage, maintaining a consistent distributed data view on all devices, and obtaining a distributed data storage architecture.

[0014] S2, based on the distributed data storage architecture, inputs the target data into the machine learning model for feature extraction and analysis to obtain data classification results and fine-grained data labels; Specifically, metadata information of the target data, including basic attributes of the data, storage location, access frequency, data format, etc., is extracted from the distributed data storage architecture to form an initial feature set. The initial feature set is subjected to dimensionality reduction processing, and a dimensionality reduction algorithm such as principal component analysis or linear discriminant analysis is used to map the high-dimensional initial feature set to a low-dimensional feature space to obtain a feature vector after dimensionality reduction. The feature vector after dimensionality reduction is input into a pre-trained machine learning model to obtain a deep feature representation of the target data. The machine learning model includes a three-layer convolutional neural network and a two-layer fully connected network structure. The convolutional neural network extracts the local pattern features of the target data through a local receptive field and a weight sharing mechanism, and gradually extracts higher-level and more abstract feature representations in different convolutional layers. After passing through the convolutional layer and the pooling layer, the obtained feature map is expanded into a one-dimensional vector, and then the global feature relationship between the data is combined and learned through a fully connected network to obtain a deep feature representation. Based on the deep feature representation, the target data is clustered and data with similar features are aggregated together to obtain preliminary data categories. The data of each preliminary category is analyzed in a fine-grained manner. According to the preliminary data category, select the corresponding tag set from the predefined tag library as the candidate tag. The semantic similarity between the candidate tag and the content of the target data is calculated to determine the degree of association between each tag and the data. The semantic similarity calculation uses methods such as word vector model and sentence vector model to map the tag and data to the same semantic space, and then calculate the cosine similarity or Euclidean distance to obtain the similarity score corresponding to each candidate tag. Based on the similarity score, the candidate tags are screened and sorted, and the tag with the highest score is retained as the final fine-grained data tag. The preliminary data category and fine-grained data tag are combined to form a complete data classification result and fine-grained data tag output. The data classification result mainly reflects the macro-category division of the data, while the fine-grained data tag enriches the semantic information of the data, enabling the data management system to perform refined management and operation on data of different types and characteristics.

[0015] S3, based on the data classification results and fine-grained data tags, conducts a comprehensive analysis of network conditions, device performance, and user behavior to obtain a data synchronization strategy; It should be noted that, based on the data classification results, the target data is prioritized according to factors such as importance and timeliness to generate a data priority list. At the same time, the network operation status is monitored in real time, and the main monitoring indicators include network bandwidth, delay, and packet loss rate. The size of the network bandwidth determines the rate of data transmission, the delay affects the real-time nature of data synchronization, and the packet loss rate reflects the stability of network transmission. The real-time data of the indicators are integrated and calculated to generate a comprehensive score reflecting the current network status to obtain the network status score. The performance indicators of each device are collected, including CPU usage, memory usage, and storage capacity. The device performance indicators reflect the current operating status and available resources of the device, and can help the system reasonably allocate data storage and computing tasks during the data synchronization process. For example, when the CPU and memory usage of a device are high, the synchronization task is assigned to other devices with better performance to avoid excessive load on the device. The user's behavioral characteristics are analyzed, including the user's data access frequency and usage pattern. User behavioral characteristics can help the system understand the user's demand preferences for different data and the user's usage habits in different time periods. The data priority list, network status score, device performance indicators, and user behavioral characteristics are input into the decision tree model. The decision tree model generates a preliminary data synchronization strategy based on different input conditions and rules. Perform conflict detection and adjustment on the preliminary synchronization strategy. For example, if the synchronization order of some data between multiple devices conflicts, or the synchronization operation causes some devices to overload, etc., the conflict is detected, and the strategy is adjusted and optimized according to the specific situation to generate an optimized synchronization strategy. The optimized synchronization strategy is matched with fine-grained data tags. Fine-grained data tags provide detailed attribute and feature information of the data. The details of the synchronization strategy are fine-tuned according to these tags, such as specific types of data should be synchronized in a specific time period, or some sensitive data needs to be encrypted before synchronization, etc. Finally, the data synchronization strategy is obtained.

[0016] S4, based on the data synchronization strategy, executes the two-phase commit protocol and vector clock concurrency control for cross-device operations to obtain the distributed transaction processing result; Specifically, a cross-device operation request is received, the operation content is parsed, and the type of operation is identified, such as data writing, modification or deletion operation, and the unique identifier of the target data. According to the target data identifier, the actual storage location of the data is found in the distributed data storage architecture, and which devices are involved in the current operation request are determined to obtain a list of devices involved. Based on the data synchronization strategy, the list of devices involved is prioritized to obtain the device processing order. The device priority is based on factors such as the current load of the device, the importance of the data or the network conditions, so as to ensure that conflicts and delays can be minimized during the transaction processing. After the sorting is completed, a preparation request is sent to all devices involved. The content of the preparation request includes the operation type, target data and other related information of the current transaction, requiring the device to prepare for the upcoming data change, such as locking data, allocating resources, etc. In the preparation stage, all devices involved respond to the preparation request according to their own status. The content of the response includes whether the device can perform transaction processing in the current state, whether the relevant data or resources have been successfully locked, etc. After the system receives the preparation response from all devices, all response results are summarized and analyzed to determine the result of the preparation stage. If all devices return a ready response, it means that the transaction can continue, and the system enters the commit stage. Otherwise, if any device fails to complete the preparation, the system will enter the rollback phase, notify all devices to terminate the operation, release the allocated resources, and restore the system to the state before the operation to ensure data consistency and integrity. In the commit phase, a commit instruction is sent to all devices, requiring each device to actually write the previously prepared data changes to the database or perform corresponding operations. After receiving the commit instruction, each device performs the final write of the data and feeds back the operation results to the system. The system summarizes the final transaction status based on the commit results of each device. If all devices successfully commit, the transaction operation is successfully completed; if any device fails to commit, the system will record the reason for the failure and take corresponding remedial measures if necessary. At the same time, based on the final transaction status, the vector clock of each device is updated. The vector clock is an important tool for tracking and managing the event sequence and data version between different devices in a distributed system. The execution order and data version of transactions on each device are recorded through the updated vector clock. The updated vector clock is used to compare the data versions on each device to detect potential concurrency conflicts. Concurrency conflict detection is an important step to ensure data consistency, because in a distributed environment, multiple devices may operate on the same data at the same time, resulting in data inconsistency or loss. After the conflict is detected, according to the preset conflict resolution strategy, select the appropriate data merge algorithm or conflict resolution algorithm, such as the last write wins strategy based on timestamp, the merge strategy based on version number, etc., to resolve the data conflict.Finally, a complete distributed transaction processing result is obtained, which includes the execution status of the transaction and also reflects the data consistency and conflict resolution between devices.

[0017] S5, inputting the distributed transaction processing result into the data feature analyzer for structural feature and repetitive pattern analysis to obtain the compression algorithm selection result and compression parameters; The distributed transaction processing results are input into the data feature analyzer for data type identification, and various types contained in the data are identified, such as text data, image data, audio data, etc. The type distribution of the entire data set is understood through data type identification. Based on the data type distribution, the data is processed in blocks, and the data of the same or similar types are aggregated together to form multiple data block sets. A sliding window analysis is performed on the data block set, and repeated byte sequences are slidingly identified in the data block to find the repeated patterns in the data. Through the statistics of the repeated patterns, it is determined which data blocks contain a large amount of repeated information to form the repeated pattern statistics. Based on the statistical results, the repetition of each data block is scored, and the data redundancy index is calculated to reflect the potential of data compression, that is, the higher the redundancy of the data, the higher the compression ratio can be obtained through compression. The entropy value of the data block set is calculated. The entropy value is an important indicator for evaluating the randomness of data. The higher the entropy value, the more random the data is and the less likely it is to be compressed. The lower the entropy value, the more regular information there is in the data and the greater the compression potential. Through the comprehensive analysis of the data redundancy index and the data entropy value, the preliminary compression algorithm selection result is generated. Based on the preliminary algorithm selection results, data sampling compression tests are performed on different algorithms. During the test, the sample data is compressed using different compression algorithms, and the compression rate and compression speed indicators of each algorithm are recorded. The compression rate indicates the efficiency of the compression algorithm in compressing data, that is, the ratio of the compressed data volume to the original data volume; the compression speed indicates the speed at which the algorithm processes data. The test results are input into the multi-objective optimization algorithm, and the final compression algorithm selection results are generated by comprehensively considering factors such as compression rate, compression speed, and resource consumption. According to the compression algorithm selection results, the corresponding parameter range is selected from the preset parameter library to generate a candidate compression parameter set, including dictionary size, window size, block size, etc. Grid search optimization is performed on the candidate parameter set. Grid search is a hyperparameter optimization method that finds the parameter combination that can maximize the compression effect by exhaustively searching in the parameter space. The parameters with the best compression rate and speed performance are selected from the results of the grid search to form the optimal compression parameters.

[0018] S6, based on the compression algorithm selection result and compression parameters, compresses the target data in the distributed data storage architecture, and performs adaptive routing selection and load balancing on the compressed data transmission path to obtain a multi-device data flow management solution.

[0019] Among them, according to the compression algorithm selection result, the corresponding compression algorithm is applied to the target data to obtain the compressed data. Different compression algorithms are applied to different types of data to make full use of the advantages of each algorithm and improve data compression efficiency and compression ratio. According to the selected compression parameters, the compressed data is processed in blocks, and the compressed data is divided into several small data blocks to obtain a set of compressed data blocks, thereby reducing the delay and packet loss risk of large data blocks during network transmission. The data fingerprint is calculated for the set of compressed data blocks to generate a unique identifier for each data block. The data fingerprint is based on a hash algorithm, such as SHA-256 or MD5, to calculate a unique identifier for each data block. Using the unique identifier of the data block, a distributed hash table is constructed, which associates the unique identifier of each data block with its storage location to obtain a data block location index. Based on the data block location index, the network topology is analyzed. By analyzing the connection relationship between each device and the network bandwidth, delay, etc., a connection relationship diagram between devices is drawn. Based on the connection relationship diagram, the minimum spanning tree algorithm is applied to determine an optimized data transmission path. The minimum spanning tree algorithm aims to select a path that minimizes transmission overhead. This path can not only cover all nodes that need to be transmitted, but also avoid unnecessary repeated transmission and path detours, thereby improving the efficiency of data transmission. According to the current load status of each device, the optimized data transmission path is weighted to obtain a weighted transmission path. A dynamic routing table is constructed based on the weighted transmission path. The dynamic routing table can update the transmission path between devices in real time, and automatically adjust the routing strategy according to changes in network conditions to form an adaptive routing strategy. The adaptive routing strategy can dynamically adjust the data transmission path according to the current network status and device resource utilization, avoid network congestion and transmission bottlenecks, and improve the stability and reliability of data transmission. The adaptive routing strategy is combined with the device performance index to execute the load balancing algorithm. The load balancing algorithm reasonably allocates the tasks and data transmission traffic of each device according to the device's CPU usage, memory usage, storage capacity and other indicators. In this way, the system can evenly distribute data transmission tasks among multiple devices to prevent some devices from failing to transmit data or performance degradation due to overload. The compressed data block set, data block location index, adaptive routing strategy and task allocation scheme are integrated to generate a multi-device data flow management scheme.

[0020] In one example, a cross-device data storage layer is constructed using distributed soft bus technology, and target data is stored in shards and backed up redundantly to obtain a distributed data storage architecture, including: A cross-device data storage layer is constructed using distributed soft bus technology, the target data is divided into data blocks to obtain multiple data blocks, and the multiple data blocks are allocated to devices to obtain a data block distribution plan; According to the data block distribution plan, multiple data blocks are respectively transmitted to corresponding devices to obtain initial data distribution results; Perform redundancy analysis on the initial data distribution results to obtain data redundancy requirements, and based on the data redundancy requirements, replicate multiple data blocks to obtain redundant data blocks; Allocate redundant data blocks to other devices different from the devices where the original data blocks are located, and obtain redundant backup results; Based on the initial data distribution results and redundant backup results, a distributed index structure is constructed to obtain a global data index, and the global data index is synchronized to all participating devices to obtain a distributed data storage architecture.

[0021] In this example, a cross-device data storage layer is constructed using distributed soft bus technology. Distributed soft bus technology uses virtualized communication channels to achieve seamless data transmission and sharing between devices. The target data is divided into data blocks, and a whole block of data (such as a large multimedia file or database) is broken down into several small data blocks. Suppose the target data is , is divided into Data blocks ,Right now: , each data block The size of the data block is defined according to actual needs, such as fixed-size blocks (such as 1 MB per data block) or dynamic-size blocks (divided according to the characteristics of the data content). are assigned to different devices. Assuming there is Devices All have different storage capacity, computing power and network connection status. Data block distribution scheme It is represented as an allocation matrix, where Represents a data block Is it assigned to a device? The value 1 indicates allocation and 0 indicates unallocation. For example, for three data blocks and two devices , a distribution scheme for: , which means the data block Assigned to device , data block Assigned to the device and , and the data block is assigned to the device . According to the data block distribution scheme , the data blocks are transmitted to the corresponding devices respectively to form the initial data distribution result. In this process, the transmission channel of the distributed soft bus technology can automatically select the optimal transmission path to ensure that the data blocks can reach the destination device quickly and reliably. The initial data distribution result is represented by the following set: , which indicates that the data block Transferred to the device , data block Transferred to the device and , data block Transferred to the device After the initial data distribution is completed, the redundancy analysis of the current initial data distribution results is performed to evaluate whether the current data distribution can maintain data integrity and availability in the event of certain device failures. Data redundancy is calculated using the following formula: ; in, Represents a data block The redundancy of the data block is the number of copies of the data block on different devices. In order to improve the fault tolerance of the data, if the redundancy of some data blocks ( is a preset minimum redundancy threshold), the data block is replicated according to the redundancy requirement. and The redundancy is insufficient, so copy them to other devices separately, such as Copy to the device , and the data block Copy to the device , get the redundant data block set: , and get the redundant backup result. Data block Store on device and , data block Still on the device and , and the data block Store on device and Based on the initial data distribution results and redundant backup results, a global data index structure is constructed to manage the location and copy information of all data blocks. The index structure is represented by a distributed hash table or other forms of index trees. The key to the index structure is to efficiently map the unique identifier of each data block (such as a hash value) to the corresponding device list, thereby supporting fast search and retrieval. For example, for a data block , whose entry in the global data index is represented as: Index , which means that each data block Corresponding to a group of devices , these devices store data blocks The global data index structure is synchronized to all participating devices to ensure that each device can access the latest global data view, ultimately resulting in a distributed data storage architecture.

[0022] In one example, based on the distributed data storage architecture, the target data is input into the machine learning model for feature extraction and analysis to obtain data classification results and fine-grained data tags, including: Extract metadata information of target data from the distributed data storage architecture to obtain an initial feature set, and perform dimensionality reduction processing on the initial feature set to obtain a feature vector after dimensionality reduction; The reduced feature vector is input into a pre-trained machine learning model to obtain a deep feature representation. The machine learning model includes a three-layer convolutional neural network and a two-layer fully connected network. Based on the deep feature representation, cluster analysis is performed on the target data to obtain preliminary data categories, and according to the preliminary data categories, corresponding label sets are selected from the predefined label library to obtain candidate labels; Calculate the semantic similarity between the candidate tags and the target data content to obtain a similarity score, and filter and sort the candidate tags based on the similarity score to obtain the final fine-grained data labeling; The preliminary data categories and fine-grained data labels are combined to obtain data classification results and fine-grained data labels.

[0023] In this example, metadata information of the target data is extracted from the distributed data storage architecture. Metadata information refers to descriptive information about the data itself, including the size, format, creation time, modification time, data type, access frequency, data owner, etc. of the data. Assume that there is a distributed data storage system that stores a large number of different types of data files, such as documents, images, audio, video, etc. Each data file Each has a corresponding metadata vector , which contains all metadata information describing the data. The metadata vector set of all data is represented as ,in is the number of data files. Each metadata vector Represented as a Dimensional vector: ,in, Represents the first characteristics, such as file size, file type, etc. That is the initial feature set. Perform dimensionality reduction. While retaining the main information of the data, reduce the number of features to simplify calculations and improve the training efficiency of the model. Assume that principal component analysis is used for dimensionality reduction, and the high-dimensional initial feature set Transformed into a set of feature vectors in a low-dimensional feature space , each feature vector after dimensionality reduction It is expressed as: .in, is the dimension size after dimensionality reduction, usually , Representation data No. Through dimensionality reduction processing, we get a set of eigenvectors The reduced feature vector is input into the pre-trained machine learning model to extract the deep feature representation of the data. The machine learning model contains three layers of convolutional neural networks and two layers of fully connected networks. Convolutional neural networks process data with spatial structures, such as images and time series data. Each layer of convolutional neural network performs convolution operations and nonlinear activation on the input feature vector to extract the local pattern characteristics of the data. Assume that the input to the first The feature vector of the layer convolutional neural network is , then The output of the layer convolutional neural network is: .in, Indicates The weight matrix of the convolution kernel of the layer, is the bias term, * represents the convolution operation, is a nonlinear activation function, such as ReLU. After feature extraction through three layers of convolutional neural network, high-order features of feature vectors are extracted layer by layer. Through two layers of fully connected networks, the features output by the convolutional layer are mapped to a higher-dimensional feature space to obtain a deep feature representation of the data. , expressed as: ; in, and are the weight matrix and bias term of the fully connected layer, respectively. is the activation function of the fully connected layer, and the deep feature representation of each data is obtained Based on deep feature representation , cluster the target data. Aggregate data with similar characteristics into the same category to form a preliminary classification result of the data. Assume that the K-means clustering algorithm is used to divide the data into preliminary categories. Each category It is composed of several data with similar characteristics, namely: ,category Contains all feature representations At the cluster center Nearby data, where It is Cluster centers. After obtaining the preliminary data categories, select the corresponding label set from the predefined label library. The label library contains all labels pre-defined by the system. These labels are keywords, category identifiers, etc. that describe the data content. According to the data characteristics in each preliminary category, select the label set that best matches the category from the label library as the candidate label. Calculate the semantic similarity between the candidate label and the content of the target data. The calculation of semantic similarity is achieved through the word vector model in natural language processing (such as Word2Vec, BERT, etc.). Assume that the word vector of each candidate label is , the semantic vector of the data content is , then the similarity score between the data and the label Expressed as the cosine similarity between the two: ; in, and Represents vectors and The symbol · represents the vector dot product. By calculating the similarity score between each data and the candidate label, all candidate labels are screened and sorted, and several labels with the highest similarity scores are retained as the final fine-grained data labels. The preliminary data categories and the final fine-grained data labels are combined to obtain the complete data classification results and fine-grained data labels. The preliminary data categories reflect the macro classification of the data, for example, image data is divided into categories such as landscapes, people, and animals; while fine-grained data labels reveal the specific information of the data, such as specific geographical locations, names of people, types of objects, etc.

[0024] In one example, based on the data classification results and fine-grained data tags, a comprehensive analysis of network conditions, device performance, and user behavior is performed to obtain a data synchronization strategy, including: Prioritize the target data based on the data classification results to obtain a data priority list, and monitor the network bandwidth, delay, and packet loss rate in real time to obtain a network status score; Collect the CPU usage, memory usage, and storage capacity of each device to obtain device performance indicators, and analyze the user's data access frequency and usage pattern to obtain user behavior characteristics; The data priority list, network status score, device performance index and user behavior characteristics are input into the decision tree model to obtain the preliminary synchronization strategy; The preliminary synchronization strategy is subjected to conflict detection and adjustment to obtain an optimized synchronization strategy, and the optimized synchronization strategy is matched with the fine-grained data marker to obtain a data synchronization strategy.

[0025] In this example, the value and timeliness requirements of different types of data in specific application scenarios are considered, and the target data is prioritized according to its importance and timeliness. , each data Corresponding to a classification result and fine-grained data labeling . Define a priority scoring function , whose value is determined by many factors, such as the importance, timeliness and frequency of use of the data. The priority scoring function is expressed as: ; in, Representation data The importance of data classification is usually determined by and fine-grained data labeling Sure; Indicates the timeliness of the data and measures the urgency with which the data needs to be synchronized; Indicates the frequency of data access, reflecting the intensity of users’ demand for the data; It is the weight parameter of each factor, which is used to balance the impact of different factors on priority. By calculating the priority score of all data , get the data priority list , where data is sorted from high to low according to priority scores. High-priority data will be processed first during the synchronization process to ensure that important data can be synchronized to all devices in a timely manner. While obtaining the data priority list, the network status is monitored in real time. The network status consists of indicators such as bandwidth, latency, and packet loss rate, which are obtained in real time through network monitoring tools. The network status is represented as a triple ,in Indicates at time The network bandwidth in Mbps; Indicates network delay in ms; It represents the packet loss rate, which is a value between 0 and 1. Based on these three indicators, a network status scoring function is defined , used to evaluate the current network status: ; in, are weight parameters, which respectively represent the influence of bandwidth, delay and packet loss rate on the network status score; and are the maximum reference values ​​for bandwidth and latency respectively. The value of this scoring function is between 0 and 1, and the higher the value, the better the network condition. Collect the performance indicators of each device, including CPU usage, memory usage, and storage capacity. Assume that the device set is , each device The performance index is represented as a vector: ; in Indicates the device CPU usage in percentage. Indicates memory usage in MB; Represents storage capacity in GB. By regularly collecting performance indicators, we can understand the resource status of each device. At the same time, we can analyze the user's behavior characteristics, including the user's access frequency and usage pattern for different data. Assume that the user set is , each user The behavioral characteristics are expressed as ,in Indicates user Data over a period of time By analyzing the behavior characteristics, we can understand which data users have a high demand for. , Network Status Rating , Equipment performance indicators and user behavior characteristics Input into the pre-trained decision tree model to generate a preliminary data synchronization strategy. The decision tree model maps the input data to different policy choices through a series of decision rules. Each node of the decision tree represents a decision condition, such as "whether the network status score is higher than a certain threshold" or "whether the CPU usage of the target device is lower than a certain value". Through the step-by-step judgment of these conditions, the decision tree outputs a preliminary data synchronization strategy. Conflict detection and adjustment are performed on the preliminary synchronization strategy. Conflict detection simulates the synchronization process and analyzes the load changes of each device. If it is detected that the CPU usage or memory occupancy of a device exceeds the set threshold, the strategy needs to be adjusted to reallocate some synchronization tasks to other devices. After conflict detection and adjustment are completed, the optimized data synchronization strategy is obtained. In order to improve the accuracy of the synchronization strategy, the optimized synchronization strategy is matched with fine-grained data tags. Fine-grained data tags reflect the specific content and attribute information of the data, such as the sensitivity of the data, file type, etc. By combining the synchronization strategy with the data tag, a suitable synchronization strategy is selected for each data block. For example, for sensitive data, encrypted transmission and synchronization to devices with higher security are preferred; for non-sensitive data, delayed synchronization is selected or synchronization is temporarily not performed when the network condition is poor. Based on the data priority list, network condition score, device performance indicators and user behavior characteristics, and after calculation of the decision tree model, conflict detection and fine-grained data tag matching, a comprehensive data synchronization strategy is obtained.

[0026] In one example, based on the data synchronization strategy, a two-phase commit protocol and vector clock concurrency control are performed on cross-device operations to obtain distributed transaction processing results, including: Receive cross-device operation requests, parse the operation content, obtain the operation type and target data identifier, and locate the data location in the distributed data storage architecture according to the target data identifier to obtain a list of involved devices; Based on the data synchronization strategy, the list of devices involved is prioritized to obtain the device processing order, and a preparation request is sent to all devices in the list of devices involved to enter the preparation phase of the two-phase commit protocol; Receive the preparation response of each device, summarize and analyze the response results, obtain the preparation phase results, and decide whether to enter the submission phase based on the preparation phase results. If all devices are ready, enter the submission phase, otherwise enter the rollback phase; In the commit phase, a commit instruction is sent to all devices, and the commit result is received to obtain the final transaction status. Based on the final transaction status, the vector clock of each device is updated to obtain an updated vector clock. The updated vector clock is used to compare the data versions of each device, detect potential concurrency conflicts, obtain conflict detection results, and execute data merging or conflict resolution algorithms based on the conflict detection results to obtain distributed transaction processing results.

[0027] In this example, a cross-device operation request is received and the operation content is parsed. The operation request contains the operation type (such as read, write, delete, etc.), the target data identifier, and the data operation details involved. The operation type determines whether the request is a read-write operation or data deletion, while the self-labeled data identifier uniquely identifies the specific data item that needs to be operated, such as a unique path to a file or a record ID in a database. According to the target data identifier, the actual storage location of the data is found in the distributed data storage architecture, and a list of all devices involved is determined. Assume that the target data identifier is , the storage location of each data block in the distributed data storage architecture is determined by a mapping function Indicates that: .in, Representation data List of devices where Indicates devices involved. The location of the target data is located through the mapping function, and a list of all devices involved is obtained. Based on the pre-set data synchronization strategy, the list of devices involved is prioritized to obtain the processing order of the devices. For example, when the current network bandwidth of a device is relatively high or the CPU load is low, it may be preferentially selected to perform write operations. After determining the device processing order, a prepare request is sent to all devices in the list of involved devices to enter the preparation phase of the two-phase commit protocol. In the preparation phase, each device processes the prepare request, including locking the target data resources, pre-allocating computing and storage resources, etc. If the device is able to complete the prepare request, it will return a ready response, otherwise it will return a failed response. After the system receives the prepare responses from all devices, it summarizes and analyzes the response results to obtain the results of the entire preparation phase. Define a device preparation status variable ,in Indicates the device Ready, Indicates the device Preparation failed. The preparation status of all devices is expressed as follows: ; in, Represents the sum of the readiness status of all devices. If , it means that all devices are ready to enter the submission phase; if , it means that some devices are not ready and need to enter the rollback phase. When the system determines that all devices are ready, it sends a commit command to all devices and enters the commit phase of the two-phase commit protocol. In the commit phase, all devices will perform actual data write, update, or delete operations and return the operation results to the coordinator (the master device that initiated the operation request). At this time, the commit status of each device is represented as ,in Indicates successful submission. Indicates that the submission failed. The submission status of all devices is shown as: ,in, Indicates the total submission status of all devices. , it means that all devices have successfully completed the commit operation and the transaction is successful; if , it means that some devices have failed to commit and the transaction will be rolled back. After the commit phase is completed, the vector clock of each device is updated according to the final transaction status. Vector clock is a commonly used tool in distributed systems to track the event sequence and data version on different devices. Each device There is a vector clock ,in Indicates the device The local timestamp of a device. After completing a data operation, the first Quantity Will automatically increase: The device synchronizes the updated vector clock to other devices. In this way, all devices can understand the order of data operations on other devices and form a globally consistent data version view. Using the updated vector clock, the data versions on all devices are compared to detect potential concurrency conflicts. Concurrency conflict detection is a key step in maintaining data consistency in distributed systems. Suppose two devices and For the same data If modifications are made at different time points, their vector clocks may appear as follows: . In this case, it is determined that a concurrency conflict has occurred and conflict resolution measures need to be taken. There are usually multiple strategies for conflict resolution, such as last write first, priority rules based on timestamps, etc. Select an appropriate conflict resolution algorithm according to the application scenario to ensure data consistency. When a concurrency conflict is detected, perform data merging or conflict resolution operations according to the specific conflict resolution strategy. Assuming that a timestamp-based conflict resolution algorithm is selected, the version with a later timestamp update is selected as the final data version. Compare the modification timestamps of the data on each device, select the latest version as the final version, and distribute it to all relevant devices for updating.

[0028] In one example, the distributed transaction processing result is input into a data feature analyzer for structural feature and repetitive pattern analysis to obtain a compression algorithm selection result and compression parameters, including: Input the distributed transaction processing results into the data feature analyzer to identify the data type, obtain the data type distribution, and perform data block processing based on the data type distribution to obtain a data block set; Perform sliding window analysis on the data block set to identify repeated byte sequences, obtain repeated pattern statistics, and calculate the data repetition score based on the repeated pattern statistics to obtain a data redundancy index; Calculate the entropy value of the data block set, evaluate the randomness of the data, obtain the data entropy value, and generate the preliminary algorithm selection result based on the data redundancy index and the data entropy value; Based on the preliminary algorithm selection results, data sampling compression tests are conducted to obtain the compression rate and speed indicators of each algorithm. The compression algorithm selection results are obtained by using a multi-objective optimization algorithm and comprehensively considering the compression rate, speed and resource consumption. According to the compression algorithm selection result, a corresponding parameter range is selected from a preset parameter library to obtain a candidate compression parameter set, and a grid search optimization is performed on the candidate compression parameter set to obtain the optimal compression parameter.

[0029] In this example, different types of data are extracted from the distributed transaction processing results. Each data block contains multiple data types, such as text, pictures, audio, video, etc. Suppose there is a data set , where each data There are different metadata information, such as file extension, data format, size, etc., which are used to identify data types. The data feature analyzer uses this metadata information, combined with machine learning models or rule matching methods, to identify each data type. Classify into a specific type, such as text type (TXT, PDF), image type (JPG, PNG), audio type (MP3, WAV), etc., and finally get the type distribution of the entire data set ,in Indicates The data types and their proportions are divided into blocks according to the data type distribution. Different types of data have different block strategies. For example, text data is divided into blocks by line or paragraph, while image data is divided into blocks by resolution or image size. Divide Data blocks , then the block set of the entire data set is expressed as: , where each data block Indicates that from the data Separated from Data blocks. Sliding window analysis is performed on the set of data blocks to identify repeated byte sequences in the data. Sliding window analysis is a pattern recognition method that identifies repeated byte sequences by sliding a window byte by byte in a data block. Assume that the size of the sliding window is , for data blocks , the subsequence formed when the sliding window slides over it is: ,in, Indicates that the window is in the data block Middle The byte sequence at position For data blocks The length of When it appears multiple times in a data block, determine the data block There are repeated patterns. By counting the repeated byte sequences in all data blocks, the repeated pattern statistics of the entire data block set are obtained. Based on the repeated pattern statistics, the repetition score of the data is calculated. The repetition score of the data indicates how much data is repeated in the entire data set. Suppose data block The repeatability is , whose value indicates the The number of identical subsequences. Defines the overall repetition index of the data block set for: ; in, Indicates the total number of data block sets, For data blocks Repeatability score. Repeatability index The larger the entropy value, the more duplicate data there is in the data block set, which has a higher compression potential. The entropy value of the data block set is calculated to evaluate the randomness of the data. The entropy value is an important indicator to measure the randomness and information content of the data. The higher the entropy value of the data, the more random the data is and the less likely it is to be compressed; the lower the entropy value, the more predictable patterns there are in the data and the better it can be compressed. The entropy value is calculated using the following formula: ; in, Represents a data block The entropy value of is the set of all possible characters in the data block, For characters In the data block The overall entropy value of the data block set is obtained by averaging the entropy values ​​of all data blocks. , reflecting the randomness level of the entire data set. Based on the data redundancy index and data entropy Generate preliminary compression algorithm selection results. Data blocks with high repetition and low entropy are more suitable for dictionary-based compression algorithms (such as LZ77 or BWT). For data with high entropy, select Huffman coding and other compression algorithms suitable for highly random data. To verify the effectiveness of the preliminary algorithm selection results, perform data sampling compression tests. Randomly extract several sample data from the data block set, test them using different compression algorithms, and record the compression rate and compression speed of each algorithm. Compression rate It indicates the ratio of the amount of compressed data to the amount of original data before compression, and is defined as: ; in, is the size of the compressed data, The size of the data before compression. Compression speed Indicates the amount of data that can be compressed per unit time. After obtaining the compression rate and compression speed indicators of different algorithms through testing, these results are input into the multi-objective optimization algorithm. The multi-objective optimization algorithm aims to comprehensively consider the three factors of compression rate, speed and resource consumption to select the optimal compression algorithm. The lower the compression rate, the better the compression effect, the faster the compression speed, the higher the efficiency of the algorithm, and the resource consumption includes the occupation of resources such as CPU and memory. Assuming that the compression rate and resource consumption are minimized and the compression speed is maximized, the weight coefficient is used to calculate the optimal compression algorithm. Each objective is weighted, and the final optimization objective is: Minimize ; in, Represents a resource consumption indicator. The multi-objective optimization algorithm can balance the trade-offs between different objectives and find the optimal compression algorithm that meets the conditions. According to the optimal compression algorithm, the corresponding parameter range is selected from the preset parameter library to obtain a candidate compression parameter set. In order to find the optimal compression parameters, the candidate compression parameter set is optimized by grid search. Grid search is an exhaustive search method that traverses the parameter space, tests the impact of each parameter combination on the compression effect, and finds the optimal parameter combination that can maximize the compression rate and speed.

[0030] In one example, based on the compression algorithm selection result and compression parameters, target data in a distributed data storage architecture is compressed, and adaptive routing and load balancing are performed on the compressed data transmission path to obtain a multi-device data flow management solution, including: According to the compression algorithm selection result, the corresponding compression algorithm is applied to the target data to obtain compressed data, and based on the compression parameters, the compressed data is divided into blocks to obtain a set of compressed data blocks; Perform data fingerprint calculation on the compressed data block set to obtain a unique identifier for each data block, and use the unique identifier of the data block to build a distributed hash table to obtain the data block location index; Based on the data block location index, the network topology is analyzed to obtain the connection relationship diagram between devices, and the minimum spanning tree algorithm is applied to the connection relationship diagram to obtain the optimized data transmission path; According to the current load status of each device, the optimized data transmission path is weighted to obtain a weighted transmission path, and a dynamic routing table is constructed based on the weighted transmission path to obtain an adaptive routing strategy; Combine the adaptive routing strategy with the device performance indicators, execute the load balancing algorithm, and obtain the task allocation plan; The compressed data block set, data block location index, adaptive routing strategy and task allocation scheme are integrated to obtain a multi-device data flow management scheme.

[0031] In this example, according to the compression algorithm selection result, the corresponding compression algorithm is applied to the target data for compression processing. The compression algorithm is selected based on the type, structure and repetitiveness of the data. Assuming that the target data After being compressed, the compressed data is obtained ,in Compress , Indicates the parameter set of the compression algorithm, such as dictionary size, window size, etc. , divide it into blocks according to the compression parameters to form a set of compressed data blocks The purpose of data chunking is to break down large compressed data into smaller units so that they can be managed and transmitted more efficiently in a multi-device environment. The chunking strategy is determined by the size of the data, the access frequency, and the characteristics of the compression algorithm. For example, text data is chunked according to a fixed size, while image data is chunked according to image resolution or content characteristics. Each data chunk are stored and transmitted independently. For each compressed data block Calculate the data fingerprint to obtain a unique identifier for each data block. Data fingerprint is a calculation result based on a hash function that can map different data blocks to a unique fixed-length value, which is used to verify data integrity and locate the location of data blocks. The data fingerprint is ,but: Hash ; Among them, the Hash function is used to generate a unique identifier for the data block These data fingerprints are used to build a distributed hash table, which maps the unique identifier of each data block to its location in the distributed storage system. A distributed hash table is an efficient data structure that can quickly find the location of data blocks in a large distributed system. Each entry is represented as: , where DHT Represents a data block Hash fingerprint of Corresponding storage location It is a device or a collection of devices that records the storage location of data blocks. After the distributed hash table is built, the network topology is analyzed based on the location index DHT of the data block to understand the connection relationship between devices. Network topology analysis helps the system understand network parameters such as bandwidth, latency, and stability between devices, so as to better plan data transmission paths. The connection relationship between devices is represented as an undirected graph ,in, Represents a collection of devices. Represents the network connection edge between devices. The edge weight is an indicator such as bandwidth and delay. Apply the minimum spanning tree algorithm to find the optimal data transmission path covering all involved devices. The minimum spanning tree algorithm (such as Prim or Kruskal algorithm) can minimize the total transmission cost while ensuring that all devices are connected. Let the minimum spanning tree be ,in , represents the optimal data transmission path. According to the current load status of each device, the optimized data transmission path The weights are assigned to obtain the weighted transmission path. The load status of each device is determined by its CPU usage, memory usage, network bandwidth and other indicators. Assume that the device The load state is , define a device load weight function: ; The higher the load, the lower the weight of the device in the data transmission path. According to the load weight of each device, each edge in the minimum spanning tree is weighted to form a weighted transmission path. , used to reflect the load balancing status of each device. Based on weighted transmission path , build a dynamic routing table. The dynamic routing table can update the data transmission strategy in real time according to the current network status and device load, ensuring that data can be transmitted efficiently and reliably in a complex network environment. Dynamic routing table Represented as a mapping relationship: ; in, Represents a data block Data fingerprint Corresponding transmission path , which is based on a weighted transmission path Dynamic adjustment. After building the dynamic routing table, combine the adaptive routing strategy with the device performance indicators, execute the load balancing algorithm, and obtain the optimal task allocation plan. Load balancing algorithms such as consistent hashing algorithm and minimum connection algorithm can reasonably allocate transmission tasks to different devices. Based on the performance indicators of the device (such as CPU utilization, memory usage) and the transmission path in the dynamic routing table, the transmission tasks of the data blocks are evenly distributed to each device to ensure that the overall load of the system is evenly distributed, improve data transmission efficiency and device utilization. Integrate compressed data block sets, data block location indexes, adaptive routing strategies and task allocation plans to form a complete multi-device data flow management solution, which can efficiently manage and transmit data in a multi-device environment to maximize system performance and resource utilization.

[0032] Reference Figure 2 , this embodiment provides a data management system based on OpenHarmony, including: Construction module 1 is used to construct a cross-device data storage layer using distributed soft bus technology, perform shard storage and redundant backup of target data, and obtain a distributed data storage architecture; Extraction module 2, used to input the target data into the machine learning model for feature extraction and analysis based on the distributed data storage architecture, and obtain data classification results and fine-grained data tags; Analysis module 3, used to conduct a comprehensive analysis of network conditions, device performance and user behavior based on data classification results and fine-grained data tags to obtain a data synchronization strategy; Execution module 4, used to execute two-phase commit protocol and vector clock concurrency control on cross-device operations based on data synchronization strategy to obtain distributed transaction processing results; Processing module 5, used for inputting the distributed transaction processing result into the data feature analyzer for structural feature and repetitive pattern analysis, and obtaining the compression algorithm selection result and compression parameters; The compression module 6 is used to compress the target data in the distributed data storage architecture based on the compression algorithm selection result and compression parameters, and to perform adaptive routing selection and load balancing on the compressed data transmission path to obtain a multi-device data flow management solution.

[0033] In this embodiment, for the specific implementation of each unit in the above system embodiment, please refer to the above method embodiment, which will not be repeated here.

[0034] Reference Figure 3 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a display screen, an input system, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0035] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0036] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0037] In summary, a data management method based on OpenHarmony is proposed in the embodiment of the present invention. The distributed soft bus technology is used to build a cross-device data storage layer, which realizes the shard storage and redundant backup of data, and improves the reliability and access efficiency of data. The machine learning model is used to extract and analyze the features of the data, realize the automatic classification and fine-grained marking of the data, and dynamically generate the data synchronization strategy by comprehensively analyzing the network status, device performance and user behavior, thereby improving the adaptability and efficiency of data synchronization. The two-phase commit protocol and the vector clock concurrency control mechanism are introduced to ensure the consistency of distributed transactions and the accuracy of concurrency control. The data feature analyzer is used to analyze the structural features and repetitive patterns, realize the intelligent selection and parameter optimization of the compression algorithm, and improve the data compression efficiency. Through the adaptive routing selection and load balancing algorithm, the transmission path of the compressed data is optimized, and the management efficiency of the multi-device data stream is improved. The overall solution fully considers the distributed characteristics of OpenHarmony, realizes the optimization of the whole process from data storage, classification, synchronization, transaction processing to compressed transmission, and significantly improves the data management performance in a distributed environment.

[0038] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0039] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, system, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, system, article or method. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, system, article or method including the element.

[0040] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A data management method based on OpenHarmony, characterized in that: The following steps are involved: Use distributed soft bus technology to build a cross-device data storage layer, perform shard storage and redundant backup of target data, and obtain a distributed data storage architecture; Based on the distributed data storage architecture, the target data is input into a machine learning model for feature extraction and analysis to obtain data classification results and fine-grained data labels; Based on the data classification results and the fine-grained data tags, a comprehensive analysis is performed on the network status, device performance and user behavior to obtain a data synchronization strategy; Based on the data synchronization strategy, a two-phase commit protocol and vector clock concurrency control are performed on cross-device operations to obtain a distributed transaction processing result; Inputting the distributed transaction processing result into a data feature analyzer for structural feature and repetitive pattern analysis to obtain a compression algorithm selection result and compression parameters; Based on the compression algorithm selection result and the compression parameters, the target data in the distributed data storage architecture is compressed, and the compressed data transmission path is adaptively routed and load balanced to obtain a multi-device data flow management solution.

2. The OpenHarmony-based data management method according to claim 1, characterized in that: The distributed soft bus technology is used to construct a cross-device data storage layer, and the target data is stored in slices and backed up redundantly to obtain a distributed data storage architecture, including: Using distributed soft bus technology to build a cross-device data storage layer, dividing the target data into data blocks to obtain multiple data blocks, and allocating the multiple data blocks to devices to obtain a data block distribution plan; According to the data block distribution scheme, the plurality of data blocks are respectively transmitted to corresponding devices to obtain an initial data distribution result; Performing redundancy analysis on the initial data distribution result to obtain data redundancy requirements, and based on the data redundancy requirements, duplicating the multiple data blocks to obtain redundant data blocks; Allocating the redundant data blocks to other devices different from the devices where the original data blocks are located, to obtain a redundant backup result; Based on the initial data distribution result and the redundant backup result, a distributed index structure is constructed to obtain a global data index, and the global data index is synchronized to all participating devices to obtain the distributed data storage architecture.

3. The OpenHarmony-based data management method according to claim 2, characterized in that: Based on the distributed data storage architecture, the target data is input into the machine learning model for feature extraction and analysis to obtain data classification results and fine-grained data tags, including: Extracting metadata information of the target data from the distributed data storage architecture to obtain an initial feature set, and performing dimensionality reduction processing on the initial feature set to obtain a feature vector after dimensionality reduction; Inputting the dimensionally reduced feature vector into a pre-trained machine learning model to obtain a deep feature representation, wherein the machine learning model includes a three-layer convolutional neural network and a two-layer fully connected network; Based on the deep feature representation, cluster analysis is performed on the target data to obtain preliminary data categories, and according to the preliminary data categories, a corresponding label set is selected from a predefined label library to obtain candidate labels; Calculating semantic similarity between the candidate tags and the content of the target data to obtain a similarity score, and screening and sorting the candidate tags based on the similarity score to obtain a final fine-grained data tag; The preliminary data categories and the fine-grained data labels are combined to obtain the data classification results and the fine-grained data labels.

4. The OpenHarmony-based data management method according to claim 3, characterized in that: The method of comprehensively analyzing the network status, device performance and user behavior according to the data classification result and the fine-grained data tag to obtain a data synchronization strategy includes: Prioritize the target data based on the data classification result to obtain a data priority list, and monitor the network bandwidth, delay and packet loss rate in real time to obtain a network status score; Collect the CPU usage, memory usage, and storage capacity of each device to obtain device performance indicators, and analyze the user's data access frequency and usage pattern to obtain user behavior characteristics; Inputting the data priority list, the network status score, the device performance index and the user behavior characteristics into a decision tree model to obtain a preliminary synchronization strategy; The preliminary synchronization strategy is subjected to conflict detection and adjustment to obtain an optimized synchronization strategy, and the optimized synchronization strategy is matched with a fine-grained data tag to obtain a data synchronization strategy.

5. The OpenHarmony-based data management method according to claim 1, characterized in that: The method of executing a two-phase commit protocol and vector clock concurrency control on cross-device operations based on the data synchronization strategy to obtain a distributed transaction processing result includes: Receiving a cross-device operation request, parsing the operation content, obtaining an operation type and a target data identifier, and locating a data location in the distributed data storage architecture according to the target data identifier to obtain a list of devices involved; Based on the data synchronization strategy, the list of devices involved is prioritized to obtain a device processing order, and a preparation request is sent to all devices in the list of devices involved to enter the preparation phase of the two-phase commit protocol; Receive the preparation response of each device, summarize and analyze the response results, obtain the preparation phase result, and decide whether to enter the submission phase according to the preparation phase result. If all devices are ready, enter the submission phase, otherwise enter the rollback phase; In the commit phase, a commit instruction is sent to all devices, and a commit result is received to obtain a final transaction status, and based on the final transaction status, a vector clock of each device is updated to obtain an updated vector clock; The updated vector clock is used to compare the data versions of each device, detect potential concurrency conflicts, obtain conflict detection results, and execute data merging or conflict resolution algorithms based on the conflict detection results to obtain the distributed transaction processing results.

6. The OpenHarmony-based data management method according to claim 1, characterized in that: The step of inputting the distributed transaction processing result into a data feature analyzer for structural feature and repetitive pattern analysis to obtain a compression algorithm selection result and compression parameters includes: Inputting the distributed transaction processing result into a data feature analyzer to identify the data type to obtain data type distribution, and performing data block processing based on the data type distribution to obtain a data block set; Performing sliding window analysis on the data block set to identify repeated byte sequences, obtain repeated pattern statistics, and calculate the data repetition score based on the repeated pattern statistics to obtain a data redundancy index; Calculating the entropy value of the data block set, evaluating the randomness of the data, obtaining the data entropy value, and generating a preliminary algorithm selection result according to the data redundancy index and the data entropy value; Based on the preliminary algorithm selection results, data sampling compression test is performed to obtain compression rate and speed indicators of each algorithm, and a multi-objective optimization algorithm is used to comprehensively consider compression rate, speed and resource consumption to obtain a compression algorithm selection result; According to the compression algorithm selection result, a corresponding parameter range is selected from a preset parameter library to obtain a candidate compression parameter set, and a grid search optimization is performed on the candidate compression parameter set to obtain an optimal compression parameter.

7. The OpenHarmony-based data management method according to claim 1, characterized in that: The method of compressing the target data in the distributed data storage architecture based on the compression algorithm selection result and the compression parameters, and adaptively selecting routing and load balancing the compressed data transmission path to obtain a multi-device data flow management solution includes: According to the compression algorithm selection result, the corresponding compression algorithm is applied to the target data to obtain compressed data, and based on the compression parameter, the compressed data is processed into blocks to obtain a set of compressed data blocks; Performing data fingerprint calculation on the compressed data block set to obtain a unique identifier for each data block, and constructing a distributed hash table using the unique identifier of the data block to obtain a data block position index; Based on the data block location index, the network topology is analyzed to obtain a connection relationship diagram between devices, and a minimum spanning tree algorithm is applied to the connection relationship diagram to obtain an optimized data transmission path; According to the current load status of each device, weights are assigned to the optimized data transmission paths to obtain weighted transmission paths, and based on the weighted transmission paths, a dynamic routing table is constructed to obtain an adaptive routing strategy; Combining the adaptive routing strategy with the device performance index, executing the load balancing algorithm, and obtaining a task allocation solution; The compressed data block set, the data block location index, the adaptive routing strategy and the task allocation scheme are integrated to obtain the multi-device data flow management scheme.

8. A data management system based on OpenHarmony, characterized in that: For implementing the steps of the method according to any one of claims 1 to 7, the system comprises: A construction module is used to construct a cross-device data storage layer using distributed soft bus technology, perform shard storage and redundant backup of target data, and obtain a distributed data storage architecture; An extraction module, used to input the target data into a machine learning model for feature extraction and analysis based on the distributed data storage architecture, to obtain data classification results and fine-grained data tags; An analysis module, configured to comprehensively analyze network conditions, device performance, and user behavior based on the data classification results and the fine-grained data tags to obtain a data synchronization strategy; An execution module, configured to execute a two-phase commit protocol and vector clock concurrency control on cross-device operations based on the data synchronization strategy to obtain a distributed transaction processing result; A processing module, used for inputting the distributed transaction processing result into a data feature analyzer for structural feature and repetitive pattern analysis to obtain a compression algorithm selection result and compression parameters; A compression module is used to compress the target data in the distributed data storage architecture based on the compression algorithm selection result and the compression parameters, and to perform adaptive routing selection and load balancing on the compressed data transmission path to obtain a multi-device data flow management solution.

9. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images