Multi-threaded concurrent processing method and system based on adaptive entropy segmentation and interactive differential learning

Abstract

The present invention relates to the field of computer science and technology, and in particular to a multi-threaded concurrent processing method and system based on adaptive entropy segmentation and interactive differential learning, in which the pre-processed data is dynamically divided by an adaptive entropy segmentation algorithm, and the data processing path is optimized by an interactive differential learning algorithm. Finally, the system realizes the backtracking and correction of abnormal data through a multi-level error correction backtracking mechanism, ensuring the consistency and accuracy of the final data. The present invention effectively solves the problems of uneven load, insufficient optimization of data processing paths, and poor abnormal data processing in traditional multi-threaded concurrent processing, and greatly improves the processing efficiency and data reliability of the system.

Description

Technical Field

[0001] The present invention relates to the field of computer science and technology, and in particular to a multi-threaded concurrent processing method and system based on adaptive entropy segmentation and interactive differential learning. Background Art

[0002] With the rapid development of computer technology, multi-threaded concurrent design plays a vital role in improving system performance and response speed. In the Beidou Internet of Vehicles industry, with the increasing number of vehicles equipped with positioning terminals, a large number of terminals are connected to the system, generating massive amounts of data. In the process of parsing, analyzing and storing these data, traditional multi-threaded concurrent processing methods often face performance bottlenecks. Although simple multi-threaded design can alleviate performance pressure to a certain extent, it can easily lead to confusion in the order of terminal data, which in turn causes errors in matching the front and back positioning data points, and ultimately leads to inaccurate data analysis results.

[0003] In addition, the multi-threaded processing methods in the prior art (Chinese invention patent, publication number: CN117492981A, name: A solution acceleration method and device based on GPU multi-threaded concurrent technology), such as GPU multi-threaded concurrent processing, usually rely on simple division and allocation of tasks. However, this method often has defects such as uneven computing load, data consistency problems, and insufficient fault recovery capabilities when dealing with large-scale data processing. These problems mainly stem from the deficiencies of the prior art in task allocation, data processing path optimization, and abnormal data processing, which makes the system prone to performance bottlenecks and data processing errors under high load conditions. Summary of the invention

[0004] In view of the many problems existing in the above-mentioned prior art, the present invention provides a multi-thread concurrent processing method and system based on adaptive entropy segmentation and interactive differential learning. The present invention dynamically divides the pre-processed data into task blocks through an adaptive entropy segmentation algorithm, realizes data exchange and path optimization among multiple threads by using an interactive differential learning algorithm, and ensures the consistency and accuracy of data through a multi-level error correction and backtracking mechanism. The present invention not only improves the processing efficiency of the system, but also significantly enhances the reliability of data processing.

[0005] A multi-threaded concurrent processing method based on adaptive entropy segmentation and interactive differential learning comprises the following steps:

[0006] Receive positioning data from Beidou Internet of Vehicles terminals, the positioning data including geographic location data, timestamp data and sensor data, generate dynamic label data for the positioning data, and pre-process the positioning data, including removing noise and redundant information, to generate pre-processed data;

[0007] The preprocessed data is divided into multiple entropy optimization task blocks by using an adaptive entropy segmentation algorithm, a shadow task block is generated for each entropy optimization task block, and an entropy balancing load scheduling algorithm is used to allocate the entropy optimization task blocks and shadow task blocks to different computing nodes, and at the same time, the entropy optimization task blocks and shadow task blocks are prepared as task blocks to be executed and shadow task blocks to be executed;

[0008] Decomposing the task block to be executed into multiple subtask units, performing data exchange and learning among multiple threads through an interactive differential learning algorithm, optimizing the processing path of the subtask units, generating first concurrent processing data, and performing streaming data analysis on the first concurrent processing data to generate second concurrent processing data;

[0009] Performing a global consistency check on the second concurrently processed data, marking abnormal data, and backtracking and correcting the abnormal data through a multi-level error correction backtracking mechanism to generate correction data and integrate it with the second concurrently processed data into final consistency data;

[0010] The final consistency data is integrated and deduplicated to generate the final integrated data, and the final integrated data is fed back to the data storage module to generate processing log data and complete the data storage and application preparation.

[0011] Preferably, the adaptive entropy segmentation algorithm comprises the following steps:

[0012] Calculate the entropy value H(x) of the preprocessed data,

[0013] H(x)=-∑p(x i )logp(x i )

[0014] Wherein, the entropy value H(x) is the entropy value of the preprocessed data; p(x i ) is each data point x in the preprocessed data i The probability of

[0015] The preprocessed data is divided into a plurality of entropy optimization task blocks according to the entropy value, so that the entropy value of each entropy optimization task block is within a preset threshold range.

[0016] Preferably, the shadow task block is synchronized with the entropy optimization task block in real time on the distributed computing node, and the shadow task block is used to take over the processing of the entropy optimization task block when a computing node fails.

[0017] Preferably, the entropy balancing load scheduling algorithm comprises the following steps:

[0018] Calculate the current entropy fluctuation H of each computing node in real time node (t), the calculation expression is:

[0019] Hnode (t)=|H prev (t)-H curr (t)|

[0020] Wherein, the entropy fluctuation H node (t) is the entropy fluctuation of the computing node at time t; H prev (t) is the entropy value of the previous moment; H curr (t) is the entropy value at the current moment;

[0021] The entropy optimization task blocks and the shadow task blocks are preferentially assigned to computing nodes with smaller entropy value fluctuations to ensure load balance.

[0022] Preferably, the interactive differential learning algorithm comprises the following steps:

[0023] Each thread generates differential data D when executing its subtask unit diff , indicating the difference between the calculation result of the current thread and the calculation result of other threads;

[0024] The thread learns the differential data D diff , dynamically adjust its calculation path, optimize processing efficiency, and generate the first concurrent processing data.

[0025] Preferably, the differential data D diff The calculation expression is:

[0026]

[0027] In the formula, the differential data D diff is differential data; and are the calculation results of thread i and thread j at time t respectively; n is the number of threads involved in the calculation.

[0028] Preferably, the streaming data analysis analyzes the first concurrently processed data by using a time series linkage prediction algorithm, specifically comprising the following steps:

[0029] Generate time series feature data, and extract the time series features of the first concurrently processed data;

[0030] Based on the time series feature data, a linkage prediction model is used to predict the trend of the second concurrently processed data, and the streaming analysis path is dynamically adjusted.

[0031] Preferably, the multi-level error correction backtracking mechanism comprises the following steps:

[0032] Go back to the subtask unit stage and make local corrections to abnormal data;

[0033] Tracing back to the shadow task block stage, resynchronizing the shadow task block with the entropy optimization task block;

[0034] Correction data is generated, and the correction data is integrated with the second concurrently processed data to generate final consistency data.

[0035] Preferably, the result integration and feedback comprises the following steps:

[0036] Identifying and deleting duplicate data on the final consistency data;

[0037] Merge the deduplicated data to generate the final integrated data;

[0038] The final integrated data is stored in a data storage module, and processing log data is generated.

[0039] A system for implementing the multi-threaded concurrent processing method based on adaptive entropy segmentation and interactive differential learning, the system comprising:

[0040] A data receiving module is used to receive positioning data from a Beidou vehicle networking terminal, wherein the positioning data includes geographic location data, timestamp data and sensor data;

[0041] A dynamic label generation and data preprocessing module, used to generate dynamic label data for the positioning data, and preprocess the positioning data to remove noise and redundant information to generate preprocessed data;

[0042] A task segmentation and management module, for dividing the preprocessed data into a plurality of entropy optimization task blocks by using an adaptive entropy segmentation algorithm, generating a shadow task block for each entropy optimization task block, and allocating the entropy optimization task blocks and the shadow task blocks to different computing nodes by using an entropy balancing load scheduling algorithm, and preparing the entropy optimization task blocks and the shadow task blocks as task blocks to be executed and shadow task blocks to be executed;

[0043] A concurrent processing and data analysis module, used for decomposing the task block to be executed into multiple subtask units, performing data exchange and learning among multiple threads through an interactive differential learning algorithm, optimizing the processing path of the subtask unit, generating first concurrent processing data, and performing streaming data analysis on the first concurrent processing data to generate second concurrent processing data;

[0044] A data verification and backtracking module, used to perform a global consistency check on the second concurrently processed data, mark abnormal data, and backtrack and correct the abnormal data through a multi-level error correction backtracking mechanism to generate correction data, and integrate the correction data with the second concurrently processed data into final consistency data;

[0045] The data integration and feedback module is used to integrate and deduplicate the final consistency data to generate final integrated data, and feed the final integrated data back to the data storage module to generate processing log data and complete data storage and application preparation.

[0046] Compared with the prior art, the advantages and beneficial effects of the present invention are:

[0047] The present invention realizes dynamic task division based on data entropy value through adaptive entropy segmentation algorithm, optimizes the load balance of task blocks, reduces load fluctuations between computing nodes, and thus solves the problem of uneven load in traditional multi-threaded processing;

[0048] The present invention realizes dynamic adjustment and learning of difference data between threads through an interactive differential learning algorithm, optimizes the processing path, improves the accuracy and consistency of data processing, and overcomes the inefficiency problem caused by the fixed data processing path in the traditional method;

[0049] The present invention realizes layer-by-layer backtracking and correction from subtasks to shadow task blocks through a multi-level error correction backtracking mechanism, thereby enhancing the fault tolerance of the system and solving the limitations of traditional technologies in abnormal data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of the process of the present invention;

[0051] Figure 2 It is a flow chart of the adaptive entropy segmentation algorithm in the present invention;

[0052] Figure 3 It is a flow chart of the interactive differential learning algorithm in the present invention;

[0053] Figure 4 It is a flow chart of the multi-level error correction backtracking mechanism in the present invention;

[0054] Figure 5 It is a structural block diagram of the system of the present invention. DETAILED DESCRIPTION

[0055] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is obvious that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0056] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0057] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0058] like Figure 1 As shown, a multi-threaded concurrent processing method based on adaptive entropy segmentation and interactive differential learning includes the following steps:

[0059] Receive positioning data from the Beidou Internet of Vehicles terminal, the positioning data includes geographic location data, timestamp data and sensor data, generate dynamic label data for the positioning data, and pre-process the positioning data, including removing noise and redundant information, to generate pre-processed data;

[0060] In the present invention, the process of receiving positioning data from the Beidou Internet of Vehicles terminal is the starting point of the entire multi-threaded concurrent processing method and a key link in the data processing chain. Positioning data generally includes the vehicle's geographic location data, timestamp data, and sensor data. These data are generated in real time in the Beidou Internet of Vehicles and transmitted to the system for further processing. In order to effectively respond to the efficient processing requirements of a large amount of real-time data, the present invention introduces a multi-threaded concurrent processing method of adaptive entropy segmentation and interactive differential learning to ensure that the system can maintain efficient and accurate processing capabilities in a complex multi-threaded environment.

[0061] After receiving the positioning data, the system performs preliminary cleaning and standardization on the data through the preprocessing module. Specifically, preprocessing includes removing noise and redundant information to ensure the quality and consistency of the data. Noise removal refers to the system applying a series of filters and algorithms to eliminate data anomalies caused by sensor errors or signal interference. Redundant information removal is to reduce the redundancy of data storage and processing by detecting and deleting duplicate or unnecessary data entries, thereby improving the efficiency of subsequent data processing.

[0062] After completing the preprocessing of the data, the system generates dynamic label data, which is one of the core innovations of this method. The generation of dynamic label data is based on the geographic location, timestamp and sensor information of the positioning data, and the label is dynamically calculated in combination with the context of the current data stream. The dynamic label not only contains static location information and time information, but also reflects the real-time status of the data in the system, such as the dynamic characteristics of the vehicle's moving speed, acceleration, direction, etc. Through these dynamic labels, the system can manage and schedule subsequent data processing tasks more flexibly. The generation of dynamic labels is based on an adaptive entropy segmentation algorithm. The system classifies and labels positioning data with different characteristics by calculating the entropy value of the positioning data, so that each data block can be optimally allocated according to its characteristics in subsequent processing.

[0063] An adaptive entropy segmentation algorithm is used to divide the preprocessed data into multiple entropy optimization task blocks, a shadow task block is generated for each entropy optimization task block, and an entropy balancing load scheduling algorithm is used to allocate the entropy optimization task blocks and shadow task blocks to different computing nodes, and at the same time, the entropy optimization task blocks and shadow task blocks are prepared as task blocks to be executed and shadow task blocks to be executed;

[0064] In the multi-threaded concurrent processing framework of the present invention, after preliminary cleaning and standardization, the pre-processed data needs to be further divided into multiple task blocks so as to be efficiently processed in a distributed computing environment. To achieve this goal, the present invention adopts an adaptive entropy segmentation algorithm to divide the pre-processed data into multiple entropy optimized task blocks. The adaptive entropy segmentation algorithm determines the complexity and information content of the data by calculating the entropy value of the pre-processed data, and dynamically adjusts the size and distribution of the task blocks accordingly. Entropy, as an indicator of system uncertainty, is used here to ensure that each task block contains a balanced computational complexity, so that in a multi-threaded environment, the task load assigned to each computing node is more uniform.

[0065] Specifically, the adaptive entropy segmentation algorithm first calculates the entropy value of each preprocessed data block, which evaluates the complexity of the data. For example, in vehicle data processing, the driving paths and sensor readings of different vehicles may vary greatly. By calculating the entropy value, the system can identify which data blocks contain more dynamic changes or abnormal situations. According to the size of the entropy value, the system divides the data into multiple entropy optimized task blocks, so that these task blocks are balanced in terms of data volume and computational complexity. This operation not only helps to balance the load of computing nodes, but also improves the overall efficiency of data processing.

[0066] After generating the entropy optimization task block, the system further generates a corresponding shadow task block for each entropy optimization task block. The shadow task block is an exact copy of the original task block, which is specially used for fault tolerance and fault recovery. By generating the shadow task block, the system can immediately switch to the corresponding shadow task block for processing when a certain computing node fails during the task execution process, thereby ensuring the continuity of the task and the reliability of data processing. This shadow task block mechanism is a key measure of the present invention in terms of high availability and data consistency assurance.

[0067] The system distributes entropy-optimized task blocks and shadow task blocks to different computing nodes through the entropy-balanced load scheduling algorithm. The core of the entropy-balanced load scheduling algorithm is to monitor the current entropy fluctuations of each computing node in real time and dynamically adjust the distribution of task blocks according to these fluctuations. The algorithm ensures that the task load processed by each computing node matches its current computing capacity, thereby avoiding excessive load imbalance between computing nodes. For example, when the load of a computing node is high, the algorithm will give priority to assigning new tasks to nodes with lighter loads. Through this dynamic scheduling mechanism, the system can achieve load balancing in a multi-threaded concurrent environment, thereby maximizing the overall computing efficiency.

[0068] After being assigned to the computing nodes, the entropy optimized task blocks and shadow task blocks are prepared as task blocks to be executed and shadow task blocks to be executed, respectively. This preparation process includes operations such as allocating computing resources and initializing task parameters to ensure that each task block can run in the best state during execution. This stage is the prelude to the entire multi-threaded concurrent processing and lays a solid foundation for subsequent parallel computing and data processing. Through the steps of adaptive entropy segmentation, shadow task block generation, entropy balanced load scheduling and task block preparation, the present invention effectively improves the efficiency and reliability of data processing, ensuring that the system can run stably in a high-load, multi-task concurrent environment.

[0069] like Figure 2 As shown, preferably, the adaptive entropy segmentation algorithm comprises the following steps:

[0070] Calculate the entropy value H(x) of the preprocessed data,

[0071] H(x)=-Σp(x i )logp(x i )

[0072] Wherein, the entropy value H(x) is the entropy value of the preprocessed data; p(x i ) is each data point x in the preprocessed data i The probability of

[0073] The preprocessed data is divided into a plurality of entropy optimization task blocks according to the entropy value, so that the entropy value of each entropy optimization task block is within a preset threshold range.

[0074] The entropy calculation of the preprocessed data is the basis of the entire segmentation process. The entropy value H(x) is an indicator to measure the uncertainty of data. In information theory, the entropy value is usually used to quantify the amount of information. In the present invention, the system calculates the probability p(x) of each data point in the preprocessed data. i ), and then substitute these probability values ​​into the above entropy calculation formula. The entropy value of the data is obtained. Specifically, H(x) represents the entropy value of the preprocessed data, which indicates the complexity and amount of information in the data set, and p(x i ) represents each data point x in the preprocessed data i Probability of occurrence.

[0075] By calculating the entropy value of preprocessed data, the system can quantify the amount of information and complexity of each data block. For example, in a preprocessed data set composed of sensor data from different vehicles, the data of different vehicles may contain different degrees of variation or uncertainty. Through entropy calculation, the system can identify which data blocks contain higher information and complexity, thus providing a basis for subsequent task allocation.

[0076] Next, the system divides the preprocessed data into multiple entropy optimization task blocks based on the calculated entropy value. The task block division is based on a preset entropy value threshold range to ensure that the entropy value of each entropy optimization task block is within this range. In this way, the system can assign data with similar information volume and complexity to the same task block, thereby achieving balanced distribution of tasks.

[0077] This entropy-based task block division is very effective in multi-threaded concurrent processing. First, it ensures that each task block contains an appropriate amount of information, so that different task blocks can fully utilize the system's computing resources when executed in parallel, avoiding the computing bottleneck caused by excessive information in some task blocks. In addition, the entropy-optimized task block division enables the system to handle various data loads more flexibly, especially when dealing with complex and changing data sets, which can significantly improve the system's processing efficiency.

[0078] For example, in a multi-threaded environment, if a task block contains too much high-entropy data (i.e., large amount of information and high complexity), it may cause the task to occupy more computing resources when processing, thereby slowing down the processing speed of the entire system. However, through the adaptive entropy segmentation algorithm, the system can decompose these high-entropy data into multiple task blocks before task allocation, and evenly distribute them to different computing nodes for execution, thereby avoiding the concentrated occupation of resources and improving the overall concurrent processing capability of the system.

[0079] Preferably, the shadow task block is synchronized with the entropy optimization task block in real time on the distributed computing node, and the shadow task block is used to take over the processing of the entropy optimization task block when a computing node fails.

[0080] The shadow task block and the entropy optimization task block are synchronized in real time on the distributed computing nodes, which means that during the entire process of task execution, the shadow task block always maintains a consistent state with the entropy optimization task block, thus providing the system with redundant support and the basis for fault switching.

[0081] In principle, the entropy optimization task block is the main computing task block divided by the system according to the adaptive entropy segmentation algorithm, and each task block is assigned to a distributed computing node for execution. However, in actual operation, a major challenge faced by distributed systems is the failure or performance fluctuation of computing nodes. If a node fails to execute a task due to hardware failure, network delay or other reasons, the overall performance and reliability of the system will be seriously affected. In order to solve this problem, the present invention designs a shadow task block mechanism.

[0082] The shadow task block is an exact copy of the entropy optimization task block. When the task is assigned, the shadow task block is assigned to different computing nodes together with the entropy optimization task block. Real-time synchronization is the core of this mechanism, that is, the shadow task block is always consistent with the entropy optimization task block during execution. This synchronization is achieved through continuous status updates and data exchange, ensuring that the shadow task block can reflect the current progress and status of the entropy optimization task block in real time.

[0083] When the system detects that a computing node fails, the shadow task block will immediately take over the processing of the entropy optimization task block and continue to complete the unfinished computing tasks. Since the shadow task block is always synchronized with the entropy optimization task block, the switching process does not require reinitialization or reloading of data, which greatly shortens the fault recovery time and ensures task continuity and system stability.

[0084] From the effect point of view, the introduction of shadow task blocks significantly improves the fault tolerance and reliability of the system. In a multi-threaded concurrent processing environment, the interruption or failure of a task will cause significant delays and waste of resources in the entire computing process, and the shadow task block mechanism effectively prevents this problem through redundant design and real-time synchronization. For example, when processing large real-time data streams, if a computing node suddenly fails, the traditional system may take seconds or even minutes to reallocate tasks and resume computing, which not only affects the response speed of the system, but may also cause data loss or processing delays. In the present invention, the shadow task block can complete fault switching within milliseconds to ensure the continuity of data processing and the accuracy of the results.

[0085] For example, suppose a computing task involves processing the real-time location information of a group of vehicles, which is divided into multiple entropy-optimized task blocks and assigned to different computing nodes for processing. If a node is unable to continue the task due to network interruption, the shadow task block will immediately take over the task and continue to process the vehicle location information, ensuring that all data can be processed in a timely manner without any data omissions or calculation errors due to node failure.

[0086] Preferably, the entropy balancing load scheduling algorithm comprises the following steps:

[0087] Calculate the current entropy fluctuation H of each computing node in real time node (t), the calculation expression is:

[0088] H node (t)=|H prev (t)-H curr (t)|

[0089] Wherein, the entropy fluctuation H node (t) is the entropy fluctuation of the computing node at time t; H prev (t) is the entropy value of the previous moment; H curr (t) is the entropy value at the current moment;

[0090] The entropy optimization task blocks and the shadow task blocks are preferentially assigned to computing nodes with smaller entropy value fluctuations to ensure load balance.

[0091] The entropy balance load scheduling algorithm is a key component to achieve efficient utilization of computing resources and system performance optimization. The core of the algorithm is to monitor the entropy fluctuation of each computing node in real time and dynamically adjust the allocation of task blocks to ensure that the system achieves load balancing in a multi-threaded concurrent environment, thereby maximizing the overall processing efficiency.

[0092] Such as H node (t)=|H prev (t)-H curr (t)|, by calculating this difference, the system can quantify the changes in the node's task load at different time points. If the entropy value fluctuates greatly, it means that the node's task load has changed significantly in a short period of time, which may be due to the sudden increase or decrease of certain tasks; if the entropy value fluctuates slightly, it indicates that the node's task load is relatively stable and the processing capacity is relatively balanced.

[0093] The entropy-balanced load scheduling algorithm uses the calculation results of this entropy fluctuation to guide the allocation of task blocks. Specifically, when the system needs to allocate new entropy-optimized task blocks and shadow task blocks to computing nodes, the algorithm prioritizes those nodes with smaller entropy fluctuations for allocation. The purpose of this is to utilize those computing nodes with relatively balanced and stable loads to avoid processing bottlenecks and resource waste caused by drastic fluctuations in task loads.

[0094] This strategy is very effective in multi-threaded concurrent processing. First, it ensures load balancing of each computing node in the system through real-time monitoring and dynamic adjustment, avoiding the problem of overloading some nodes or idle resources of some nodes. Second, it prioritizes assigning tasks to nodes with smaller entropy fluctuations, and can also reduce the frequency of task switching and reallocation, thereby improving the overall processing efficiency and stability of the system.

[0095] For example, in a multi-threaded system that processes complex real-time data streams, suppose that some computing nodes have a large increase in load due to receiving a large number of high-entropy data blocks at the previous moment, and the entropy fluctuation also increases. In this case, the system uses the entropy balance load scheduling algorithm to allocate the newly generated entropy optimization task blocks to those nodes with smaller entropy fluctuations and relatively stable loads. This allocation strategy not only avoids further increasing the pressure on nodes with higher loads, but also ensures the balance of the overall system processing tasks.

[0096] Preferably, the differential data D diff The calculation expression is:

[0097]

[0098] In the formula, the differential data D diff is differential data; and are the calculation results of thread i and thread j at time t respectively; n is the number of threads involved in the calculation.

[0099] By calculating the differential data between multiple threads, the calculation path of each thread is dynamically adjusted to optimize the overall processing efficiency and accuracy of the system. diff Represents the difference between the calculation results of different threads at a certain time point t.

[0100] The formula calculates the total absolute differences between all threads by summing them up, and derives the overall diversity of the system at a certain moment. This diversity data reflects the degree of consistency or inconsistency between threads in the current state of the system.

[0101] In principle, the interactive differential learning algorithm uses differential data to evaluate whether the current thread's calculation path is optimal and adjust it when necessary. Specifically, when the system detects that the differential data between threads is large, it indicates that there are significant differences in the calculation results of different threads. This may be due to the calculation path of some threads deviating from expectations or due to the randomness of the input data. In this case, the system will reduce the differences between threads by adjusting algorithm parameters or reallocating computing resources, thereby improving the consistency and accuracy of the overall calculation.

[0102] The effect of this algorithm is particularly evident in a multi-threaded concurrent processing environment. By calculating and adjusting differential data in real time, the system can dynamically optimize the calculation path of the thread, reducing calculation errors and performance degradation caused by path deviation or inconsistency. For example, when processing large data sets, different threads may take different calculation paths due to differences in data characteristics, resulting in deviations in the final results. By introducing differential data calculation, the system can quickly identify and correct these deviations to ensure that the calculation results of all threads tend to be consistent.

[0103] For example, suppose a concurrent processing system is responsible for analyzing data from multiple sensors. The readings of these sensors are assigned to different threads for processing. If at a certain moment, the readings of some sensors are abnormal or mutated, causing the calculation results of some threads to be significantly different from those of other threads, the differential data D diff The system can reduce these differences by adjusting the calculation path of abnormal threads, reallocating computing resources, or applying additional processing steps to ensure the accuracy of the final analysis results.

[0104] Through this dynamic adjustment mechanism, the interactive differential learning algorithm not only improves the efficiency of multi-threaded concurrent processing, but also enhances the robustness of the system when processing complex and variable data. This method is particularly suitable for application scenarios that require high precision and high consistency, such as real-time data analysis, financial risk assessment and other fields. In these applications, maintaining consistency between threads and reducing calculation deviations are crucial to ensuring the reliability of the final results. By real-time monitoring and adjusting differential data, the present invention successfully achieves efficient and accurate data processing in a multi-threaded concurrent environment.

[0105] Decomposing the task block to be executed into multiple subtask units, performing data exchange and learning among multiple threads through an interactive differential learning algorithm, optimizing the processing path of the subtask units, generating first concurrent processing data, and performing streaming data analysis on the first concurrent processing data to generate second concurrent processing data;

[0106] like Figure 3 As shown, preferably, the interactive difference learning algorithm includes the following steps:

[0107] Each thread generates differential data D when executing its subtask unit diff , indicating the difference between the calculation result of the current thread and the calculation result of other threads;

[0108] The thread learns the differential data D diff , dynamically adjust its calculation path, optimize processing efficiency, and generate the first concurrent processing data.

[0109] Each thread generates a set of differential data when executing its assigned subtask unit. The generation of differential data is based on the difference between the calculation results of the current thread and the calculation results of other threads. These differences are quantified by calculating the differential values ​​between threads. The differential data reflects the consistency or difference of the results of each thread when processing similar or related data. By monitoring these differential data in real time, the system can quickly identify which threads have significantly different calculation results from other threads, and then determine whether these differences are caused by deviation in calculation path, data anomaly, or uneven system load.

[0110] After obtaining the differential data, the system will dynamically adjust the calculation path of each thread by learning the differential data. The core of the adjustment is to optimize the calculation path to reduce the differences between threads and improve the consistency and efficiency of the overall processing. Specifically, when the calculation results of a thread are significantly different from those of other threads, the system may take the following measures: adjust the calculation strategy of the thread, reallocate its computing resources, or add additional correction steps to the calculation path. This dynamic adjustment enables each thread to work better together when processing complex data, avoiding performance bottlenecks or result deviations caused by inconsistent calculation paths.

[0111] The effect of the interactive differential learning algorithm is particularly prominent in a multi-threaded concurrent processing environment. Through continuous dynamic adjustment, the system can always maintain efficient and accurate calculation results during the execution of high-concurrency tasks. Especially when processing highly dynamic or highly uncertain data sets, the algorithm can significantly improve the coordination between threads and reduce calculation deviations caused by differences in data characteristics. For example, when processing financial transaction data, different threads may execute trading strategies based on different market data. Through the interactive differential learning algorithm, the system can ensure that the trading decisions of all threads are consistent, reduce strategy conflicts and execution deviations, and thus improve the stability and profitability of the overall trading system.

[0112] For example, suppose a concurrent processing system is analyzing a set of data from different sensors that are used to monitor the operating status of industrial equipment in real time. If a thread receives abnormal sensor data and its calculation results are significantly different from those of other threads, the system will detect this anomaly through differential data. At this time, the interactive differential learning algorithm will automatically adjust the calculation path of the thread, perhaps by recalibrating the data input, optimizing the calculation model, or allocating more computing resources, so that the calculation results of the thread are consistent with other threads again. This real-time adjustment not only ensures that the system accurately monitors the status of the equipment, but also improves the fault tolerance and robustness of the system.

[0113] Preferably, the streaming data analysis analyzes the first concurrently processed data by using a time series linkage prediction algorithm, specifically comprising the following steps:

[0114] Generate time series feature data, and extract the time series features of the first concurrently processed data;

[0115] Based on the time series feature data, a linkage prediction model is used to predict the trend of the second concurrently processed data, and the streaming analysis path is dynamically adjusted.

[0116] When processing the first concurrent processing data, the system generates time series feature data. The core of this process is to extract the time series features in the data. Time series features refer to the change patterns of data in the time dimension. These patterns may include periodic fluctuations, trend changes, sudden anomalies, etc. In a multi-threaded concurrent environment, the data streams processed by different threads often have complex time characteristics. For example, some data may gradually increase over time, or the amount of data may suddenly surge when certain events occur. By analyzing these time series features, the system can identify the potential rules in the data and provide a basis for subsequent prediction and processing.

[0117] After generating the time series feature data, the system uses the linkage prediction model to predict the trend of the second concurrent processing data based on these feature data. The design of the linkage prediction model takes into account the linkage and time series characteristics between data, that is, it not only focuses on the changes in a single thread or a single data stream, but also analyzes the correlation and mutual influence between different data streams. For example, in a logistics system, there is a complex linkage relationship between the shipment volume, traffic conditions and market demand of different warehouses, and the combined effect of these factors determines the future logistics flow. Through the linkage prediction model, the system can model these complex relationships and predict the trend of future data changes.

[0118] The output of the linkage prediction model is a trend prediction of the second concurrently processed data, and this prediction result is used to dynamically adjust the streaming analysis path. The purpose of dynamic adjustment is to optimize the data processing path based on the prediction results, so that the system can respond to data changes in a timely manner and avoid computing bottlenecks or delays caused by improper paths. For example, if the prediction results show that the flow of a certain data will increase significantly in a certain period of time in the future, the system can adjust the allocation of processing resources in advance, increase the computing power of the data flow, or optimize the processing path of the data flow to ensure that processing efficiency can still be maintained under high load conditions.

[0119] This time-series linkage prediction algorithm is very effective in multi-threaded concurrent processing. First, it can significantly reduce processing delays and resource waste by accurately predicting future trends in data streams. Second, dynamically adjusting the streaming analysis path ensures that the system can respond flexibly and maintain efficient operation when faced with complex and changing real-time data. Especially when processing highly volatile or sudden data, such as in application scenarios such as financial market data analysis and traffic flow monitoring, this algorithm can effectively improve the system's response speed and processing accuracy.

[0120] For example, in a real-time stock trading system, market data changes very quickly and in a complex manner, and the price fluctuations of different stocks are often interrelated and affected by multiple factors. Through the time series linkage prediction algorithm, the system can analyze the price change pattern of each stock in real time and predict the price trend in the next few minutes. Based on this prediction, the system can dynamically adjust the trading strategy and optimize the data processing path, thereby maintaining a competitive advantage in the ever-changing market environment.

[0121] Performing a global consistency check on the second concurrently processed data, marking abnormal data, and backtracking and correcting the abnormal data through a multi-level error correction backtracking mechanism to generate correction data and integrate it with the second concurrently processed data into final consistency data;

[0122] The principle of global consistency check is that the large amount of data processed by the system in a multi-threaded concurrent environment may be deviated or erroneous due to competition between threads, inconsistency of data flow or imbalance of system load. Therefore, after generating the second concurrent processing data, the system needs to perform a global consistency check on these data. The global consistency check compares the calculation results of each thread to check whether the data conforms to the expected logical and statistical characteristics, and identifies those data entries that do not conform to global consistency. Specifically, the system verifies each data point according to predefined verification rules and standards, and marks abnormal data that is significantly different from other data or violates the expected pattern.

[0123] For the marked abnormal data, the system uses a multi-level error correction backtracking mechanism to process it. The multi-level error correction backtracking mechanism is an adaptive data correction method that checks and corrects errors in the data layer by layer by tracing back the data generation path. First, the system traces back to the step where the abnormal data first appeared and analyzes the possible causes of the error, such as calculation errors, data conflicts, or resource competition. Then, based on the backtracking, the system gradually corrects each link that may have errors and corrects the abnormal data through technical means such as recalculation, data smoothing, or error compensation. After multi-level backtracking and correction, the system generates corrected data, which, after fine-tuning, has higher accuracy and consistency than the original second concurrent processing data.

[0124] After the correction data is generated, the system integrates it with the original second concurrent processing data to finally generate consistent data. This integration process is not just a simple data merge, but also includes operations such as deduplication, conflict resolution, and data consistency checks to ensure that the final consistent data not only maintains the integrity of the calculation results of each thread, but also eliminates all contradictions and errors in the data.

[0125] The effect of this process is particularly important in a multi-threaded concurrent processing environment. The introduction of global consistency checking and multi-level error correction and backtracking mechanisms effectively solves the problem of data inconsistency caused by concurrent processing, especially when processing complex and dynamically changing data sets, which can significantly improve the final data quality and reliability of the system. For example, in a financial data analysis system, multiple threads simultaneously process data streams from different markets. If there is no consistency checking and error correction mechanism, incorrect trading decisions may be made. Through the global consistency checking and backtracking mechanism in the present invention, the system can promptly identify and correct any potential errors after data processing is completed, ensuring that the final data analysis results are accurate and reliable.

[0126] For example, in a real-time weather forecast system, different threads process data from different sensors, which are integrated to predict weather conditions. If a sensor provides incorrect data due to a fault, the traditional multi-threaded system may not be able to detect and correct the problem in time, resulting in deviations in the prediction results. In the present invention, the system quickly identifies abnormal data through global consistency checks, and corrects the data through a multi-level backtracking mechanism. The weather forecast results finally generated are not only accurate, but also reflect the actual weather changes in a timely manner.

[0127] like Figure 4 As shown, preferably, the multi-level error correction backtracking mechanism includes the following steps:

[0128] Go back to the subtask unit stage and make local corrections to abnormal data;

[0129] Tracing back to the shadow task block stage, resynchronizing the shadow task block with the entropy optimization task block;

[0130] Correction data is generated, and the correction data is integrated with the second concurrently processed data to generate final consistency data.

[0131] The first step of the multi-level error correction backtracking mechanism is to trace back to the subtask unit stage and make local corrections to abnormal data. In multi-threaded concurrent processing, each thread is assigned to multiple subtask units, which independently perform specific computing tasks and generate corresponding data results. However, due to competition conditions, resource conflicts, or calculation errors in a multi-threaded environment, some subtask units may generate abnormal data. Tracing back to the subtask unit stage means that the system can identify and locate the source of these abnormal data and make local corrections to them. This correction may involve recalculating certain key steps, adjusting calculation parameters, or correcting errors by referring to the results of other subtask units. By correcting the data to the minimum extent, the system can improve the accuracy of the data without affecting the overall calculation process.

[0132] The system further traces back to the shadow task block stage and resynchronizes the shadow task block with the entropy optimization task block. The shadow task block plays a role of fault tolerance and redundancy in the present invention, and its design purpose is to be able to take over the tasks of the entropy optimization task block when a computing node fails. However, in actual execution, the shadow task block and the entropy optimization task block may be out of sync due to data transmission delays, calculation errors or uneven loads. This asynchrony may cause the results of the shadow task block to be inconsistent with those of the entropy optimization task block, thereby affecting the accuracy of the overall data. By tracing back to the shadow task block stage, the system can detect and correct these asynchrony problems to ensure that the data of the shadow task block and the entropy optimization task block are always consistent. The core of this step is to resynchronize the two task blocks so that their calculation results can be seamlessly integrated, thereby providing a basis for subsequent data integration.

[0133] The system generates correction data and integrates these correction data with the second concurrent processing data to generate final consistency data. The correction data is the product of the first two backtracking steps. They are the results obtained after correcting abnormal data and synchronizing task blocks. These correction data are considered to be more accurate and reliable. In the final data integration process, the correction data is merged with the second concurrent processing data to ensure that the calculation results of the entire system meet the highest quality standards. The integration process is not only a simple merger of data, but also includes removing redundancy, eliminating contradictions, and performing a final consistency check to ensure that all data is logically and statistically consistent with expectations. The output of this step, the final consistency data, is high-quality data after multi-level error correction and strict verification, which can be used in subsequent application scenarios such as data analysis, decision support, etc.

[0134] The multi-level error correction backtracking mechanism is effective in multi-threaded concurrent processing. First, it can effectively handle abnormal data problems caused by multi-threaded competition conditions or calculation errors by backtracking and correcting layer by layer. Secondly, the synchronous correction mechanism of the shadow task block and the entropy optimization task block ensures that the system can maintain data consistency when failures or uneven loads occur. Finally, by integrating the correction data with the second concurrent processing data, the system can output highly consistent and accurate final data. This mechanism is particularly suitable for application scenarios that require high reliability and accuracy, such as real-time financial trading systems, complex industrial control systems, and large-scale scientific computing environments.

[0135] For example, in a high-frequency trading system, different threads are responsible for processing trading data from different markets, and these data are finally aggregated to generate trading strategies. If the data processed by a thread is abnormal (for example, extreme values ​​caused by market fluctuations), the system will go back to the subtask unit stage that processes these data to make local corrections, and synchronize the shadow task block with the entropy optimization task block to ensure that the processing results of all threads are consistent. The final generated correction data is integrated with other normal data to form highly consistent trading strategy data, ensuring the accuracy and reliability of trading decisions.

[0136] The final consistency data is integrated and deduplicated to generate the final integrated data, and the final integrated data is fed back to the data storage module to generate processing log data and complete the data storage and application preparation.

[0137] Preferably, the result integration and feedback comprises the following steps:

[0138] Identifying and deleting duplicate data on the final consistency data;

[0139] Merge the deduplicated data to generate the final integrated data;

[0140] The final integrated data is stored in a data storage module, and processing log data is generated.

[0141] The principle of integration and deduplication is that in a multi-threaded concurrent processing environment, duplicate data entries may be generated because different threads may process similar data blocks at the same time, or because the data is calculated and transferred multiple times during the processing process. If these duplicate data are not effectively processed in the system, they may lead to data redundancy and affect the accuracy and efficiency of the final result. Therefore, identifying and deleting duplicate data is an important part of the data integration process.

[0142] In the specific implementation process, the system first identifies duplicate data for the final consistency data. This step usually detects duplicates based on the unique identifier of the data (such as timestamp, data source, hash value of data content, etc.). For some complex data sets, the system may also use advanced algorithms such as content comparison and pattern matching to further identify data that is not exactly the same but essentially duplicate. After identifying duplicate data, the system will mark it and delete it from the data set, retaining the unique and most representative data entries.

[0143] The deduplicated data will be merged to generate the final integrated data. Data merging is not just a simple splicing operation, it also includes data normalization, conflict resolution, and sorting. For example, if multiple threads process the same data block differently, the system needs to use a merge operation to determine which version of the data is more accurate, or to merge multiple versions of the data into a more complete result. Through this data merging, the system ensures that the final output data is both non-duplicate and comprehensive.

[0144] The integrated data will be stored in the data storage module and the processing log data will be generated. The data storage module is responsible for storing the final integrated data in a persistent storage in an efficient and reliable manner for subsequent retrieval and application. The processing log data records the key operations and events in the entire data processing process, including the steps of data reception, processing, deduplication, and merging. These logs not only provide an important basis for subsequent auditing and troubleshooting, but also provide data support for the continuous optimization and improvement of the system.

[0145] The effect of integration and deduplication is particularly important in a multi-threaded concurrent processing environment. By eliminating duplicate data, the system significantly reduces data redundancy and improves the utilization of storage and computing resources. At the same time, data merging and storage ensure that the final output data has high consistency and high reliability. This is crucial for application scenarios that rely on large-scale concurrent data processing, such as real-time financial data analysis, large-scale sensor network data processing, or complex industrial control systems. For example, in a real-time stock trading system, the system may receive the same transaction information from multiple data sources. If deduplication is not performed, the system may misjudge market trends and lead to inaccurate trading decisions. Through the data deduplication and integration processing in the present invention, the system can ensure that each trading decision is based on accurate, non-redundant data, thereby improving the success rate of transactions and market response speed.

[0146] like Figure 5 As shown, a system for implementing the multi-threaded concurrent processing method based on adaptive entropy segmentation and interactive differential learning, the system comprising:

[0147] The data receiving module is used to receive positioning data from the Beidou Internet of Vehicles terminal, and the positioning data includes geographic location data, timestamp data and sensor data. This module is responsible for receiving positioning data from the Beidou Internet of Vehicles terminal, and the positioning data includes geographic location data, timestamp data and sensor data. These data serve as the starting point of the entire processing flow and will enter the next step of analysis after preprocessing.

[0148] The dynamic label generation and data preprocessing module is used to generate dynamic label data for the positioning data, and preprocess the positioning data to remove noise and redundant information to generate preprocessed data; after receiving the positioning data, the module performs preliminary processing on the data to generate dynamic label data, and remove noise and redundant information to ensure that the subsequently processed data has high quality. This step ensures the consistency and accuracy of the data, laying the foundation for subsequent task segmentation.

[0149] The task segmentation and management module is used to divide the preprocessed data into multiple entropy optimization task blocks using an adaptive entropy segmentation algorithm, generate a shadow task block for each entropy optimization task block, and use an entropy balancing load scheduling algorithm to allocate the entropy optimization task block and the shadow task block to different computing nodes, and prepare the entropy optimization task block and the shadow task block as a task block to be executed and a shadow task block to be executed; using the adaptive entropy segmentation algorithm, the preprocessed data is divided into multiple entropy optimization task blocks. In order to ensure the fault tolerance and load balancing of the calculation, each task block generates a corresponding shadow task block, and these task blocks are allocated to different computing nodes through the entropy balancing load scheduling algorithm. The core of this module is to ensure that the system can run efficiently in concurrent processing through reasonable task allocation, avoiding resource waste and computing bottlenecks.

[0150] The concurrent processing and data analysis module is used to decompose the task block to be executed into multiple subtask units, exchange and learn data between multiple threads through an interactive differential learning algorithm, optimize the processing path of the subtask unit, generate the first concurrent processing data, and perform streaming data analysis on the first concurrent processing data to generate the second concurrent processing data; the task block is further decomposed into multiple subtask units, and exchange and learn data between multiple threads through an interactive differential learning algorithm to optimize the processing path of each subtask. This module generates the first concurrent processing data by dynamically adjusting the processing path, performs streaming data analysis on it, and generates the second concurrent processing data. The key to this step is to improve the flexibility of system processing and the accuracy of data processing.

[0151] The data verification and backtracking module is used to perform a global consistency check on the second concurrently processed data, mark abnormal data, and backtrack and correct the abnormal data through a multi-level error correction backtracking mechanism to generate correction data, and integrate the correction data with the second concurrently processed data into final consistency data; this module performs a global consistency check on the second concurrently processed data, marks and identifies abnormal data. Through the multi-level error correction backtracking mechanism, the system can backtrack and correct abnormal data, generate correction data, and finally integrate it with the second concurrently processed data into consistency data. Through this mechanism, the system improves the fault tolerance of data processing and ensures the integrity and reliability of the final data.

[0152] The data integration and feedback module is used to integrate and deduplicate the final consistency data, generate the final integrated data, and feed the final integrated data back to the data storage module, generate processing log data and complete the storage and application preparation of the data. In the last step, the module integrates and deduplicates the consistency data, generates the final integrated data, stores it in the data storage module, and generates processing log data. Through this process, the system ensures the accuracy of data storage and prepares for subsequent data applications.

[0153] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware.

[0154] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A multi-threaded concurrent processing method based on adaptive entropy segmentation and interactive differential learning, characterized in that: The following steps are involved: Receive positioning data from Beidou Internet of Vehicles terminals, the positioning data including geographic location data, timestamp data and sensor data, generate dynamic label data for the positioning data, and pre-process the positioning data, including removing noise and redundant information, to generate pre-processed data; The preprocessed data is divided into a plurality of entropy optimization task blocks by using an adaptive entropy segmentation algorithm, a shadow task block is generated for each entropy optimization task block, and an entropy balancing load scheduling algorithm is used to allocate the entropy optimization task blocks and the shadow task blocks to different computing nodes, and the entropy optimization task blocks and the shadow task blocks are prepared as task blocks to be executed and shadow task blocks to be executed, and the entropy balancing load scheduling algorithm includes the following steps: Calculate the current entropy fluctuation H of each computing node in real time node (t), the calculation expression is: H noed (t)=|H prev (t)-H curr (t)| Wherein, the entropy fluctuation H node (t) is the entropy fluctuation of the computing node at time t; H prev (t) is the entropy value of the previous moment; H curr (t) is the entropy value at the current moment; Prioritize the allocation of the entropy optimization task block and the shadow task block to computing nodes with smaller entropy value fluctuations to ensure load balance; Decomposing the task block to be executed into multiple subtask units, performing data exchange and learning among multiple threads through an interactive differential learning algorithm, optimizing the processing path of the subtask units, generating first concurrent processing data, and performing streaming data analysis on the first concurrent processing data to generate second concurrent processing data; Performing a global consistency check on the second concurrently processed data, marking abnormal data, and backtracking and correcting the abnormal data through a multi-level error correction backtracking mechanism to generate correction data and integrate it with the second concurrently processed data into final consistency data; The final consistency data is integrated and deduplicated to generate the final integrated data, and the final integrated data is fed back to the data storage module to generate processing log data and complete the data storage and application preparation.

2. The multi-threaded concurrent processing method based on adaptive entropy segmentation and interactive differential learning according to claim 1 is characterized in that: The adaptive entropy segmentation algorithm comprises the following steps: Calculate the entropy value H(x) of the preprocessed data, H(x)=-∑p(x i )logp(x i ) Wherein, the entropy value H(x) is the entropy value of the preprocessed data; p(x i ) is each data point x in the preprocessed data i The probability of The preprocessed data is divided into a plurality of entropy optimization task blocks according to the entropy value, so that the entropy value of each entropy optimization task block is within a preset threshold range.

3. The multi-threaded concurrent processing method based on adaptive entropy segmentation and interactive differential learning according to claim 1, characterized in that: The shadow task block is synchronized with the entropy optimization task block in real time on the distributed computing node, and the shadow task block is used to take over the processing of the entropy optimization task block when a computing node fails.

4. The multi-threaded concurrent processing method based on adaptive entropy segmentation and interactive differential learning according to claim 1, characterized in that: The interactive differential learning algorithm comprises the following steps: Each thread generates differential data D when executing its subtask unit diff , indicating the difference between the calculation result of the current thread and the calculation result of other threads; The thread learns the differential data D diff , dynamically adjust its calculation path, optimize processing efficiency, and generate the first concurrent processing data.

5. The multi-threaded concurrent processing method based on adaptive entropy segmentation and interactive differential learning according to claim 4 is characterized in that: The differential data D diff The calculation expression is: In the formula, the differential data D diff is differential data; and are the calculation results of thread i and thread j at time t respectively; n is the number of threads involved in the calculation.

6. The multi-threaded concurrent processing method based on adaptive entropy segmentation and interactive differential learning according to claim 1, characterized in that: The streaming data analysis analyzes the first concurrently processed data by using a time series linkage prediction algorithm, specifically including the following steps: Generate time series feature data, and extract the time series features of the first concurrently processed data; Based on the time series feature data, a linkage prediction model is used to predict the trend of the second concurrently processed data, and the streaming analysis path is dynamically adjusted.

7. The multi-threaded concurrent processing method based on adaptive entropy segmentation and interactive differential learning according to claim 1, characterized in that: The multi-level error correction backtracking mechanism includes the following steps: Go back to the subtask unit stage and make local corrections to abnormal data; Tracing back to the shadow task block stage, resynchronizing the shadow task block with the entropy optimization task block; Correction data is generated, and the correction data is integrated with the second concurrently processed data to generate final consistency data.

8. The multi-threaded concurrent processing method based on adaptive entropy segmentation and interactive differential learning according to claim 1, characterized in that: The result integration and feedback includes the following steps: Identifying and deleting duplicate data on the final consistency data; Merge the deduplicated data to generate the final integrated data; The final integrated data is stored in a data storage module, and processing log data is generated.

9. A system for implementing the multi-threaded concurrent processing method based on adaptive entropy segmentation and interactive differential learning according to any one of claims 1 to 8, characterized in that: The system comprises: A data receiving module is used to receive positioning data from a Beidou vehicle networking terminal, wherein the positioning data includes geographic location data, timestamp data and sensor data; A dynamic label generation and data preprocessing module, used to generate dynamic label data for the positioning data, and preprocess the positioning data to remove noise and redundant information to generate preprocessed data; A task segmentation and management module, for dividing the preprocessed data into a plurality of entropy optimization task blocks by using an adaptive entropy segmentation algorithm, generating a shadow task block for each entropy optimization task block, and allocating the entropy optimization task blocks and the shadow task blocks to different computing nodes by using an entropy balancing load scheduling algorithm, and preparing the entropy optimization task blocks and the shadow task blocks as task blocks to be executed and shadow task blocks to be executed; A concurrent processing and data analysis module, used for decomposing the task block to be executed into multiple subtask units, performing data exchange and learning among multiple threads through an interactive differential learning algorithm, optimizing the processing path of the subtask unit, generating first concurrent processing data, and performing streaming data analysis on the first concurrent processing data to generate second concurrent processing data; A data verification and backtracking module, used to perform a global consistency check on the second concurrently processed data, mark abnormal data, and backtrack and correct the abnormal data through a multi-level error correction backtracking mechanism to generate correction data, and integrate the correction data with the second concurrently processed data into final consistency data; The data integration and feedback module is used to integrate and deduplicate the final consistency data to generate final integrated data, and feed the final integrated data back to the data storage module to generate processing log data and complete data storage and application preparation.

Images