A data stream ordered processing method and device, storage medium and electronic equipment

Abstract

The application discloses a data stream ordered processing method and device, a storage medium and an electronic equipment, and relates to the technical field of automobile diagnosis. In the process of automobile remote diagnosis, a frame sequence number field of a target data packet meeting a write condition is taken as a valid / ready flag of a data block through an atomic write mechanism of a ready bit, and it is stipulated that this operation sequence must be written last, that is, the target frame sequence number is stored as metadata together with a user datagram protocol data packet itself, a target data packet meeting the write condition is obtained, and the target frame sequence number is written last when the target data packet meeting the write condition is written into a data buffer area of a target memory block, so as to serve as an atomic flag of a ready data block, prevent multi-thread conflicts, enable a processing thread to safely, accurately and efficiently judge whether a data block is readable without locking, and thus ensure that user datagram protocol data packets are ordered and reliable in the process of automobile remote diagnosis, and logical errors are avoided.

Description

Technical Field

[0001] This application relates to the field of automotive diagnostic technology, and more specifically, to a method, apparatus, storage medium, and electronic device for orderly processing of data streams. Background Technology

[0002] In scenarios highly sensitive to latency, such as remote vehicle diagnostics, User Datagram Protocol (UDP) is typically used for point-to-point communication to minimize network latency. UDP is a protocol that operates at the transport layer in the Open Systems Interconnection (OSI) model.

[0003] During remote vehicle diagnostics, the inherent unordered and unreliable nature of the UDP protocol may cause diagnostic commands and response data to be processed out of order, leading to logical errors.

[0004] Therefore, ensuring the orderly processing of data and avoiding logical errors during remote vehicle diagnostics is a problem that this application urgently needs to solve. Summary of the Invention

[0005] In view of this, this application discloses a data stream ordered processing method, apparatus, storage medium and electronic device, which aims to ensure the ordered processing of data and avoid logical errors during the process of remote vehicle diagnostics.

[0006] To achieve the above objectives, the disclosed technical solution is as follows:

[0007] The first aspect of this application discloses a method for ordered processing of data streams, the method comprising:

[0008] Acquire the data stream to be processed;

[0009] For each User Datagram Protocol (UDP) data packet in the data stream to be processed, obtain the frame sequence number corresponding to the UDP data packet;

[0010] The initial data packet corresponding to the frame sequence number is stored in the target memory block of a pre-initialized shared memory circular array; wherein, the shared memory circular array is a circular buffer constructed from fixed memory blocks with fixed array values; the target memory block is determined by direct indexing of the shared memory circular array;

[0011] The ready bit atomic write mechanism is used to write target data packets that meet the write conditions into the data buffer of the target memory block without locking; wherein, the ready bit atomic write mechanism is a write mechanism that writes the data packet into the data buffer of the target memory block and then writes the frame sequence number of the data packet as the ready flag.

[0012] The ready status of the target data packets in the data buffer is determined, and the corresponding operation is performed according to the determination result until the processing of all user datagram protocol data packets in the data stream to be processed is completed.

[0013] A second aspect of this application discloses a data stream ordered processing apparatus, the apparatus comprising:

[0014] The first acquisition unit is used to acquire the data stream to be processed;

[0015] The second acquisition unit is used to acquire the frame sequence number corresponding to each User Datagram Protocol (UDP) data packet in the data stream to be processed.

[0016] A storage unit is used to store the initial data packet corresponding to the frame sequence number into the target memory block of a pre-initialized shared memory circular array; wherein, the shared memory circular array is a circular buffer area constructed from fixed memory blocks with fixed array values; the target memory block is determined by direct indexing of the shared memory circular array;

[0017] The write unit is used to write target data packets that meet the write conditions into the data buffer of the target memory block without locking through the atomic write mechanism of the ready bit; wherein, the atomic write mechanism of the ready bit is a write mechanism in which the frame sequence number of the data packet is written as the ready flag after the data packet is written into the data buffer of the target memory block.

[0018] The judgment operation unit is used to judge the ready status of the target data packet in the data buffer and execute the corresponding operation according to the judgment result until the processing of all user datagram protocol data packets in the data stream to be processed is completed.

[0019] A third aspect of this application discloses a storage medium comprising stored instructions, wherein, when the instructions are executed, the device in which the storage medium resides executes the data stream ordered processing method as described in the first aspect.

[0020] The fourth aspect of this application discloses an electronic device including a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors using the data stream ordered processing method as described in the first aspect.

[0021] As can be seen from the above technical solutions, this application discloses a data stream ordered processing method, device, storage medium and electronic device. In the process of remote vehicle diagnosis, the fixed storage location of the user data packet, i.e. the target memory block, is calculated by taking the frame sequence number corresponding to the user data packet modulo the fixed array value through the direct indexing method of the shared memory ring array. When performing placement and search operations in the data structure, the time required is constant regardless of the amount of data, i.e., it does not increase with the growth of the input scale, thereby achieving a time complexity of O(1). Furthermore, through an atomic write mechanism for the ready bit, the frame sequence number field of the target data packet that meets the write conditions is used as the valid / ready flag of the data block. It is stipulated that the operation order must be last, that is, the target frame sequence number is stored as metadata together with the User Datagram Protocol (UDP) data packet itself to obtain the target data packet that meets the write conditions. When writing the target data packet that meets the write conditions to the data buffer of the target memory block, the target frame sequence number is written last. This serves as the atomic flag of the data block being ready, preventing multi-threaded conflicts. This allows the processing thread to safely, accurately, and efficiently determine whether a data block is readable without locking, thereby ensuring that UDP data packets are ordered and reliable during remote vehicle diagnostics and avoiding logical errors. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0023] Figure 1 This is an architecture diagram of remote vehicle diagnostics disclosed in an embodiment of this application;

[0024] Figure 2 This is a schematic diagram of the structure of a data stream ordered processing system disclosed in an embodiment of this application;

[0025] Figure 3 This is a flowchart illustrating an ordered data stream processing method disclosed in an embodiment of this application;

[0026] Figure 4 This is a schematic diagram of the structure of a data stream ordered processing device disclosed in an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of the structure of the electronic device disclosed in the embodiments of this application. Detailed Implementation

[0028] The embodiments of this application will now be described with reference to the accompanying drawings.

[0029] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0030] As the background technology indicates, in scenarios highly sensitive to latency, such as remote vehicle diagnostics, UDP is typically used for point-to-point communication to minimize network latency. In remote vehicle diagnostics, UDP is commonly used... Figure 1 The architecture diagram. Figure 1 This includes a network server, a C-end connector, a B-end connector, and a diagnostic tool. The B-end connector connects to the network server, the diagnostic tool, and the C-end connector. Specifically, the B-end connector connects to the C-end connector via a direct UDP connection; the C-end connector connects to both the network server and the vehicle. Automotive diagnostics has very high latency requirements; for example, the diagnostic tool will time out if it doesn't receive a response within 55 milliseconds. In remote diagnostics, to reduce network latency, UDP communication is often used, establishing a direct point-to-point UDP connection between the B and C-end devices to achieve minimal network latency.

[0031] refer to Figure 2 The diagram shown is a structural schematic of a data stream ordered processing system disclosed in an embodiment of this application. The data stream ordered processing system includes a receiving thread, a processing thread, and a pre-initialized shared memory circular array.

[0032] The receiving thread is connected to both the processing thread and the shared memory circular array.

[0033] Shared memory circular array:

[0034] During the initialization of the data stream ordered processing system, a contiguous block of physical memory is pre-allocated, and this physical memory is logically divided into N fixed memory blocks (struct packets) of equal size.

[0035] Each fixed memory block stores a potentially arriving UDP packet and its metadata. This array is logically considered a ring buffer. The expected frame sequence number (_seq) is initialized to 1 (expected_seq=1), and this is also repeated after reconnection between the B and C devices. This process constitutes the initialization of the shared memory ring array. _seq is the frame sequence number of the data frame the application currently expects to receive.

[0036] It should be noted that the UDP packets that may arrive are the data that are expected to be received. For example, if the current _seq is 1 and the circular array can store 100 packets, then the UDP packets that may arrive are the packets with frame sequence numbers from 1 to 100, and the metadata is the payload data in the UDP packets.

[0037] Because UDP is unordered, but the application layer of a data stream processing system requires ordered data, each data frame must have a frame sequence number (this frame sequence number is unique and sequentially incrementing). The frame sequence number to be processed is the frame sequence number of the data frame that the application currently expects to receive; the frame sequence number to be processed is a dynamically changing value.

[0038] A circular buffer, also known as a cyclic buffer or circular buffer, is a data structure used to represent a fixed-size buffer that is connected end to end, and is suitable for caching data streams.

[0039] The structure of a fixed memory block is defined as follows:

[0040] The structure of each fixed memory block (struct packet) is designed as follows, balancing versatility and performance:

[0041] struct packet{

[0042] uint32_t _seq;

[0043] uint8_t data_buf[PACK_FIX_DATA_LEN];

[0044] int long_data_len;

[0045] uint8_t* long_data_ptr;

[0046] };

[0047] The parameters in the fixed memory block structure have the following meanings:

[0048] The _seq key field: stores the sequence number of the data packet frame written to the target memory block;

[0049] The data_buf main data area is a fixed-length array that stores the original data frames and can accommodate the vast majority of data packets.

[0050] long_data_len is an indicator of the total length of the data;

[0051] long_data_ptr; Optional extended area: only points to additional storage (non-standard storage path) when the data is too long (i.e. the data length is greater than the preset length, the preset length is set according to the actual situation, and this application does not make a specific limit).

[0052] It's important to note that the key design principle of fixed memory blocks is to store the target frame sequence number as metadata along with the User Datagram Protocol (UDP) data packet itself. This yields target data packets that meet the write conditions. Furthermore, when writing these matching data packets to the data buffer of the target memory block, the target frame sequence number is written last, serving as an atomicity marker indicating that the data block is ready. The fixed memory block layout also avoids the uncertainties caused by memory fragmentation, has simple logic, and is easier to verify and test.

[0053] Storing `_seq` as metadata along with the UDP packet itself is to prevent multi-threaded conflicts. To improve efficiency, a lock-free design is used. If `_seq` is updated first, the following possibility exists: The thread is switched by the system immediately after writing to `_seq`, and the processing thread detects new data being saved (a new `_seq` has been written to the circular array) and processes the new data, which can lead to multi-threaded conflicts.

[0054] The workflow of the receiving thread includes steps such as packet reception and parsing, preliminary filtering, direct address calculation, lock-free writing and deduplication, and waking up the processing thread. The specific flow of each step is as follows:

[0055] Reception and parsing: Receive UDP packets and parse out the corresponding UDP packet sequence number (seq), payload, and other data;

[0056] Preliminary filtering:

[0057] If the sequence number (seq) of a UDP packet is less than the expected sequence number of the frame to be processed (expected_seq), the UDP packet is determined to be an outdated packet, and is directly discarded or deleted.

[0058] If the seq corresponding to the UDP packet is greater than or equal to the sequence number of the frame to be processed, then calculate the difference (diff) between the seq corresponding to the UDP packet and the sequence number of the frame to be processed; if diff≥N, then it is determined to be an out-of-range packet and discarded; otherwise, proceed to subsequent processing; where N is a fixed array size value.

[0059] The initial data packets are obtained through the above preliminary filtering.

[0060] Direct address calculation (core step): Calculate the fixed location where the data packet should be stored according to the formula memory block index. This fixed location is the target memory block ((packets[index])) of the shared memory circular array. The formula memory block index is shown in formula (1):

[0061] index=seq%N(1)

[0062] Where index is the index of the target memory block; seq is the sequence number of the UDP packet; % is the modulo operator; and N is a fixed array size.

[0063] The initial data packet is stored in the target memory block of the shared memory circular array obtained by formula (1) above.

[0064] The calculation of this formula (1) is a deterministic O(1) operation, requiring no lookup. By using the direct indexing method of the shared memory circular array, that is, by taking the frame sequence number corresponding to the User Datagram Protocol (UDP) data packet modulo the fixed array value using formula (1) to calculate the fixed storage location of the UDP data packet, i.e., the target memory block, the time required for placement and lookup operations in the data structure is constant regardless of the amount of data, i.e., it does not increase with the growth of the input size, thus achieving a time complexity of O(1).

[0065] To better understand the process of calculating the fixed location where the data packet should be stored based on the memory block index formula, an example is provided here:

[0066] For example, a circular array can store 100 data frames, so N is 100. Now a data frame with frame number 101 is received, so index = 101%100 = 1. This data frame is stored at index 1 of the circular array. At this time, a data frame with frame number 105 arrives, so 105%100 = 5. This data frame is stored at index 5.

[0067] Lock-free writing and deduplication:

[0068] Check the target frame number in the target memory block (packets[index]).

[0069] If the target frame sequence number equals the UDP packet's corresponding frame sequence number (seq), then the packet corresponding to the target frame sequence number is determined to be a duplicate packet, and the duplicate packet is deleted.

[0070] If the target frame sequence number is not equal to the UDP packet's corresponding frame sequence number seq, determine that the packet corresponding to the target frame sequence number is a target packet that meets the write conditions, write the content of the target packet that meets the write conditions to the data buffer (data_buf) of the target memory block without locking (or use long_data_ptr if necessary), and finally update the target frame sequence number = seq.

[0071] It should be noted that the lock-free design and high cache friendliness (continuous memory access) enable the system to handle higher network packet rates.

[0072] Wake up the processing thread: After writing the contents of the target data packet that meets the writing conditions into the data buffer, notify the processing thread that there is new data to process.

[0073] The workflow of the processing thread includes sequential scanning, ready status determination, response, and memory reclamation, as detailed below:

[0074] Sequential scanning: Start checking from the memory block corresponding to the current frame sequence number, and calculate the target data packet index in the data buffer according to formula (2);

[0075] index=expected_seq%N(2);

[0076] Where index is the index of the memory block corresponding to the sequence number of the frame to be processed; expected_seq is the sequence number of the frame to be processed; % is the modulo operator; and N is a fixed array size.

[0077] Readiness status assessment:

[0078] Check if the target frame sequence number (packets[index]._seq) corresponding to the target packet index in the data buffer is equal to the expected frame sequence number (expected_seq);

[0079] If packets[index]._seq = expected_seq, the result indicates that the target data packet corresponding to the target frame number is in a valid / ready state. The target data packet is then processed, and expected_seq is incremented by 1 (i.e., expected_seq++). The process then loops back to the "sequential scan" step to continue checking the next frame number to be processed. This process continuously processes all ready data packets with consecutive frame numbers.

[0080] If packets[index]._seq ≠ expected_seq, it means the data packet with the sequence number of the frame to be processed has not yet arrived or has been lost, and the loop should be exited here.

[0081] By employing an atomic write mechanism for the ready bit, the frame sequence number field of the target data packet that meets the write conditions is used as the valid / ready flag of the data block. Furthermore, the operation order is stipulated to be the last to be written. Specifically, the target frame sequence number is stored as metadata along with the User Datagram Protocol (UDP) data packet itself to obtain the target data packet that meets the write conditions. When writing the target data packet that meets the write conditions to the data buffer of the target memory block, the target frame sequence number is written last, serving as the atomic flag indicating that the data block is ready. This prevents multi-threaded conflicts and allows processing threads to safely, accurately, and efficiently determine whether a data block is readable without the need for locking.

[0082] Response and memory reclamation:

[0083] After processing a batch of User Datagram Protocol (UDP) packets, a unified acknowledgement (ACK) message can be sent. Memory blocks do not need to be explicitly released; when the sequence number (seq) of a subsequent new packet is not equal to the sequence number (_seq) of the original packet, the receiving thread directly overwrites and reuses it. Essentially, memory blocks are reused cyclically within a circular array.

[0084] It's important to note that explicit memory release refers to the programmer actively calling a release function (such as `free()` in C or `delete` in C++) in the code to return the heap memory previously allocated using methods like `malloc()`, `calloc()`, `realloc()`, or `new` to the system or memory allocator, thus preventing memory leaks. This one-time static memory allocation trades for long-term high performance at runtime, making it suitable for embedded systems or resource-constrained automotive environments.

[0085] This application relies solely on a pre-allocated, fixed shared memory circular array throughout the entire lifecycle of UDP packet reception, sorting, and processing, completely avoiding runtime dynamic memory allocation and deallocation. Dynamic memory allocation involves system calls and heap management, incurring significant overhead. The processing flow without dynamic memory management reduces CPU time consumption and improves execution efficiency. This application minimizes processing latency on the critical path by eliminating dynamic memory management, eliminating buffer insertion sorting, and greatly reducing lock contention.

[0086] Throughout the entire lifecycle of UDP packet reception, sorting, and processing, this application relies solely on a pre-allocated fixed circular array, completely avoiding the system architecture and methods of runtime dynamic memory allocation and release, making processing simpler and more convenient.

[0087] In this embodiment of the application, during the remote vehicle diagnostic process, the receiving thread of the data stream ordered processing system calculates the fixed storage location of the User Datagram Protocol (UDP) data packet, i.e. the target memory block, by taking the frame sequence number corresponding to the UDP data packet modulo the fixed array value through the direct indexing method of the shared memory circular array. When placing and searching in the data structure, the time required is constant regardless of the amount of data, i.e., it does not increase with the growth of the input size, thereby achieving a time complexity of O(1). Furthermore, the processing thread of the data stream ordered processing system uses the atomic write mechanism of the ready bit to use the frame sequence number field of the target data packet that meets the write conditions as the valid / ready flag of the data block. It also stipulates that the operation order must be last, that is, the target frame sequence number is stored as metadata together with the User Datagram Protocol (UDP) data packet itself to obtain the target data packet that meets the write conditions. When writing the target data packet that meets the write conditions to the data buffer of the target memory block, the target frame sequence number is written last. This serves as the atomic flag of the data block being ready, preventing multi-threaded conflicts. This allows the processing thread to safely, accurately, and efficiently determine whether a data block is readable without locking, thereby ensuring that UDP data packets are ordered and reliable during remote vehicle diagnostics and avoiding logical errors.

[0088] refer to Figure 3 The diagram shown is a flowchart illustrating a data stream ordered processing method disclosed in an embodiment of this application. This data stream ordered processing method mainly includes the following steps:

[0089] S301: Obtain the data stream to be processed.

[0090] The data stream to be processed is the data stream that has not undergone frame sequence number parsing. The data stream to be processed includes multiple UDP packets.

[0091] S302: For each UDP packet in the data stream to be processed, obtain the seq corresponding to the UDP packet.

[0092] S303: Store the initial data packet corresponding to the seq of the UDP packet into the target memory block of the pre-initialized shared memory circular array; wherein, the shared memory circular array is a circular buffer constructed from fixed memory blocks with fixed array values; the target memory block is determined by direct indexing of the shared memory circular array.

[0093] The target memory block of the shared memory circular array is determined by the direct indexing method of the shared memory circular array, i.e., by the above formula (1).

[0094] The specific initialization process of the shared memory circular array is as follows:

[0095] First, physical memory is allocated for the shared memory ring array. Then, the physical memory in the shared memory ring array is divided into fixed-size struct packets of equal size (N), and the sequence number of the frames to be processed is initialized to 1. Each fixed-size struct packet is used to store a potentially arriving UDP packet and its metadata. This array is logically treated as a ring buffer. The sequence number of the frames to be processed (_seq) is initialized to 1 (expected_seq=1), and the sequence number of the frames to be processed is also initialized to 1 after the B / C end devices reconnect. The above process is the initialization process of the shared memory ring array. _seq is the sequence number of the data frame that the application currently expects to receive.

[0096] The initialization process of the shared memory circular array is the same as in the above embodiment. Figure 2 The initialization process for the shared memory circular array is the same and can be referred to; it will not be repeated here.

[0097] In this embodiment, outdated and out-of-range data packets are deleted to obtain accurate initial data. Furthermore, deleting outdated and out-of-range data packets reduces the load on the database or storage, improving the overall system operating efficiency.

[0098] The specific process of storing the initial data packet into the target memory block of the shared memory circular array, as shown in A1-A3, mainly includes the following steps:

[0099] A1: Filter the frame sequence numbers corresponding to the UDP packets to obtain the initial data packets corresponding to the frame sequence numbers; the initial data packets are the data packets after removing outdated and out-of-range data packets.

[0100] The execution process and principle of A1 are the same as those in the above embodiments. Figure 2 The initial filtering process and execution principle are the same, which can be referred to, and will not be repeated here.

[0101] A2: Calculate the target memory block of the pre-initialized shared memory circular array for the initial data packet based on the memory block index formula.

[0102] A3: Store the initial data packet into the target memory block.

[0103] The execution process and principle of A2-A3 are the same as those in the above embodiments. Figure 2 The execution process and principle of direct address calculation are the same, which can be referred to, and will not be repeated here.

[0104] The specific process of obtaining the initial data packet is shown in B1-B5.

[0105] B1: Compare the sequence number of the frame corresponding to the User Datagram Protocol (UDP) data packet with the sequence number of the frame to be processed.

[0106] B2: If the sequence number of the frame corresponding to the User Datagram Protocol (UDP) data packet is less than the sequence number of the frame to be processed, determine that the data packet corresponding to the sequence number is an outdated data packet and delete it.

[0107] B3: If the sequence number of the frame corresponding to the User Datagram Protocol (UDP) data packet is greater than or equal to the sequence number of the frame to be processed, calculate the difference between the sequence number of the UDP data packet and the sequence number of the frame to be processed.

[0108] B4: If the difference is greater than or equal to the fixed array size (N), determine that the data packet corresponding to the frame sequence number is an out-of-range data packet and delete it.

[0109] B5: If the difference is less than the fixed array size, the initial data packet corresponding to the frame sequence number of the User Datagram Protocol data packet is obtained.

[0110] The execution process and execution principle of B1-B5 are the same as those in the above embodiments. Figure 2 The initial filtering process is the same as the previous one and can be referred to; it will not be repeated here.

[0111] In this embodiment, the target memory block of the initial data packet in the pre-initialized shared memory circular array is calculated according to the memory block index formula. That is, the fixed storage location of the user data packet is calculated by taking the frame sequence number corresponding to the user data packet modulo the fixed array value. When performing placement and search operations in the data structure, the time required is constant regardless of the amount of data, that is, it does not increase with the growth of the input size, thereby achieving a time complexity of O(1).

[0112] S304: The target data packet that meets the write conditions is written to the data buffer of the target memory block without lock through the atomic write mechanism of the ready bit; wherein, the atomic write mechanism of the ready bit is a write mechanism that writes the data packet to the data buffer of the target memory block and then writes the frame sequence number of the data packet as the ready flag.

[0113] Specifically, the process of writing target data packets that meet the write conditions to the data buffer of the target memory block without locking, through the ready bit atomic write mechanism, is shown in C1-C4, and mainly includes the following steps:

[0114] C1: Get the target frame number in the target memory block.

[0115] C2: If the target frame sequence number matches the frame sequence number corresponding to the UDP packet, determine that the packet corresponding to the target frame sequence number is a duplicate packet and delete it.

[0116] C3: If the target frame sequence number is inconsistent with the frame sequence number corresponding to the UDP data packet, determine that the data packet corresponding to the target frame sequence number is the target data packet that meets the writing conditions.

[0117] C4: Writes the target data packet that meets the write conditions to the data buffer (data_buf) of the target memory block without locking.

[0118] The execution process and execution principle of C1-C4 are the same as those in the above embodiments. Figure 2 The lock-free writing and deduplication process and principle are the same, which can be referred to, and will not be repeated here.

[0119] In this embodiment, the atomic write mechanism of the ready bit is used to use the frame sequence number field of the target data packet that meets the write conditions as the valid / ready flag of the data block. It is also stipulated that the operation order must be last. That is, the target frame sequence number is stored as metadata together with the User Datagram Protocol (UDP) data packet itself to obtain the target data packet that meets the write conditions. When the target data packet that meets the write conditions is written to the data buffer of the target memory block, the target frame sequence number is written last. This serves as the atomic flag of the data block being ready, preventing multi-threaded conflicts and enabling the processing thread to safely, accurately, and efficiently determine whether a data block is readable without the need for locking.

[0120] S305: Determine the ready status of the target data packet in the data buffer and perform the corresponding operation based on the determination result until the processing of all user datagram protocol data packets in the pending data stream is completed.

[0121] The process of determining the ready status of target data packets in the data buffer and executing corresponding operations based on the determination result, as shown in D1-D5, mainly includes the following steps:

[0122] D1: Calculate the storage location of the target data packet in the data buffer based on the sequence number of the frame to be processed.

[0123] D2: Get the target frame sequence number (packets[index]._seq) of the target data packet stored in the storage location.

[0124] The ready status is determined based on the target frame number and the frame number to be processed.

[0125] D3: If the target frame sequence number is equal to the frame sequence number to be processed, obtain the judgment result that the target data packet corresponding to the target frame sequence number is in a valid / ready state.

[0126] D4: Based on the determination result that the target data packet is in a valid / ready state, perform response processing on the target data packet corresponding to the target frame sequence number that is in the correct ready state.

[0127] In D4, after processing a batch of consecutive User Datagram Protocol (UDP) packets, an ACK message is sent after processing the batch of consecutive UDP packets.

[0128] It's important to note that the data buffer is a fixed-length array, forming a circular queue. Outdated data is simply overwritten by new data; there is no dynamic memory release. Frequent memory allocation and deallocation incurs significant system overhead and unpredictable time consumption, making it unsuitable for scenarios with extremely high latency requirements.

[0129] For example, if there are 15 consecutive data packets in the buffer, then after processing 1-10 packets, an ACK is sent, and after processing 11-15 packets, another ACK is sent.

[0130] If the buffer receives data 1-10 and 12-15 (11 was not received), then the data to be processed is 1-10. Then an Ack is given; the remaining data 12-15 will not be processed until 11 is received.

[0131] In extreme cases, the buffer may contain a large amount of data, such as 1000 packets, which would take a long time to process. This could result in delayed ACK responses, causing the sender to not receive the ACK in time and leading to unnecessary retransmissions. Therefore, it is necessary to wait until the data packets are processed before responding.

[0132] This scheme does not respond with an ACK for every data item, as this would increase the number of ACKs and waste unnecessary bandwidth. In fact, as long as the receiver receives an ACK for 10, it can be determined that data 1-10 has been received because it processes continuous data.

[0133] D5: If the target frame sequence number is not equal to the frame sequence number to be processed, the judgment result of the data packet with the frame sequence number to be processed not responding is obtained, and the data loop processing in the shared memory circular array is terminated.

[0134] The execution process and principle of D1-D5 are the same as those in the above embodiments. Figure 2 The execution process and principle of the ready state judgment are the same, which can be referred to, and will not be repeated here.

[0135] In this embodiment, after processing all User Datagram Protocol (UDP) packets, an acknowledgment message is sent to all UDP packets. The ACK mechanism is implemented through the application layer to detect whether the packets have been successfully delivered. If no ACK is received, retransmission can be triggered, thereby compensating for the lack of reliability inherent in UDP.

[0136] In this embodiment, during remote vehicle diagnostics, the fixed storage location of the User Datagram Protocol (UDP) data packet, i.e., the target memory block, is calculated by taking the frame sequence number of the UDP data packet modulo a fixed array value using the direct indexing method of a shared memory circular array. During placement and lookup operations in the data structure, the time required is constant regardless of the data volume, meaning it does not increase with the input size, thus achieving a time complexity of O(1). Furthermore, an atomic write mechanism for the ready bit is used to use the frame sequence number field of the target data packet that meets the write conditions as a valid / ready flag for the data block. This mechanism stipulates that the write operation must be performed last. Specifically, the target frame sequence number is stored as metadata along with the UDP data packet itself to obtain the target data packet that meets the write conditions. When writing the target data packet that meets the write conditions to the data buffer of the target memory block, the target frame sequence number is written last, serving as an atomic flag indicating that the data block is ready. This prevents multi-threaded conflicts, allowing the processing thread to safely, accurately, and efficiently determine whether a data block is readable without locking, thereby ensuring that the UDP data packets are ordered and reliable during remote vehicle diagnostics and avoiding logical errors.

[0137] Based on the above embodiments Figure 2 The present application discloses a method for orderly processing data streams, and also provides a corresponding apparatus for such orderly processing. Figure 4 As shown, the data stream ordered processing device mainly includes:

[0138] The first acquisition unit 401 is used to acquire the data stream to be processed;

[0139] The second acquisition unit 402 is used to acquire the frame sequence number corresponding to each User Datagram Protocol (UDP) data packet in the data stream to be processed.

[0140] Storage unit 403 is used to store the initial data packet corresponding to the frame sequence number into the target memory block of the pre-initialized shared memory circular array; wherein, the shared memory circular array is a circular buffer area constructed from fixed memory blocks with fixed array values; the target memory block is determined by direct indexing of the shared memory circular array;

[0141] The write unit 404 is used to write the target data packet that meets the write conditions into the data buffer of the target memory block without locking through the atomic write mechanism of the ready bit; wherein, the atomic write mechanism of the ready bit is a write mechanism that writes the data packet into the data buffer of the target memory block and then writes the frame sequence number of the data packet as the ready flag.

[0142] The judgment operation unit 405 is used to judge the ready status of the target data packet in the data buffer and execute the corresponding operation according to the judgment result until the processing of all user datagram protocol data packets in the data stream to be processed is completed.

[0143] Furthermore, the storage unit 403 includes:

[0144] The filtering module is used to filter frame sequence numbers to obtain the initial data packets corresponding to the frame sequence numbers; the initial data packets are the data packets after removing outdated data packets and out-of-range data packets;

[0145] The first calculation module is used to calculate the target memory block of the pre-initialized shared memory circular array of the initial data packet according to the memory block index formula.

[0146] The storage module is used to store the initial data packet into the target memory block.

[0147] Furthermore, the filtering module includes:

[0148] The comparison submodule is used to compare the frame sequence number corresponding to the User Datagram Protocol (UDP) data packet with the sequence number of the frame to be processed.

[0149] The first deletion determination submodule is used to determine that the data packet corresponding to the frame sequence number is an outdated data packet and delete it if the frame sequence number corresponding to the User Datagram Protocol data packet is less than the frame sequence number to be processed.

[0150] The calculation submodule is used to calculate the difference between the sequence number of the frame corresponding to the User Datagram Protocol (UDP) data packet and the sequence number of the frame to be processed if the sequence number of the frame corresponding to the UDP data packet is greater than or equal to the sequence number of the frame to be processed.

[0151] The second deletion submodule is used to determine if the difference is greater than or equal to the fixed array size, and then delete the data packet corresponding to the frame sequence number as an out-of-range data packet.

[0152] The `get` submodule is used to obtain the initial data packet corresponding to the frame sequence number if the difference is less than the fixed array size.

[0153] Furthermore, the write unit 404 includes:

[0154] The first acquisition module is used to acquire the target frame sequence number in the target memory block;

[0155] The deletion determination module is used to determine that if the target frame sequence number is consistent with the frame sequence number, the data packet corresponding to the target frame sequence number is a duplicate data packet and is deleted.

[0156] The determination module is used to determine the data packet corresponding to the target frame sequence number as a target data packet that meets the writing conditions if the target frame sequence number is inconsistent with the frame sequence number.

[0157] The write module is used to write the target frame sequence number of the target data packet that meets the write conditions to the data buffer of the target memory block without locking.

[0158] Furthermore, the judgment operation unit 405 includes:

[0159] The second calculation module is used to calculate the storage location of the target data packet in the data buffer based on the sequence number of the frame to be processed;

[0160] The third calculation module is used to obtain the target frame sequence number of the target data packet stored in the storage location;

[0161] The judgment module is used to determine the ready status by comparing the target frame sequence number with the frame sequence number to be processed.

[0162] The second acquisition module is used to obtain the judgment result that the target data packet corresponding to the target frame sequence number is in a valid / ready state if the target frame sequence number is equal to the frame sequence number to be processed.

[0163] The processing module is used to respond to the target data packet corresponding to the target frame sequence number that is in the correct ready state based on the judgment result that the target data packet is in a valid / ready state;

[0164] The third acquisition module is used to determine if the target frame sequence number is not equal to the sequence number of the frame to be processed, and to end the data loop processing in the shared memory circular array.

[0165] Furthermore, the processing module includes:

[0166] The sending submodule is used to send a positive response message to the target data packet corresponding to the target frame sequence number that is in the correct ready state;

[0167] The operation submodule is used to perform an overwrite and reuse operation when it is detected that the frame sequence number of the target data packet corresponding to the new correct ready state target frame sequence number is not equal to the data packet frame sequence number of the target memory block.

[0168] Furthermore, the storage unit 403 for the initialization process of the shared memory circular array includes:

[0169] The allocation module is used to allocate physical memory for the shared memory circular array;

[0170] The partitioning module is used to divide the physical memory in the shared memory circular array into fixed memory blocks of equal size, and initialize the sequence number of the frame to be processed to 1.

[0171] In this embodiment, during remote vehicle diagnostics, the fixed storage location of the User Datagram Protocol (UDP) data packet, i.e., the target memory block, is calculated by taking the frame sequence number of the UDP data packet modulo a fixed array value using the direct indexing method of a shared memory circular array. During placement and lookup operations in the data structure, the time required is constant regardless of the data volume, meaning it does not increase with the input size, thus achieving a time complexity of O(1). Furthermore, an atomic write mechanism for the ready bit is used to use the frame sequence number field of the target data packet that meets the write conditions as a valid / ready flag for the data block. This mechanism stipulates that the write operation must be performed last. Specifically, the target frame sequence number is stored as metadata along with the UDP data packet itself to obtain the target data packet that meets the write conditions. When writing the target data packet that meets the write conditions to the data buffer of the target memory block, the target frame sequence number is written last, serving as an atomic flag indicating that the data block is ready. This prevents multi-threaded conflicts, allowing the processing thread to safely, accurately, and efficiently determine whether a data block is readable without locking, thereby ensuring that the UDP data packets are ordered and reliable during remote vehicle diagnostics and avoiding logical errors.

[0172] This application embodiment also provides a storage medium, the storage medium including stored instructions, wherein, when the instructions are executed, the device where the storage medium is located is controlled to perform the data stream ordered processing method described above.

[0173] This application also provides an electronic device, the structural schematic diagram of which is shown below. Figure 5 As shown, it specifically includes a memory 501 and one or more instructions 502, wherein one or more instructions 502 are stored in the memory 501 and are configured to be executed by one or more processors 503 to perform the above-mentioned data stream ordered processing method.

[0174] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0175] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0176] The steps in the methods of the various embodiments of this application can be adjusted, combined, or deleted according to actual needs.

[0177] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0178] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0179] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for ordered processing of data streams, characterized in that, The method includes: Acquire the data stream to be processed; For each User Datagram Protocol (UDP) data packet in the data stream to be processed, obtain the frame sequence number corresponding to the UDP data packet; The initial data packet corresponding to the frame sequence number is stored in the target memory block of a pre-initialized shared memory circular array; wherein, the shared memory circular array is a circular buffer constructed from fixed memory blocks with fixed array values; the target memory block is determined by direct indexing of the shared memory circular array; The ready bit atomic write mechanism is used to write target data packets that meet the write conditions into the data buffer of the target memory block without locking; wherein, the ready bit atomic write mechanism is a write mechanism that writes the data packet into the data buffer of the target memory block and then writes the frame sequence number of the data packet as the ready flag. The ready status of the target data packets in the data buffer is determined, and the corresponding operation is performed according to the determination result until the processing of all user datagram protocol data packets in the data stream to be processed is completed.

2. The method according to claim 1, characterized in that, Storing the initial data packet corresponding to the frame sequence number into the target memory block of a pre-initialized shared memory circular array includes: The frame sequence number is filtered to obtain the initial data packet corresponding to the frame sequence number; wherein, the initial data packet is the data packet after removing outdated data packets and out-of-range data packets; The target memory block of the pre-initialized shared memory circular array is calculated based on the memory block index formula; The initial data packet is stored in the target memory block.

3. The method according to claim 2, characterized in that, The frame sequence number is filtered to obtain the initial data packet corresponding to the frame sequence number, including: Compare the sequence number of the frame corresponding to the User Datagram Protocol (UDP) data packet with the sequence number of the frame to be processed; If the sequence number of the frame corresponding to the User Datagram Protocol (UDP) data packet is less than the sequence number of the frame to be processed, the data packet corresponding to the sequence number is determined to be an outdated data packet and deleted. If the sequence number of the frame corresponding to the User Datagram Protocol (UDP) data packet is greater than or equal to the sequence number of the frame to be processed, calculate the difference between the sequence number of the UDP data packet and the sequence number of the frame to be processed. If the difference is greater than or equal to the fixed array size, the data packet corresponding to the frame sequence number is determined to be an out-of-range data packet and deleted; If the difference is less than the fixed array size, the initial data packet corresponding to the frame sequence number is obtained.

4. The method according to claim 1, characterized in that, The ready bit atomic write mechanism writes target data packets that meet the write conditions to the data buffer of the target memory block without locking, including: Obtain the target frame sequence number in the target memory block; If the target frame sequence number is the same as the frame sequence number, the data packet corresponding to the target frame sequence number is determined to be a duplicate data packet and deleted. If the target frame sequence number is inconsistent with the frame sequence number, the data packet corresponding to the target frame sequence number is determined to be a target data packet that meets the writing conditions. Write the target frame sequence number of the target data packet that meets the writing conditions into the data buffer of the target memory block without locking.

5. The method according to claim 1, characterized in that, The ready status of the target data packet in the data buffer is determined, and the corresponding operation is performed according to the determination result, including: Calculate the storage location of the target data packet in the data buffer based on the sequence number of the frame to be processed; Obtain the target frame sequence number of the target data packet stored at the storage location; The target frame sequence number and the frame sequence number to be processed are used to determine the ready state; If the target frame sequence number is equal to the frame sequence number to be processed, the result of determining whether the target data packet corresponding to the target frame sequence number is in a valid / ready state is obtained; Based on the determination result that the target data packet is in a valid / ready state, the target data packet corresponding to the target frame sequence number that is in the correct ready state is responded to. If the target frame sequence number is not equal to the frame sequence number to be processed, the judgment result of the data packet of the frame sequence number to be processed not responding is obtained and the data loop processing in the shared memory circular array ends.

6. The method according to claim 5, characterized in that, Responding to the target data packet corresponding to the target frame sequence number that is in the correct and ready state includes: Send a positive response message to the target data packet corresponding to the target frame sequence number that is in the correct ready state; When it is detected that the frame number of the target data packet corresponding to the new correct and ready state target frame number is not equal to the frame number of the data packet of the target memory block, an overwrite and reuse operation is performed.

7. The method according to claim 1, characterized in that, The initialization process of a shared-memory circular array includes: Allocate physical memory for the shared-memory circular array; The physical memory in the shared memory circular array is divided into fixed memory blocks of equal size, and the sequence number of the frame to be processed is initialized to 1.

8. A data stream ordered processing device, characterized in that, The device includes: The first acquisition unit is used to acquire the data stream to be processed; The second acquisition unit is used to acquire the frame sequence number corresponding to each User Datagram Protocol (UDP) data packet in the data stream to be processed. A storage unit is used to store the initial data packet corresponding to the frame sequence number into the target memory block of a pre-initialized shared memory circular array; wherein, the shared memory circular array is a circular buffer area constructed from fixed memory blocks with fixed array values; the target memory block is determined by direct indexing of the shared memory circular array; The write unit is used to write target data packets that meet the write conditions into the data buffer of the target memory block without locking through the atomic write mechanism of the ready bit; wherein, the atomic write mechanism of the ready bit is a write mechanism in which the frame sequence number of the data packet is written as the ready flag after the data packet is written into the data buffer of the target memory block. The judgment operation unit is used to judge the ready status of the target data packet in the data buffer and execute the corresponding operation according to the judgment result until the processing of all user datagram protocol data packets in the data stream to be processed is completed.

9. A storage medium, characterized in that, The storage medium includes stored instructions, wherein, when the instructions are executed, the device containing the storage medium is controlled to perform the data stream ordered processing method as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes a memory, and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors as described in any one of claims 1 to 7.

Images