Data detection system and method with acquisition and transmission characteristics

Abstract

The invention discloses a data detection system and method with collection and transmission characteristics, and relates to the field of data detection and transmission, and the method comprises the steps: constructing a first data frame sequence of a target service at N collection timestamps, constructing a second data frame sequence according to the original data frames of the first data frame sequence, aligning the first data frame sequence and the second data frame sequence, constructing N data frame sequence pairs, identifying a plurality of dislocation data frames in the corresponding ternary data frames based on the N data frame sequence pairs, constructing a standard sequence number mapping table according to the plurality of dislocation data frames, constructing a third data frame sequence based on the standard sequence number mapping table, and identifying the third data frame sequence according to the standard sequence number mapping table. According to the method, the target positions of the misplaced data frames are traced based on the original collection sequence numbers, the correct arrangement sequence of the misplaced data frames is reconstructed, recognition and recovery of the disordered data frames are achieved, and the accuracy of data transmission is remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data detection transmission, in particular to a data detection system and method with acquisition and transmission characteristics. BACKGROUND

[0002] In a distributed data acquisition and real-time transmission system, the sequential integrity of data frames directly affects the accuracy of subsequent processing of target services. For example, in high-timeliness scenarios such as real-time sensor acquisition, audio / video stream transmission, or industrial control signal reporting, if the arrival order of data frames is disordered due to network jitter, multi-path routing, or buffer delay, it will cause serious problems such as state misjudgment, picture tearing, or control instruction misplacement.

[0003] Existing technologies usually rely on the sequence number mechanism of the transmission layer protocol (such as TCP) or the sorting method based on the arrival timestamp of the receiving end to restore the data sequence. For example, the document with the existing technology publication number CN119675822A discloses a modbus data transmission anti-disorder method, device, equipment and medium based on a lora module, which can solve the problem of packet loss in production applications.

[0004] However, such existing technologies have obvious limitations, i.e., relying only on sequence numbers or timestamps cannot determine whether the data frame at a certain position is truly "belonging to the position", especially in the case of packet loss retransmission, clock drift, or intermediate node buffer disturbance, the wrong frame may be mistakenly considered as correctly arrived. SUMMARY

[0005] In view of the deficiencies of the prior art, the present application provides a data detection system and method with acquisition and transmission characteristics, which solves the technical problems proposed in the background art by introducing data frame disorder detection based on content consistency as a criterion.

[0006] To achieve the above purpose, the present application is realized by the following technical scheme:

[0007] In a first aspect, the present application discloses a data detection method with acquisition and transmission characteristics, comprising the following steps:

[0008] S1, constructing a first data frame sequence of a target service at N acquisition time stamps; wherein the first data frame sequence represents the time sequence of the original data frames;

[0009] S2, constructing a second data frame sequence of the target service at N arrival time stamps according to the original data frames of the first data frame sequence; wherein the second data frame sequence represents the time sequence of N three-element data frames;

[0010] S3, aligning the first data frame sequence and the second data frame sequence to construct N data frame sequence pairs;

[0011] S4, identifying a plurality of misaligned data frames in the corresponding triadic data frames of the N data frame sequences pair;

[0012] S5, constructing a standard sequence number mapping table according to the plurality of misaligned data frames;

[0013] S6, constructing a third data frame sequence based on the standard sequence number mapping table, and adaptively deriving the standard timing data frame based on the reordering frame number of the third data frame sequence.

[0014] In some specific embodiments, constructing the first data frame sequence of the target service at N acquisition time stamps comprises:

[0015] S11, obtaining the original data frame of the target service at N acquisition time stamps;

[0016] S12, assigning N monotonically increasing acquisition sequence numbers to N acquisition time stamps according to the time order of N acquisition time stamps;

[0017] S13, arranging the original data frame in ascending order according to the N monotonically increasing acquisition sequence numbers to generate the first data frame sequence.

[0018] In some specific embodiments, constructing the second data frame sequence of the target service at N arrival time stamps comprises:

[0019] S21, extracting the payload of the original data frame, and encapsulating the payload with its corresponding acquisition time stamp and acquisition sequence number into a triadic data frame;

[0020] S22, traversing the original data frame of N acquisition time stamps, repeatedly performing the encapsulation until N triadic data frames are generated;

[0021] S23, sending the triadic data frame to the receiving end in sequence based on the order of N acquisition sequence numbers; wherein a buffer area for temporarily storing the triadic data frame is established in the receiving end;

[0022] S24, recording the N arrival time stamps of the triadic data frame in the buffer area;

[0023] S25, assigning N monotonically increasing first arrival sequence numbers to N arrival time stamps according to the time order of N arrival time stamps;

[0024] S26, arranging the triadic data frame in ascending order according to the N monotonically increasing first arrival sequence numbers to generate the second data frame sequence;

[0025] In some specific embodiments, aligning the first data frame sequence and the second data frame sequence to construct the N data frame sequence pair comprises:

[0026] S31, in the first data frame sequence, anchor the collection sequence number and its original data frame by sequence number;

[0027] S32, in the second data frame sequence, anchor the first arrival sequence number and its ternary data frame by sequence number

[0028] S33, if the collection sequence number and the first arrival sequence number are the same, pair the corresponding original data frame and the ternary data frame, and construct the data frame sequence pair;

[0029] The data frame sequence pair is represented as (P i s , P i r ), wherein:

[0030] P i s represents the original data frame corresponding to the i-th collection sequence number in the first data frame sequence;

[0031] P i r represents the ternary data frame corresponding to the i-th first arrival sequence number in the second data frame sequence.

[0032] In some specific embodiments, based on N data frame sequence pairs, a plurality of misplaced data frames are identified in the corresponding ternary data frames, including:

[0033] S41, anchor any data frame sequence pair, and extract its original data frame and ternary data frame;

[0034] S42, calculate the content similarity of the original data frame and the ternary data frame;

[0035] S43, if the content similarity is less than the consistency threshold, identify the corresponding ternary data frame in the second data frame sequence as a misplaced data frame, otherwise, mark it as a synchronous data frame.

[0036] In some specific embodiments, calculating the content similarity of the original data frame and the ternary data frame includes:

[0037] S421, extract the payload in the original data frame at the i-th expected position of the first data frame sequence;

[0038] S422, convert the extracted payload of the original data frame into a first feature vector through a hash function;

[0039] S423, extract the payload in the ternary data frame at the i-th real position of the second data frame sequence;

[0040] S424, convert the extracted payload of the ternary data frame into a second feature vector through a hash function;

[0041] S425, calculate the content similarity of the first feature vector and the second feature vector.

[0042] In some specific embodiments, a standard sequence number mapping table is constructed according to a plurality of misaligned data frames, including:

[0043] S51, for the misaligned data frame, anchoring the ternary data frame in which it is located;

[0044] S52, extracting the pre-encapsulation collection sequence number in the ternary data frame;

[0045] S53, defining the pre-encapsulation collection sequence number as the target arrival sequence number of the misaligned data frame;

[0046] S54, replacing the first arrival sequence number of the misaligned data frame in the second data frame sequence based on the second arrival sequence number;

[0047] S55, traversing the data frame sequence pair, repeatedly replacing the first arrival sequence number of the misaligned data frame until the sequence numbers of all misaligned data frames are replaced;

[0048] S56, aggregating the first arrival sequence number of the synchronous data frame and the second arrival sequence number of the misaligned data frame to generate a standard sequence number mapping table.

[0049] In some specific embodiments, adaptive derivation of standard timing data frames is performed based on the number of reordered frames of the third data frame sequence, including:

[0050] S61, based on the standard sequence number mapping table, reordering N ternary data frames in the second data frame sequence to generate a third data frame sequence;

[0051] S62, calculating the number of reordered frames of the third data frame sequence in the buffer;

[0052] S63, obtaining the number of data frames expected to be generated by the target service within a service period;

[0053] S64, comparing the number of expected generated data frames with the minimum derivation number of the buffer;

[0054] S65, if the number of expected generated data frames is less than the minimum derivation number of the buffer, then the third data frame sequence is derived at once when the number of reordered frames is equal to the number of expected generated data frames;

[0055] S66, if the number of expected generated data frames is not less than the minimum derivation number of the buffer, then the third data frame sequence is derived immediately when the number of reordered frames is greater than or equal to the minimum derivation number of the buffer, and the reordering continues until the data period of the target service ends.

[0056] The application provides a data detection method with collection transmission characteristics, which has the following beneficial effects:

[0057] The application forms a comparable structure between the ideal sequence and the actual sequence by constructing the first data frame sequence of the sending end and the second data frame sequence of the receiving end, accurately identifies the sequence dislocation caused by network jitter, multi-path transmission or buffer delay by using the consistency judgment of the data frame content similarity, traces the target position of the dislocated data frame based on the original collection sequence number, reconstructs the correct arrangement sequence, realizes the identification and recovery of the disordered data frame, and significantly improves the accuracy of data transmission.

[0058] Further, the export strategy of the third data frame sequence is adaptively adjusted according to the comparison between the expected number of data frames generated in the business cycle and the minimum export number: for short-cycle small-batch business, the complete frame reordering is completed and then the data frames are exported at one time, so as to guarantee the timing integrity; for long-cycle or large-data-volume business, the data frames are output immediately after the minimum export number is met, so as to balance the data integrity and real-time requirement, and improve the response efficiency in diversified business scenarios.

[0059] In a second aspect, the application provides a data detection system with collection transmission characteristics, which is used for executing the detection system of any one of the first aspect, and the detection system comprises:

[0060] A first sequence unit is configured to construct a first data frame sequence of a target business at N collection time stamps; wherein the first data frame sequence represents the time sequence of the original data frames;

[0061] A second sequence unit is configured to construct a second data frame sequence of the target business at N arrival time stamps according to the original data frames of the first data frame sequence; wherein the second data frame sequence represents the time sequence of the N ternary data frames;

[0062] A sequence pair unit is configured to align the first data frame sequence and the second data frame sequence, and construct N data frame sequence pairs;

[0063] A dislocation identification unit is configured to identify a plurality of dislocated data frames in the corresponding ternary data frames based on the N data frame sequence pairs;

[0064] A mapping construction unit is configured to construct a standard sequence number mapping table according to the plurality of dislocated data frames;

[0065] A reordering unit is configured to construct a third data frame sequence based on the standard sequence number mapping table, and adaptively export the standard timing data frames based on the reordering frame number of the third data frame sequence.

[0066] Compared with the prior art, the data detection system with the collection and transmission characteristics has the same beneficial effects as the data detection method with the collection and transmission characteristics, and thus will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 A flowchart of the data detection method with the collection and transmission characteristics is shown.

[0068] Figure 2 A flowchart of the construction of the data frame sequence pair is shown.

[0069] Figure 3 A flowchart of the identification of the error data frame is shown.

[0070] Figure 4 A flowchart of the generation of the standard sequence number mapping table is shown.

[0071] Figure 5 A block diagram of the data detection system with the collection and transmission characteristics is shown. DETAILED DESCRIPTION

[0072] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0073] Embodiment 1: Please refer to Figures 1 to 4 The present application provides a data detection method with collection and transmission characteristics, comprising the following steps:

[0074] S1, constructing a first data frame sequence of a target service at N collection time stamps; wherein the first data frame sequence is characterized as a time sequence of original data frames;

[0075] S2, constructing a second data frame sequence of the target service at N arrival time stamps according to the original data frames of the first data frame sequence; wherein the second data frame sequence is characterized as a time sequence of N three-element data frames;

[0076] S3, aligning the first data frame sequence and the second data frame sequence to construct N data frame sequence pairs;

[0077] S4, identifying a plurality of dislocated data frames in the corresponding three-element data frames based on the N data frame sequence pairs;

[0078] S5, constructing a standard sequence number mapping table according to the plurality of dislocated data frames;

[0079] S6, constructing a third data frame sequence based on the standard sequence number mapping table, and adaptively deriving the standard timing data frame based on a reordering frame number of the third data frame sequence.

[0080] In the embodiment, by constructing the first data frame sequence of the target service at the sending end and the second data frame sequence at the receiving end, the sequence dislocation of the ternary data frame in the transmission process is identified through alignment of the sequences, and the sequence recovery and adaptive derivation of the data frame are realized based on the collection sequence number of the original timing, thereby guaranteeing the transmission integrity of the data stream in the high-time-efficiency target service.

[0081] Exemplarily, the step S1 further includes several specific application steps:

[0082] S11, obtaining original data frames of a target service at N collection time stamps;

[0083] Exemplarily, the target service is real-time sensor data collection, audio / video stream transmission, or industrial control signal reporting; the original data frame represents an unencapsulated data unit generated by the target service. The collection time stamps are provided by a high-precision clock source, forming N collection time stamps.

[0084] S12, according to the time sequence of the N collection time stamps, assigning N monotonically increasing collection sequence numbers thereto;

[0085] Specifically, the original data frame with the smallest collection time stamp is assigned a collection sequence number 1, and then collection sequence numbers 2 to N are sequentially assigned in the time increasing order, forming N collection sequence numbers.

[0086] S13, arranging the original data frames in ascending order according to the N monotonically increasing collection sequence numbers, to generate a first data frame sequence. Specifically, the first data frame sequence represents an ideal transmission sequence of the data frames at the N original data frames at the sending end.

[0087] In the embodiment, the traceable data frame arrangement sequence is established according to the time sequence of the collection time stamps, so that the original data frames are organized into a sequence structure with a clear time sequence, ensuring that the original data frames at the sending end can maintain a consistent timing when output, thereby providing a reference for sequence comparison.

[0088] Exemplarily, the step S2 in the embodiment further includes several specific application steps:

[0089] S21, extracting the payload of the original data frame, and encapsulating the payload, the corresponding collection time stamp, and the collection sequence number as a ternary data frame;

[0090] Specifically, the payload represents a minimum semantic data unit that can represent a target service state, i.e., core service data content generated by the target service, without a transport layer or a link layer encapsulation header; the extraction can be performed by analyzing a protocol format or a preset data offset position of the original data frame. The encapsulation operation includes: attaching the collection timestamp and the collection sequence number as metadata to a frame header or a frame tail of the payload to form a data structure with timing and sequence identification.

[0091] S22, traversing the original data frames of the N collection timestamps, repeatedly performing the encapsulation until N triple data frames are generated;

[0092] Specifically, the triple data frame includes three fields: the payload of the original data frame, the collection timestamp, and the collection sequence number, which supports unpacking and sequence analysis at the receiving end.

[0093] S23, sequentially sending the triple data frame to the receiving end based on the sequence of the N collection sequence numbers; wherein, a buffer area for temporarily storing the triple data frame is established in the receiving end;

[0094] S24, recording N arrival timestamps of the triple data frame in the buffer area;

[0095] S25, according to the time sequence of the N arrival timestamps, assigning N first arrival sequence numbers that are monotonically increasing;

[0096] It should be noted that in this embodiment, the arrival timestamp and the collection timestamp should be consistent in sequence, i.e., the original data frame of the first collection timestamp is first sent after being encapsulated into a triple data frame, and is first received at the receiving end; therefore, the first collection timestamp should correspond to the generation and reception time of the same data frame as the first arrival timestamp.

[0097] Further, the first arrival sequence number and the collection sequence number are assigned using the same sorting logic, i.e., the first arrival sequence number and the collection sequence number are both natural sequence numbers of sequence elements in sequence positions. Therefore, the first arrival sequence number and the collection sequence number correspond one-to-one and have equal values.

[0098] S26, arranging the triple data frame in ascending order according to the N first arrival sequence numbers that are monotonically increasing, to generate a second data frame sequence; specifically, the second data frame sequence represents the actual arrival sequence of data at the receiving end.

[0099] In this embodiment, by encapsulating the payload, the collection timestamp, and the collection sequence number into a triple data frame, the service information and timing information of the original data frame are retained; at the same time, based on the second data frame sequence generated by the arrival timestamp sorting, a data stream based on the actual arrival sequence is formed, providing a comparable premise for identifying misaligned data frames.

[0100] Exemplarily, in the present embodiment, the step S3 further comprises several specific application steps, including:

[0101] S31, in the first data frame sequence, anchoring the acquisition sequence number and its original data frame by sequence number;

[0102] S32, in the second data frame sequence, anchoring the first arrival sequence number and its ternary data frame by sequence number;

[0103] S33, if the acquisition sequence number is the same as the first arrival sequence number, pairing the corresponding original data frame and ternary data frame to construct the data frame sequence pair;

[0104] The data frame sequence pair is represented as (P i s , P i r ), wherein:

[0105] P i s represents the original data frame corresponding to the i-th acquisition sequence number in the first data frame sequence;

[0106] P i r represents the ternary data frame corresponding to the i-th first arrival sequence number in the second data frame sequence.

[0107] Specifically, the data frame sequence pair is used to judge whether the sending data frame and the receiving data frame at the same sequence position are consistent; it should be noted that in the present embodiment, the acquisition timestamp and the sending timestamp are equal in time reference, i.e. both of them are derived from the same high-precision clock source without introducing clock offset or drift. The high-precision clock source can be a GPS synchronous clock, a PTP (IEEE1588) master clock or an atomic clock; the consistent time reference ensures the high alignment of the acquisition time and the packaging / sending time, avoiding the time misplacement caused by the non-synchronization of local clocks.

[0108] In the present embodiment, by pairing the first data frame sequence and the second data frame sequence one by one according to the sequence number position, the same position alignment between the sending order and the receiving order is established; the alignment makes the original data frame and the ternary data frame at the same sequence position can be compared for content consistency, providing a structured basis for the detection of sequence misplacement.

[0109] Exemplarily, in the present embodiment, the step S4 further comprises several specific application steps, including:

[0110] S41, anchoring any data frame sequence pair, extracting its original data frame and ternary data frame;

[0111] S42, calculate the content similarity between the original data frame and the ternary data frame;

[0112] S43, if the content similarity is less than the consistency threshold, identify the corresponding ternary data frame as a misaligned data frame in the second data frame sequence, otherwise, mark it as a synchronous data frame.

[0113] wherein the consistency threshold is expressed as: Sim i <1-∈;wherein ∈=10 6 , the minimum deviation used in hash comparison.

[0114] Specifically, in an ideal case, the i-th collected data frame should be completely consistent in content with the i-th arrived ternary data frame, and the similarity is 1; but in the actual scene, slight differences can be accepted on the premise of allowing signal noise, coding jitter or transmission disturbance; if the content difference is significant, it means that the ternary data frame at the i-th position does not arrive in sequence, and the sequence misalignment occurs.

[0115] In this embodiment, by comparing the content similarity between the original data frame and the ternary data frame in the data frame sequence pair, the content consistency is used as the basis for determining whether the received data frame is misaligned, which can effectively identify the misaligned data frame caused by the sequence disturbance in the transmission process.

[0116] wherein the application step S42 further comprises:

[0117] S421, extract the payload in the original data frame at the i-th expected position of the first data frame sequence;

[0118] S422, convert the extracted payload of the original data frame into a first feature vector through a hash function;

[0119] Exemplarily, the first feature vector can be represented as a fixed-length binary sequence;

[0120] wherein the feature extraction can use MD5 or SHA1 hash algorithm to map the variable-length payload into a 128-bit or 160-bit bit vector;

[0121] S423, extract the payload in the ternary data frame at the i-th real position of the second data frame sequence;

[0122] S424, convert the extracted payload of the ternary data frame into a second feature vector through a hash function;

[0123] S425, calculate the content similarity of the first feature vector and the second feature vector.

[0124] Exemplarily, in this embodiment, the content similarity is preferably the cosine similarity, and its calculation formula is:

[0125]

[0126] wherein, Sim i represents the content similarity of the ith pair of data frame sequences, the value closer to 1 indicates higher content consistency, V s represents the first feature vector corresponding to the ith original data frame, V r represents the second feature vector corresponding to the ith triple data frame; V s ·V r represents the dot product of the first feature vector and the second feature vector, ||V s ||and||V r ||respectively represent the L2 norm of the first feature vector and the second feature vector.

[0127] In this embodiment, by extracting the payloads in the original data frames and the triple data frames and converting them into fixed-length feature vectors, the comparison of content similarity is converted into numerical calculation based on hash features, improving the efficiency of similarity determination.

[0128] Exemplarily, in this embodiment, the step S5 further includes several specific application steps, including:

[0129] S51, for the misaligned data frame, anchoring the triple data frame where it is located;

[0130] S52, extracting the collection sequence number before encapsulation in the triple data frame;

[0131] S53, defining the collection sequence number before encapsulation as the target arrival sequence number of the misaligned data frame;

[0132] S54, based on the second arrival sequence number, replacing the first arrival sequence number thereof in the second data frame sequence;

[0133] S55, traversing the pair of data frame sequences, repeatedly replacing the first arrival sequence number of the misaligned data frame until the sequence numbers of all misaligned data frames are replaced;

[0134] S56, summarizing the first arrival sequence numbers of the synchronous data frames and the second arrival sequence numbers of the misaligned data frames to generate a standard sequence number mapping table.

[0135] In this embodiment, by extracting the collection sequence number carried by the misaligned data frame and taking it as the target arrival sequence number, and combining the original arrival sequence number of the synchronous data frame, a relationship structure reflecting the ideal order of the data frame and the actual position is constructed, realizing the position tracing of the misaligned data frame.

[0136] Exemplarily, in this embodiment, the step S6 further includes several specific application steps, including:

[0137] S61, reorder the N triple data frames in the second data frame sequence based on the standard sequence number mapping table to generate a third data frame sequence;

[0138] S62, calculate the reordered frame number of the third data frame sequence in the buffer;

[0139] Exemplarily, the reordered frame number represents the number of data frames corresponding to the continuous and non-missing target sequence numbers after completing the sequence recovery based on the standard sequence number mapping table. It is not simply "the number of sorted ones", but refers to the number of data frames that have been arranged in the correct logical order and can constitute a complete segment.

[0140] S63, obtain the number of data frames expected to be generated by the target service within the service period;

[0141] Specifically, the number of expected generated data frames represents the total number M of original data frames planned to be generated by the target service in this service period, which can be determined in advance through service configuration parameters, period timers or first frame metadata.

[0142] Exemplarily, the scenario of the number of expected generated data frames can be defined as follows:

[0143] In the scene of real-time sensor collected scene data transmission: trigger sampling once every 10ms, collect 32 frames each time → M=32;

[0144] In the scene of video stream transmission: one GOP (group of pictures) contains 15 frames → M=15;

[0145] In the data transmission scene of industrial control signal reporting: 8 signal frames are reported fixedly every scanning period → M=8;

[0146] S64, compare the number of expected generated data frames with the minimum export number of the buffer;

[0147] S65, if the number of expected generated data frames is less than the minimum export number of the buffer, export the third data frame sequence once when the reordered frame number is equal to the number of expected generated data frames;

[0148] S66, if the number of expected generated data frames is not less than the minimum export number of the buffer, export the third data frame sequence immediately when the reordered frame number is greater than or equal to the minimum export number of the buffer, and continue to perform the reordering until the data period of the target service ends.

[0149] In the embodiment, the three-element data frames are reordered by a standard sequence number mapping table to generate a third data frame sequence conforming to the original collection sequence, and the export timing of the data frames is dynamically determined according to the matching relationship between the service cycle length and the export capability, so that the adaptive control of the data output rhythm in different service scenarios is realized under the premise of guaranteeing the data timing integrity.

[0150] To sum up, the application establishes the first data frame sequence based on the collection time at the sending end, constructs the second data frame sequence based on the actual arrival sequence at the receiving end, and forms the data frame sequence pair that can be compared bit by bit by aligning the sequence number positions of the two sequences. Further, based on the consistency of the content similarity, the position dislocation data frames caused by transmission disturbance are identified, and then the original timing collection sequence number is traced back to reconstruct the correct arrangement sequence. Finally, the export timing of the data frames is dynamically decided in combination with the service cycle, so that the recovery and output of the dislocation data frames are realized under the premise of guaranteeing the timing integrity, and the transmission integrity of the target service in the complex communication environment such as network jitter, multi-path transmission or buffer delay is effectively improved.

[0151] Embodiment 2: refer to Figure 5 The embodiment of the application also provides a data detection system with collection and transmission characteristics, which is used to implement the method embodiments and has been described above. The terms "module", "unit", "sub-unit" and the like used below can be a combination of software and / or hardware that can realize the predetermined function. Although the system described in the following embodiments is preferably realized in software, the realization of hardware or the combination of software and hardware is also possible and conceived.

[0152] As Figure 2 shown, Figure 2 is a structural block diagram of a data detection system with collection and transmission characteristics, which is used to implement the detection system of the method embodiments, and the detection system comprises:

[0153] A first sequence unit is configured to construct a first data frame sequence of the target service at N collection time stamps, wherein the first data frame sequence represents the timing sequence of the original data frames;

[0154] A second sequence unit is configured to construct a second data frame sequence of the target service at N arrival time stamps according to the original data frames of the first data frame sequence, wherein the second data frame sequence represents the timing sequence of the N three-element data frames;

[0155] A sequence pair unit is configured to align the first data frame sequence and the second data frame sequence to construct N data frame sequence pairs;

[0156] A dislocation identification unit is configured to identify a plurality of dislocation data frames in the corresponding three-element data frames based on the N data frame sequence pairs.

[0157] a mapping construction unit configured to construct a standard sequence number mapping table according to the plurality of misaligned data frames;

[0158] a reordering unit configured to construct a third data frame sequence based on the standard sequence number mapping table, and to adaptively derive the standard timing data frames based on a reordering frame number of the third data frame sequence.

[0159] In the system, the first sequence unit constructs the first data frame sequence, the second sequence unit constructs the second data frame sequence, the sequence pair unit constructs the N data frame sequence pairs, the misalignment identification unit identifies the plurality of misaligned data frames, the mapping construction unit constructs the standard sequence number mapping table, and the reordering unit constructs the third data frame sequence and adaptively derives the standard timing data frames based on a reordering frame number of the third data frame sequence, thereby solving the problem of data frame misalignment.

[0160] The above-described embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the above-described embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable systems. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server, or data center to another via wired (such as infrared, wireless, microwave, etc.) or wireless means.

[0161] The above describes only the specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. A data detection method with acquisition and transmission characteristics, characterized in that, include: S1. Construct the first data frame sequence of the target service at N collection timestamps; wherein, the first data frame sequence is represented as the time sequence of the original data frames; S2. Based on the original data frames of the first data frame sequence, construct the second data frame sequence of the target service at N arrival timestamps; wherein, the second data frame sequence is represented as a time sequence of N three-dimensional data frames; S3. Align the first data frame sequence and the second data frame sequence to construct N data frame sequence pairs; S4. Based on N data frame sequence pairs, identify several misaligned data frames in their corresponding three-dimensional data frames; S5. Construct a standard sequence number mapping table based on several misaligned data frames; S6. Based on the standard sequence number mapping table, construct the third data frame sequence, and adaptively export the standard time-series data frames based on the number of reordered frames in the third data frame sequence.

2. The data detection method with acquisition and transmission characteristics according to claim 1, characterized in that, Construct the first data frame sequence of the target service at N collection timestamps, including: S11. Obtain the original data frames of the target service at N collection timestamps; S12. Assign N monotonically increasing collection sequence numbers to the N collection timestamps according to their time order. S13. Arrange the original data frames in ascending order according to the N monotonically increasing acquisition sequence numbers to generate the first data frame sequence.

3. The data detection method with acquisition and transmission characteristics according to claim 2, characterized in that, Construct the second data frame sequence for the target service at N arrival timestamps, including: S21. Extract the payload of the original data frame and encapsulate the payload and its corresponding acquisition timestamp and acquisition sequence number into a three-dimensional data frame. S22. Traverse the original data frames of N collection timestamps and repeat the encapsulation process until N three-dimensional data frames are generated. S23. Based on the order of N acquisition sequence numbers, send three-dimensional data frames to the receiving end sequentially; wherein, the receiving end has a buffer for temporarily storing the three-dimensional data frames. S24. Record N arrival timestamps of the three-dimensional data frame in the buffer; S25. Assign N monotonically increasing first arrival numbers to the N arrival timestamps according to their time order. S26. Arrange the three data frames in ascending order according to the N monotonically increasing first arrival sequence numbers to generate the second data frame sequence.

4. The data detection method with acquisition and transmission characteristics according to claim 3, characterized in that, Align the first data frame sequence and the second data frame sequence to construct N data frame sequence pairs, including: S31. In the first data frame sequence, anchor the acquisition sequence number and its original data frame sequentially. S32. In the second data frame sequence, anchor the first arrival sequence number and its three data frames sequentially. S33. If the acquisition sequence number is the same as the first arrival sequence number, then the corresponding original data frame is paired with the three-dimensional data frame to construct the data frame sequence pair. The data frame sequence pair is represented as (P) i s ,P i r ),in: P i s This represents the original data frame corresponding to the i-th acquisition sequence number in the first data frame sequence; P i r This represents the three-dimensional data frame corresponding to the i-th first arrival sequence number in the second data frame sequence.

5. A data detection method with acquisition and transmission characteristics according to claim 4, characterized in that, Based on N data frame sequence pairs, identify several misaligned data frames in their corresponding three-dimensional data frames, including: S41. Anchor any data frame sequence pair and extract its original data frame and three-dimensional data frame; S42. Calculate the content similarity between the original data frame and the three-dimensional data frame; S43. If the content similarity is less than the consistency threshold, then in the second data frame sequence, the corresponding three-dimensional data frame is identified as a misaligned data frame; otherwise, it is recorded as a synchronized data frame.

6. A data detection method with acquisition and transmission characteristics according to claim 5, characterized in that, Calculating the content similarity between the original data frame and the three-dimensional data frame includes: S421. Extract the payload from the original data frame at the i-th expected position in the first data frame sequence; S422. Convert the payload of the extracted original data frame into a first feature vector using a hash function; S423. Extract the payload from the three-dimensional data frame at the i-th real position in the second data frame sequence; S424. Convert the payload of the extracted three-dimensional data frame into a second feature vector using a hash function; S425. Calculate the content similarity between the first feature vector and the second feature vector.

7. A data detection method with acquisition and transmission characteristics according to claim 1, characterized in that, Based on several misaligned data frames, construct a standard sequence number mapping table, including: S51. For the misaligned data frame, anchor the three-dimensional data frame in which it is located; S52. Extract the acquisition sequence number before encapsulation from the three data frames; S53. Define the acquisition sequence number before encapsulation as the target arrival sequence number of the misaligned data frame. S54. Based on the second arrival sequence number, replace its first arrival sequence number in the second data frame sequence; S55. Traverse the data frame sequence pairs and repeatedly replace the first arrival sequence number of the misaligned data frame until the sequence number of all misaligned data frames has been replaced. S56. Summarize the first arrival sequence number of the synchronized data frames and the second arrival sequence number of the misaligned data frames to generate a standard sequence number mapping table.

8. The data detection method with acquisition and transmission characteristics according to claim 1, characterized in that, Adaptive derivation of standard time-series data frames based on the number of reordered frames in the third data frame sequence, including: S61. Based on the standard sequence number mapping table, reorder the N three-dimensional data frames in the second data frame sequence to generate the third data frame sequence. S62. Calculate the number of reordered frames in the buffer of the third data frame sequence; S63. Obtain the number of data frames that the target service is expected to generate within the service cycle; S64. Compare the expected number of data frames to be generated with the minimum number of data frames to be exported from the buffer; S65. If the expected number of data frames to be generated is less than the minimum number of data frames to be exported from the buffer, then when the number of reordered frames is equal to the expected number of data frames to be generated, the third data frame sequence is exported all at once. S66. If the expected number of data frames to be generated is not less than the minimum number of exported frames in the buffer, then when the number of reordered frames is greater than or equal to the minimum number of exported frames in the buffer, the third data frame sequence is immediately exported, and the reordering continues until the data cycle of the target service ends.

9. A data detection system with acquisition and transmission characteristics, used to perform the detection system according to any one of claims 1 to 8, characterized in that, The detection system includes: The first sequence unit is used to construct the first data frame sequence of the target service at N collection timestamps; wherein, the first data frame sequence is characterized as the time sequence of the original data frames; The second sequence unit is used to construct a second data frame sequence of the target service at N arrival timestamps based on the original data frames of the first data frame sequence; wherein, the second data frame sequence is characterized as a time sequence of N three-dimensional data frames; Sequence pairing unit, used to align the first data frame sequence and the second data frame sequence to construct N data frame sequence pairs; The misalignment detection unit is used to identify several misaligned data frames in their corresponding three-dimensional data frames based on N data frame sequence pairs. The mapping construction unit is used to construct a standard sequence number mapping table based on several misaligned data frames; The reordering unit is used to construct a third data frame sequence based on the standard sequence number mapping table, and to adaptively export standard time-series data frames based on the number of reordered frames in the third data frame sequence.

Images