Data stream collection method and system based on ring detection in source code dynamic testing

Abstract

The application provides a data stream collection method and system based on loop detection in source code dynamic testing, comprising: collecting data stream from a target machine, the data stream being a mark point sequence data output by a tested program in a source code dynamic testing process; detecting the newly collected data stream based on loop characteristics in real time to identify loop body data and loop times; processing redundant loop body data and recording effective data stream to a local memory. In the data collection process of the source code dynamic testing, the loop detection algorithm is used to detect the data stream in the buffer of the collection system, and a new lossless compression idea for large data stream collection is adopted under the premise of retaining the time sequence characteristics of the data stream. The method is simple, efficient and applicable, can greatly compress the data flow of the large data stream generated in the long-time collection engineering application, and is very suitable for improving the collection effective data amount and collection time of the high-speed collection system under the condition of limited memory and disk resources.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data flow processing, in particular to a data flow collection method and system based on ring detection in source code dynamic testing. BACKGROUND

[0002] At present, many technologies related to data or data flow processing compression are basically aimed at streaming media or general compression algorithms, and the methods used are too complex or are specific to application scenarios. These methods are not suitable for collecting data sources in source code dynamic testing applications. The data sources in source code dynamic testing are mainly marker point information inserted in the source code control flow graph. These marker point information will be output to the collection system device in real time according to a certain timing during the long execution of the tested source code. These data have large amounts of data and a large number of repeated data pieces.

[0003] The patent document CN201811029895.6 discloses a streaming media video compression method based on inter-frame correlation, which includes: a video compression sensing coding based on bidirectional prediction; the process is as follows: the inter-frame correlation and intra-frame correlation of the video are introduced at the video encoding end to divide the reasonable sampling rate for the block images of the non-reference frame; first, the same projection matrix is used to obtain the measurement vectors of the corresponding image blocks of the reference frame and the non-reference frame, and the image blocks are divided into three types according to the energy difference value: substantially unchanged blocks, slowly changing blocks and rapidly changing blocks; second, the intra-frame correlation is used to correct the preliminary classification result; then the final classification result is obtained, and different sampling rates are allocated to different types of image blocks for sampling processing, and the number of measurement vector points of the current image block is adaptively adjusted to obtain the measurement vector.

[0004] The patent document CN202211018311.1 discloses a data cross-process compression storage method and system based on big data, which solves the problem that the existing cross-process data transmission method cannot determine the priority of object data and performs compression transmission based on the priority, thereby increasing the load of the cross-process data transmission method; the method comprises the following steps: acquiring a set of tasks to be compressed, loading a target compression task set, determining the priority order of the target compression task set in the compression queue by a model, forming a sorted compression queue, and performing compression processing on the sorted compression queue by at least one group of service processes to obtain a set of data compressed and stored; the priority order of the target compression task set in the compression queue is determined by a priority determination model to form a sorted compression queue, and the data is processed based on the priority of different data collection terminals to improve the efficiency of data compression and storage.

[0005] The above two schemes are too complex and not suitable for collecting data sources in source code dynamic testing applications. SUMMARY

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a data stream acquisition method and system based on loop detection in dynamic source code testing.

[0007] A data stream acquisition method based on loop detection in dynamic source code testing according to the present invention includes the following steps:

[0008] Step S1: Collect data stream from the target machine. The data stream is the sequence of marker points output by the program under test during the source code dynamic testing process.

[0009] Step S2: Real-time detection of newly acquired data streams based on loop features to identify loop body data and loop count;

[0010] Step S3: Process redundant loop body data and record the valid data stream to local memory.

[0011] Preferably, the marker points are instrumented based on the control flow diagram of the source code being tested.

[0012] Preferably, step S2 includes:

[0013] Step S201: Determine data D i Is it duplicate data? If the result is yes, then record D. i Last position and D i Current position; if the judgment result is negative, record D. i Map the current position;

[0014] Step S202: Determine subsequent data D i+k If the data is duplicated, and the result is no, then set the loop termination state; if the result is yes, then determine whether to enter the loop: Condition 1: Data D i+k Current position minus D i Current position equals data D i+k Last occurrence position minus D i Last occurrence position; Condition 2: Data D i+k Current position minus D i Current position equals data D i Current position minus D i Last location;

[0015] If condition two is met, start the loop and initialize the current loop start position and loop count.

[0016] Step S203: After starting the loop, process the subsequent data D i+k+jIf the condition in step S202 is not met, the loop is not established, and the loop is re-established to enter step S201, otherwise, the step S204 is entered to analyze the length of the loop and the number of repetitions in the loop mode;

[0017] Step S204: If the data D i+k+j The current position minus the current loop start position plus 1 equals the data D i The current position minus the data D i The last position indicates the end of a loop, and the current loop number is incremented by 1; if the data D i+k+j The current position minus the current loop start position equals the data D i The current position minus the data D i The last position indicates the start of a new loop, and the current loop start position is updated to D i+k+j The current position.

[0018] Preferably, the step S3 comprises:

[0019] Step S301: In the non-loop start state, record the data stream to the local memory;

[0020] Step S302: In the loop start state, when the current loop number is 0, the loop start position data is not recorded to the local memory;

[0021] Step S303: In the loop mode, if the subsequent position data passes the loop judgment, the data is still in the loop and is not recorded to the local memory; otherwise, the loop judgment is not passed, the loop number and the accumulated non-loop data are recorded to the local memory in the non-loop start state, and the step S301 is entered.

[0022] Preferably, before the collection is completed, if in the loop mode, the loop number and the accumulated non-complete loop data are recorded to the local memory.

[0023] According to the data stream collection system based on loop detection in the source code dynamic test provided by the application, comprising the following modules:

[0024] Data collection module: collecting data stream from the target machine, wherein the data stream is the mark point sequence data output by the tested program in the source code dynamic test process;

[0025] Data recognition module: detecting the newly collected data stream based on the loop characteristics in real time, and recognizing the loop body data and the loop number;

[0026] Data processing module: processing the redundant loop body data, and recording the effective data stream to the local memory.

[0027] Preferably, the mark point is inserted based on the control flow diagram of the tested source code.

[0028] Preferably, the data recognition module comprises:

[0029] An initial judgment module: judging whether the data D i is repeated data, if the result is yes, recording D i last position and D i current position; if the result is no, recording D i mapping current position;

[0030] A loop start judgment module: judging whether the subsequent data D i+k is repeated data, if the result is no, setting a loop end state; if the result is yes, judging whether to enter a loop: condition one: data D i+k current position minus D i current position equals data D i+k last occurrence position minus D i last occurrence position; condition two: data D i+k current position minus D i current position equals data D i current position minus D i last occurrence position;

[0031] if condition two is met, starting a loop, initializing a current loop start position and a loop number;

[0032] A loop detection judgment module: after starting a loop, judging whether the subsequent data D i+k+j satisfies a loop detection condition: if not, the loop is invalid, re-establishing the loop to make an initial judgment, otherwise, continuing to analyze the loop length and the repetition number under the loop mode;

[0033] A loop update module: if data D i+k+j current position minus current loop start position plus 1 equals data D i current position minus data D i last position, it means that a loop ends, the current loop number is increased by 1; if data D i+k+j current position minus current loop start position equals data D i current position minus data D i last position, it means that a new loop starts, the current loop start position is updated to D i+k+j current position.

[0034] Preferably, the data processing module comprises:

[0035] In a non-loop start state, recording data streams to a local memory;

[0036] In the loop start state, when the current loop number is 0, the loop start position data is not recorded to the local memory;

[0037] In the loop mode, if the subsequent position data passes the loop judgment, the data is still in the loop and is not recorded to the local memory; otherwise, the loop number and the accumulated non-loop data are recorded to the local memory in the non-loop start state.

[0038] Preferably, before the collection ends, if in the loop mode, the loop number and the accumulated non-complete loop data are recorded to the local memory.

[0039] Compared with the prior art, the present application has the following beneficial effects:

[0040] The present application solves the problem of improving the collection time of the collection system and the engineering application ability of large data collection in special scenarios under the premise of preserving the time sequence characteristics of the data stream by using the loop detection based data stream collection system and method. BRIEF DESCRIPTION OF DRAWINGS

[0041] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments with reference to the attached drawings:

[0042] Figure 1 Flow chart of the loop detection based data stream collection method in the source code dynamic test.

[0043] Figure 2 Flow chart of the loop detection based data stream collection method in the source code dynamic test. DETAILED DESCRIPTION

[0044] The present application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present application. These all belong to the protection scope of the present application.

[0045] As shown in Figure 1 and 2 A loop detection based data stream collection method and system in a source code dynamic test are provided according to the present application, the method comprising the following steps: S1: collecting a data stream from a target machine, the data stream being a mark point sequence data output by a tested program in a source code dynamic test process; the mark point being inserted based on a control flow graph of the tested source code; S2: detecting the newly collected data stream based on loop characteristics in real time to identify loop body data and loop number; specifically comprising: S201: judging whether the data D is in the loop; S202: if the data D is in the loop, recording the loop number and the loop body data; S203: if the data D is not in the loop, recording the data D to the local memory.i whether the data is repetitive, if the result is yes, record D i last position and D i current position; if the result is no, record D i map the current position; step S202: judge the subsequent data D i+k whether the data is repetitive, if the result is no, set the loop end state; if the result is yes, determine whether to enter the loop: condition one: data D i+k current position minus D i current position equals data D i+k last occurrence position minus D i last occurrence position; condition two: data D i+k current position minus D i current position equals data D i current position minus D i last occurrence position; if condition two is met, start the loop, initialize the current loop start position and the number of loops; step S203: after starting the loop, the subsequent data D i+k+j is judged: if condition one in step S202 is not met, the loop is not established, and the loop entry step S201 is re-established, otherwise continue to step S204 to analyze the loop length and the number of repetitions in the loop mode; step S204: if data D i+k+j current position minus current loop start position plus 1 equals data D i current position minus data D i last position indicates the end of a loop, and the current loop number is incremented by 1; if data D i+k+j current position minus current loop start position equals data D i current position minus data D i last position indicates the start of a new loop, and the current loop start position is updated to D i+k+j current position. Step S3: process redundant loop body data and record valid data stream to local memory, specifically including: step S301: in a non-loop start state, record the data stream to the local memory; step S302: in a loop start state, when the current loop number is 0, the loop start position data is not recorded to the local memory; step S303: in the loop mode, if the subsequent position data passes the loop judgment, the data is still in the loop and is not recorded to the local memory; otherwise, it does not pass the loop judgment, enters the non-loop start state, records the loop number and the accumulated non-loop data to the local memory, and continues to step S301. Before the collection is completed, if in the loop mode, the loop number and the accumulated non-complete loop data are recorded to the local memory.

[0046] The application also provides a data stream collection system based on loop detection in source code dynamic testing, comprising a data collection module: collecting data stream from a target machine, wherein the data stream is a mark point sequence data output by a tested program in a source code dynamic testing process; the mark point is inserted based on a control flow chart of the tested source code; a data identification module: detecting the newly collected data stream based on loop features, identifying loop body data and loop times; and a data processing module: processing redundant loop body data and recording effective data stream to a local memory.

[0047] The data identification module comprises an initial judgment module: judging whether the data D i is repeated data, if the judgment result is yes, recording D i last position and D i current position; if the judgment result is no, recording D i mapping current position; a loop start judgment module: judging whether the subsequent data D i+k is repeated data, if the judgment result is no, setting a loop end state; if the judgment result is yes, judging whether to enter a loop: condition one: data D i+k last position minus D i last position equals data D i+k last position minus D i last position; condition two: data D i+k last position minus D i last position equals data D i last position minus D i last position; if condition two is met, starting a loop, initializing a current loop start position and loop times; a loop detection judgment module: after starting a loop, judging the subsequent data D i+k+j ; if the loop condition is not met, the loop is not established, re-establishing the loop to make an initial judgment, otherwise, continuing to analyze the loop length and repetition times in a loop mode; a loop updating module: if data D i+k+j last position minus current loop start position plus 1 equals data D i last position minus data D i last position, it indicates that a loop is ended, and the current loop times are added by 1; if data D i+k+j last position minus current loop start position equals data D i last position minus data D i last position, it indicates that a new loop is started, and the current loop start position is updated to D i+k+j current position.

[0048] The data processing module comprises: in a non-cyclic opening state, recording data flow to local memory; in a cyclic opening state, when the current cycle number is 0, the cycle start position data is not recorded to the local memory; in the cycle mode, if the subsequent position data passes the cycle judgment, the data is still in the cycle and is not recorded to the local memory; otherwise, the cycle judgment is not passed, the cycle number and the accumulated non-cyclic data are recorded to the local memory. Before the acquisition is completed, if in the cycle mode, the cycle number and the accumulated non-complete cycle data are recorded to the local memory.

[0049] In more detail, the data flow acquisition method based on loop detection in the source code dynamic test provided by the application comprises the following steps: 1: the acquisition system acquires data flow [D0D1…D i ] from a target machine, the data flow being a sequence of mark points output by a tested program in a source code dynamic test process, the mark points being inserted based on a control flow graph of the tested source code; 2: detecting the newly acquired data flow [D i+1 D i+2 …D i+k …D i+k+j …] based on loop characteristics in real time, identifying loop body data and cycle number; 3: the acquisition system processes redundant continuous loop data and records effective data flow to local memory.

[0050] Specifically, the step 2 comprises the following steps:

[0051] Step S201: judging whether the data D i is repeated data, if yes, recording D i last position and D i current position, otherwise, recording D i mapping current position.

[0052] Step S202: if the subsequent D i+k is also repeated data (k=1, 2, 3, 4, …), judging whether to enter a cycle: condition one, data D i+k current position minus D i current position is equal to data D i+k last occurrence position minus D i last occurrence position; condition two, data D i+k current position minus D i current position is equal to data D i current position minus D i last occurrence position; if condition two is satisfied, the cycle is started, and the current cycle start position and cycle number are initialized.

[0053] Step S203: after the cycle is started, the subsequent data D i+k+jIf the condition one is not met in step S202, the loop is not established and needs to be re-established to enter step S201, otherwise, continue to enter step S204 to analyze the length of the loop and the number of repetitions in the loop mode.

[0054] Step S204: If condition three data D i+k+j The current position minus the current loop start position plus 1 equals data D i The current position minus data D i The last position indicates the end of a loop, and the current loop number is incremented by 1; if condition four data D i+k+j The current position minus the current loop start position equals data D i The current position minus data D i The last position indicates the start of a new loop, and the current loop start position is updated to D i+k+j The current position.

[0055] For example:

[0056]

[0057] The step 3 comprises the following steps:

[0058] Step S301: In the non-loop start state, record the data stream to the local memory.

[0059] Step S302: In the loop start state, when the current loop number is 0, the loop start position data is not recorded to the local memory.

[0060] Step S303: In the loop mode, if the subsequent position data passes the loop judgment, the data is still in the loop and is not recorded to the local memory, otherwise, it does not pass the loop judgment and enters the non-loop start state, and the loop number and the accumulated non-loop data are recorded to the local memory, and continue to enter step S301.

[0061] Step S304: Before the acquisition system acquisition is completed, if in the loop mode, the loop number and the accumulated non-complete loop data are recorded to the local memory.

[0062] In the data acquisition process of the source code dynamic test, the loop detection algorithm is used to detect the data stream in the buffer of the acquisition system, and a new idea of lossless compression for large data stream acquisition is provided on the premise of preserving the time sequence characteristics of the data stream. The technical method is simple, efficient and applicable, and can greatly compress the data flow of the large data stream generated in the long-time acquisition engineering application, and is very suitable for improving the effective data amount and acquisition time of the high-speed acquisition system under the condition of limited memory and disk resources.

[0063] The specific embodiments of the present application are described above. It needs to be understood that the present application is not limited to the specific embodiments described above, and various changes or modifications can be made by those skilled in the art within the scope of the claims, which does not affect the essential content of the present application. The embodiments of the present application and the features in the embodiments can be combined with each other at will without conflict.

Claims

1. A data stream acquisition method based on loop detection in dynamic source code testing, characterized in that, Includes the following steps: Step S1: Collect data stream from the target machine. The data stream is the sequence of marker points output by the program under test during the source code dynamic testing process. Step S2: Real-time detection of newly acquired data streams based on loop features to identify loop body data and loop count; Step S3: Process redundant loop body data and record valid data streams to local memory; Step S2 includes: Step S201: Determine data D i Is it duplicate data? If the result is yes, then record D. i Last position and D i Current position; if the judgment result is negative, record D. i Map the current position; Step S202: Determine subsequent data D i+k If the data is duplicated, and the result is no, then set the loop termination state; if the result is yes, then determine whether to enter the loop: Condition 1: Data D i+k Current position minus D i Current position equals data D i+k Last occurrence position minus D i Last occurrence position; Condition 2: Data D i+k Current position minus D i Current position equals data D i Current position minus D i Last location; If condition two is met, start the loop and initialize the current loop start position and loop count. Step S203: After starting the loop, process the subsequent data D i+k+j Perform loop detection and judgment: If condition one in step S202 is not met, the loop is not valid, the loop is reconstructed and the process proceeds to step S201; otherwise, continue to step S204 to analyze the loop length and number of repetitions under the loop pattern. Step S204: If data D i+k+j The current position minus the current loop start position plus 1 equals the data D. i Current position minus data D i The previous position indicates the end of one loop, and the current loop count is incremented by 1; if data D i+k+j The current position minus the current loop start position equals the data D. i Current position minus data D i The previous position indicates the start of a new loop; the current loop start position is updated to D. i+k+j Current location.

2. The data stream acquisition method based on loop detection in dynamic source code testing according to claim 1, characterized in that, The marker points are instrumented based on the control flow diagram of the source code being tested.

3. The data stream acquisition method based on loop detection in dynamic source code testing according to claim 1, characterized in that, Step S3 includes: Step S301: In the non-loop start state, record the data stream to local memory; Step S302: When the loop is enabled, if the current loop count is 0, the data at the start of the loop is not recorded in the local memory; Step S303: In the loop mode, if the subsequent position data passes the loop determination, the data is still in the loop and is not recorded in the local memory; otherwise, it fails the loop determination, enters the non-loop enabled state, records the loop count and the accumulated non-loop data in the local memory, and continues to step S301.

4. The data stream acquisition method based on loop detection in dynamic source code testing according to claim 3, characterized in that, Before the data collection ends, if in loop mode, the number of loops and the accumulated incomplete loop data will be recorded in local memory.

5. A data stream acquisition system based on loop detection in dynamic source code testing, characterized in that, Includes the following modules: Data acquisition module: Acquires data streams from the target machine, wherein the data streams are the sequence of marker points output by the program under test during the source code dynamic testing process; Data recognition module: Real-time detection of newly acquired data streams based on loop features, identifying loop body data and loop count; Data processing module: processes redundant loop data and records valid data streams to local memory; The data recognition module includes: Initial judgment module: Judgment data D i Is it duplicate data? If the result is yes, then record D. i Last position and D i Current position; if the judgment result is negative, record D. i Map the current position; The judgment module is activated repeatedly: it judges subsequent data D. i+k If the data is duplicated, and the result is no, then set the loop termination state; if the result is yes, then determine whether to enter the loop: Condition 1: Data D i+k Current position minus D i Current position equals data D i+k Last occurrence position minus D i Last occurrence position; Condition 2: Data D i+k Current position minus D i Current position equals data D i Current position minus D i Last location; If condition two is met, start the loop and initialize the current loop start position and loop count. Loop detection and judgment module: After starting the loop, it checks subsequent data D. i+k+j Perform loop detection and judgment: if the loop condition is not met, the loop is invalid, the loop is reconstructed and the initial judgment is performed; otherwise, continue to analyze the loop length and repetition count under the loop pattern. Circular update module: If data D i+k+j The current position minus the current loop start position plus 1 equals the data D. i Current position minus data D i The previous position indicates the end of one loop, and the current loop count is incremented by 1; if data D i+k+j The current position minus the current loop start position equals the data D. i Current position minus data D i The previous position indicates the start of a new loop; the current loop start position is updated to D. i+k+j Current location.

6. The data stream acquisition system based on loop detection in dynamic source code testing according to claim 5, characterized in that, The marker points are instrumented based on the control flow diagram of the source code being tested.

7. The data stream acquisition system based on loop detection in dynamic source code testing according to claim 5, characterized in that, The data processing module includes: In non-looping mode, the recorded data stream is stored in local memory; When the loop is active and the current loop count is 0, the data at the beginning of the loop is not recorded in local memory. When entering loop mode, if subsequent position data passes the loop determination, the data will remain in the loop and will not be recorded in local memory; otherwise, if the loop determination is not passed, the non-loop enabled state will be entered, and the loop count and accumulated non-loop data will be recorded in local memory.

8. The data stream acquisition system based on loop detection in dynamic source code testing according to claim 7, characterized in that, Before the data collection ends, if in loop mode, the number of loops and the accumulated incomplete loop data will be recorded in local memory.

Images