A method and system for desensitizing drive test data

Abstract

The present invention belongs to the field of high-precision map production, and specifically provides a method and system for desensitizing road test data, wherein the method includes S1, parsing road test data to form a multidimensional structure array, and obtaining the specific position of the timestamp and the relationship between the timestamp and other multidimensional arrays; S2, finding the retrieval timestamp corresponding to the sensitive information in the road test data through a universal player; S3, synchronously retrieving the timestamps in each array through the retrieval timestamp, if a time stamp is consistent with the retrieval timestamp, then recording the timestamp as a sensitive timestamp, and the corresponding relationship between the timestamp and other multidimensional arrays; S4, removing the data content corresponding to the sensitive timestamp segment and then reorganizing to obtain desensitized data. This solution can effectively remove and reorganize complex road test data structures, thereby realizing the screening of sensitive information of the original data, and this solution is particularly suitable for MF4 and H5 data.

Description

Technical Field

[0001] The present invention relates to the technical field of high-precision map production, and more specifically, to a method and system for desensitizing road test data. Background Art

[0002] In the field of high-precision map production, road test data is a very important element, but according to topographic maps, it is an important national confidential or secret surveying and mapping result, which belongs to the category of state secrets. Point cloud and H5 data need to be desensitized before they can be given to companies without surveying and mapping qualifications to study sensors and other algorithms. That is, desensitization is very important. At present, general data desensitization technology can only simply desensitize common data formats such as MP4 and AVI, and cannot perform data desensitization on H5 and MF4 (point cloud data). As we all know, data desensitization actually removes and reorganizes data according to the time axis corresponding to sensitive scenes. The data structure of H5 and MF4 in the automotive industry is very complex, and the road test data structure of each company is different, which is similar to black box information. Therefore, it is very important to desensitize the MF4 and H5 data that need to be used for road tests and comply with national laws. Summary of the invention

[0003] The present invention aims to solve the technical problem in the prior art that point cloud data cannot be desensitized.

[0004] The present invention provides a method for desensitizing drive test data, comprising the following steps:

[0005] S1, parsing the drive test data to form a multi-dimensional structure array, and obtaining the specific location of the timestamp and the relationship between it and other multi-dimensional arrays;

[0006] S2, finding the retrieval timestamp corresponding to the sensitive information in the drive test data through a universal player;

[0007] S3, synchronously searching the timestamps in the arrays through the search timestamp, and if a timestamp is consistent with the search timestamp, recording the timestamp as a sensitive timestamp, and the corresponding relationship between the timestamp and other multidimensional arrays;

[0008] S4, remove the data content corresponding to the sensitive timestamp segment and then reassemble to obtain the desensitized data.

[0009] Preferably, the drive test data is H5 data or MF4 data.

[0010] Preferably, S2 specifically includes: obtaining a retrieval time stamp of a corresponding position of the sensitive location and / or identification in the drive test data obtained by querying on the universal player and writing it into an Excel table.

[0011] Preferably, the S3 specifically includes: synchronously writing the pictures involving sensitive location identifications in the drive test process into a new array by retrieving the timestamp.

[0012] Preferably, the S4 specifically includes: reorganizing the multidimensional array through the pointer index of the new array, judging and skipping the multidimensional array of sensitive information in step S3, and finally obtaining N+1 segments of desensitized coherent drive test data.

[0013] The present invention also provides a drive test data desensitization processing system, wherein the detection and extraction system is used to implement the drive test data desensitization processing method, including:

[0014] A parsing module is used to parse the drive test data to form a multi-dimensional structure array, and obtain the specific location of the timestamp and the relationship between it and other multi-dimensional arrays;

[0015] A retrieval module, used to find a retrieval timestamp corresponding to the sensitive information in the drive test data through a universal player;

[0016] A judgment and screening module, used for synchronously searching the timestamps in the arrays through the search timestamp, and if a timestamp is consistent with the search timestamp, recording the timestamp as a sensitive timestamp and the corresponding relationship between the timestamp and other multidimensional arrays;

[0017] The regeneration module is used to remove the data content corresponding to the sensitive timestamp segment and then reassemble it to obtain the desensitized data.

[0018] The present invention also provides an electronic device, comprising a memory and a processor, wherein the processor is used to implement the steps of the road test data desensitization processing method when executing a computer management program stored in the memory.

[0019] The present invention also provides a computer-readable storage medium on which a computer management program is stored. When the computer management program is executed by a processor, the steps of the road test data desensitization processing method are implemented.

[0020] Beneficial effect: The present invention provides a method and system for desensitizing drive test data, wherein the method includes S1, parsing the drive test data to form a multidimensional structure array, and obtaining the specific position of the timestamp and the relationship between the timestamp and other multidimensional arrays; S2, finding the retrieval timestamp corresponding to the sensitive information in the drive test data through a universal player; S3, synchronously retrieving the timestamps in each array through the retrieval timestamp, and if a time stamp is consistent with the retrieval timestamp, then recording the timestamp as a sensitive timestamp and the corresponding relationship between the timestamp and other multidimensional arrays; S4, removing the data content corresponding to the sensitive timestamp segment and then reorganizing to obtain desensitized data. This solution can effectively remove and reorganize complex drive test data structures, thereby realizing the screening of sensitive information in the original data, and this solution is particularly suitable for MF4 and H5 data. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A flowchart of a drive test data desensitization processing method provided by the present invention;

[0022] Figure 2 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0023] Figure 3 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0024] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0025] Figure 1 The present invention provides a method and system for desensitizing drive test data, wherein the method includes S1, parsing the drive test data to form a multidimensional structure array, and obtaining the specific position of the timestamp and the relationship between the timestamp and other multidimensional arrays; S2, finding the retrieval timestamp corresponding to the sensitive information in the drive test data through a universal player; S3, synchronously retrieving the timestamps in each array through the retrieval timestamp, if a time stamp is consistent with the retrieval timestamp, then recording the timestamp as a sensitive timestamp, and the corresponding relationship between the timestamp and other multidimensional arrays; S4, removing the data content corresponding to the sensitive timestamp segment and then reorganizing to obtain desensitized data. This solution can effectively remove and reorganize complex drive test data structures, thereby realizing the screening of sensitive information in the original data, and this solution is particularly suitable for MF4 and H5 data.

[0026] Specifically, firstly, we deeply analyze the point cloud, H5 and other road test data to obtain a relatively clear multi-dimensional data structure diagram. What does the structure of each dimension in the diagram represent? It can be printed out from the program, and the meaning of this layer of data can be clarified. Find the specific position of the timestamp data in the multi-dimensional data structure diagram, and the position of the array. Use the timestamp data as the time synchronization information of the data in this format, and find the relationship with other dimensional data. According to the found relationship, the corresponding sensitive image information is synchronously processed by processing in time. The data structure after desensitization is to generate coherent and segmented data. It can be seen from the file size that the size of the data before and after desensitization is reasonable.

[0027] The optional solution is for point cloud data, i.e. MF4 data. The specific process of desensitization is as follows:

[0028] 1. First, use an open source general tool to parse this data format tool; then, find the location of the timestamp data in this data structure and the relationship between it and other data; then, record the timestamp of the sensitive location or logo seen on the general player that can view this data, that is, retrieve the timestamp in N segments and put it into an Excel table;

[0029] 2. Synchronize the writing of pictures involving sensitive location identification in the drive test process into the array by retrieving the timestamp. Specifically, the timestamp in the data is retrieved by the recorded timestamp to make a match. If a consistent timestamp is encountered, this timestamp and other data groups are recorded using the corresponding relationship between the timestamp and other arrays.

[0030] 3. Reorganize the multidimensional array through the pointer index of the array, determine the multidimensional array of sensitive information that was skipped in the previous step, and finally obtain N+1 segments of desensitized and coherent road test data.

[0031] Through the above steps, the point cloud and other corresponding H5 or .bag video files are desensitized, which makes up for the fact that other methods only pause the video files by replacing frames, that is, the processed video files cannot be played continuously, and the processed data lacks further research on sensors and other algorithms.

[0032] The optional solution for H5 data is as follows:

[0033] Get the HS file data and index it according to the recorded CSV sensitive time. If it is a sensitive time period, that is, a sensitive timestamp, generate the corresponding n time periods for retention. Reorganize and generate n+1 new H5 files according to the retained time periods, which are the desensitized data.

[0034] The embodiment of the present invention further provides a drive test data desensitization processing system, and the detection and extraction system is used to implement the drive test data desensitization processing method, which specifically includes:

[0035] A parsing module is used to parse the drive test data to form a multi-dimensional structure array, and obtain the specific location of the timestamp and the relationship between it and other multi-dimensional arrays;

[0036] A retrieval module, used to find a retrieval timestamp corresponding to the sensitive information in the drive test data through a universal player;

[0037] A judgment and screening module, used for synchronously searching the timestamps in the arrays through the search timestamp, and if a timestamp is consistent with the search timestamp, recording the timestamp as a sensitive timestamp and the corresponding relationship between the timestamp and other multidimensional arrays;

[0038] The regeneration module is used to remove the data content corresponding to the sensitive timestamp segment and then reassemble it to obtain the desensitized data.

[0039] See also Figure 2 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 2 As shown, an embodiment of the present invention provides an electronic device, including a memory 1310, a processor 1320, and a computer program 1311 stored in the memory 1310 and executable on the processor 1320. When the processor 1320 executes the computer program 1311, the following steps are implemented: S1, parsing drive test data to form a multi-dimensional structure array, and obtaining the specific position of the timestamp and the relationship between the timestamp and other multi-dimensional arrays;

[0040] S2, finding the retrieval timestamp corresponding to the sensitive information in the drive test data through a universal player;

[0041] S3, synchronously searching the timestamps in the arrays through the search timestamp, and if a timestamp is consistent with the search timestamp, recording the timestamp as a sensitive timestamp, and the corresponding relationship between the timestamp and other multidimensional arrays;

[0042] S4, remove the data content corresponding to the sensitive timestamp segment and then reassemble to obtain the desensitized data.

[0043] See also Figure 3 Schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. Figure 3 As shown, this embodiment provides a computer-readable storage medium 1400, on which a computer program 1411 is stored. When the computer program 1411 is executed by a processor, the following steps are implemented: S1, parsing drive test data to form a multi-dimensional structure array, and obtaining the specific position of the timestamp and the relationship between the timestamp and other multi-dimensional arrays;

[0044] S2, finding the retrieval timestamp corresponding to the sensitive information in the drive test data through a universal player;

[0045] S3, synchronously searching the timestamps in the arrays through the search timestamp, and if a timestamp is consistent with the search timestamp, recording the timestamp as a sensitive timestamp, and the corresponding relationship between the timestamp and other multidimensional arrays;

[0046] S4, remove the data content corresponding to the sensitive timestamp segment and then reassemble to obtain the desensitized data.

[0047] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0048] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0049] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0050] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0051] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0052] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0053] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for desensitizing drive test data, characterized in that: The following steps are involved: S1, parsing the drive test data to form a multi-dimensional structure array, and obtaining the specific location of the timestamp and the relationship between it and other multi-dimensional arrays; S2, finding N retrieval timestamps corresponding to the sensitive information in the drive test data through a universal player; S3, synchronously searching the timestamps in the arrays through the search timestamp, and if a timestamp is consistent with the search timestamp, recording the timestamp as a sensitive timestamp, and the corresponding relationship between the timestamp and other multidimensional arrays; S4, after removing the data content corresponding to the sensitive timestamp segment, reorganize to obtain the desensitized data; The S3 specifically includes: synchronously writing the pictures involving sensitive location identifications in the drive test process into a new array by retrieving the timestamp; The S4 specifically includes: reorganizing the multidimensional array through the pointer index of the new array, judging and skipping the multidimensional array of sensitive information in step S3, and finally obtaining N+1 segments of desensitized coherent drive test data.

2. The drive test data desensitization processing method according to claim 1, characterized in that: The drive test data is H5 data or MF4 data.

3. The drive test data desensitization processing method according to claim 1, characterized in that: The S2 specifically includes: obtaining the retrieval timestamp of the corresponding position of the sensitive location and / or identification in the drive test data obtained by querying on the universal player and recording it in an Excel table.

4. A drive test data desensitization processing system, characterized in that: The drive test data desensitization processing system is used to implement the drive test data desensitization processing method according to any one of claims 1 to 3, including: A parsing module is used to parse the drive test data to form a multi-dimensional structure array, and obtain the specific location of the timestamp and the relationship between it and other multi-dimensional arrays; A retrieval module, used to find N retrieval timestamps corresponding to the sensitive information in the drive test data through a universal player; A judgment and screening module, used for synchronously searching the timestamps in the arrays through the search timestamp, and if a timestamp is consistent with the search timestamp, recording the timestamp as a sensitive timestamp and the corresponding relationship between the timestamp and other multidimensional arrays; The regeneration module is used to remove the data content corresponding to the sensitive timestamp segment and then reassemble it to obtain the desensitized data; The judgment and screening module is used to: synchronously write the pictures involving sensitive location identifications in the drive test process into a new array by retrieving the timestamp; The regeneration module is used to reorganize the multidimensional array through the pointer index of the new array, determine the multidimensional array of sensitive information skipped in step S3, and finally obtain N+1 segments of desensitized coherent drive test data.

5. An electronic device, characterized in that: It comprises a memory and a processor, wherein the processor is used to implement the steps of the drive test data desensitization processing method as described in any one of claims 1 to 3 when executing a computer management program stored in the memory.

6. A computer-readable storage medium, characterized in that: A computer management program is stored thereon, and when the computer management program is executed by a processor, the steps of the drive test data desensitization processing method according to any one of claims 1 to 3 are implemented.

Images