Sensor data recording method and recording system, electronic equipment and vehicle

Abstract

The invention provides a sensor data recording method and system, electronic equipment and a vehicle. The sensor data recording method comprises the following steps: acquiring distributed data; detecting whether a trigger signal of a trigger event exists in the distributed data or not; when it is detected that the trigger signal exists, determining a trigger moment corresponding to the trigger signal; reading first sensor data of a first preset duration before the triggering moment from the data buffer area, and reading second sensor data of a second preset duration after the triggering moment from the data temporary storage area; and performing format processing on the first sensor data and the second sensor data to obtain a recording file. According to the method, the memory occupancy rate of the obtained recorded file is reduced, the quality of the recorded data is improved, the data redundancy is reduced, and the data transmission time is shortened.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle information processing and transmission, and in particular to a sensor data recording method, a recording system, an electronic device and a vehicle. Background Art

[0002] In the related art, the sensor data recording process includes rolling recording and manual recording. The above methods all have certain defects: in the automatic recording mode, all sensor data need to be stored and reported, and a large part of the sensor data is normal data, which is of little significance for the analysis and troubleshooting of unmanned driving problems, resulting in high information redundancy, which not only occupies memory but also affects the efficiency of information transmission and reception. Manual triggering depends on the driver's control. If an emergency occurs, it is difficult for the driver to trigger the button in time. Even if the driver can trigger the button in time, he can only get the sensor data after the emergency occurs, and it is difficult to trace the cause of the incident. In addition, because the sensor data has a large data structure, it occupies a large storage space and requires a long network transmission time when transmitted.

[0003] Therefore, the existing sensor data recording method has high information redundancy and cannot record in time, resulting in low quality of recorded data. In addition, the sensor data occupies a large amount of memory space and the transmission time is also long, which is a problem that needs to be solved urgently. Summary of the invention

[0004] The present invention aims to improve at least one of the technical problems existing in the prior art.

[0005] Therefore, an object of the present invention is to provide a method for recording sensor data.

[0006] Another object of the present invention is to provide a sensor data recording system.

[0007] Another object of the present invention is to provide an electronic device.

[0008] Another object of the present invention is to provide a computer-readable storage medium.

[0009] Another object of the present invention is to provide a vehicle.

[0010] To achieve the above-mentioned purpose, the first aspect of the present invention provides a method for recording sensor data, the recording method comprising: acquiring distributed data; detecting whether there is a trigger signal of a trigger event in the distributed data; when the presence of a trigger signal is detected, determining the trigger time corresponding to the trigger signal; reading first sensor data of a first preset time length before the trigger time from a data buffer, and reading second sensor data of a second preset time length after the trigger time from a data temporary storage area; and obtaining a recording file after formatting the first sensor data and the second sensor data.

[0011] The sensor data recording method provided by the present invention is applicable to a data recording method for automatically recording sensor data according to a trigger signal during an unmanned driving process. Specifically, firstly, distributed data is obtained, and the distributed data includes a plurality of sensor data and master control data. During the process of obtaining the distributed data, whether a trigger signal of a trigger event is received is detected. If a trigger signal of a trigger event is received, the trigger time corresponding to the trigger signal is determined, and the first sensor data of the first preset time length before the trigger time is read out from the data buffer, and then the second sensor data of the second preset time length after the trigger time is read out from the data temporary storage area. The first sensor data and the second sensor data are the sensor data of a certain time period before and after the trigger event. Therefore, the recording of the normal driving data of the vehicle is avoided, and the sensor data of a certain time period before and after the trigger event can be accurately obtained. The obtained recording file reduces information redundancy, reduces memory occupancy, and improves the quality of the recorded data. Then, the first sensor data and the second sensor data are formatted to reduce the data volume of the sensor data, reduce the memory space occupied by the sensor data, and reduce the transmission time.

[0012] The sensor data recording method provided by the present invention, the first sensor data and the second sensor data in the obtained recording file are sensor data of a certain time period before and after the triggering event, so that the obtained recording file can realize the analysis and troubleshooting of unmanned driving problems, reduce memory occupancy, improve the quality of recorded data, reduce data redundancy, and also reduce the time of data transmission.

[0013] In addition, the above technical solution provided by the present invention may also have the following additional technical features:

[0014] In some technical solutions, optionally, first sensor data of a first preset time length before the trigger moment is read from the data buffer, and second sensor data of a second preset time length after the trigger moment is read from the data buffer, including: taking the trigger moment as the first end moment, determining the first start moment according to the first preset time length, and reading the sensor data between the first start moment and the first end moment from the data buffer as the first sensor data; taking the trigger moment as the second start moment, determining the second end moment according to the second preset time length, and reading the sensor data between the second start moment and the second end moment from the data buffer as the second sensor data.

[0015] In this technical solution, the process of determining the first sensor data is to first take the trigger moment as the first end moment and the first end moment as the end time, determine the first start moment, that is, the start time, through the first preset time length, obtain the sensor data in the time period before the trigger signal according to the first start moment and the first end moment, and then read it out from the data buffer and use it as the first sensor data; the process of determining the second sensor data is to first take the trigger moment as the second start moment, that is, take the second start moment as the start time, determine the second end moment, that is, the end time, through the second preset time length, obtain the sensor data in the time period after the trigger signal according to the second start moment and the second end moment, and then read it out from the data buffer and use it as the second sensor data.

[0016] The recording file includes the sensor data in the two time periods before and after the trigger signal. Because the sensor data in these two time periods before and after the trigger event are of important research significance and are used for tracing and playback of the trigger event, the recording file in this solution includes this part of key sensor data, which can not only reduce memory usage and data redundancy, but also improve the quality of recorded data.

[0017] In some technical schemes, optionally, the first sensor data and the second sensor data are formatted to obtain a recording file, including: obtaining first point cloud data of the first sensor data and second point cloud data in the second sensor data; octree encoding the first point cloud data and the second point cloud data according to a first compression ratio to obtain first point cloud compressed data and second point cloud compressed data; obtaining first video stream data of the first sensor data and second video stream data in the second sensor data; hard encoding the first video stream data and the second video stream data according to a second compression ratio to obtain first video stream compressed data and second video stream compressed data; merging the first point cloud compressed data and the second point cloud compressed data to obtain target point cloud data; merging the first video stream compressed data and the second video stream compressed data to obtain target video stream data; and obtaining a recording file based on the target point cloud data and the target video stream data.

[0018] In this technical solution, since the point cloud data occupies a high amount of memory, it is necessary to compress the first point cloud data and the second point cloud data in the first sensor data and the second sensor data. Specifically, octree encoding is used for compression with a first compression ratio to obtain compressed first point cloud compressed data and second point cloud compressed data. The point cloud data is encoded using octree to effectively compress the saved file size, thereby occupying less storage space and having a shorter network transmission time.

[0019] Since the video stream data also occupies a high amount of memory, it is necessary to obtain the first video stream data of the first sensor data and the second video stream data of the second sensor data for compression. Specifically, the first video stream data and the second video stream data are hard-coded according to the second compression ratio to obtain compressed first video stream compressed data and second video stream compressed data. By hard-coding the compressed video stream data, the saved file size can be effectively compressed, thereby occupying less storage space and shortening the network transmission time.

[0020] Furthermore, the first point cloud compressed data and the second point cloud compressed data of the same data type are combined to obtain target point cloud data, the first video stream compressed data and the second video stream compressed data of the same data type are combined to obtain target video stream data, and finally a recording file is obtained according to the target point cloud data and the target video stream data. The recording file thus obtained has a smaller memory usage and a shorter transmission time.

[0021] It should be noted that the recording file includes not only target point cloud data and target video stream data, but also data from other sensors and master control data. Data from other sensors include inertial measurement unit data, real-time positioning system data, millimeter wave raw data, video stream data, vehicle-to-vehicle data, scattered point data, concentrated millimeter wave data, path point data, target box data, smart box data, etc. These data are saved in the data buffer or data temporary storage area in binary form. Since these data occupy a small amount of memory, they are not compressed to save data processing time.

[0022] In some technical solutions, optionally, the first compression ratio is 7:1; and the second compression ratio is 10:1.

[0023] In this technical solution, point cloud data is compressed by octree encoding, and the first compression ratio is 7:1, which reduces memory usage. Video stream data is compressed by hard coding, that is, H264 video encoding, and the second compression ratio is 10:1, which reduces memory usage.

[0024] In some technical schemes, optionally, the first sensor data includes at least one of the following: inertial measurement unit data, real-time positioning system data, millimeter wave raw data, video stream data, vehicle-to-vehicle data, scattered data, concentrated millimeter wave data, path point data, target box data, and smart box data; the second sensor data includes at least one of the following: inertial measurement unit data, real-time positioning system data, millimeter wave raw data, video stream data, vehicle-to-vehicle data, scattered data, concentrated millimeter wave data, path point data, target box data, and smart box data; the value range of the first preset time length is: greater than or equal to 8s, and less than or equal to 12s; the value range of the second preset time length is: greater than or equal to 8s, and less than or equal to 12s.

[0025] In this technical solution, the first sensor data and the second sensor data both include at least one of inertial measurement unit data, real-time positioning system data, millimeter wave raw data, video stream data, vehicle-to-vehicle data, scattered point data, concentrated millimeter wave data, path point data, target frame data, and smart box data. The above data are collected using corresponding multiple sensors, and the trigger signal based on the trigger event is used as the key to start recording, without relying on the driver's human reaction speed. Multiple sensors are set in the vehicle to facilitate the collection of sensor data during the actual driving process of the vehicle. Multiple sensors such as laser radar, millimeter wave radar, GPS positioning sensor, camera module, inertial navigation sensor, body data acquisition module, etc. are used for vehicle driving sensors. For example, laser radar is used to collect point cloud data in the vehicle environment, millimeter wave radar is used to collect millimeter wave radar data in the vehicle environment, positioning sensor is used to collect latitude and longitude data of positioning sensor, camera module is used to collect video stream data of the vehicle environment, inertial navigation sensor is used to collect inertial data of the vehicle (such as vehicle speed, acceleration, angular velocity, etc.), and body data acquisition module is used to collect body bus data of the vehicle.

[0026] Further, the value range of the first preset time length is: greater than or equal to 8s and less than or equal to 12s; the value range of the second preset time length is: greater than or equal to 8s and less than or equal to 12s. The specific value of the first preset time length and the value of the second preset time length are set according to actual application.

[0027] It can be understood that the value of the first preset time length and the value of the second preset time length corresponding to different trigger events may be the same or different. In actual applications, when the value of the first preset time length and the value of the second preset time length are the same, the value of the first preset time length and the value of the second preset time length are both 10 seconds. This value is more in line with the rationality of vehicle data collection and improves the utilization of data.

[0028] In some technical solutions, optionally, detecting whether there is a trigger signal of a trigger event in the distributed data also includes: when no trigger signal is detected, storing the distributed data in a data buffer and recording a first storage duration; when the first storage duration exceeds a preset storage duration, deleting the distributed data from the data buffer; wherein the trigger event includes at least one of the following: parking due to obstacles, manual takeover, deceleration parking and unmanned remote control.

[0029] In this technical solution, when no trigger signal is detected, the distributed data will be stored in the data buffer and a first storage duration will be recorded. The first storage duration is used to indicate the storage duration of the distributed data in the data buffer. When the first storage duration exceeds the preset storage duration, the distributed data will be deleted to release storage space in the data buffer.

[0030] The triggering event includes at least one of the following: parking due to obstacles, manual takeover, deceleration parking and remote control. Specifically, parking due to obstacles includes emergency braking, etc.; manual takeover includes the event that the vehicle switches from unmanned state to manned state; deceleration parking includes the event that the vehicle reduces the driving speed to stop. When it is judged that the difference between the current frame speed and the previous frame speed of the vehicle is greater than 10fps (frames per second), the deceleration parking condition is met; remote control, that is, switching from unmanned state to remote control state, is an event in which the vehicle is originally in an unmanned driving state and is converted to remote control.

[0031] Specifically, when the vehicle is an unmanned mining vehicle and any of the following situations occurs: stopping due to an obstacle, manual takeover, slowing down and stopping, or unmanned remote control, the vehicle-side data collection mechanism will be triggered, and upon receiving a trigger signal corresponding to any of the above trigger events, data recording will begin.

[0032] The second aspect of the present invention provides a sensor data recording system, including: an acquisition module, used to acquire distributed data; a detection module, used to detect whether there is a trigger signal of a trigger event in the distributed data; a determination module, used to determine the trigger time corresponding to the trigger signal when the presence of the trigger signal is detected; a reading module, used to read first sensor data of a first preset time length before the trigger time from a data buffer, and read second sensor data of a second preset time length after the trigger time from a data temporary storage area; a processing module, used to format the first sensor data and the second sensor data to obtain a recording file.

[0033] The sensor data recording system provided by the present invention is suitable for unmanned vehicles. Specifically, firstly, distributed data is obtained, and the distributed data includes multiple sensor data and main control data. During the process of obtaining the distributed data, whether the trigger signal of the trigger event is received is detected. If the trigger signal of the trigger event is received, the trigger time corresponding to the trigger signal is determined, and the first sensor data of the first preset time length before the trigger time is read out from the data buffer, and then the second sensor data of the second preset time length after the trigger time is read out from the data temporary storage area. The first sensor data and the second sensor data are the sensor data of a certain time period before and after the trigger event. Therefore, the recording of the normal driving data of the vehicle is avoided, and the sensor data of a certain time period before and after the trigger event can be accurately obtained, thereby reducing information redundancy, reducing memory occupancy, and improving the quality of recorded data. Then, the first sensor data and the second sensor data are formatted to reduce the data volume of the sensor data, reduce the memory space occupied by the sensor data, and shorten the transmission time.

[0034] The sensor data recording system provided by the present invention obtains the first sensor data and the second sensor data in the recording file, which are sensor data of a certain time period before and after the triggering event, so that the obtained recording file realizes the analysis and troubleshooting of unmanned driving problems, reduces memory occupancy, improves the quality of recorded data, reduces data redundancy, and also reduces the time of data transmission.

[0035] A third aspect of the present invention provides an electronic device, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein when the program or instruction is executed by the processor, the steps of the sensor data recording method in any of the above technical solutions are implemented.

[0036] In this technical solution, the electronic device includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the steps of the sensor data recording method in any of the above-mentioned technical solutions are implemented. Therefore, all the beneficial effects of the sensor data recording method having any of the above-mentioned technical solutions are no longer stated one by one here.

[0037] A fourth aspect of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the sensor data recording method in any of the above technical solutions are implemented.

[0038] In this technical solution, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the sensor data recording method in any of the above-mentioned technical solutions are implemented. Therefore, all the beneficial effects of the sensor data recording method with any of the above-mentioned technical solutions are no longer stated one by one here.

[0039] A fifth aspect of the present invention provides a vehicle, including a controller, on which a set of instruction sets are stored, and the instruction sets execute the steps of the sensor data recording method in any of the above technical solutions.

[0040] In this technical solution, the vehicle includes a controller storing a set of instruction sets, which execute the steps of the sensor data recording method in any of the above-mentioned technical solutions. Therefore, all the beneficial effects of the sensor data recording method of any of the above-mentioned technical solutions are no longer stated one by one here.

[0041] Additional aspects and advantages of the technical solutions of the present invention will become apparent in the following description or will be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 One of the flowcharts of the method for recording sensor data according to one embodiment of the present invention is shown;

[0043] Figure 2 A second flowchart of a method for recording sensor data according to an embodiment of the present invention is shown;

[0044] Figure 3 A third flowchart of a method for recording sensor data according to an embodiment of the present invention is shown;

[0045] Figure 4 A structural block diagram of a sensor data recording system according to an embodiment of the present invention is shown;

[0046] Figure 5 A structural block diagram of an electronic device according to an embodiment of the present invention is shown;

[0047] Figure 6 A diagram showing an input and output interface of a sensor data recording system according to an embodiment of the present invention is shown;

[0048] Figure 7 A schematic diagram showing a method for recording sensor data according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0049] In order to more clearly understand the above-mentioned purposes, features and advantages of the embodiments of the present invention, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0050] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the embodiments of the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0051] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the embodiments of the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0052] like Figure 1 As shown, according to an embodiment of the first aspect of the present invention, a method for recording sensor data is provided, comprising the following steps:

[0053] Step 102: Obtain distributed data;

[0054] Step 104: Detect whether there is a trigger signal of a trigger event in the distributed data;

[0055] Step 106: When a trigger signal is detected, determining a trigger time corresponding to the trigger signal;

[0056] Step 108: reading the first sensor data of a first preset time length before the triggering moment from the data buffer, and reading the second sensor data of a second preset time length after the triggering moment from the data temporary storage area;

[0057] Step 110: Format the first sensor data and the second sensor data to obtain a recording file.

[0058] The sensor data recording method provided by the present invention is applicable to a data recording method for automatically recording sensor data according to a trigger signal during an unmanned driving process. Specifically, firstly, distributed data is obtained, and the distributed data includes a plurality of sensor data and master control data. During the process of obtaining the distributed data, whether a trigger signal of a trigger event is received is detected. If a trigger signal of a trigger event is received, the trigger time corresponding to the trigger signal is determined, and the first sensor data of the first preset time length before the trigger time is read out from the data buffer, and then the second sensor data of the second preset time length after the trigger time is read out from the data temporary storage area. The first sensor data and the second sensor data are the sensor data of a certain time period before and after the trigger event. Therefore, the recording of the normal driving data of the vehicle is avoided, and the sensor data of a certain time period before and after the trigger event can be accurately obtained. The obtained recording file reduces information redundancy, reduces memory occupancy, and improves the quality of the recorded data. Then, the first sensor data and the second sensor data are formatted to reduce the data volume of the sensor data, reduce the memory space occupied by the sensor data, and reduce the transmission time.

[0059] The sensor data recording method provided by the present invention, the first sensor data and the second sensor data in the obtained recording file are sensor data of a certain time period before and after the triggering event, so that the obtained recording file can realize the analysis and troubleshooting of unmanned driving problems, reduce memory occupancy, improve the quality of recorded data, reduce data redundancy, and also reduce the time of data transmission.

[0060] like Figure 2 As shown, according to a sensor data recording method of an embodiment of the present invention, first sensor data of a first preset time length before a trigger moment is read from a data buffer, and second sensor data of a second preset time length after a trigger moment is read from a data temporary storage area, specifically comprising the following steps:

[0061] Step 202: taking the triggering moment as the first ending moment, determining the first starting moment according to the first preset duration, and reading the sensor data between the first starting moment and the first ending moment from the data buffer as the first sensor data;

[0062] Step 204: taking the triggering time as the second starting time, determining the second ending time according to the second preset duration, and reading the sensor data between the second starting time and the second ending time from the data temporary storage area as the second sensor data.

[0063] In this embodiment, the process of determining the first sensor data is to first use the trigger moment as the first end moment and the first end moment as the end time, determine the first start moment, that is, the start time, through the first preset time length, obtain the sensor data in the time period before the trigger signal according to the first start moment and the first end moment, and then read it out from the data buffer as the first sensor data; the process of determining the second sensor data is to first use the trigger moment as the second start moment, that is, use the second start moment as the start time, determine the second end moment, that is, the end time, through the second preset time length, obtain the sensor data in the time period after the trigger signal according to the second start moment and the second end moment, and then read it out from the data buffer as the second sensor data.

[0064] The recording file includes the sensor data in the two time periods before and after the trigger signal. Because the sensor data in these two time periods before and after the trigger event are of important research significance and are used for tracing and playback of the trigger event, the recording file in this solution includes this part of key sensor data, which can not only reduce memory usage and data redundancy, but also improve the quality of recorded data.

[0065] like Figure 3 As shown, according to a sensor data recording method of an embodiment of the present invention, the first sensor data and the second sensor data are formatted to obtain a recording file, which specifically includes the following steps:

[0066] Step 302: Acquire first point cloud data of the first sensor data and second point cloud data of the second sensor data;

[0067] Step 304: performing octree encoding on the first point cloud data and the second point cloud data according to a first compression ratio to obtain first point cloud compressed data and second point cloud compressed data;

[0068] Step 306: Acquire first video stream data of the first sensor data and second video stream data of the second sensor data;

[0069] Step 308: hard-encode the first video stream data and the second video stream data according to the second compression ratio to obtain first video stream compressed data and second video stream compressed data;

[0070] Step 310: merging the first point cloud compressed data and the second point cloud compressed data to obtain target point cloud data;

[0071] Step 312: merging the first video stream compressed data and the second video stream compressed data to obtain target video stream data;

[0072] Step 314: Obtain a recording file according to the target point cloud data and the target video stream data.

[0073] In this embodiment, since the point cloud data occupies a high amount of memory, the first point cloud data and the second point cloud data in the first sensor data and the second sensor data need to be compressed. Specifically, octree encoding is used for compression with a first compression ratio to obtain compressed first point cloud compressed data and second point cloud compressed data. The point cloud data is encoded using octree to effectively compress the saved file size, thereby occupying less storage space and having a shorter network transmission time.

[0074] Since the video stream data also occupies a high amount of memory, it is necessary to obtain the first video stream data of the first sensor data and the second video stream data of the second sensor data for compression. Specifically, the first video stream data and the second video stream data are hard-coded according to the second compression ratio to obtain compressed first video stream compressed data and second video stream compressed data. By hard-coding the compressed video stream data, the saved file size can be effectively compressed, thereby occupying less storage space and shortening the network transmission time.

[0075] Furthermore, the first point cloud compressed data and the second point cloud compressed data of the same data type are combined to obtain target point cloud data, the first video stream compressed data and the second video stream compressed data of the same data type are combined to obtain target video stream data, and finally a recording file is obtained according to the target point cloud data and the target video stream data. The recording file thus obtained has a smaller memory usage and a shorter transmission time.

[0076] It should be noted that the recording file includes not only target point cloud data and target video stream data, but also data from other sensors and master control data. Data from other sensors include inertial measurement unit data, real-time positioning system data, millimeter wave raw data, video stream data, vehicle-to-vehicle data, scattered point data, concentrated millimeter wave data, path point data, target box data, smart box data, etc. These data are saved in the data buffer or data temporary storage area in binary form. Since these data occupy a small amount of memory, they are not compressed to save data processing time.

[0077] The first point cloud data and the second point cloud data are compressed using octree encoding. Octree encoding is currently the most popular encoding method in GPCC. (The most common way to encode geometric information in Geometry-based Point Cloud Compression is also the basis of TISOUP encoding (a coding standard). After octree encoding, the geometric information of the point cloud will be converted into binary placeholder information to achieve the purpose of encoding, and then combined with the corresponding entropy coding technology to further reduce the bit rate.

[0078] The detailed process of using octree encoding for the first point cloud data and the second point cloud data is as follows: (1) Preprocessing. Removing outliers: The first point cloud data and the second point cloud data may contain some outliers that do not contribute to the representation of the overall shape and therefore need to be removed. Downsampling: In order to reduce the amount of data and improve the encoding efficiency, the first point cloud data and the second point cloud data can be downsampled; (2) Constructing an octree. Determining the bounding box: First, determine a minimum cuboid containing all point cloud data, called a bounding box. The origin coordinates of the bounding box consist of the minimum values ​​of the coordinates of each point in the point cloud in the three dimensions of x, y, and z. Coordinate conversion: With the origin of the bounding box as the reference, the original coordinates of each point in the point cloud are converted into relative coordinates relative to the origin of the coordinates. Out-of-loop quantization: Quantize the geometric coordinates of the points, which mainly plays a role in scaling. After quantization, the geometric information of some points may be the same. Whether to remove these points with repeated geometric information is determined based on the parameters. Octree partitioning: Take the bounding box as the root node and divide it into eight equal parts, generating eight sub-cubes as its child nodes. Eight bits are used to represent the occupancy information of a child node, which is called a space occupancy code. If there is a point in a child cube, it means that the child node is occupied, and the corresponding occupancy bit takes the value of 1, otherwise it takes the value of 0. The occupied child cube continues to be divided until the leaf node obtained by the division is a 1×1×1 unit cube; (3) Octree encoding. Encoding space occupancy code: During the octree division process, a series of space occupancy codes are generated. These occupancy codes represent the occupancy of each node in the octree. Encoding the number of leaf node points: When a leaf node contains the first point cloud data and the second point cloud data (especially when the point cloud contains duplicate points), the number of these points needs to be encoded. Entropy coding: In order to further reduce the bit rate, the space occupancy code and the number of points contained in the leaf node can be entropy encoded. Entropy coding is a lossless compression method that can compress data based on the statistical characteristics of the data.

[0079] There are five advantages of using octree to encode the first point cloud data and the second point cloud data: (1) Efficient compression. Octree encoding recursively divides the three-dimensional space into smaller subspaces and only stores the occupied subspace information, thereby achieving efficient compression of point cloud data. This method can significantly reduce the bandwidth and storage space required to store and transmit point cloud data; (2) Simple and intuitive: The structure of octree encoding is relatively simple and easy to understand and implement. It divides the three-dimensional space into a series of cubes (or sub-cubes) and uses binary bits to represent the occupancy of each cube, thereby intuitively representing the spatial distribution of point cloud data; (3) Support for fast query: Since octree encoding organizes the first point cloud data and the second point cloud data into a hierarchical data structure, it can support fast point cloud query and retrieval operations. For example, the first point cloud data and the second point cloud data in a specific area can be found by traversing the octree; (4) Easy to expand: The octree encoding method has good scalability. As the point cloud data increases or the resolution improves, the depth of the octree can be increased or the partition threshold can be adjusted accordingly to adapt to different application scenarios and data requirements; (5) Strong adaptability and high flexibility. The octree is not only applicable to static first point cloud data and second point cloud data, but also can process dynamic first point cloud data and second point cloud data. When the first point cloud data and the second point cloud data change, the octree can flexibly adjust its structure to adapt to the new data distribution.

[0080] The first video stream data and the second video stream data are compressed by hard coding, that is, H264 video coding. H264 is a new generation of digital video compression format proposed by the International Organization for Standardization (ISO) and the International Telecommunication Union (ITU) after MPEG4. H264 video single frame coding and jpeg_ code logic (JointPhotographic Experts Group, an image compression standard) add I frame (IntraPicture, intra-frame coded image frame), P frame (Predictive-Frame, predictive coded image frame) and B frame (Bi-Directional Interpolated PredictionFrame, bidirectional predictive coded image frame) logic to solve the problem of spatial redundancy, and do difference operations between frames to solve temporal redundancy. H264 coding, also known as Advanced Video Coding (AVC), is a block-based video compression standard and a widely used high-precision video recording, compression and publishing format. It achieves efficient compression by removing spatial and temporal redundancy in video.

[0081] The first video stream data and the second video stream data are hard-coded, that is, the detailed process of H264 encoding is as follows: (1) Preprocessing: Image segmentation: The image to be encoded is divided into multiple 16×16 macroblocks (MacroBlock), which is the basic unit of H264 encoding. Macroblocks can also be further divided into smaller sub-blocks, such as 8×8 or 4×4 sub-blocks, for more refined encoding. Frame type determination: Determine the frame type of the image according to the encoding requirements. H264 defines three frame types: I frame (Intra Picture, intra-frame coded image frame), P frame (Predictive-Frame, predictive coded image frame) and B frame (Bi-Directional Interpolated PredictionFrame, bidirectional predictive coded image frame). I frame does not depend on other frames and contains complete image information; P frame uses the previous I frame or P frame for predictive encoding; B frame refers to both the previous frame and the following frame for encoding; (2) Intra-frame compression: Remove spatial redundancy: Use the similarity within the image for compression. By predictive coding, transform coding (such as discrete cosine transform DCT) ，The full name is Discrete Cosine Transform), quantization and other steps to remove spatial redundant information in the image. Entropy coding: Entropy coding is performed on the quantized coefficients, such as Huffman coding or arithmetic coding, to further reduce the amount of data; (3) Inter-frame compression: Motion estimation and compensation: Compression is performed using the temporal correlation between adjacent frames. Through motion estimation, the best matching block between the current frame and the reference frame is found, and the motion vector is calculated. Then, motion compensation technology is used to predict the current frame based on the motion vector. Residual coding: The prediction error (residual) is calculated and the residual is encoded. Residual coding usually includes steps such as transform coding, quantization and entropy coding; (4) Post-processing: Loop filtering: In order to improve the clarity and accuracy of the decoded image, loop filtering technology is usually used to reduce block effects and quantization noise. Code stream organization: The encoded data is organized into a code stream in a certain format for storage or transmission.

[0082] Using H264 to encode video data brings many benefits: (1) High compression rate. H264 uses a variety of advanced coding technologies, such as intra-frame prediction, inter-frame prediction, transform coding and quantization, to achieve a high compression rate. Under the same image quality, the compression rate of H264 is much higher than that of previous video coding standards, such as MPEG-2 (Moving Picture Experts Group-2) and MPEG-4 (Moving Picture Experts Group-4).

[0083] (2) Good network affinity. The H264 encoding standard has good network adaptability and can be well adapted to IP and wireless network applications. It supports a variety of network transmission protocols and error recovery mechanisms, and can provide high-quality video transmission when the network is unstable or bandwidth is limited.

[0084] (3) Support multiple resolutions and bit rates. The H264 encoding standard supports various video encoding requirements from low resolution to high resolution and from low bit rate to high bit rate. This enables H264 to be widely used in different video application scenarios, such as live web video, video on demand, high-definition television broadcasting, etc.

[0085] (4) High-quality image reconstruction: Since H264 uses advanced coding technology and post-processing technology, such as loop filtering, it can provide high-quality image reconstruction. Even at low bit rates, H264 can maintain good image clarity and detail.

[0086] Since the performance of the controller hardware model Orin domain has the characteristics of powerful processing power, efficient graphics processing unit, large memory bandwidth, multi-core architecture and low power design. Therefore, hard coding is used for video stream data for several reasons: (1) Real-time requirements. During the unmanned driving process, video stream data needs to be processed in real time. Hard coding can significantly increase the encoding speed and ensure that the delay of the video stream is minimized; (2) Reduce the CPU burden. Through hard coding, the compression and processing of video stream data can be completed by dedicated hardware, thereby reducing the burden on the CPU and improving the efficiency of the overall system; (3) Higher encoding efficiency. Hard coding generally has higher compression efficiency, which can reduce the file size while maintaining high image quality, making it easier to store and transmit.

[0087] In some embodiments, optionally, the first compression ratio is 7:1; and the second compression ratio is 10:1.

[0088] In this embodiment, the first point cloud data and the second point cloud data are compressed by octree encoding, and the first compression ratio is 7:1, which reduces memory usage. The first video stream data and the second video stream data are compressed by hard coding, that is, H264 video encoding, and the second compression ratio is 10:1, which reduces memory usage.

[0089] The compression ratios of point cloud data (first point cloud data, second point cloud data) and video stream data (first video stream data, second video stream data) are shown in Table 1. As can be seen from Table 1, the use of octree encoding and H264 encoding (hard encoding) can effectively compress the size of the saved file, thereby occupying less storage space and shortening the network transmission time.

[0090] Table 1 Comparison of compression ratios of point cloud data and video stream data

[0091]

[0092] In some embodiments, optionally, the first sensor data includes at least one of the following: inertial measurement unit data, real-time positioning system data, millimeter wave raw data, video stream data, vehicle-to-vehicle data, scattered point data, concentrated millimeter wave data, path point data, target box data, and smart box data; the second sensor data includes at least one of the following: inertial measurement unit data, real-time positioning system data, millimeter wave raw data, video stream data, vehicle-to-vehicle data, scattered point data, concentrated millimeter wave data, path point data, target box data, and smart box data; the value range of the first preset time length is: greater than or equal to 8s, and less than or equal to 12s; the value range of the second preset time length is: greater than or equal to 8s, and less than or equal to 12s.

[0093] In this embodiment, the first sensor data and the second sensor data both include at least one of inertial measurement unit data, real-time positioning system data, millimeter wave raw data, video stream data, vehicle-to-vehicle data, scattered point data, concentrated millimeter wave data, path point data, target frame data, and smart box data. The above data are collected using corresponding multiple sensors, and the trigger signal based on the trigger event is used as the key to start recording, without relying on the driver's human reaction speed. Multiple sensors are set in the vehicle to facilitate the collection of sensor data during the actual driving process of the vehicle. Multiple sensors such as laser radar, millimeter wave radar, GPS positioning sensor, camera module, inertial navigation sensor, body data acquisition module, etc. are used for vehicle driving sensors. For example, laser radar is used to collect point cloud data in the vehicle environment, millimeter wave radar is used to collect millimeter wave radar data in the vehicle environment, positioning sensor is used to collect latitude and longitude data of positioning sensor, camera module is used to collect video stream data of the vehicle environment, inertial navigation sensor is used to collect inertial data of the vehicle (such as vehicle speed, acceleration, angular velocity, etc.), and body data acquisition module is used to collect body bus data of the vehicle.

[0094] Further, the value range of the first preset time length is: greater than or equal to 8s and less than or equal to 12s; the value range of the second preset time length is: greater than or equal to 8s and less than or equal to 12s. The specific value of the first preset time length and the value of the second preset time length are set according to actual application.

[0095] It can be understood that the value of the first preset time length and the value of the second preset time length corresponding to different trigger events may be the same or different. In actual applications, when the value of the first preset time length and the value of the second preset time length are the same, the value of the first preset time length and the value of the second preset time length are both 10 seconds. This value is more in line with the rationality of vehicle data collection and improves the utilization of data.

[0096] In some embodiments, optionally, detecting whether there is a trigger signal of a trigger event in the distributed data also includes: when no trigger signal is detected, storing the distributed data in a data buffer and recording a first storage duration; when the first storage duration exceeds a preset storage duration, deleting the distributed data from the data buffer; wherein the trigger event includes at least one of the following: parking due to obstacles, manual takeover, deceleration parking and unmanned remote control.

[0097] In this embodiment, when no trigger signal is detected, the distributed data will be stored in the data buffer and a first storage duration will be recorded. The first storage duration is used to indicate the storage duration of the distributed data in the data buffer. When the first storage duration exceeds the preset storage duration, the distributed data will be deleted to release storage space in the data buffer.

[0098] The triggering event includes at least one of the following: parking due to obstacles, manual takeover, deceleration parking and unmanned switching to remote control. Specifically, parking due to obstacles includes emergency braking, etc.; manual takeover includes the event that the vehicle switches from unmanned state to manned state; deceleration parking includes the event that the vehicle reduces the driving speed to stop. Specifically, when it is judged that the difference between the current frame speed and the previous frame speed of the vehicle is greater than 10.0, the deceleration parking condition is met; switching from unmanned state to remote control state means that the vehicle is originally in an unmanned driving state and is converted to remote control.

[0099] Specifically, when the vehicle is an unmanned mining vehicle and any of the following situations occurs: stopping due to an obstacle, manual takeover, slowing down and stopping, or unmanned remote control, the vehicle-side data collection mechanism will be triggered, and upon receiving a trigger signal corresponding to any of the above trigger events, data recording will begin.

[0100] like Figure 4 As shown, an embodiment of the second aspect of the present invention provides a sensor data recording system 400, including: an acquisition module 410, used to acquire distributed data; a detection module 420, used to detect whether there is a trigger signal of a trigger event in the distributed data; a determination module 430, used to determine the trigger time corresponding to the trigger signal when the presence of the trigger signal is detected; a reading module 440, used to read the first sensor data of a first preset time length before the trigger time from the data buffer, and read the second sensor data of a second preset time length after the trigger time from the data temporary storage area; a processing module 450, used to obtain a recording file after formatting the first sensor data and the second sensor data.

[0101] The sensor data recording system provided by the present invention is suitable for unmanned vehicles. Specifically, firstly, distributed data is obtained, and the distributed data includes multiple sensor data and main control data. During the process of obtaining the distributed data, whether the trigger signal of the trigger event is received is detected. If the trigger signal of the trigger event is received, the trigger time corresponding to the trigger signal is determined, and the first sensor data of the first preset time length before the trigger time is read out from the data buffer, and then the second sensor data of the second preset time length after the trigger time is read out from the data temporary storage area. The first sensor data and the second sensor data are the sensor data of a certain time period before and after the trigger event. Therefore, the recording of the normal driving data of the vehicle is avoided, and the sensor data of a certain time period before and after the trigger event can be accurately obtained, thereby reducing information redundancy, reducing memory occupancy, and improving the quality of recorded data. Then, the first sensor data and the second sensor data are formatted to reduce the data volume of the sensor data, reduce the memory space occupied by the sensor data, and shorten the transmission time.

[0102] The sensor data recording system 400 provided by the present invention, the first sensor data and the second sensor data in the obtained recording file are sensor data of a certain time period before and after the triggering event, so that the obtained recording file realizes the analysis and troubleshooting of unmanned driving problems, reduces memory occupancy, improves the quality of recorded data, reduces data redundancy, and also reduces the time of data transmission.

[0103] like Figure 5 As shown, the embodiment of the third aspect of the present invention provides an electronic device 500, including a processor 510, a memory 520, and a program or instruction stored in the memory and executable on the processor, and when the program or instruction is executed by the processor, the steps of the sensor data recording method in any of the above embodiments are implemented. Therefore, the technical effects of any of the above embodiments of the first aspect are achieved, and no further description is given here.

[0104] The embodiment of the fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the sensor data recording method in any of the above embodiments are implemented. Therefore, the technical effects of any of the above embodiments of the first aspect are achieved, and no further description is given here.

[0105] The fifth aspect of the present invention provides a vehicle, the vehicle includes a controller, a set of instruction sets are stored on the controller, and the instruction sets execute the steps of the sensor data recording method in any of the above embodiments. Therefore, the technical effects of any of the above first aspects are achieved, which will not be repeated here.

[0106] The vehicle is specifically an engineering vehicle, which is used in the direction of unmanned driving, such as an unmanned mining car.

[0107] like Figure 6 and Figure 7 As shown, according to a sensor data recording system of a specific embodiment provided by the present invention, Figure 6 It is the IO (Input Output) interface diagram of the sensor data recording system, also known as the system IO interface diagram of the data acquisition system. Figure 7 The schematic diagram of the sensor data recording method. The input is DDS topic (Data Distribution Service topic) and configuration file. The DDS topic contains all sensor data and master control data. Before running, the detailed system operation parameters and the storage path of the recorded files on the vehicle side are obtained by reading the configuration file.

[0108] The output is divided into two types: DDS topic and ROSbag (Robot Operating System bag) file. The system parsing end can parse and store binary data, and republish the DDS (Data Distribution Service) topic containing all sensor data and master control data. It can also be converted into ROSbag format to facilitate the use of data through the ROS system (Robot Operating System).

[0109] The purpose of unmanned driving triggered vehicle-side data collection is to trigger the vehicle-side data collection mechanism and start recording data when the mine car encounters any of the following situations: stopping due to obstacles, manual takeover, slowing down and stopping, or unmanned remote control. Unmanned driving technology is becoming more and more mature, and the development of perception functions in unmanned driving depends on sensor data. Sensor data recording is an important part of the unmanned driving tool chain. Sensor data recording is to obtain sensor data and store the recording results on the vehicle side after recording. The recording results of triggered events are extremely important for the development and optimization of unmanned driving functions.

[0110] The existing sensor data recording process includes rolling recording and manual recording: in the rolling recording mode, all sensor data will be recorded and then stored on the vehicle side; in the manual recording mode, when encountering specific scenarios, the corresponding data will be saved by manually recording the data.

[0111] The existing rolling recording and manual recording methods of sensor data both have certain defects: in the rolling recording method, because all sensor data need to be stored, and a large part of the sensor data is normal data, it is of little significance to the improvement of unmanned driving functions, resulting in high information redundancy, and occupying memory and affecting efficiency. Manual triggering relies on manual control. If an emergency occurs, it is difficult to record data in time. Only sensor data after the emergency occurs can be obtained, and it is difficult to trace the cause of the incident. Therefore, compared with related technologies, the sensor data recording method provided by the present invention reduces information redundancy, reduces memory occupancy, and improves the quality of recorded data.

[0112] In the present invention, the terms "first", "second", and "third" are used for descriptive purposes only and should not be understood as indicating or implying relative importance; the term "plurality" refers to two or more, unless otherwise clearly defined. The terms "installed", "connected", "connected", "fixed", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; "connected" can be a direct connection or an indirect connection through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0113] In the description of the present invention, it is necessary to understand that the directions or positional relationships indicated by terms such as “upper”, “lower”, “left”, “right”, “front” and “back” are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or unit referred to must have a specific direction, be constructed and operated in a specific orientation, and therefore, cannot be understood as a limitation on the present invention.

[0114] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "specific embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0115] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for recording sensor data, characterized in that: The recording method comprises: Get distributed data; Detecting whether there is a trigger signal of a trigger event in the distributed data; When the trigger signal is detected, determining a trigger time corresponding to the trigger signal; Reading first sensor data of a first preset time length before the triggering moment from the data buffer, and reading second sensor data of a second preset time length after the triggering moment from the data temporary storage area; The first sensor data and the second sensor data are formatted to obtain a recording file.

2. The sensor data recording method according to claim 1, characterized in that: The step of reading the first sensor data of a first preset time length before the triggering moment from the data buffer, and reading the second sensor data of a second preset time length after the triggering moment from the data temporary storage area, comprises: Taking the triggering moment as the first ending moment, determining a first starting moment according to a first preset duration, and reading sensor data between the first starting moment and the first ending moment from the data buffer as the first sensor data; The triggering moment is used as a second starting moment, a second ending moment is determined according to a second preset duration, and sensor data between the second starting moment and the second ending moment is read from the data temporary storage area as the second sensor data.

3. The sensor data recording method according to claim 1, characterized in that: The step of formatting the first sensor data and the second sensor data to obtain a recording file includes: Acquire first point cloud data from the first sensor data and second point cloud data from the second sensor data; Performing octree encoding on the first point cloud data and the second point cloud data according to a first compression ratio to obtain first point cloud compressed data and second point cloud compressed data; Acquire first video stream data of the first sensor data and second video stream data of the second sensor data; Hard-coding the first video stream data and the second video stream data according to a second compression ratio to obtain first video stream compressed data and second video stream compressed data; Merging the first point cloud compressed data and the second point cloud compressed data to obtain target point cloud data; Combining the first video stream compressed data and the second video stream compressed data to obtain target video stream data; A recording file is obtained according to the target point cloud data and the target video stream data.

4. The sensor data recording method according to claim 3, characterized in that: include: The first compression ratio is 7:1; The second compression ratio is 10:

1.

5. The method for recording sensor data according to any one of claims 1 to 4, characterized in that: The first sensor data includes at least one of the following: inertial measurement unit data, real-time positioning system data, millimeter wave raw data, video streaming data, vehicle-to-vehicle data, scattered point data, concentrated millimeter wave data, waypoint data, target box data, smart box data; The second sensor data includes at least one of: inertial measurement unit data, real-time positioning system data, millimeter wave raw data, video streaming data, vehicle-to-vehicle data, scattered point data, concentrated millimeter wave data, waypoint data, target box data, smart box data; The value range of the first preset time length is: greater than or equal to 8s and less than or equal to 12s; The value range of the second preset time length is: greater than or equal to 8s and less than or equal to 12s.

6. The method for recording sensor data according to any one of claims 1 to 4, characterized in that: The detecting whether there is a trigger signal of a trigger event in the distributed data further includes: When the trigger signal is not detected, storing the distributed data in the data buffer and recording a first storage duration; When the first storage duration exceeds a preset storage duration, deleting the distributed data from the data buffer; The triggering event includes at least one of the following: stopping due to an obstacle, manual takeover, slowing down and stopping, and remote control.

7. A sensor data recording system, characterized in that: include: Acquisition module, used to acquire distributed data; A detection module, used to detect whether there is a trigger signal of a trigger event in the distributed data; A determination module, configured to determine a triggering time corresponding to the triggering signal when the triggering signal is detected; A reading module, used for reading first sensor data of a first preset time length before the triggering moment from a data buffer, and reading second sensor data of a second preset time length after the triggering moment from a data temporary storage area; A processing module is used to obtain a recording file after formatting the first sensor data and the second sensor data.

8. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the sensor data recording method as claimed in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the sensor data recording method according to any one of claims 1 to 6 are implemented.

10. A vehicle, characterized in that: include: A controller having a set of instruction sets stored thereon, wherein the instruction sets execute the steps of the sensor data recording method according to any one of claims 1 to 6.

Images