Vehicle-mounted data saving method, vehicle-mounted data acquisition method and device

Abstract

The application provides a vehicle-mounted data saving method and device and a vehicle-mounted data acquisition method, and relates to the field of vehicle-mounted data saving. The method comprises the following steps: acquiring vehicle-mounted data and video image data; selecting target vehicle-mounted data in the vehicle-mounted data according to a preset metadata capacity and key degree attribute values of each vehicle-mounted data, so that the capacity of the target vehicle-mounted data is less than or equal to the preset metadata capacity and the total key degree attribute value of the target vehicle-mounted data is maximum; and packing the video image data and the associated target vehicle-mounted data into a video code stream and saving the video code stream. The method can realize the effect of saving the video image data and the vehicle-mounted data in association, and can save high-value vehicle-mounted data under the constraint of a limited metadata capacity.

Description

Technical Field

[0001] This invention relates to the field of vehicle data storage, and particularly to a vehicle data storage method, a vehicle data acquisition method, and an apparatus. Background Technology

[0002] In related technologies, vehicle data and vehicle video are usually stored separately, which can easily lead to problems such as loss of associated files, mismatch, or damage to alignment information. Summary of the Invention

[0003] The purpose of this invention is to provide a method for storing vehicle data, a method for acquiring vehicle data, and an apparatus that can achieve the effect of storing video image data and vehicle data together, while also storing high-value vehicle data under the constraint of limited metadata capacity.

[0004] To solve the above-mentioned technical problems, the present invention provides a method for storing vehicle data, comprising: Acquire vehicle data and video image data; Based on the preset metadata capacity and the criticality attribute values ​​of each of the vehicle data, target vehicle data is selected from the vehicle data, so that the capacity of the target vehicle data is less than or equal to the preset metadata capacity and the total criticality attribute value of the target vehicle data is maximized. The video image data and its associated target vehicle data are packaged into a video stream and saved.

[0005] Optionally, the criticality attribute of the vehicle data includes the preset priority of the vehicle data, the correlation value determined based on the correlation between vehicle events and the vehicle data, and the freshness value determined based on the collection time of the vehicle data; The method further includes: The preset priority, relevance value, and freshness value of the vehicle data are weighted to obtain the criticality attribute value of the vehicle data.

[0006] Optionally, before selecting the target vehicle data from the vehicle data based on the preset metadata capacity and the criticality attribute values ​​of each of the vehicle data, the method further includes: Retrieve historical vehicle data corresponding to the last time vehicle data was successfully saved from video image data; Determine whether the video image data corresponds to the target frame type; If so, then use the full amount of current vehicle data to set a snapshot of the vehicle data; If not, then use the change value of vehicle data between the current vehicle data and the historical vehicle data to set incremental vehicle data; The step of selecting target vehicle data from the vehicle data based on the preset metadata capacity and the criticality attribute values ​​of each of the vehicle data includes: Based on the preset metadata capacity and the criticality attribute values ​​of each of the vehicle data, the target vehicle data is selected from the snapshot vehicle data or the incremental vehicle data.

[0007] Optionally, setting incremental vehicle data using the vehicle data change value between the current vehicle data and the historical vehicle data includes: The incremental vehicle data is determined by the change in vehicle data values ​​that are greater than a preset threshold.

[0008] Optionally, the step of packaging the video image data and its associated target vehicle data into a video stream includes: The target vehicle data is encapsulated into metadata units; The metadata unit is inserted into a preset position in the access unit corresponding to the video image data to obtain the video stream.

[0009] Optionally, the target vehicle data is encapsulated into metadata units, including: Determine the type of the target vehicle data; If the target vehicle data is a bus message, then a timestamp reference value is set according to the time when the video image data is generated, the time offset value of the acquisition time of the bus message relative to the timestamp reference value is determined, and the bus message itself, along with the message identifier, message type flag, data length code, and time offset value of the bus message, are encapsulated into a data entry. If the target vehicle data is a status signal, then the status signal and its signal identifier are encapsulated as the data entry; Set header information according to the status version number and type of the target vehicle data; The metadata unit is encapsulated by the metadata identifier, the header information, the timestamp base value, the number of data entries, and the data entries themselves.

[0010] Optionally, it also includes: The frame identifier of the video image data and the target vehicle data are jointly signed to obtain a signature value; The step of packaging the video image data and its associated target vehicle data into a video stream includes: The video image data, along with the associated target vehicle data and the signature value, are packaged into a video stream.

[0011] Optionally, it also includes: During video transcoding, the metadata of the video image data is either preserved or passed through. Alternatively, the metadata can be extracted from the video image data, and the metadata can be re-added to the video image data when the video image data has been transcoded.

[0012] The present invention also provides a method for acquiring vehicle data, comprising: Acquire video image data; the metadata of the video image data includes vehicle data, and the vehicle data is saved into the metadata according to the above-described vehicle data saving method; The vehicle data is obtained from the metadata of the video image data, and the vehicle data is time-aligned according to the timestamp of the video image data.

[0013] Optionally, obtaining the vehicle data from the metadata of the video image data includes: The vehicle data is obtained from the metadata of the video image data, and it is determined whether the video image data corresponds to the target frame type. If so, the vehicle data is set as the baseline vehicle data; If not, then the vehicle data is recovered based on the previous benchmark vehicle data.

[0014] Optionally, the vehicle data is obtained from the metadata of the video image data, including: The vehicle data and signature value are obtained from the metadata of the video image data, and the vehicle data is verified using the frame identifier of the video image data and the signature value.

[0015] The present invention also provides an in-vehicle data storage device, comprising: The data acquisition module acquires vehicle data and video image data; The vehicle data selection module is used to select target vehicle data from the vehicle data according to the preset metadata capacity and the key attribute value of each vehicle data, so that the capacity of the target vehicle data is less than or equal to the preset metadata capacity and the total key attribute value of the target vehicle data is maximized. The vehicle data storage module is used to package the video image data and its associated target vehicle data into a video stream and save it.

[0016] The present invention also provides an in-vehicle data acquisition device, comprising: A video image data acquisition module is used to acquire video image data; the metadata of the video image data includes vehicle data, and the vehicle data is saved into the metadata according to the above-described vehicle data saving method; The vehicle data acquisition module is used to acquire the vehicle data from the metadata of the video image data, and to time-align the vehicle data according to the timestamp of the video image data.

[0017] This invention provides a method for storing vehicle-mounted data, comprising: acquiring vehicle-mounted data and video image data; selecting target vehicle-mounted data from the vehicle-mounted data according to a preset metadata capacity and the criticality attribute value of each of the vehicle-mounted data, such that the capacity of the target vehicle-mounted data is less than or equal to the preset metadata capacity and the total criticality attribute value of the target vehicle-mounted data is maximized; and packaging the video image data and the associated target vehicle-mounted data into a video stream and storing it.

[0018] The beneficial effects of this invention are as follows: This invention can acquire vehicle-mounted data and video image data, and can use the vehicle-mounted data as metadata for the video image data, packaging it together with the video image data into a video stream and saving it, thus achieving the effect of associated saving of video image data and vehicle-mounted data. Simultaneously, considering the limited metadata space for video image data, this invention can select target vehicle-mounted data from the vehicle-mounted data based on a preset metadata capacity and the criticality attribute values ​​of each vehicle-mounted data point. This ensures that the capacity of the target vehicle-mounted data is less than or equal to the preset metadata capacity and that the total criticality attribute values ​​of the target vehicle-mounted data are maximized. This allows for the preservation of high-value vehicle-mounted data under the constraint of limited metadata capacity.

[0019] The present invention also provides a vehicle data acquisition method, a vehicle data storage device, and a vehicle data acquisition device, which have the above-mentioned beneficial effects. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a method for storing vehicle data according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the decision-making process for generating snapshots and incremental fragments, as provided in an embodiment of the present invention. Figure 3 A flowchart of a vehicle data acquisition method provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an in-vehicle data storage and retrieval system provided in an embodiment of the present invention; Figure 5A timing diagram of the interaction between an encoder and an injection module is provided in an embodiment of the present invention; Figure 6 A structural block diagram of an in-vehicle data storage device provided in an embodiment of the present invention; Figure 7 This is a structural block diagram of an in-vehicle data acquisition device provided in an embodiment of the present invention; Figure 8 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] In post-incident verification, in-vehicle systems typically require synchronized playback of video footage and vehicle status. In related technologies, in-vehicle data and video are usually saved as separate files, with data matching later using timestamps. However, this method is prone to problems such as lost or mismatched files, or corrupted alignment information during file copying, clip editing, transcoding, and distribution. Alternatively, related technologies can use custom frame structures to stitch in-vehicle data and video. However, custom frames disrupt the standard video bitstream, requiring in-vehicle video to be played and transcoded using a dedicated parser, making it difficult to reuse general-purpose players and transcoding toolchains.

[0024] In view of this, regarding the technical problem of how to associate and save in-vehicle data and in-vehicle video, this invention provides an in-vehicle data saving method. This method uses in-vehicle data as metadata for video image data, associates and packages the video image data with the associated in-vehicle data into a video stream, and saves it. Furthermore, it can save high-value in-vehicle data within the constraint of limited metadata capacity. This achieves the associated saving of in-vehicle data and in-vehicle video image data while avoiding damage to the standard video stream structure of in-vehicle video, thereby enabling the reuse of universal players and universal transcoding toolchains.

[0025] The following describes the vehicle data storage method provided in this embodiment. Please refer to... Figure 1 , Figure 1 A flowchart of a vehicle data storage method provided in an embodiment of the present invention, the method may include: S11. Acquire vehicle data and video image data.

[0026] In this embodiment, the vehicle data refers to data characterizing the vehicle's operating status. Vehicle data may include bus messages and status signals. Bus messages are communication messages sent by the vehicle's internal control unit via the vehicle bus (such as CAN bus, Controller Area Network). Status signals are signal data characterizing vehicle operating indicators, such as vehicle speed, engine speed, and fuel level. The methods for acquiring bus messages and status signals can refer to relevant vehicle control technologies.

[0027] Furthermore, this embodiment can set up a status table to continuously record the vehicle's operating status. After obtaining the latest vehicle data, this embodiment can use the vehicle data to update the status table. In this way, when the vehicle data saving operation is triggered, the vehicle data representing the latest vehicle status can be obtained from the status table and saved.

[0028] S12. Select target vehicle data from the vehicle data according to the preset metadata capacity and the criticality attribute values ​​of each vehicle data, so that the capacity of the target vehicle data is less than or equal to the preset metadata capacity and the total criticality attribute value of the target vehicle data is maximized.

[0029] In this embodiment, considering the wide variety of vehicle data, but the limited metadata that can be stored in video image data, and the possibility that it may be difficult to save all the vehicle data under extreme conditions, this embodiment can filter the vehicle data to be saved to obtain high-value target vehicle data, thereby storing as much high-value vehicle data as possible based on the limited metadata capacity of video image data.

[0030] This step selects target vehicle data from the vehicle data based on the preset metadata capacity and the criticality attribute values ​​of each vehicle data point. The goal is to ensure that the total volume of the target vehicle data is less than or equal to the preset metadata capacity and that the total criticality attribute value of the target vehicle data is maximized. The preset metadata capacity constrains the maximum volume of the vehicle data to be stored and can be set based on the available metadata capacity of the video image data. The criticality attribute value characterizes the preservation value of the vehicle data; a higher criticality attribute value indicates higher preservation value and higher preservation priority, while a lower value indicates lower preservation value and lower preservation priority. The selection process for target vehicle data can be represented as follows: ; in, This represents the byte budget for each frame of metadata (i.e., the preset metadata capacity, in bytes, typically 256-1024). This represents the set of candidate entries, which contains the vehicle data to be filtered. The key attribute value of the vehicle data (dimensionless, with a value range of ). ), This refers to the volume of in-vehicle data (in bytes). The above formula means that within the budget... Selecting a subset of items under the premise of This maximizes the total value.

[0031] The following is a detailed introduction to criticality attribute values. Criticality attribute values ​​are determined based on the criticality attributes of the vehicle data. Criticality attributes are inherent properties of the vehicle data and are used to measure its preservation value. Criticality attributes can be fixed static attributes or dynamically adjustable dynamic attributes. For example, in one possible scenario, the value attributes of vehicle data may include the data's preset priority, relevance value, and freshness value, where: The preset priority is a static attribute that is preset based on the type of vehicle data.

[0032] The correlation value can be determined based on the correlation between vehicle events and onboard data, and can change with the vehicle's status, making it a dynamic attribute. Vehicle events refer to operational events detected by the onboard system during vehicle operation, such as turning events and emergency braking events. For example, during emergency braking, the onboard system can detect the event. At this time, the correlation of state signals related to the emergency braking event, such as vehicle speed, brake pressure, and ABS status (Anti-lock Braking System), will be increased, while the correlation of state signals unrelated to the emergency braking event, such as air conditioning signals, can remain low.

[0033] The freshness value can be set based on the data collection time of the vehicle, and it can be calculated in real time over time, making it a dynamic attribute. For example, if a message was received just 1 millisecond ago, it is very "fresh" and has a high freshness value. If a message was received 100 milliseconds ago, its freshness value is low.

[0034] Furthermore, the weights of various key attributes in vehicle data differ. Therefore, corresponding weights can be preset for each key attribute. When calculating the key attribute value, the corresponding weights are used to weight each key attribute. For example, the preset priority, relevance value, and freshness value of vehicle data can be weighted to obtain the key attribute value of the vehicle data. The key attribute value can be represented as: ; in: The event correlation value (dimensionless, with values ​​ranging from 0 to 100). ).

[0035] Preset priority for signals or messages (dimensionless, value) ).

[0036] Freshness value (dimensionless, taking values) ).

[0037] These are the weighting coefficients, and .

[0038] Furthermore, the vehicle data saving operation can be triggered when each frame of video image data is generated. In this embodiment, the video encoder responsible for encoding the vehicle video can be monitored. As soon as it is determined that the video encoder has completed encoding a frame of video image data, a vehicle data saving operation is triggered.

[0039] S13. Package the video image data and its associated target vehicle data into a video stream and save it.

[0040] In this embodiment, video image data and its associated target vehicle data can be packaged into a video stream, thereby achieving the effect of associating and saving video image data with target vehicle data. To achieve the associative saving of vehicle data and video image data without disrupting the standard video stream of the vehicle video, the target vehicle data can be saved as metadata of the video image data.

[0041] For example, in a video stream, one frame of video image data corresponds to one access unit. This access unit stores both the image data and the metadata of the video image data. Since the vehicle data is additional metadata embedded into the video image data in this embodiment, to avoid affecting the normal parsing of the video image data, the vehicle data can be encapsulated into a separate metadata unit and inserted into a preset position in the access unit corresponding to the video image data. In this way, a general video player can ignore the metadata unit containing the vehicle data when parsing the access unit of the video image data, and perform normal parsing of the other data in the access unit. A dedicated parser can then parse the aforementioned metadata unit to obtain the required vehicle data. The same principle applies when using a general transcoding toolchain. Therefore, this embodiment can achieve the associated storage of vehicle data and vehicle video while avoiding damage to the standard video stream of the vehicle video, thus avoiding the shortcomings of not being able to reuse general video players and general transcoding toolchains.

[0042] Specifically, metadata units can be custom-constructed or configured using the metadata unit structures provided in the encoding standard. For example, target vehicle data can be encapsulated into metadata units using H.264 / H.265 encoded Supplemental Enhancement Information (SEI) or AV1 encoded Metadata OBU. H.264 / H.265 encoding and AV1 encoding are different video encoding formats.

[0043] Based on this, packaging video image data and its associated target vehicle data into a video stream can include: S131. Encapsulate the target vehicle data into metadata units.

[0044] S132. Insert the metadata unit into the preset position in the access unit corresponding to the video image data to obtain the video bitstream.

[0045] After inserting the metadata unit, the video stream can be obtained based on the video image data, thus obtaining the in-vehicle video and completing the association and storage of in-vehicle data and in-vehicle video.

[0046] The structure and encapsulation steps of the metadata unit are described below. Taking H.264 / H.265 as an example, this embodiment can preferentially use the SEI message of type user_data_unregistered as the carrier unit, with the payload starting with a 16-byte UUID to identify the data type. In this embodiment, the target vehicle data can be set as a data entry and stored in this metadata unit. To facilitate cross-version compatibility and independent parsing, the metadata unit payload adopts a self-describing structure: ; in: This indicates a fixed UUID, 128 bits, used to identify vehicle data metadata.

[0047] The header information is 4 bytes long and includes the version number and payload type (Type). Type indicates the format of subsequent items, such as 0x01 = raw message, 0x02 = status signal.

[0048] This is the frame display timestamp base value (4-8 bytes, in milliseconds or 90kHz) for the video image data corresponding to this metadata unit. The frame display timestamp base value is the display time of the video image data on the video timeline, which can be understood as "the moment when this frame will be displayed". For example, in a video with a total length of 1 hour, the frame at 5 minutes and 30 seconds has a pts_base value that is the value after converting "5 minutes and 30 seconds" into the agreed time unit.

[0049] The number of vehicle data contained in this unit (2 bytes, value range) ).

[0050] This is an array of items, which allows you to set vehicle data as data items and store them in the items array.

[0051] In addition, since vehicle data can include both bus messages and status signals, this embodiment also provides two data entry formats to facilitate differentiation between these two types, as follows: When Type=0x01 (raw message mode), the format of each message entry is as follows (compatible with CAN / CAN-FD, standard frame / extended frame): ; in: The message identifier (4 bytes, including standard ID / extended ID encoding).

[0052] This is a flag bit (1 byte, indicating whether to extend the frame, whether to use CAN-FD, etc.).

[0053] Data length code (1 byte, value range) ).

[0054] The message data payload (0-64 bytes, length depends on DLC).

[0055] The time offset relative to pts_base (2 bytes, in milliseconds, value range) This embodiment uses the frame display timestamp base value of the video image data as the time starting point. Subsequently, each data entry only needs to write the time offset relative to this time starting point, such as how many milliseconds / clock units later it is. During parsing, the pts_base is used to add this time offset to restore the vehicle data to the video timeline, thereby achieving time alignment.

[0056] When Type=0x02 (status signal mode), it can be represented by signal identifier and numerical code: ; in, This is a semantic field identifier (2 bytes), which is a unique internal number or variable ID assigned to vehicle indicators (such as vehicle speed, engine speed, fuel level) to indicate what indicator this value represents. For example, number 1001 represents "current vehicle speed", number 1002 represents "engine speed", number 1003 represents "remaining fuel percentage", and so on.

[0057] The normalized or quantized numerical encoding (4 bytes).

[0058] Based on this, the target vehicle data can be encapsulated into metadata units, which may include: S1311. Determine the type of target vehicle data.

[0059] S1312. If the target vehicle data is a bus message, then set the timestamp reference value according to the time of generating the video image data, determine the time offset value of the bus message acquisition time relative to the timestamp reference value, and encapsulate the bus message itself and the bus message identifier, message type flag, data length code, and time offset value into a data entry.

[0060] S1313. If the target vehicle data is a status signal, then the status signal and its signal identifier are encapsulated into a data entry.

[0061] S1314. Set header information according to the status version number and type of the target vehicle data.

[0062] S1315. Encapsulate the metadata identifier, header information, timestamp base value, number of data entries, and data entries of the metadata unit into a metadata unit.

[0063] The insertion position of the metadata unit (SEI) is described below. Taking H.264 / H.265 bitstreams as an example, the SEI can be preferentially inserted before the VCL NAL unit within the corresponding access unit (H.265 uses the prefix SEI; H.264 uses the SEI NAL), so that the decoder can obtain the corresponding metadata simultaneously when outputting the frame. The structure of the access unit after insertion satisfies the constraint that "the SEI is located in the same access unit and does not disrupt the original VCL order," that is: ; Furthermore, to support verification of credibility, this embodiment can also sign the metadata unit. To achieve association, this embodiment can jointly sign the frame identifier of the video image data and the metadata unit payload, that is: ; in: This is a metadata summary for the k-th frame.

[0064] For frame identifier (integer, such as frame number or PTS value).

[0065] Metadata payload (byte stream).

[0066] For hash functions (such as SHA-256).

[0067] After obtaining the signature, the digest and frame identifier are written together into the metadata payload. In this way, the parser can detect tampering and determine the presence of missing metadata based on the digest.

[0068] Based on this, the method may also include: S1331. Sign the frame identifier of the video image data and the target vehicle data together to obtain the signature value.

[0069] Accordingly, packaging video image data and its associated target vehicle data into a video stream may include: S1332. Package the video image data and its associated target vehicle data and signature value into a video stream.

[0070] Based on the above embodiments, the present invention can acquire vehicle-mounted data and video image data, and can use the vehicle-mounted data as metadata for the video image data, packaging it together with the video image data into a video stream and saving it, thus achieving the effect of associating and saving video image data with vehicle-mounted data. Meanwhile, considering the limited metadata space for video image data, the present invention can select target vehicle-mounted data from the vehicle-mounted data based on a preset metadata capacity and the criticality attribute values ​​of each vehicle-mounted data, ensuring that the capacity of the target vehicle-mounted data is less than or equal to the preset metadata capacity and that the total criticality attribute values ​​of the target vehicle-mounted data are maximized. This allows for the preservation of high-value vehicle-mounted data under the constraint of limited metadata capacity.

[0071] Based on the above embodiments, to further conserve metadata space and meet the storage requirements of important vehicle data, this embodiment can also provide a method for hierarchical storage of snapshot data and incremental data. This method utilizes temporal redundancy to reduce metadata volume, thereby prioritizing the retention of critical vehicle status change data even under capacity constraints. The hierarchical storage method for snapshot data and incremental data is described below: S21. Acquire vehicle data and video image data.

[0072] S22. Obtain the historical vehicle data corresponding to the last time the vehicle data was successfully saved in the video image data.

[0073] In this embodiment, historical vehicle data refers to the full amount of vehicle data at the time when vehicle data was successfully saved to the video image data last time. As mentioned above, this embodiment can use a status table to continuously record the latest vehicle data. Therefore, when vehicle data was successfully saved to the video image data last time, the status table can be recorded separately to obtain historical vehicle data.

[0074] S23. Determine whether the video image data corresponds to the target frame type.

[0075] S24. If so, then use the full amount of current vehicle data to set a snapshot of the vehicle data; S25. If not, then use the change value of vehicle data between the current vehicle data and the historical vehicle data to set the incremental vehicle data; In steps S23-S25, the snapshot vehicle data can be set based on the full amount of current vehicle data, which refers to the vehicle data recorded in the status table when the video image data is generated. Incremental vehicle data can be set based on the change values ​​between the current vehicle data and historical vehicle data. To reduce metadata volume, this embodiment can set incremental vehicle data only based on the vehicle data change values ​​corresponding to significantly changed vehicle data; that is, incremental vehicle data is determined using vehicle data change values ​​greater than a preset threshold.

[0076] This embodiment can, when determining the target frame type corresponding to the video image data, set snapshot vehicle data (snapshot segments) based on the full amount of current vehicle data and inject the snapshot vehicle data into the video image data; and can, when determining that the video image data does not correspond to the target frame type, determine incremental vehicle data (incremental segments) based on the vehicle data change value between the current vehicle data and historical vehicle data and inject the incremental vehicle data into the video image data. The aforementioned target frame type can be a keyframe type (I-frame).

[0077] In this way, this embodiment can save as complete as possible vehicle data in key frames, while saving incremental vehicle data in non-key frames. It is understood that since not all vehicle data changes at every moment, saving incremental vehicle data in non-key frames can reduce data volume. Furthermore, by saving snapshot vehicle data in each key frame, this application can use the snapshot vehicle data in the key frames as a reference and realign it when parsing the key frames. Thus, if metadata is missing or dropped in a non-key frame, the system only needs to wait for the next set of snapshot vehicle data and realign it when parsing the next set of snapshot vehicle data, avoiding error accumulation.

[0078] For details, please refer to Figure 2 , Figure 2 This is a flowchart illustrating the snapshot and incremental fragment generation decision-making process provided in an embodiment of the present invention. The system represents the state table as a state vector at time t. It contains m signal or semantic fields. A snapshot is written at the keyframe. Write the change amount at non-keyframe locations The change can be expressed as: ; in, It is a vector of change; This is the current state vector; This is the state vector from the last successful embedding.

[0079] To reduce metadata volume, this embodiment can write increments only to components that have changed significantly, and use a change mask. To indicate which components are written, the change mask can be represented as: ; in, The write flag (0 or 1) for the j-th component; The change of the j-th component; The threshold value for the change of the j-th component can be set according to the signal accuracy, such as a vehicle speed threshold of 1 km / h or an acceleration threshold of 0.1g. This is an indicator function; it takes the value 1 if the condition is met, and 0 otherwise.

[0080] Furthermore, the snapshot vehicle data can simultaneously write a status version number and a frame display timestamp (PTS) baseline for realignment and recovery in case of missing metadata or dropped processing links. The parser can then restore the status table based on the video frame display timestamp (PTS) as the main timeline. If missing metadata or dropped processing links occur, the system can wait for the next set of snapshot vehicle data for realignment, preventing error accumulation.

[0081] S26. Based on the preset metadata capacity and the criticality attribute values ​​of each vehicle data, select the target vehicle data from the snapshot vehicle data or incremental vehicle data, so that the capacity of the target vehicle data is less than or equal to the preset metadata capacity and the total criticality attribute value of the target vehicle data is maximized.

[0082] In this step, due to the limited size of metadata, after obtaining the above snapshot vehicle data or incremental vehicle data, the target vehicle data can also be filtered to improve the preservation value of the vehicle data.

[0083] S27. Package the video image data and its associated target vehicle data into a video stream and save it.

[0084] In short, this embodiment supports the following combinations regarding frame type strategies: The keyframe snapshot strategy writes snapshot fragments near keyframes, including a full snapshot of key state fields and the state version number, which facilitates quick acquisition of the complete state after random location.

[0085] The non-critical frame incremental strategy writes incremental fragments near non-critical frames, only writing changed fields or critical messages to reduce the average metadata volume.

[0086] The frame-by-frame complete strategy writes all packets within the time window of each frame near the frame and is only enabled when the bit rate and budget allow or in scenarios using "spanning multiple frames / multiple SEI bearers".

[0087] Based on the above embodiments, to reduce the risk of data loss during video transcoding, this embodiment can also provide two transcoding methods. In one possible case, the method may further include: S31. When performing video transcoding, the metadata of the video image data is retained or passed through. S32, or, extract metadata from the video image data and re-add metadata to the video image data when the video image data has been transcoded.

[0088] In related technologies, stream editing or container transcoding often does not alter the content of the NAL (Network Abstraction Layer Unit), and the vehicle metadata can be retained along with the stream. However, to reduce the risk of metadata loss, this solution still provides the following transcoding strategy: 1. Use a toolchain configuration that supports passing through unknown SEIs or retaining metadata to ensure that metadata units are not discarded during transcoding.

[0089] 2. Extract metadata before transcoding and re-inject it after transcoding to ensure that the output bitstream still carries the aligned metadata.

[0090] This effectively ensures that vehicle data is not lost due to video transcoding, thereby improving the reliability of vehicle data storage.

[0091] Based on the above embodiments, the vehicle data acquisition method provided in this embodiment will be described below. Please refer to... Figure 3 , Figure 3 A flowchart of a vehicle data acquisition method provided in an embodiment of the present invention, the method comprising: S41. Acquire video image data; the metadata of the video image data contains vehicle data, and the vehicle data is saved to the metadata according to the vehicle data saving method described above. S42. Obtain vehicle data from the metadata of video image data, and time-align the vehicle data according to the timestamp of the video image data.

[0092] In this embodiment, video image data can be obtained from the vehicle-mounted video, and vehicle-mounted data can be obtained from the metadata of the video image data. For example, the metadata unit that records vehicle-mounted data can be obtained from the video image data access unit, and vehicle-mounted data can be obtained from the metadata unit. Since the video image data has a timestamp, such as the frame display timestamp PTS, the vehicle-mounted data can be time-aligned based on the timestamp of the video image data, thereby realizing the restoration of the vehicle-mounted data timing.

[0093] Furthermore, to reduce the size of metadata, this embodiment can inject snapshot vehicle data into video image data of the target frame type and incremental vehicle data into video image data of non-target frame types. Based on the changes in vehicle data recorded by the snapshot and incremental vehicle data, the original vehicle data at the time of recording the incremental vehicle data can be recovered. The target frame type can be, for example, a keyframe. Therefore, after obtaining vehicle data from the metadata of the video image data, this embodiment can determine whether the video image data corresponds to the target frame type; if so, the vehicle data can be set as the baseline vehicle data; if not, the vehicle data can be recovered based on the previous baseline vehicle data.

[0094] Based on this, obtaining vehicle data from the metadata of video image data can include: S4211. Obtain vehicle data from the metadata of video image data and determine whether the video image data corresponds to the target frame type; S4212. If so, set the vehicle data to the baseline vehicle data. S4213. If not, then perform data recovery on the vehicle data based on the previous benchmark vehicle data.

[0095] Furthermore, the metadata of the video image data can also record the signature value corresponding to the vehicle data. This signature value is based on the frame identifier settings of the vehicle data and the video image data. Therefore, this embodiment can also obtain the vehicle data and signature value from the metadata of the video image data, and use the frame identifier and signature value of the video image data to verify the vehicle data.

[0096] Based on this, obtaining vehicle data from the metadata of video image data can include: S4221. Obtain vehicle data and signature value from the metadata of video image data, and verify the vehicle data using the frame identifier and signature value of the video image data.

[0097] Based on the above embodiments, the specific structure of an in-vehicle data storage and retrieval system provided in this embodiment is described below with reference to specific schematic diagrams. Please refer to... Figure 4 , Figure 4 This is a schematic diagram of the structure of an in-vehicle data storage and retrieval system provided in an embodiment of the present invention. The system mainly consists of a bus status mirroring module, a metadata packaging module, a capacity control and selection module, a code stream injection module, and a parsing and verification module.

[0098] Core module functions: The bus status mirroring module is responsible for receiving bus messages, parsing out available fields, maintaining a status table, and recording the status version embedded in the previous frame.

[0099] The metadata packaging module is responsible for generating snapshot fragments (snapshot vehicle data) or incremental fragments (incremental vehicle data), and performing compression, verification, and optional signing on the fragments.

[0100] Within each frame's byte budget, the capacity control and selection module selects a subset of vehicle data to be written from the candidate entry set, prioritizing critical messages or semantics through a priority mechanism. The candidate entry set consists of snapshot vehicle data or incremental vehicle data.

[0101] When the video encoder outputs a bitstream, the bitstream injection module inserts metadata units near the corresponding frames to ensure that the bitstream still conforms to the standard and can be decoded and played after injection.

[0102] The parsing and verification module extracts metadata units and parses the payload during the decoding process, restores the status table with the video frame display timestamp (PTS) as the main timeline, and outputs the aligned vehicle status sequence.

[0103] Module collaboration process: During the encoding phase, the bus status mirroring module continuously receives messages and updates the status table. Whenever the video encoder outputs a frame, the system determines whether the frame is a keyframe. If so, the metadata packaging module generates a snapshot segment; otherwise, an incremental segment is generated. Subsequently, the capacity control and selection module filters entries within the byte budget, and finally, the stream injection module inserts the metadata unit into the access unit corresponding to the frame. Please refer to [reference needed]. Figure 5 , Figure 5 This is a timing diagram of the interaction between an encoder and an injection module provided in an embodiment of the present invention.

[0104] During the parsing phase, the parsing and verification module extracts metadata units from the bitstream and restores the status table with the frame display timestamp (PTS) as the main timeline. If missing metadata or verification failure is detected, the system waits for the next snapshot segment to be re-aligned.

[0105] The beneficial effects of this invention are as follows: 1. Metadata is embedded into the video stream to form a strong binding. The video file still carries the corresponding vehicle status data during the copying, editing and distribution process, reducing the risk of mismatch and loss.

[0106] 2. Snapshots and incremental hierarchical coding utilize temporal redundancy to reduce metadata volume, enabling the priority preservation of critical state changes even under capacity-constrained conditions.

[0107] 3. The item selection and priority control mechanism under budget constraints makes it easier to retain event-related information even when the byte budget is insufficient, thus improving the integrity of evidence during critical periods.

[0108] 4. Metadata fragment digests and optional signatures support tamper detection and missing data recovery alignment at the parsing end, improving verification credibility and robustness.

[0109] 5. While ordinary media players can usually play the video normally, a dedicated parser can extract metadata and align the playback, making the engineering adaptation cost more controllable.

[0110] The following describes the vehicle data storage device, vehicle data acquisition device, electronic device, computer-readable storage medium, and computer program product provided in the embodiments of the present invention. The vehicle data storage device, vehicle data acquisition device, electronic device, computer-readable storage medium, and computer program product described below can be referred to in correspondence with the vehicle data storage method and vehicle data acquisition method described above.

[0111] Please refer to Figure 6 , Figure 6 This is a structural block diagram of an in-vehicle data storage device provided in an embodiment of the present invention. The device may include: Data acquisition module 601 acquires vehicle data and video image data; The vehicle data selection module 602 is used to select target vehicle data from vehicle data according to the preset metadata capacity and the key attribute value of each vehicle data, so that the capacity of the target vehicle data is less than or equal to the preset metadata capacity and the total key attribute value of the target vehicle data is maximized. The vehicle data storage module 603 is used to package video image data and its associated target vehicle data into a video stream and save it.

[0112] Optionally, the value attributes of vehicle data include the preset priority of vehicle data, the correlation value determined based on the correlation between vehicle events and vehicle data, and the freshness value determined based on the collection time of vehicle data; The device may also include: The criticality attribute value determination module is used to weight the preset priority, relevance value, and freshness value of the vehicle data to obtain the criticality attribute value of the vehicle data.

[0113] Optionally, the device may further include: The historical vehicle data acquisition module is used to acquire the historical vehicle data corresponding to the last time vehicle data was successfully saved in video image data; The frame type determination module is used to determine whether the video image data corresponds to the target frame type; The snapshot data setting module is used to set snapshot vehicle data using all current vehicle data if the condition is met. The incremental data setting module is used to set incremental vehicle data if no, by using the change value of vehicle data between the current vehicle data and the historical vehicle data; The vehicle data selection module 602 can be used for: Based on the preset metadata capacity and the criticality attribute values ​​of each vehicle data, select the target vehicle data from snapshot vehicle data or incremental vehicle data.

[0114] Optionally, the incremental data setting module can be used for: Incremental vehicle data is determined by using changes in vehicle data that exceed a preset threshold.

[0115] Optionally, the vehicle data storage module 603 may include: The encapsulation submodule is used to encapsulate the target vehicle data into metadata units; The insertion submodule is used to insert metadata units into preset positions in the access units corresponding to video image data to obtain the video bitstream.

[0116] Optionally, encapsulated submodules can be used for: Determine the type of target vehicle data; If the target vehicle data is a bus message, then the timestamp reference value is set according to the time of generating the video image data, the time offset value of the bus message acquisition time relative to the timestamp reference value is determined, and the bus message itself, along with the bus message identifier, message type flag, data length code, and time offset value, are encapsulated into a data entry. If the target vehicle data is a status signal, then the status signal and its identifier are encapsulated into a data entry; Set header information based on the status version number and type of the target vehicle data; The metadata identifier, header information, timestamp baseline value, number of data entries, and data entries of a metadata unit are encapsulated into a metadata unit.

[0117] Optionally, the device may further include: The signature module is used to jointly sign the frame identifier of the video image data and the target vehicle data to obtain a signature value; The vehicle data storage module 603 can be used for: The video image data, along with its associated target vehicle data and signature value, is packaged into a video stream.

[0118] Optionally, the device may further include: The transcoding module is used to retain or pass through the metadata of video image data during video transcoding; or, to extract metadata from video image data and re-add metadata to the video image data when the transcoding is completed.

[0119] Please refer to Figure 7 , Figure 7 This is a structural block diagram of an in-vehicle data acquisition device provided in an embodiment of the present invention. The device may include: The video image data acquisition module 701 is used to acquire video image data; the metadata of the video image data includes vehicle data, and the vehicle data is saved to the metadata according to the vehicle data saving method described above. The vehicle data extraction module 702 is used to obtain vehicle data from the metadata of video image data and to perform time alignment of the vehicle data according to the timestamp of the video image data.

[0120] Optionally, the vehicle data extraction module 702 may include: The acquisition submodule is used to obtain vehicle data from the metadata of video image data and determine whether the video image data corresponds to the target frame type; The baseline setting submodule is used to set the vehicle data as baseline vehicle data if the condition is met. The incremental data recovery submodule is used to recover vehicle data based on the previous baseline vehicle data if no.

[0121] Optionally, the device may further include: The verification module is used to obtain vehicle data and signature values ​​from the metadata of video image data, and to verify the vehicle data using the frame identifier and signature value of the video image data.

[0122] Please refer to Figure 8 , Figure 8 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. The present invention provides an electronic device 10, including a processor 11 and a memory 12; wherein, the memory 12 is used to store a computer program; the processor 11 is used to execute the vehicle data storage method or vehicle data acquisition method provided in the foregoing embodiment when executing the computer program.

[0123] For details regarding the specific process of the above-mentioned vehicle data storage method or vehicle data acquisition method, please refer to the corresponding content provided in the foregoing embodiments, which will not be repeated here.

[0124] Furthermore, the memory 12, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, and the storage method can be temporary storage or permanent storage.

[0125] In addition, the electronic device 10 also includes a power supply 13, a communication interface 14, an input / output interface 15, and a communication bus 16; wherein, the power supply 13 is used to provide operating voltage for each hardware device on the electronic device 10; the communication interface 14 can create a data transmission channel between the electronic device 10 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this invention, and is not specifically limited here; the input / output interface 15 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0126] This invention also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the vehicle data storage method or vehicle data acquisition method as described in the above embodiments.

[0127] Since the embodiments of the computer program product part correspond to the embodiments of the vehicle data storage method and the vehicle data acquisition method part, please refer to the description of the embodiments of the vehicle data storage method and the vehicle data acquisition method part for the embodiments of the computer program product part, and will not be repeated here.

[0128] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the vehicle data storage method or vehicle data acquisition method described in the above embodiments.

[0129] Since the embodiments of the computer-readable storage medium portion correspond to the embodiments of the vehicle data storage method and the vehicle data acquisition method portion, the embodiments of the storage medium portion can be found in the descriptions of the embodiments of the vehicle data storage method and the vehicle data acquisition method portion, and will not be repeated here.

[0130] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0131] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0132] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0133] The vehicle data storage method, vehicle data acquisition method, device, and electronic device provided by this invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. It should be noted that those skilled in the art can make several improvements and modifications to this invention without departing from the principles of this invention, and these improvements and modifications also fall within the protection scope of this invention.

Claims

1. A method for storing vehicle-mounted data, characterized in that, include: Acquire vehicle data and video image data; Based on the preset metadata capacity and the criticality attribute values ​​of each of the vehicle data, target vehicle data is selected from the vehicle data, so that the capacity of the target vehicle data is less than or equal to the preset metadata capacity and the total criticality attribute value of the target vehicle data is maximized. The video image data and its associated target vehicle data are packaged into a video stream and saved.

2. The vehicle data storage method according to claim 1, characterized in that, The criticality attributes of the vehicle data include the preset priority of the vehicle data, the correlation value determined based on the correlation between vehicle events and the vehicle data, and the freshness value determined based on the collection time of the vehicle data. The method further includes: The preset priority, relevance value, and freshness value of the vehicle data are weighted to obtain the keyness attribute value of the vehicle data.

3. The vehicle data storage method according to claim 1, characterized in that, Before selecting the target vehicle data from the vehicle data based on the preset metadata capacity and the criticality attribute values ​​of each of the vehicle data, the process also includes: Retrieve historical vehicle data corresponding to the last time vehicle data was successfully saved from video image data; Determine whether the video image data corresponds to the target frame type; If so, then use the full amount of current vehicle data to set a snapshot of the vehicle data; If not, then incremental vehicle data is set using the vehicle data change value between the current vehicle data and the historical vehicle data; The step of selecting target vehicle data from the vehicle data based on the preset metadata capacity and the criticality attribute values ​​of each of the vehicle data includes: Based on the preset metadata capacity and the criticality attribute values ​​of each of the vehicle data, the target vehicle data is selected from the snapshot vehicle data or the incremental vehicle data.

4. The vehicle data storage method according to claim 3, characterized in that, The step of setting incremental vehicle data using the vehicle data change value between the current vehicle data and the historical vehicle data includes: The incremental vehicle data is determined by the change in vehicle data values ​​that are greater than a preset threshold.

5. The vehicle data storage method according to claim 1, characterized in that, The step of packaging the video image data and its associated target vehicle data into a video stream includes: The target vehicle data is encapsulated into metadata units; The metadata unit is inserted into a preset position in the access unit corresponding to the video image data to obtain the video stream.

6. The vehicle data storage method according to claim 5, characterized in that, The target vehicle data is encapsulated into metadata units, including: Determine the type of the target vehicle data; If the target vehicle data is a bus message, then a timestamp reference value is set according to the time when the video image data is generated, the time offset value of the acquisition time of the bus message relative to the timestamp reference value is determined, and the bus message itself, along with the message identifier, message type flag, data length code, and time offset value of the bus message, are encapsulated into a data entry. If the target vehicle data is a status signal, then the status signal and its signal identifier are encapsulated as the data entry; Set header information according to the status version number and type of the target vehicle data; The metadata unit is encapsulated by the metadata identifier, the header information, the timestamp base value, the number of data entries, and the data entries themselves.

7. The vehicle data storage method according to claim 1, characterized in that, Also includes: The frame identifier of the video image data and the target vehicle data are jointly signed to obtain a signature value; The step of packaging the video image data and its associated target vehicle data into a video stream includes: The video image data, along with the associated target vehicle data and the signature value, are packaged into a video stream.

8. The vehicle data storage method according to claim 1, characterized in that, Also includes: During video transcoding, the metadata of the video image data is either preserved or passed through. Alternatively, the metadata can be extracted from the video image data, and the metadata can be re-added to the video image data when the video image data has been transcoded.

9. A method for acquiring vehicle-mounted data, characterized in that, include: Acquire video image data; the metadata of the video image data includes vehicle data, and the vehicle data is saved to the metadata according to the vehicle data saving method as described in any one of claims 1 to 8; The vehicle data is obtained from the metadata of the video image data, and the vehicle data is time-aligned according to the timestamp of the video image data.

10. The vehicle data acquisition method according to claim 9, characterized in that, The step of obtaining the vehicle data from the metadata of the video image data includes: The vehicle data is obtained from the metadata of the video image data, and it is determined whether the video image data corresponds to the target frame type. If so, the vehicle data is set as the baseline vehicle data; If not, then the vehicle data is recovered based on the previous benchmark vehicle data.

11. The vehicle data acquisition method according to claim 9, characterized in that, Obtaining the vehicle data from the metadata of the video image data includes: The vehicle data and signature value are obtained from the metadata of the video image data, and the vehicle data is verified using the frame identifier of the video image data and the signature value.

12. A vehicle-mounted data storage device, characterized in that, include: The data acquisition module acquires vehicle data and video image data; The vehicle data selection module is used to select target vehicle data from the vehicle data according to the preset metadata capacity and the key attribute value of each vehicle data, so that the capacity of the target vehicle data is less than or equal to the preset metadata capacity and the total key attribute value of the target vehicle data is maximized. The vehicle data storage module is used to package the video image data and its associated target vehicle data into a video stream and save it.

13. A vehicle-mounted data acquisition device, characterized in that, include: A video image data acquisition module is used to acquire video image data; the metadata of the video image data includes vehicle data, and the vehicle data is saved to the metadata according to the vehicle data saving method as described in any one of claims 1 to 8; The vehicle data acquisition module is used to acquire the vehicle data from the metadata of the video image data, and to time-align the vehicle data according to the timestamp of the video image data.

Images