Automatic driving multimode data processing method and device, storage medium and electronic equipment

By judging whether the data task is running for the first time and using the mongodb log library to record the topic partition offset, the breakpoint transmission of the autonomous driving data is solved, and the problem of restarting after abnormal data tasks in the autonomous driving system is saved, saving labor and time costs. Improve the efficiency and reliability of data processing.

CN119066042APending Publication Date: 2024-12-03GUANGZHOU AUTOMOBILE GROUP CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202310626973.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

When the autonomous driving system processes multi-mode data, the data task needs to be restarted after abnormality, resulting in manual intervention, wasting manpower and time costs. When the data volume is large, the start-stop cost is high, and the ability to continue transmission of breakpoints is lacking.

Method used

By determining whether the data task is running for the first time, obtaining or pushing autonomous driving data to the corresponding kafka topic partition, using the mongodb log library to record and restore the topic partition offset, realize automatic breakpoint transmission of data, and avoid restarting the data task.

Benefits of technology

The interruption of data tasks can be achieved without manual intervention, saving manpower and time costs, reducing the cost of multiple starts and stops of data tasks, and improving the reliability and efficiency of autonomous driving data processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066042A_ABST
    Figure CN119066042A_ABST
Patent Text Reader

Abstract

The invention relates to an automatic driving multi-mode data processing method and device and a storage medium, and the method comprises the steps: judging whether a data task is operated for the first time or not when the data task is started; if the data task is operated for the first time, acquiring automatic driving data, and pushing the automatic driving data to a corresponding kafka theme partition according to the type of the automatic driving data; if the data task is not operated for the first time, acquiring a corresponding theme partition offset from a mongodb log library, acquiring automatic driving data, and pushing the automatic driving data to a corresponding kafka theme partition according to the type of the automatic driving data and the theme partition offset; the automatic driving data comprises at least one type of data, and each type of data corresponds to one kafka theme partition. According to the invention, automatic breakpoint resuming of the automatic driving data can be realized without manual intervention.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of multi-mode data processing technology, and specifically to an autonomous driving multi-mode data processing method and its device, storage medium, and electronic device. Background Art

[0002] We are now in an era of rapid development of the automotive industry. In the automotive revolution, the transformation from traditional fuel vehicles to new energy electric vehicles has been carried out. With the empowerment of intelligent, technological and digital capabilities in the automotive industry, it has gained greater room for improvement and provided feasibility for the development of autonomous driving. The autonomous driving system needs to process a large amount of data collected from various sensors, including lidar, cameras, GPS, radar, inertial measurement units and other sensors. In order to effectively process this data, multi-mode data processing is required, and then it is provided to the algorithm model of autonomous driving for deep learning to continuously improve the reliability of the algorithm model. Therefore, autonomous driving multi-mode data requires a large amount of data migration. If the data task is abnormal, in the case of no primary key constraint on the data, the traditional solution is to delete the processed data and restart the data task. Without the ability to resume transmission from breakpoints, manual intervention is required, which wastes manpower and time costs in the process; especially in the processing of multi-mode data for autonomous driving, due to the increasing amount of data, the cost of starting and stopping the data task will become higher and higher. Summary of the invention

[0003] The purpose of this application is to propose a multi-mode data processing method for autonomous driving and its device, as well as a computer-readable storage medium, to achieve breakpoint resumption of autonomous driving data.

[0004] To achieve the above objectives, the present application provides a method for processing multi-mode data for autonomous driving, including:

[0005] When a data task is started, determining whether the data task is being run for the first time;

[0006] If the data task is run for the first time, the autonomous driving data is obtained, and the autonomous driving data is pushed to the corresponding Kafka topic partition according to the type of the autonomous driving data;

[0007] If the data task is not run for the first time, the corresponding topic partition offset is obtained from the MongoDB log library, and the autonomous driving data is obtained, and the autonomous driving data is pushed to the corresponding Kafka topic partition according to the type of the autonomous driving data and the topic partition offset;

[0008] The autonomous driving data includes at least one type of data, and each type of data corresponds to a Kafka topic partition.

[0009] In some embodiments, determining whether the data task is being run for the first time includes:

[0010] Determine whether there is a log corresponding to the data task in the mongodb log library, and whether the log is in a completed state. If not, the data task is running for the first time. If so, the data task is not running for the first time.

[0011] In some embodiments, the method further comprises:

[0012] When the data task is stopped, the preset checkpoint mechanism is triggered, the current topic partition offset corresponding to the data task is obtained, and the current topic partition offset corresponding to the data task is written into the log corresponding to the data task.

[0013] In some embodiments, the method further comprises:

[0014] In the process of pushing the autonomous driving data to the corresponding kafka topic partition, the checkpoint mechanism is periodically triggered according to a preset time interval to obtain the current topic partition offset corresponding to the data task, and the current topic partition offset corresponding to the data task is written into the log corresponding to the data task in the mongodb log library.

[0015] In some embodiments, the method further comprises:

[0016] When the checkpoint mechanism is periodically triggered according to a preset time interval, if the checkpoint is abnormal, the data task is stopped and the abnormal data task state is written into the log corresponding to the data task in the mongodb log library.

[0017] The present application also provides an autonomous driving multi-mode data processing device, the device comprising:

[0018] A data task start module, used to determine whether the data task is being run for the first time when the data task is started;

[0019] A first data push module, configured to obtain the autonomous driving data if the data task is run for the first time, and push the autonomous driving data to a corresponding Kafka topic partition according to the type of the autonomous driving data;

[0020] The second data push module is used to obtain the corresponding topic partition offset from the MongoDB log library and the autonomous driving data if the data task is not run for the first time, and push the autonomous driving data to the corresponding Kafka topic partition according to the type of the autonomous driving data and the topic partition offset;

[0021] The autonomous driving data includes at least one type of data, and each type of data corresponds to a Kafka topic partition.

[0022] In some embodiments, the data task start module is used to:

[0023] Determine whether there is a log corresponding to the data task in the mongodb log library, and whether the log is in a completed state. If not, the data task is running for the first time. If so, the data task is not running for the first time.

[0024] In some embodiments, the device further comprises:

[0025] When the data task is stopped, the preset checkpoint mechanism is triggered, the current topic partition offset corresponding to the data task is obtained, and the current topic partition offset corresponding to the data task is written into the log corresponding to the data task.

[0026] In some embodiments, the device further comprises:

[0027] In the process of pushing the autonomous driving data to the corresponding kafka topic partition, the checkpoint mechanism is periodically triggered according to a preset time interval to obtain the current topic partition offset corresponding to the data task, and the current topic partition offset corresponding to the data task is written into the log corresponding to the data task in the mongodb log library;

[0028] If the checkpoint is abnormal, the data task is stopped and the abnormal data task status is written into the log corresponding to the data task in the mongodb log library.

[0029] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the autonomous driving multi-mode data processing method as described above.

[0030] The embodiment of the present application provides an autonomous driving multi-mode data processing device and its device, and a computer-readable storage medium, which implement autonomous driving multi-mode data processing based on Flink, such as processing of multiple types of data such as video frame images, annotation results, sensor logs, and operation logs. Each type of data has a corresponding Kafka topic partition for storage; when any data task is started, it is determined whether any data task is running for the first time; if it is the first time, the autonomous driving data is obtained, and the autonomous driving data is pushed to one or more corresponding Kafka topic partitions according to the type of the autonomous driving data; if it is not the first time, the corresponding topic partition offset is obtained from the MongoDB log library, and the autonomous driving data is obtained, and the autonomous driving data is pushed to one or more corresponding Kafka topic partitions according to the type of the autonomous driving data and the topic partition offset; through the above means, automatic breakpoint resumption of autonomous driving data is realized, and no manual intervention is required to delete the processed data and then restart the data task, saving labor costs and time costs, and providing effective protection for multiple starts and stops of data tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 This is a flowchart of a multi-mode data processing method for autonomous driving in one embodiment of the present application.

[0033] Figure 2 This is a schematic diagram of the principle of distributing multi-mode data of autonomous driving to various topic partitions of Kafka in one embodiment of the present application.

[0034] Figure 3 This is a schematic diagram of starting, stopping, and resuming breakpoint transmission of a data task in an embodiment of the present application.

[0035] Figure 4 This is a schematic diagram of the offset change during multiple starts and stops of a data task in an embodiment of the present application.

[0036] Figure 5 This is a schematic diagram of a multi-mode data processing device for autonomous driving in one embodiment of the present application. DETAILED DESCRIPTION

[0037] The detailed description of the drawings is intended as an illustration of some embodiments of the present application, and is not intended to represent the only form in which the present application can be implemented. It should be understood that the same or equivalent functions can be accomplished by different embodiments intended to be included in the spirit and scope of the present application.

[0038] An embodiment of the present application provides a method for processing multi-mode data for autonomous driving, see Figure 1 , the method of this embodiment includes the following steps:

[0039] Step S10, when the data task is started, determining whether the data task is being run for the first time;

[0040] Step S20: if the data task is run for the first time, the autonomous driving data is obtained, and the autonomous driving data is pushed to a corresponding Kafka topic partition according to the type of the autonomous driving data;

[0041] Step S30: if the data task is not run for the first time, obtain the corresponding topic partition offset from the MongoDB log library, obtain the autonomous driving data, and push the autonomous driving data to the corresponding Kafka topic partition according to the type of the autonomous driving data and the topic partition offset;

[0042] The autonomous driving data includes at least one type of data, and each type of data corresponds to a Kafka topic partition.

[0043] Specifically, if Figure 2 As shown, the autonomous driving multi-mode data includes video frame images, annotation results, sensor logs, operation logs and other data for parsing and production, and the unstructured and structured data are processed uniformly to generate data including but not limited to json format, and the data is classified according to different data contents to form different topics, which are pushed to the corresponding topic partition of kafka; among them, when the kafka cluster is large enough, the retention time of the topic data can be appropriately extended, and the actual situation can be configured as needed.

[0044] In this embodiment, data with high requirements for timeliness and integrity are collected, and the number of partitions of the corresponding topic is configured according to the amount of data and the logical complexity of processing. If the number is large, it is recommended to configure a large number of partitions, so that the data can be evenly distributed and finally pushed to the corresponding topic partition of Kafka.

[0045] The data task status storage of this embodiment uses mongodb as the log library because this database has the advantages of fast update, strong retrieval performance, ability to store relatively complex data and scalability.

[0046] The method of this embodiment is carried out in a custom offset manner for Kafka consumption. When the data task is started, the topic-partition consumption progress of the completed checkpoint, that is, the topic partition offset (topic partition offset), will be obtained from the MongoDB log library as a prerequisite for the current data task to execute consumption. According to the progress of the last completed data task, for example: consuming data with 3 topic partitions; the last progress: the offset of the topic partition of partition-0 is 102, the offset of the topic partition of partition-1 is 101, and the offset of the topic partition of partition-2 is 100. The meaning of xy is defined as follows: x represents the number of the topic partition, and y represents the offset. Then the offset consumed by the data task will be specified to start consumption from 0-102, 1-101, and 2-100, instead of based on the offset recorded inside Kafka.

[0047] The method of this embodiment implements multi-mode data processing for autonomous driving based on Flink. Multi-mode data processing for autonomous driving can obtain multiple types of data, such as video frame images, annotation results, sensor logs, operation logs, etc. Each type of data has a corresponding Kafka topic partition for storage. When any data task is started, it is determined whether the data task is running for the first time. If it is the first time, the autonomous driving data is obtained, and the autonomous driving data is pushed to one or more corresponding Kafka topic partitions according to the type of the autonomous driving data. If it is not the first time, the corresponding topic partition offset is obtained from the mongodb log library, and the autonomous driving data is obtained, and the autonomous driving data is pushed to one or more corresponding Kafka topic partitions according to the type of the autonomous driving data and the topic partition offset. Through the above means, automatic breakpoint resumption of autonomous driving data is realized, and there is no need for manual intervention to delete the processed data and restart the data task, which effectively saves labor costs and time costs, and provides effective protection for multiple starts and stops of data tasks.

[0048] In some embodiments, the step S10 specifically includes:

[0049] Determine whether there is a log corresponding to the data task in the mongodb log library, and whether the log is in a completed state. If not, the data task is running for the first time. If so, the data task is not running for the first time.

[0050] Specifically, the mongodb log library is an important part of the method of this embodiment to realize data breakpoint resumption. The mongodb log library stores multiple logs, each log corresponds to a topic partition, and is used to record the offset of the corresponding topic partition. It should be noted that the log only records the offset of the corresponding topic partition of the consumer kafka under the normal checkpoint. When the checkpoint is completed, the current state will be written back to the log. The core fields of the mongodb log library are shown in the following table. According to the unique identifier jobId of the data task, it can be determined whether there is an offset log of this data task in the mongodb log library.

[0051]

[0052] In some embodiments, the method further comprises:

[0053] When the data task is stopped, the preset checkpoint mechanism is triggered, the current topic partition offset corresponding to the data task is obtained, and the current topic partition offset corresponding to the data task is written into the log corresponding to the data task.

[0054] Specifically, when the data task is executed for the first time, the offset of consuming Kafka is initialized to start consumption from 0. As time goes by, the data task continues until it needs to be stopped. At the moment of stopping, a Checkpoint mechanism will be triggered actively. The pre-written checkpoint function will write the offset of the current data task consuming Kafka to the corresponding log in the MongoDB log library. When the data task is triggered again next time, it will determine whether there is an offset log of this data task in the MongoDB log library based on the unique identifier of the data task, jobId. If it exists and is a completed log, the offset of this log will be used as the starting value of the specified offset of the consumption of Kafka by this data task execution; the same is true for the subsequent multiple data task starts and stops, and finally the effect of autonomous driving multi-mode data meeting breakpoint continuation is achieved, for example Figure 3 shown.

[0055] In some embodiments, the method further comprises:

[0056] In the process of pushing the autonomous driving data to the corresponding kafka topic partition, the checkpoint mechanism is periodically triggered according to the preset time interval. When the checkpoint is normal, the pre-written Checkpoint function can be called normally to obtain the current topic partition offset corresponding to the data task, and write the current topic partition offset corresponding to the data task into the log corresponding to the data task in the mongodb log library.

[0057] Furthermore, when the checkpoint mechanism is triggered, if the checkpoint is abnormal, it is impossible to obtain the current topic partition offset corresponding to the data task and write the current topic partition offset corresponding to the data task into the log corresponding to the data task in the mongodb log library. In this case, it is necessary to stop the data task immediately and write the abnormal data task status into the log corresponding to the data task in the mongodb log library.

[0058] When starting a data task, the number of data task parallelism settings is determined based on the number of topic partitions. When the number of topic partitions is too large, the parallelism of the Flink data task can be appropriately increased to ensure that one parallelism is responsible for the data processing of one topic partition as much as possible, and then the specified offset is used for the specified topic partition to perform consumption logic. The process is as follows: Figure 4 As shown, in Figure 4 In the example, the Flink data task first reads the real offset of each partition of the last data task, and then the consumption continues according to the last offset. The parallelism is set to 3, including 3 subtasks 0, 1, and 2, and there are three topic partitions, 0 partition, 1 partition, and 2 partition. The offset of real-time consumption of Kafka is recorded in the corresponding log of the MongoDB log library; the checkpoint callback function period (preset time interval) can be dynamically configured, and the preset time interval includes but is not limited to not less than 5 minutes, which can reduce the number of interactions with MongoDB. When the checkpoint callback is performed, the offset is actively submitted to Kafka, which can also reduce the number of interactions with Kafka.

[0059] It should be noted that during the execution of Flink data tasks, the way to process data is concentrated in rewriting transform processing chain operators such as richFlatMapFunction and richFilterFunction. To meet business needs, this embodiment will need to write the required operators before the data task is executed, and register them in the flink cluster environment. Finally, during the execution of the data task, they will be automatically recognized. In this embodiment, Flink uses the characteristics of different target storage components to rewrite RichSinkFunction to consume kafka data for processing and storage, such as: hive writes using parquetwriter compression format batch writes, es writes using buckrequest batch writes, etc., and there are corresponding methods according to different components to rewrite RichSinkFunction; the batch write amount and write parallelism are configurable.

[0060] In the process of executing Flink data tasks, the method of this embodiment enables the periodic checkpoint mechanism by default. Traditional checkpoint mechanisms include memory, Filesystem, Rocksdb and other storage methods. These methods provide a fast storage channel. However, most data tasks require incremental data or breakpoint-resume data, and a unified viewing method is required. Or data tasks that need real-time progress monitoring require a method that can be used not only for storage but also for fast retrieval and controllability. Therefore, the method of this embodiment rewrites CheckpointListner and CheckpointedFunction to provide a custom checkpoint mechanism for Upsert to mongodb. If the checkpoint is not completed normally, the Upsert action is not performed. In this way, the data retained in mongodb is the data in the last normal completion state.

[0061] The method of this embodiment establishes a closed-loop data task execution status record, provides an efficient and sustainable breakpoint resume capability, and reduces the cost loss caused by starting and stopping data tasks. Through the method of this embodiment, the life cycle control of the smallest unit data task is solved, providing a reliable base capability for subsequent multi-mode data processing of autonomous driving with large data volumes and large data tasks.

[0062] See also Figure 5 Corresponding to the method of the above embodiment, another embodiment of the present application further provides an autonomous driving multi-mode data processing device, the device comprising:

[0063] A data task start module, used to determine whether the data task is being run for the first time when the data task is started;

[0064] A first data push module, configured to obtain the autonomous driving data if the data task is run for the first time, and push the autonomous driving data to a corresponding Kafka topic partition according to the type of the autonomous driving data;

[0065] The second data push module is used to obtain the corresponding topic partition offset from the MongoDB log library and the autonomous driving data if the data task is not run for the first time, and push the autonomous driving data to the corresponding Kafka topic partition according to the type of the autonomous driving data and the topic partition offset;

[0066] The autonomous driving data includes at least one type of data, and each type of data corresponds to a Kafka topic partition.

[0067] In some embodiments, the data task starting module is used to:

[0068] Determine whether there is a log corresponding to the data task in the mongodb log library, and whether the log is in a completed state. If not, the data task is running for the first time. If so, the data task is not running for the first time.

[0069] In some embodiments, the apparatus further comprises:

[0070] When the data task is stopped, the preset checkpoint mechanism is triggered, the current topic partition offset corresponding to the data task is obtained, and the current topic partition offset corresponding to the data task is written into the log corresponding to the data task.

[0071] In some embodiments, the apparatus further comprises:

[0072] In the process of pushing the autonomous driving data to the corresponding kafka topic partition, the checkpoint mechanism is periodically triggered according to a preset time interval to obtain the current topic partition offset corresponding to the data task, and the current topic partition offset corresponding to the data task is written into the log corresponding to the data task in the mongodb log library;

[0073] If the checkpoint is abnormal, the data task is stopped and the abnormal data task status is written into the log corresponding to the data task in the mongodb log library.

[0074] The above-described embodiment of the multi-mode data processing device for autonomous driving is merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components of the modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the multi-mode data processing device for autonomous driving of the embodiment.

[0075] It should be noted that the autonomous driving multi-mode data processing device of the above-mentioned embodiment corresponds to the autonomous driving multi-mode data processing method of the above-mentioned embodiment. Therefore, the parts of the autonomous driving multi-mode data processing device of the above-mentioned embodiment that are not described in detail can be obtained by referring to the contents of the autonomous driving multi-mode data processing method of the above-mentioned embodiment. That is, the specific steps recorded in the autonomous driving multi-mode data processing method of the above-mentioned embodiment can be understood as the functions that can be realized by the autonomous driving multi-mode data processing device of the above-mentioned embodiment, and will not be repeated here.

[0076] Furthermore, if the autonomous driving multi-mode data processing device of the above-mentioned embodiment is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0077] Another embodiment of the present application further proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the autonomous driving multi-mode data processing method as described in the above embodiment.

[0078] Specifically, the computer-readable storage medium may include: any entity or recording medium that can carry the computer program instructions, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0079] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for processing multi-mode data for autonomous driving, characterized in that: The method comprises: When a data task is started, determining whether the data task is being run for the first time; If the data task is run for the first time, the autonomous driving data is obtained, and the autonomous driving data is pushed to the corresponding Kafka topic partition according to the type of the autonomous driving data; If the data task is not run for the first time, the corresponding topic partition offset is obtained from the MongoDB log library, and the autonomous driving data is obtained, and the autonomous driving data is pushed to the corresponding Kafka topic partition according to the type of the autonomous driving data and the topic partition offset; The autonomous driving data includes at least one type of data, and each type of data corresponds to a Kafka topic partition.

2. The method according to claim 1, characterized in that The determining whether the data task is being run for the first time includes: Determine whether there is a log corresponding to the data task in the mongodb log library, and whether the log is in a completed state. If not, the data task is running for the first time. If so, the data task is not running for the first time.

3. The method according to claim 2, characterized in that The method further comprises: When the data task is stopped, the preset checkpoint mechanism is triggered, the current topic partition offset corresponding to the data task is obtained, and the current topic partition offset corresponding to the data task is written into the log corresponding to the data task.

4. The method according to claim 3, characterized in that The method further comprises: In the process of pushing the autonomous driving data to the corresponding kafka topic partition, the checkpoint mechanism is periodically triggered according to a preset time interval to obtain the current topic partition offset corresponding to the data task, and the current topic partition offset corresponding to the data task is written into the log corresponding to the data task in the mongodb log library.

5. The method according to claim 1, characterized in that The method further comprises: When the checkpoint mechanism is periodically triggered according to a preset time interval, if the checkpoint is abnormal, the data task is stopped and the abnormal data task state is written into the log corresponding to the data task in the mongodb log library.

6. An autonomous driving multi-mode data processing device, characterized in that: The device comprises: A data task start module, used to determine whether the data task is being run for the first time when the data task is started; A first data push module, configured to obtain the autonomous driving data if the data task is run for the first time, and push the autonomous driving data to a corresponding Kafka topic partition according to the type of the autonomous driving data; The second data push module is used to obtain the corresponding topic partition offset from the MongoDB log library and the autonomous driving data if the data task is not run for the first time, and push the autonomous driving data to the corresponding Kafka topic partition according to the type of the autonomous driving data and the topic partition offset; The autonomous driving data includes at least one type of data, and each type of data corresponds to a Kafka topic partition.

7. The device according to claim 6, characterized in that The data task start module is used to: Determine whether there is a log corresponding to the data task in the mongodb log library, and whether the log is in a completed state. If not, the data task is running for the first time. If so, the data task is not running for the first time.

8. The device according to claim 7, characterized in that The device also includes: When the data task is stopped, the preset checkpoint mechanism is triggered, the current topic partition offset corresponding to the data task is obtained, and the current topic partition offset corresponding to the data task is written into the log corresponding to the data task.

9. The device according to claim 8, characterized in that The device also includes: In the process of pushing the autonomous driving data to the corresponding kafka topic partition, the checkpoint mechanism is periodically triggered according to a preset time interval to obtain the current topic partition offset corresponding to the data task, and the current topic partition offset corresponding to the data task is written into the log corresponding to the data task in the mongodb log library; If the checkpoint is abnormal, the data task is stopped and the abnormal data task status is written into the log corresponding to the data task in the mongodb log library.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for processing multi-mode data for autonomous driving according to any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Data transmitting method and device

    CN110825533A

  • Driving mode switching method and system of intelligent driving vehicle

    CN111717206A

  • Simulation test method and device of automatic driving system and storage medium

    CN112364439A

  • Message processing method and device, electronic equipment and readable storage medium

    CN113779149A

  • Semantization processing method and device for automatic driving log data and storage medium

    CN114691873A