Abnormality processing method and device, electronic equipment and storage medium
Through the collaborative work of edge computing module and cloud platform, FIFO cache queue caching and back-passing data, the high storage cost and resource utilization of Badcase processing in autonomous driving technology are solved, real-time and generalization are improved, and the Low-cost, efficient data processing and storage.
Patent Information
- Application Number
- CN202411269361.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2024-12-13
AI Technical Summary
When handling Badcase, existing autonomous driving technology has problems such as high storage costs, high resource occupation, low data value, poor real-time and poor generalization, which limits the improvement of autonomous driving technology.
Using edge computing module and cloud platform to work together, the original data flow is cached through the FIFO cache queue, and the original data written to the disk is returned to the cloud platform according to the target time period and target data requirements in the exception event processing request, to realize Concurrent processing and low-cost storage.
It improves the real-time and generalization of Badcase, reduces the CPU and IO resources of edge computing modules, reduces disk storage costs, and realizes low-cost and efficient data storage and processing.
Smart Images
Figure CN119131498A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of autonomous driving technology, and in particular to an exception handling method, device, electronic device, and storage medium. Background Art
[0002] As the application of autonomous driving technology becomes more and more widespread, the improvement and enhancement of autonomous driving technology partly relies on solving bad cases that occur in real scenarios. Bad cases can usually be understood as abnormal events that occur during the autonomous driving process, such as abnormalities when the autonomous driving vehicle is taken over, or the condition of the autonomous driving vehicle when an abnormality occurs.
[0003] The existing method of collecting Badcases requires storing all the original data, and then mining Badcases offline when needed. This method has problems such as high storage cost, high resource usage, low data value, and poor real-time performance. It will also lead to problems such as poor generalization of Badcases, which has limitations on the improvement of autonomous driving technology. Summary of the invention
[0004] The embodiments of the present application provide an exception handling method, device, electronic device, and storage medium to optimize the Badcase processing process and improve the generalization of Badcase.
[0005] The present application embodiment adopts the following technical solutions:
[0006] In a first aspect, an embodiment of the present application provides an exception handling method, which is applied to an edge computing module, and the method includes:
[0007] Receiving an original data stream collected by a road-side perception module, wherein the road-side perception module includes an image sensor and a radar sensor, and the original data stream includes a video image data stream and a laser point cloud data stream;
[0008] Recognize the original data stream in real time to obtain the perception data stream, and send it to the cloud platform;
[0009] Using a FIFO buffer queue to buffer the original data currently received in the original data stream;
[0010] receiving an abnormal event processing request issued by the cloud platform, wherein the abnormal event includes an abnormal situation of autonomous driving obtained based on the perception data stream;
[0011] According to the target time period and target data requirements in the abnormal event processing request, the currently received original data cached by the FIFO cache queue is written to the disk device and the original data written to the disk is transmitted back to the cloud platform.
[0012] In some embodiments, the real-time identification of the original data stream to obtain the perception data stream and sending it to the cloud platform includes:
[0013] Based on a pre-saved algorithm, identifying traffic participants in the raw data stream in real time;
[0014] The perception data stream of the traffic participants in the original data stream is transmitted to the cloud platform.
[0015] In some embodiments, the adopting of a FIFO buffer queue to buffer the currently received original data in the original data stream includes:
[0016] The original data in the original data stream is cached based on a memory or a disk device.
[0017] In some embodiments, writing the currently received raw data cached by the FIFO buffer queue to a disk and transmitting the raw data written to the disk device back to the cloud platform according to the target time period and the target data requirement in the abnormal event processing request includes:
[0018] According to the target recording time period in the abnormal event processing request issued by the cloud platform, writing the original data cached by the FIFO cache queue corresponding to the target recording time period into a disk device;
[0019] According to the target recording data requirements, the original data written to the disk device is transmitted back to the cloud platform.
[0020] In a second aspect, an embodiment of the present application further provides an exception handling method, which is applied to a cloud platform, and the method includes:
[0021] Based on the business rules, calculating whether an abnormal event occurs in the perception data stream, wherein the abnormal event includes an abnormal situation of autonomous driving obtained based on the perception data stream;
[0022] If an abnormal event occurs in the calculation of the perception data stream, sending the abnormal event processing request to the edge computing module;
[0023] Receive the original data written to the disk device returned by the edge computing module based on the abnormal event processing request, where the original data written to the disk device includes the original data cached in the FIFO cache queue of the edge computing module;
[0024] The original data is combined and packaged with the perception data stream to obtain an abnormal event data packet.
[0025] In some embodiments, if an abnormal event occurs in the calculation of the perception data stream, sending the abnormal event processing request to the edge computing module includes:
[0026] Based on the TCP long link, receiving the perception data stream of the edge computing module in real time;
[0027] If an abnormal event occurs in the perception data stream and needs to be recorded, the abnormal event processing request is sent to the edge computing module, and the abnormal event processing request includes the time period that needs to be recorded and the data that needs to be recorded.
[0028] In some embodiments, the method further comprises:
[0029] Receive the perception data stream uploaded by the edge computing module, where the perception data stream is obtained based on real-time recognition of the original data stream collected by the road-side or vehicle-side perception module, where the road-side or vehicle-side perception module includes an image sensor and a radar sensor, and the original data stream includes a video picture data stream and a laser point cloud data stream.
[0030] In a third aspect, an embodiment of the present application further provides an exception handling device, wherein the device comprises:
[0031] A first receiving module is used to receive the original data stream collected by the road-side perception module, the road-side perception module includes an image sensor and a radar sensor, and the original data stream includes a video image data stream and a laser point cloud data stream;
[0032] A sending module, used for obtaining a perception data stream according to the original data stream in real time identification, and sending the perception data stream to the cloud platform;
[0033] A second receiving module, configured to use a FIFO buffer queue to buffer the original data currently received in the original data stream;
[0034] A sending module, used for receiving an abnormal event processing request sent by the cloud platform, wherein the abnormal event includes an abnormal situation of autonomous driving obtained based on the perception data stream;
[0035] The writing module is used to write the currently received original data cached by the FIFO cache queue to the disk according to the target time period and target data requirements in the abnormal event processing request, and transmit the original data written to the disk back to the cloud platform.
[0036] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory arranged to store computer executable instructions, wherein the executable instructions, when executed, cause the processor to perform the above method.
[0037] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes the above method.
[0038] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: receiving the original data stream collected by the road-side perception module, wherein the road-side perception module includes an image sensor and a radar sensor, and the original data stream includes a video picture data stream and a laser point cloud data stream. According to the original data stream, the perception data stream is obtained in real time and sent to the cloud platform. The FIFO cache queue is used to cache the original data currently received in the original data stream, and the process here is concurrent processing. Receive the abnormal event processing request issued by the cloud platform, and the abnormal event includes the abnormal situation of autonomous driving obtained based on the perception data stream. Finally, according to the target time period and target data requirements in the abnormal event processing request, the currently received original data cached by the FIFO cache queue is written to the disk device and the original data written to the disk is transmitted back to the cloud platform. The original data written to the disk only includes the target time period and the target data requirements, and there is no need to save all the original data. Not only can the real-time performance of the abnormal processing be improved, but also the generalization can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0040] Figure 1 This is a flow chart of the exception handling method in the embodiment of the present application (edge computing module);
[0041] Figure 2 This is a flow chart of the exception handling method in the embodiment of the present application (cloud platform);
[0042] Figure 3 This is a schematic diagram of the structure of an exception handling device in an embodiment of the present application;
[0043] Figure 4 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION
[0044] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0045] The exception handling method in the embodiment of the present application can be used in a system architecture including a road-side perception module, an edge computing module, and a cloud platform, wherein the road-side perception module includes multiple road-side perception modules, and the multiple road-side perception modules are connected to at least one edge computing module, and the edge computing module is connected to the cloud platform. The edge computing module has the ability to process raw data streams, the road-side perception module is mainly used to collect data, and the cloud platform is used to receive the perception data stream.
[0046] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0047] The present application embodiment provides an exception handling method, such as Figure 1 As shown, a schematic diagram of the exception handling method in an embodiment of the present application is provided, and the method at least includes the following steps S110 to S150:
[0048] Step S110, receiving the original data stream collected by the road-side perception module, the road-side perception module includes an image sensor and a radar sensor, and the original data stream includes a video picture data stream and a laser point cloud data stream.
[0049] The edge computing module receives the raw data stream collected by the roadside perception module. It can be understood that the roadside perception module includes an image sensor and a radar sensor, the image sensor uses a camera, etc., and the radar sensor includes a laser radar, a millimeter wave radar, etc.
[0050] The original data stream mainly includes but is not limited to video image data stream and laser point cloud data stream. Each road-end perception module can collect the original data stream in the target intersection and road section and send it to the edge computing module.
[0051] It should be noted that the roadside sensing module can be set on a roadside single pole or other roadside equipment, and those skilled in the art can make a selection according to actual needs.
[0052] Step S120, obtaining a perception data stream based on the original data stream in real time identification, and sending it to the cloud platform.
[0053] The edge computing module obtains the perception data stream through real-time recognition based on the received raw data stream. For example, the perception data stream includes but is not limited to the vehicle position, vehicle heading angle, vehicle speed, etc. The edge computing module will continuously send the perception data stream to the cloud platform and will continue to send it throughout the entire autonomous driving process.
[0054] Step S130: Using a FIFO buffer queue to buffer the original data currently received in the original data stream.
[0055] It should be noted that there is no strict order between step S120 and step S130, and they can usually be processed concurrently. The edge computing module will use the FIFO cache queue in the edge computing module to cache the currently received original data according to the original data stream. The "original data" here refers to the "original data" in step S120.
[0056] Step S140, receiving an abnormal event processing request issued by the cloud platform, where the abnormal event includes an abnormal situation of autonomous driving obtained based on the perception data stream.
[0057] The edge computing module will decide whether to further process the currently received raw data cached in the FIFO cache queue based on the processing request received from the cloud platform.
[0058] Abnormal events, namely Badcases, are abnormal situations of autonomous driving obtained based on the perception data stream. It should be noted that the abnormal events here are detected and judged on the cloud platform, and are not performed on the edge computing module.
[0059] Step S150, according to the target time period and target data requirements in the abnormal event processing request, the currently received original data cached by the FIFO cache queue is written to the disk device and the original data written to the disk is transmitted back to the cloud platform.
[0060] The edge computing module writes the currently received original data cached by the FIFO cache queue to the disk device according to the target time period in the abnormal event processing request, which is usually the time points before and after the abnormal event occurs, and the target data requirement, which is usually the original data corresponding to the time points before and after the abnormal event occurs. The original data written to the disk is then transmitted back to the cloud platform.
[0061] It can be understood that since the perception data stream has been received on the cloud platform, the original data written to the disk can be packaged together with the perception data after processing on the cloud platform and saved as Badcase in the cloud platform.
[0062] Through the above method, the CPU and IO resource usage of the edge computing module is reduced, and the disk storage cost of the additional edge computing module is also reduced.
[0063] Through the above method, the perceived data stream is obtained according to the real-time identification of the original data stream and sent to the cloud platform; the FIFO cache queue is used to cache the original data currently received in the original data stream. When the abnormal event processing request issued by the cloud platform is received, according to the target time period and target data requirements in the abnormal event processing request, the currently received original data cached by the FIFO cache queue is written to the disk device and the original data written to the disk is transmitted back to the cloud platform. It is not necessary to store all the original data, so the real-time and generalization of obtaining Badcase can be improved.
[0064] Through the above method, the real-time perception data stream is used to calculate the Badcases that occur in the current autonomous driving in real time. The Badcases are calculated based on the real-time data stream, which improves the real-time performance of collection and recording while meeting the low-cost data storage requirements.
[0065] In one embodiment of the present application, the perception data stream is obtained according to the real-time identification of the original data stream and sent to the cloud platform, including: based on a pre-saved algorithm, real-time identification of traffic participants in the original data stream; and transmission of the perception data stream of the traffic participants in the original data stream to the cloud platform.
[0066] In order to reduce the processing complexity of the cloud platform for the original data stream collected by the road side, the edge computing module identifies the traffic participants in the original data stream in real time based on the pre-saved algorithm, and then transmits the perception data stream of the traffic participants in the original data stream to the cloud platform. The perception data can be continuously sent to the cloud platform, and the original data stream will not be directly sent to the cloud platform.
[0067] In one embodiment of the present application, the adopting of a FIFO cache queue to cache the currently received original data in the original data stream includes: caching the original data in the original data stream based on a memory or a disk device.
[0068] Use caching technology to avoid continuously occupying the CPU, IO, and disk resources of the edge computing module. The currently received raw data can be cached based on a bounded FIFO queue.
[0069] It should be noted that the edge computing module not only caches the original data in the original data stream, but also identifies the original data and obtains the perception data.
[0070] In one embodiment of the present application, according to the target time period and target data requirements in the abnormal event processing request, the currently received original data cached by the FIFO cache queue is written to the disk and the original data written to the disk device is transmitted back to the cloud platform, including: according to the target recording time period in the abnormal event processing request issued by the cloud platform, the original data cached by the FIFO cache queue corresponding to the target recording time period is written to the disk device; according to the target recording data requirements, the original data written to the disk device is transmitted back to the cloud platform.
[0071] In order to achieve the purpose of real-time collection and recording, the edge computing module finds out the original data cached in the FIFO cache queue corresponding to the target recording time period according to the target recording time period in the abnormal event processing request sent by the cloud platform, and writes it to the disk device; in this way, according to the target recording data requirements, the original data written to the disk device is transmitted back to the cloud platform. It can be understood that not all original data will be written to the disk device of the edge computing module; instead, the original data cached in the current queue is written to the disk device according to the recording information carried in the abnormal event processing request transmitted back by the cloud platform, and the original data written to the disk device is transmitted back to the cloud platform.
[0072] like Figure 2 As shown, an exception handling method is provided in an embodiment of the present application, wherein the method is applied to a cloud platform and includes:
[0073] The exception handling method in the embodiment of the present application can be used for a system architecture including a road-side / vehicle-side perception module, an edge computing module, and a cloud platform, wherein the road-side / vehicle-side perception module includes one or more, and one or more road-side / vehicle-side perception modules are connected to at least one edge computing module, and the edge computing module is connected to the cloud platform. Usually, the cloud platform and the road-side / vehicle-side perception module are in different locations, and the data received from the road-side perception module needs to be processed by the edge computing module. The edge computing module has the ability to process raw data streams, the road-side / vehicle-side perception module is mainly used to collect data, and the cloud platform is used to receive the perception data stream and form a Badcase package together with the corresponding raw data.
[0074] Step S210, based on business rules, calculate whether an abnormal event occurs in the perception data stream, and the abnormal event includes an abnormal situation of autonomous driving obtained based on the perception data stream.
[0075] The cloud platform will calculate whether there are abnormal events in the perception data stream according to business rules. Abnormal events, namely Badcases, include but are not limited to abnormal conditions of autonomous driving obtained from the perception data stream.
[0076] It should be noted that the business rules are determined based on actual business scenarios and actual business needs, and technical personnel in this field can obtain corresponding business rules based on actual business scenarios.
[0077] Step S220: If an abnormal event occurs in the calculation of the perception data stream, the abnormal event processing request is sent to the edge computing module.
[0078] If the cloud platform calculates an abnormal event in the perception data stream, it will send the abnormal event processing request to the edge computing module. The edge computing module packages the data into a Badcase according to the time period and data requirements and then sends it back to the cloud platform.
[0079] Step S230, receiving the original data written to the disk device returned by the edge computing module based on the processing request of the abnormal event, the original data written to the disk device includes the original data cached in the FIFO cache queue of the edge computing module.
[0080] In addition to receiving the perception data stream, the cloud platform also receives the original data written to the disk device that is returned by the edge computing module based on the processing request of the abnormal event. The original data written to the disk device is not the original data stream collected by the road-side / vehicle-side perception module, but the original data stream corresponding to the abnormal event determined by the cloud platform after judgment. For example, the time of the perception data stream is 13:00, and the time period of the abnormal event is 13:20-14:00, then the original data written to the disk device corresponds to the original data uploaded in the road-side / vehicle-side perception module between 13:20-14:00. In this way, the original data written to the disk device will include the original data currently cached in the FIFO cache queue of the edge computing module. Similarly, the edge computing module will simultaneously process and identify the perception data stream, and cache the original data when the abnormality occurs.
[0081] Step S240: Combine and package the original data with the perception data stream to obtain an abnormal event data packet.
[0082] On the cloud platform, the original data and the perception data stream are combined and packaged to obtain the abnormal event data packet, namely Badcase.
[0083] The roadside / vehicle-side perception module collects video images, laser point clouds and other streams from the original data and transmits them to the edge computing module. The edge computing module uses algorithms to identify traffic participants in the original data stream in real time. The edge computing transmits the identified traffic participant data stream to the cloud platform in real time. Then, the cloud platform calculates whether a Badcase occurs based on business rules. If a Badcase occurs, it sends a related processing request to the edge computing module. The edge computing module packages the data into a Badcase based on the time period and data requirements, and sends it back to the cloud platform.
[0084] Through the above method, the cloud platform calculates whether an abnormal event occurs in the perception data stream based on business rules; if the cloud platform calculates that an abnormal event occurs in the perception data stream, it sends the abnormal event processing request to the edge computing module. In this way, the cloud platform receives the original data written to the disk device and the perception data stream returned by the edge computing module based on the abnormal event processing request, and obtains the abnormal event data packet after packaging.
[0085] In one embodiment of the present application, if an abnormal event occurs in the calculation of the perception data stream, the abnormal event processing request is sent to the edge computing module, including: based on the TCP long link, the perception data stream of the edge computing module is received in real time; if an abnormal event occurs in the calculation of the perception data stream and needs to be recorded, the abnormal event processing request is sent to the edge computing module, and the abnormal event processing request includes the time period to be recorded and the data to be recorded.
[0086] The cloud platform receives the perception semantic result data from the edge computing module in real time based on the long TCP connection. The cloud platform then determines whether the Badcase needs to be recorded at the current moment according to the business rules. If it needs to be recorded, the edge computing module uses the http protocol to send a recording data request to the edge computing module. After receiving the request, the edge computing module archives the data cached by the current edge computing module to form a Badcase according to the time period and content to be recorded contained in the http request, and sends it back to the cloud platform.
[0087] In one embodiment of the present application, the method also includes: receiving a perception data stream uploaded by an edge computing module, wherein the perception data stream is obtained by real-time identification of an original data stream collected by a road-side or vehicle-side perception module, wherein the road-side or vehicle-side perception module includes an image sensor and a radar sensor, and the original data stream includes a video picture data stream and a laser point cloud data stream.
[0088] The cloud platform receives the perception data stream uploaded by the edge computing module. The perception data stream is obtained by real-time recognition of the original data stream collected by the road-side or vehicle-side perception module, and the corresponding road-side or vehicle-side perception module includes the image sensor and radar sensor on the vehicle side, or the image sensor and radar sensor on the road side. The original data stream includes the video image data stream and laser point cloud data stream on the vehicle side or road side.
[0089] The embodiment of the present application also provides an exception handling device 300, such as Figure 3 As shown, a schematic diagram of the structure of an exception handling device in an embodiment of the present application is provided, which is applied to an edge computing module. The exception handling device 300 at least includes: a first receiving module 310, a sending module 320, a second receiving module 330, a third receiving module 340 and a writing module 350, wherein:
[0090] In one embodiment of the present application, the first receiving module 310 is specifically used to: receive the original data stream collected by the road-side perception module, the road-side perception module includes an image sensor and a radar sensor, and the original data stream includes a video picture data stream and a laser point cloud data stream.
[0091] The edge computing module receives the raw data stream collected by the roadside perception module. It can be understood that the roadside perception module includes an image sensor and a radar sensor, the image sensor uses a camera, etc., and the radar sensor includes a laser radar, a millimeter wave radar, etc.
[0092] The original data stream mainly includes but is not limited to video image data stream and laser point cloud data stream. Each road-end perception module can collect the original data stream in the target intersection and road section and send it to the edge computing module.
[0093] It should be noted that the roadside sensing module can be set on a roadside single pole or other roadside equipment, and those skilled in the art can make a selection according to actual needs.
[0094] In one embodiment of the present application, the sending module 320 is specifically used to: obtain the perception data stream according to the original data stream in real time identification, and send it to the cloud platform.
[0095] The edge computing module obtains the perception data stream through real-time recognition based on the received raw data stream. For example, the perception data stream includes but is not limited to the vehicle position, vehicle heading angle, vehicle speed, etc. The edge computing module will continuously send the perception data stream to the cloud platform and will continue to send it throughout the entire autonomous driving process.
[0096] In one embodiment of the present application, the second receiving module 330 is specifically configured to: use a FIFO buffer queue to buffer the original data currently received in the original data stream.
[0097] It should be noted that there is no strict order between step S120 and step S130, and they can usually be processed concurrently. The edge computing module will use the FIFO cache queue in the edge computing module to cache the currently received original data according to the original data stream. The "original data" here refers to the "original data" in step S120.
[0098] In one embodiment of the present application, the third reception 340 is specifically used to: receive an abnormal event processing request issued by the cloud platform, and the abnormal event includes an autonomous driving abnormal situation obtained based on the perception data stream.
[0099] The edge computing module will decide whether to further process the currently received raw data cached in the FIFO cache queue based on the processing request received from the cloud platform.
[0100] Abnormal events, namely Badcases, are abnormal situations of autonomous driving obtained based on the perception data stream. It should be noted that the abnormal events here are detected and judged on the cloud platform, and are not performed on the edge computing module.
[0101] In one embodiment of the present application, the write module 350 is specifically used to: write the currently received original data cached by the FIFO cache queue to the disk device and transmit the original data written to the disk back to the cloud platform according to the target time period and target data requirements in the abnormal event processing request.
[0102] The edge computing module writes the currently received original data cached by the FIFO cache queue to the disk device according to the target time period in the abnormal event processing request, which is usually the time points before and after the abnormal event occurs, and the target data requirement, which is usually the original data corresponding to the time points before and after the abnormal event occurs. The original data written to the disk is then transmitted back to the cloud platform.
[0103] It can be understood that since the perception data stream has been received on the cloud platform, the original data written to the disk can be packaged together with the perception data after processing on the cloud platform and saved as Badcase in the cloud platform.
[0104] In one embodiment of the present application, the sending module 320 is also used to
[0105] Based on a pre-saved algorithm, identifying traffic participants in the raw data stream in real time;
[0106] The perception data stream of the traffic participants in the original data stream is transmitted to the cloud platform.
[0107] In one embodiment of the present application, the second receiving module 330 is further used to
[0108] The original data in the original data stream is cached based on a FIFO queue.
[0109] In one embodiment of the present application, the writing module 350 is also used to
[0110] According to the target recording time period in the abnormal event processing request issued by the cloud platform, writing the original data cached by the FIFO cache queue corresponding to the target recording time period into a disk device;
[0111] According to the target recording data requirements, the original data written to the disk device is transmitted back to the cloud platform.
[0112] In an embodiment of the present application, an exception handling device is also provided, which is applied to a cloud platform, and the device includes:
[0113] A calculation module, used to calculate whether an abnormal event occurs in the perception data stream based on business rules, wherein the abnormal event includes an abnormal situation of autonomous driving obtained based on the perception data stream;
[0114] A sending processing module, configured to send the abnormal event processing request to the edge computing module if an abnormal event occurs in the calculation of the perception data stream;
[0115] A receiving and processing module, configured to receive the original data written to the disk device returned by the edge computing module based on the processing request of the abnormal event, wherein the original data written to the disk device includes the original data cached in the FIFO cache queue of the edge computing module;
[0116] The packing processing module is used to combine and pack the original data with the perception data stream to obtain an abnormal event data packet.
[0117] In one embodiment of the present application, the sending processing module is also used to
[0118] Based on the TCP long link, receiving the perception data stream of the edge computing module in real time;
[0119] If an abnormal event occurs in the perception data stream and needs to be recorded, the abnormal event processing request is sent to the edge computing module, and the abnormal event processing request includes the time period that needs to be recorded and the data that needs to be recorded.
[0120] In one embodiment of the present application, it further includes: a receiving processing module for
[0121] Receive the perception data stream uploaded by the edge computing module, where the perception data stream is obtained based on real-time recognition of the original data stream collected by the road-side or vehicle-side perception module, where the road-side or vehicle-side perception module includes an image sensor and a radar sensor, and the original data stream includes a video picture data stream and a laser point cloud data stream.
[0122] It can be understood that the above-mentioned exception handling device can implement each step of the exception handling method provided in the above-mentioned embodiment, and the relevant explanations about the exception handling method are applicable to the exception handling device, which will not be repeated here.
[0123] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.
[0124] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0125] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.
[0126] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an exception handling device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:
[0127] Receiving an original data stream collected by a road-side perception module, wherein the road-side perception module includes an image sensor and a radar sensor, and the original data stream includes a video image data stream and a laser point cloud data stream;
[0128] Recognize the original data stream in real time to obtain the perception data stream, and send it to the cloud platform;
[0129] Using a FIFO buffer queue to buffer the original data currently received in the original data stream;
[0130] receiving an abnormal event processing request issued by the cloud platform, wherein the abnormal event includes an abnormal situation of autonomous driving obtained based on the perception data stream;
[0131] According to the target time period and target data requirements in the abnormal event processing request, the currently received original data cached by the FIFO cache queue is written to the disk device and the original data written to the disk is transmitted back to the cloud platform.
[0132] or,
[0133] Based on the business rules, calculating whether an abnormal event occurs in the perception data stream, wherein the abnormal event includes an abnormal situation of autonomous driving obtained based on the perception data stream;
[0134] If an abnormal event occurs in the calculation of the perception data stream, sending the abnormal event processing request to the edge computing module;
[0135] Receive the original data written to the disk device returned by the edge computing module based on the processing request of the abnormal event, wherein the original data written to the disk device includes the original data cached in the FIFO cache queue of the edge computing module;
[0136] The original data is combined and packaged with the perception data stream to obtain an abnormal event data packet.
[0137] The above application Figure 1 or Figure 3The method performed by the exception handling device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.
[0138] The electronic device may also perform Figure 1 or Figure 3 The method executed by the exception handling device in Figure 1 or Figure 3 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.
[0139] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 or Figure 3 The method executed by the exception handling device in the illustrated embodiment is specifically used to execute:
[0140] Receiving an original data stream collected by a road-side perception module, wherein the road-side perception module includes an image sensor and a radar sensor, and the original data stream includes a video image data stream and a laser point cloud data stream;
[0141] Recognize the original data stream in real time to obtain the perception data stream, and send it to the cloud platform;
[0142] Using a FIFO buffer queue to buffer the original data currently received in the original data stream;
[0143] receiving an abnormal event processing request issued by the cloud platform, wherein the abnormal event includes an abnormal situation of autonomous driving obtained based on the perception data stream;
[0144] According to the target time period and target data requirements in the abnormal event processing request, the currently received original data cached by the FIFO cache queue is written to the disk device and the original data written to the disk is transmitted back to the cloud platform.
[0145] or,
[0146] Based on the business rules, calculating whether an abnormal event occurs in the perception data stream, wherein the abnormal event includes an abnormal situation of autonomous driving obtained based on the perception data stream;
[0147] If an abnormal event occurs in the calculation of the perception data stream, sending the abnormal event processing request to the edge computing module;
[0148] Receive the original data written to the disk device returned by the edge computing module based on the processing request of the abnormal event, wherein the original data written to the disk device includes the original data cached in the FIFO cache queue of the edge computing module;
[0149] The original data is combined and packaged with the perception data stream to obtain an abnormal event data packet.
[0150] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0151] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0152] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0154] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0155] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0156] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0157] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0158] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0159] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. An exception handling method, in, Applied to the edge computing module, the method includes: Receiving an original data stream collected by a road-side perception module, wherein the road-side perception module includes an image sensor and a radar sensor, and the original data stream includes a video image data stream and a laser point cloud data stream; Recognize the original data stream in real time to obtain the perception data stream, and send it to the cloud platform; Using a FIFO buffer queue to buffer the original data currently received in the original data stream; receiving an abnormal event processing request issued by the cloud platform, wherein the abnormal event includes an abnormal situation of autonomous driving obtained based on the perception data stream; According to the target time period and target data requirements in the abnormal event processing request, the currently received original data cached by the FIFO cache queue is written to the disk device and the original data written to the disk is transmitted back to the cloud platform.
2. The method according to claim 1, in, The real-time identification according to the original data stream obtains the perception data stream and sends it to the cloud platform, including: Based on a pre-saved algorithm, identifying traffic participants in the raw data stream in real time; The perception data stream of the traffic participants in the original data stream is transmitted to the cloud platform.
3. The method according to claim 2, in, The adopting of the FIFO buffer queue to buffer the original data currently received in the original data stream comprises: The original data in the original data stream is cached based on a memory or a disk device.
4. The method according to claim 1, in, The step of writing the currently received raw data cached by the FIFO cache queue to a disk and transmitting the raw data written to the disk device back to the cloud platform according to the target time period and the target data requirement in the abnormal event processing request includes: According to the target recording time period in the abnormal event processing request issued by the cloud platform, writing the original data cached by the FIFO cache queue corresponding to the target recording time period into a disk device; According to the target recording data requirements, the original data written to the disk device is transmitted back to the cloud platform.
5. An exception handling method, in, Applied to a cloud platform, the method includes: Based on the business rules, calculating whether an abnormal event occurs in the perception data stream, wherein the abnormal event includes an abnormal situation of autonomous driving obtained based on the perception data stream; If an abnormal event occurs in the calculation of the perception data stream, sending the abnormal event processing request to the edge computing module; Receive the original data written to the disk device returned by the edge computing module based on the abnormal event processing request, where the original data written to the disk device includes the original data cached in the FIFO cache queue of the edge computing module; The original data is combined and packaged with the perception data stream to obtain an abnormal event data packet.
6. The method according to claim 5, in, If an abnormal event occurs in the calculation of the perception data stream, sending the abnormal event processing request to the edge computing module includes: Based on the TCP long link, receiving the perception data stream of the edge computing module in real time; If an abnormal event occurs in the perception data stream and needs to be recorded, the abnormal event processing request is sent to the edge computing module, and the abnormal event processing request includes the time period that needs to be recorded and the data that needs to be recorded.
7. The method according to claim 5, in, The method further comprises: Receive the perception data stream uploaded by the edge computing module, where the perception data stream is obtained based on real-time recognition of the original data stream collected by the road-side or vehicle-side perception module, where the road-side or vehicle-side perception module includes an image sensor and a radar sensor, and the original data stream includes a video picture data stream and a laser point cloud data stream.
8. An exception handling device, in, Applied to an edge computing module, the device comprises: A first receiving module is used to receive the original data stream collected by the road-side perception module, the road-side perception module includes an image sensor and a radar sensor, and the original data stream includes a video image data stream and a laser point cloud data stream; A sending module, used for obtaining a perception data stream according to the original data stream in real time identification, and sending the perception data stream to the cloud platform; A second receiving module, configured to use a FIFO buffer queue to buffer the original data currently received in the original data stream; A third receiving module is used to receive an abnormal event processing request issued by the cloud platform, wherein the abnormal event includes an abnormal situation of autonomous driving obtained based on the perception data stream; The writing module is used to write the currently received original data cached by the FIFO cache queue to the disk according to the target time period and target data requirements in the abnormal event processing request, and transmit the original data written to the disk back to the cloud platform.
9. An electronic device, include: processor; as well as A memory arranged to store computer executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 4 and / or the method of any one of claims 5 to 7.
10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, enables the electronic device to execute any of the methods of claims 1 to 4 and / or any of the methods of claims 5 to 7.