Data processing methods, devices, equipment, media and vehicles
By improving the data processing methods of autonomous driving perception software and utilizing the interaction between nodes to process image data, the problems of high latency and low throughput have been solved, achieving more efficient data processing and development.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING CO WHEELS TECH CO LTD
- Filing Date
- 2022-09-14
- Publication Date
- 2026-06-30
AI Technical Summary
Autonomous driving perception software suffers from high latency and low throughput in data processing, and its development efficiency is also low.
The image data to be processed is preprocessed by the data preprocessing node to obtain the target image data; the target extraction processing is performed by the image data processing node to obtain the target object data; post-processing is performed on each target object data to obtain the post-processing result; finally, the results are integrated by the data output node to realize data interaction between nodes.
It reduced data processing latency and improved data throughput and development efficiency.
Smart Images
Figure CN115690726B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of big data technology, and in particular to a data processing method, apparatus, equipment, medium, and vehicle. Background Technology
[0002] With the development of technology, autonomous driving perception technology has been widely used. Autonomous driving perception software typically needs to process massive image data from multiple cameras, as well as data from other sensors.
[0003] In related technologies, autonomous driving perception software suffers from high latency and low throughput due to its complex internal functions and large amount of code during data processing, resulting in low development efficiency. Summary of the Invention
[0004] To address the aforementioned technical problems, this disclosure provides a data processing method, apparatus, device, medium, and vehicle.
[0005] Firstly, this disclosure provides a data processing method, including:
[0006] The data preprocessing node preprocesses the image data to be processed to obtain the target image data.
[0007] The target image data is processed by the image data processing node to extract the target object data from the target image data.
[0008] For each target object data, the target object data is post-processed through the first post-processing node corresponding to the target object data to obtain the first post-processing result corresponding to the first post-processing node.
[0009] The data output node integrates the first post-processing results corresponding to each first post-processing node to obtain the data processing result corresponding to the image data to be processed.
[0010] Secondly, this disclosure provides a data processing apparatus, including:
[0011] The first processing module is used to preprocess the image data to be processed through the data preprocessing node to obtain the target image data;
[0012] The second processing module is used to perform target extraction processing on the target image data through the image data processing node to obtain the data of each target object in the target image data.
[0013] The third processing module is used to perform post-processing on the target object data through the first post-processing node corresponding to the target object data for each target object data, and obtain the first post-processing result corresponding to the first post-processing node.
[0014] The result integration module is used to integrate the first post-processing results corresponding to each first post-processing node through the data output node to obtain the data processing result corresponding to the image data to be processed.
[0015] Thirdly, this disclosure provides an electronic device, including:
[0016] processor;
[0017] Memory, used to store executable instructions;
[0018] The processor is used to read executable instructions from memory and execute the executable instructions to implement the data processing method of the first aspect.
[0019] Fourthly, this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the data processing method of the first aspect.
[0020] Fifthly, this disclosure provides a vehicle including the data processing device described above.
[0021] The technical solution provided in this disclosure has the following advantages compared with the prior art:
[0022] The data processing method, apparatus, device, medium, and vehicle of this disclosure can preprocess the image data to be processed through a data preprocessing node to obtain target image data. Then, the target image data is extracted through an image data processing node to obtain the target object data in the target image data. For each target object data, the target object data is further post-processed through a first post-processing node to obtain a first post-processing result corresponding to the first post-processing node. Finally, the results of the first post-processing results corresponding to each first post-processing node are integrated through a data output node to obtain the data processing result corresponding to the image data to be processed. Thus, the image data to be processed can be processed and the corresponding data processing result can be obtained through data interaction between various nodes. By interacting with nodes, data processing latency is reduced, data throughput is improved, and development efficiency is increased. Attached Figure Description
[0023] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.
[0024] Figure 1 A flowchart illustrating a data processing method provided in an embodiment of this disclosure;
[0025] Figure 2 A flowchart illustrating a communication process provided in an embodiment of this disclosure;
[0026] Figure 3 This is a schematic diagram of the structure of a node scheduling framework provided in an embodiment of the present disclosure;
[0027] Figure 4 This is a schematic diagram of a node topology provided in an embodiment of the present disclosure;
[0028] Figure 5 This is a schematic diagram of the structure of a data processing apparatus provided in an embodiment of the present disclosure;
[0029] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation
[0030] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.
[0031] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.
[0032] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.
[0033] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.
[0034] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".
[0035] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
[0036] This disclosure provides a data processing method, apparatus, device, medium, and vehicle. The following will first combine... Figures 1 to 4 The data processing method provided in the embodiments of this disclosure will be described in detail.
[0037] Figure 1 A flowchart illustrating a data processing method provided in an embodiment of this disclosure is shown.
[0038] In this embodiment of the disclosure, the data processing method can be executed by a terminal device. The terminal device can be an electronic device. The electronic device can be a device with data processing capabilities. Specifically, the electronic device can include, but is not limited to, mobile terminals such as mobile phones, in-vehicle devices, vehicle controllers, tablet computers, and wearable devices.
[0039] like Figure 1 As shown, the data processing method may include the following steps.
[0040] S110. The image data to be processed is preprocessed by the data preprocessing node in the terminal device to obtain the target image data.
[0041] In this embodiment of the disclosure, when a terminal device wants to process the image data to be processed, it can first preprocess the image data to be processed through the data preprocessing node in the terminal device to obtain the target image data.
[0042] Optionally, the data preprocessing node can be a node used to preprocess the image data to be processed. For example, preprocessing can include data filtering, data resizing, etc.
[0043] Optionally, the image data to be processed can be data that requires preprocessing.
[0044] Optionally, the target image data can be image data obtained by preprocessing the image data to be processed.
[0045] Specifically, when a terminal device wants to process the image data to be processed, it can preprocess the image data to be processed through the data preprocessing node in the terminal device to obtain the processed target image data.
[0046] S120. The target image data is processed by the image data processing node in the terminal device to obtain the target object data in the target image data.
[0047] In this embodiment of the disclosure, after obtaining the target image data, the terminal device can perform target extraction processing on the target image data through the image data processing node to obtain the data of each target object in the target image data.
[0048] Optionally, the image data processing node can be a node used for target extraction processing of image data.
[0049] Alternatively, the target extraction process can be the process of extracting target object data from image data.
[0050] Optionally, the target object data can be data corresponding to the target object. For example, the target object can be a pedestrian, lane line, obstacle, etc., without limitation.
[0051] For example, taking road image data as the target image data, the road image data can include target objects such as pedestrians, lane lines, and obstacles. The terminal device can perform target extraction processing on the target objects in the road image data through the image data processing node, thereby obtaining the target object data corresponding to each target object.
[0052] S130. For each target object data, the target object data is post-processed through the first post-processing node corresponding to the target object data in the terminal device to obtain the first post-processing result corresponding to the first post-processing node.
[0053] In this embodiment of the disclosure, after obtaining the target object data in the target image data, the terminal device can perform post-processing on the target object data through the first post-processing node corresponding to the target object data for each target object data, and obtain the first post-processing result corresponding to the first post-processing node.
[0054] Optionally, the first post-processing node can be a node used to post-process the target object data.
[0055] Optionally, each target object data corresponds to a different first post-processing node. For example, the target object data can be pedestrian data, and the corresponding first post-processing node can be a post-processing node that processes pedestrian data; the target object data can be lane line data, and the corresponding first post-processing node can be a post-processing node that processes lane line data.
[0056] Specifically, after obtaining the data of each target object in the target image data, the terminal device can perform post-processing on each target object data through the first post-processing node corresponding to each target object data, such as optimizing the extracted region, correcting misclassified regions, etc., so as to obtain the first post-processing results corresponding to each first post-processing node.
[0057] S140. The data output nodes in the terminal device integrate the first post-processing results corresponding to each first post-processing node to obtain the data processing result corresponding to the image data to be processed.
[0058] In this embodiment of the disclosure, after obtaining the first post-processing results corresponding to each first post-processing node, the terminal device can integrate the first post-processing results corresponding to each first post-processing node through the data output node to obtain the data processing result corresponding to the image data to be processed.
[0059] Optionally, the data output node can be a node used to output the results of data processing.
[0060] Optionally, result integration can be achieved by combining multiple first post-processing results.
[0061] Specifically, after obtaining the first post-processing results corresponding to each first post-processing node, the terminal device can integrate the first post-processing results corresponding to each first post-processing node through the data output node, thereby obtaining and outputting the data processing results corresponding to the image data to be processed. For example, for road image data, the first post-processing results obtained for pedestrian data, the first post-processing results obtained for lane line data, etc., can be combined to obtain the data processing results corresponding to each target object data of the road image data.
[0062] Therefore, in this embodiment of the present disclosure, the image data to be processed can be preprocessed by the data preprocessing node in the terminal device to obtain the target image data. Then, the target image data can be extracted by the image data processing node in the terminal device to obtain the target object data in the target image data. For each target object data, the target object data can be postprocessed by the first postprocessing node corresponding to the target object data in the terminal device to obtain the first postprocessing result corresponding to the first postprocessing node. Finally, the first postprocessing results corresponding to each first postprocessing node can be integrated by the data output node in the terminal device to obtain the data processing result corresponding to the image data to be processed. Thus, the image data to be processed can be processed and the corresponding data processing result can be obtained by data interaction between the nodes in the terminal device. By interacting with the nodes, the data processing latency can be reduced, the data throughput can be improved, and the development efficiency can be improved.
[0063] Optionally, before S140, the data processing method may further include: if the first post-processing node includes a first type of post-processing node, based on the second post-processing node, performing a second post-processing on the first post-processing result corresponding to the first type of post-processing node to obtain the second post-processing result corresponding to the second post-processing node.
[0064] In this embodiment of the disclosure, after obtaining the first post-processing result corresponding to the first post-processing node, if the first post-processing node includes a first type of post-processing node, the terminal device can perform further post-processing on the first post-processing result corresponding to the first type of post-processing node based on the second post-processing node to obtain the second post-processing result corresponding to the second post-processing node.
[0065] Optionally, the first type of post-processing node can be a node that needs to be post-processed again.
[0066] Optionally, the second post-processing node can be a node that performs further post-processing on the result of the first post-processing.
[0067] Specifically, after obtaining the first post-processing result corresponding to the first post-processing node, if the first post-processing node includes a first type of post-processing node, the terminal device can use the second post-processing node to perform further post-processing on the first post-processing result corresponding to the first type of post-processing node, thereby obtaining the second post-processing result corresponding to the second post-processing node. For example, the first post-processing result corresponding to the first type of post-processing node can be a post-processing result for lane lines, and the second post-processing result corresponding to the second post-processing node can be a post-processing result for vehicles on lane lines.
[0068] Optionally, S140 may specifically include: integrating the first post-processing results corresponding to each first post-processing node and the second post-processing results corresponding to each second post-processing node through the data output node to obtain the data processing result corresponding to the image data to be processed.
[0069] In this embodiment of the disclosure, after obtaining the second post-processing result corresponding to the second post-processing node, the terminal device can integrate the first post-processing results corresponding to each first post-processing node and the second post-processing results corresponding to the second post-processing node through the data output node to obtain the data processing result corresponding to the image data to be processed.
[0070] Specifically, after obtaining the second post-processing result corresponding to the second post-processing node, the terminal device can integrate the first post-processing results corresponding to each first post-processing node and the second post-processing results corresponding to each second post-processing node through the data output node, thereby obtaining and outputting the data processing result corresponding to the image data to be processed. For example, for road image data, the first post-processing result obtained for pedestrian data, the first post-processing result obtained for lane line data, and the second post-processing result obtained for vehicles on the lane lines can be combined to obtain the data processing result corresponding to each target object data of the road image data.
[0071] Therefore, in this embodiment of the disclosure, data processing latency can be reduced, data throughput can be increased, and development efficiency can be improved through inter-node interaction.
[0072] Optionally, prior to S110, the data processing method may further include: acquiring multiple image data collected by the image acquisition device through the image data reading node in the terminal device, and performing alignment processing on the multiple image data to obtain image data to be processed; wherein, the alignment processing includes performing timestamp alignment processing and frame rate filtering processing on the multiple images.
[0073] In this embodiment of the disclosure, the terminal device can acquire multiple image data collected by the image acquisition device through the image data reading node, and perform alignment processing on the multiple image data to obtain the image data to be processed.
[0074] Optionally, the image data reading node can be a node used to read image data.
[0075] Optionally, the image acquisition device can be any device used to acquire images. For example, a camera, a webcam, etc., are not limited here.
[0076] Optionally, the alignment process can be a process of aligning image data. This alignment process may include timestamp alignment and frame rate filtering.
[0077] Specifically, the terminal device can acquire multiple image data collected by the image acquisition device through the image data reading node, and perform alignment processing on the acquired multiple image data to obtain the image data to be processed.
[0078] For example, a terminal device can obtain multiple image data captured by a camera through an image data reading node and perform alignment processing. That is, based on the timestamps corresponding to multiple image data, it can remove image data whose timestamps are far apart, and it can also remove image data with excessively high frame rates based on the frame rates corresponding to multiple image data, thereby obtaining the image data to be processed.
[0079] Therefore, in this embodiment of the disclosure, multiple image data can be aligned to obtain suitable image data to be processed, thereby improving development efficiency.
[0080] Optionally, the data processing method may further include: acquiring multiple sensor data collected by the sensor through a sensor data reading node in the terminal device; and aligning the multiple sensor data based on the image data to be processed through a data alignment node in the terminal device to obtain the sensor data to be processed; wherein the alignment process includes timestamp alignment of the multiple sensor data.
[0081] In this embodiment of the disclosure, the terminal device can acquire multiple sensor data collected by the sensor through the sensor data reading node.
[0082] Optionally, the sensor data reading node can be a node used to read sensor data.
[0083] Alternatively, the sensor can be a device used to collect sensor data. For example, the sensor can be an odometer sensor, a radar sensor, etc., without limitation.
[0084] Optionally, the sensor data can be data collected by sensors. For example, sensor data can be odometer data, radar data, etc., and there is no limitation here.
[0085] For example, terminal devices can acquire odometer data collected by odometer sensors, radar data collected by radar sensors, etc., through sensor data reading nodes.
[0086] Furthermore, after acquiring multiple sensor data collected by the sensors, the terminal device can perform alignment processing on the multiple sensor data based on the image data to be processed through the data alignment node to obtain the sensor data to be processed.
[0087] Optionally, the alignment process may include timestamp alignment of data from multiple sensors.
[0088] For example, after acquiring multiple sensor data collected by the sensor, the terminal device can use a data alignment node to align the multiple sensor data according to the timestamps corresponding to the image data to be processed, thereby removing sensor data with large timestamp differences and obtaining the sensor data to be processed.
[0089] Optionally, S130 may specifically include: when the first post-processing node is a second type of post-processing node, performing post-processing on the target object data and the sensor data to be processed through the first post-processing node corresponding to the target object data to obtain the first post-processing result corresponding to the first post-processing node.
[0090] In this embodiment of the disclosure, when the first post-processing node is a second type of post-processing node, the terminal device can perform post-processing on the target object data and the sensor data to be processed through the first post-processing node corresponding to the target object data to obtain the first post-processing result corresponding to the first post-processing node.
[0091] Optionally, the second type of post-processing node can be a node that performs post-processing on the sensor data to be processed.
[0092] For example, the terminal device can obtain the first post-processing result corresponding to the first post-processing node by post-processing the pedestrian data and radar data of the first post-processing node.
[0093] Therefore, in this embodiment of the disclosure, the target object data and the sensor data to be processed can be post-processed to obtain the corresponding data processing results. By interacting between nodes, the data processing latency can be reduced, the data throughput can be improved, and the development efficiency can be increased.
[0094] In some embodiments of this disclosure, data can be transmitted between nodes of a terminal device via process communication, and each node may include a data sending node and a data receiving node.
[0095] Optionally, the data sending node can be a node that sends data.
[0096] Optionally, the data receiving node can be a node that receives data.
[0097] In some embodiments, each node can find the corresponding target thread in the module thread pool and perform data processing based on the target thread to obtain the corresponding data processing result.
[0098] The following is combined Figure 2 The communication process corresponding to the inter-process communication method is explained in detail.
[0099] Figure 2 A schematic flowchart of a communication process provided in an embodiment of this disclosure is shown.
[0100] like Figure 2 As shown, the communication process may include the following steps.
[0101] S210. Obtain the data sent by the data sending node through the scheduling node in the terminal device.
[0102] In this embodiment of the disclosure, the terminal device can obtain data sent by the data sending node through the scheduling node.
[0103] Optionally, the scheduling node can be a node used to schedule data between various nodes.
[0104] S220. The scheduling node fills the data sent by the data sending node into the corresponding message object to obtain the target message.
[0105] In this embodiment of the disclosure, after obtaining the data sent by the data sending node, the terminal device can fill the data sent by the data sending node into the corresponding message object through the scheduling node to obtain the corresponding target message.
[0106] Optionally, the message object can be a message template corresponding to the data sending node.
[0107] Optionally, the target message can be a message used for communication.
[0108] Specifically, after obtaining the data sent by the data sending node, the terminal device can fill the data sent by the data sending node into the corresponding message object through the scheduling node, that is, fill it into the corresponding message template, and obtain the corresponding target message.
[0109] S230. The scheduling node determines the data receiving node associated with the data sending node based on the node topology relationship.
[0110] In this embodiment of the disclosure, after receiving the target message, the terminal device can determine the data receiving node associated with the data sending node based on the node topology relationship through the scheduling node.
[0111] Optionally, the node topology can be the connection relationship between the nodes.
[0112] Specifically, after receiving the target message, the terminal device can determine the data receiving node associated with the data sending node by scheduling the node according to the node topology relationship, that is, the connection relationship between each node.
[0113] S240, The target message is transmitted to the data receiving node through the scheduling node.
[0114] In this embodiment of the disclosure, after determining the data receiving node associated with the data sending node, the terminal device can transmit the target message to the data receiving node through the scheduling node.
[0115] Therefore, in this embodiment of the disclosure, data transmission between nodes can be scheduled by a scheduling node, which can reduce data processing latency, increase data throughput, and thus improve development efficiency.
[0116] Optionally, S220 may specifically include: searching for the message object corresponding to the data sending node in the message object pool through the scheduling node; filling the message object with the data sent by the data sending node through the scheduling node to obtain the target message.
[0117] In this embodiment of the disclosure, after obtaining the data sent by the data sending node, the terminal device can search for the message object corresponding to the data sending node in the message object pool through the scheduling node.
[0118] Optionally, the message object pool can store multiple pools of preset message objects. Each preset message object corresponds to a different data sending node.
[0119] Specifically, after obtaining the data sent by the data sending node, the terminal device can search for the message object corresponding to that data sending node in the message object pool through the scheduling node.
[0120] Furthermore, after locating the message object, the terminal device can use the scheduling node to fill the message object with the data sent by the data sending node to obtain the target message.
[0121] Specifically, after obtaining the data sent by the data sending node, the terminal device can fill the data sent by the data sending node into the corresponding message object through the scheduling node, that is, fill it into the corresponding message template, and obtain the corresponding target message.
[0122] Therefore, in the embodiments disclosed herein, data processing latency can be reduced, data throughput can be increased, and development efficiency can be improved.
[0123] Optionally, after S220 and before S230, the data processing method may further include: storing the target message in a message queue through a scheduling node.
[0124] In this embodiment of the disclosure, after receiving the target message, the terminal device can store the target message in a message queue through a scheduling node.
[0125] Optionally, the message queue can be a queue that stores multiple messages.
[0126] Optionally, S240 may specifically include: retrieving the target message from the message queue through the scheduling node; and transmitting the target message to the data receiving node through the scheduling node.
[0127] In this embodiment of the disclosure, the terminal device can obtain a target message from the message queue through the scheduling node and pass the target message to the data receiving node associated with the data sending node.
[0128] Therefore, in this embodiment of the disclosure, data transmission between nodes can be scheduled by a scheduling node, which can reduce data processing latency, increase data throughput, and improve development efficiency.
[0129] Figure 3 A schematic diagram of a node scheduling framework provided in an embodiment of this disclosure is shown.
[0130] like Figure 3 As shown, scheduling node 301 can obtain data sent by data sending node 302. Scheduling node 301 retrieves the message object corresponding to data sending node 302 from message object pool 304, fills the message object with the data sent by data sending node 302, obtains the target message, and stores the target message in message queue 305. Scheduling node 301 retrieves the data receiving node 303 associated with data sending node 302 from node topology relationship 306, retrieves the target message from message queue 305, and passes the target message to data receiving node 303. Data sending node 302 and data receiving node 303 can obtain corresponding threads from module thread pool 307 to perform data processing.
[0131] Figure 4 A schematic diagram of a node topology provided in an embodiment of this disclosure is shown.
[0132] like Figure 4As shown, the terminal device can acquire multiple image data collected by the image acquisition device through the image data reading node 401, and perform alignment processing on the multiple image data to obtain the image data to be processed; acquire multiple sensor data collected by the sensor through the sensor data reading node 402, and perform alignment processing on the multiple sensor data based on the image data to be processed through the data alignment node 403 to obtain the sensor data to be processed; preprocess the image data to be processed through the data preprocessing node 404 to obtain the target image data; and perform target extraction processing on the target image data through the image data processing node 405 to obtain the data of each target object in the target image data; for Target object data A is post-processed by the first post-processing node A406 to obtain the first post-processing result A. Target object data B is post-processed by the first post-processing node B407 to obtain the first post-processing result B. If the first post-processing node A406 includes a first type of post-processing node, the first post-processing result A is further post-processed by the second post-processing node 408 to obtain the second post-processing result corresponding to the second post-processing node. The first post-processing result A, the first post-processing result B, and the second post-processing result are integrated by the data output node 409 to obtain the data processing result corresponding to the image data to be processed. The terminal device can initialize each of the above nodes through the initialization node 410.
[0133] Figure 5 A schematic diagram of the structure of a data processing apparatus provided in an embodiment of the present disclosure is shown.
[0134] In some embodiments of this disclosure, Figure 5 The data processing device shown can be installed within a terminal device. This terminal device can be an electronic device with data processing capabilities. Specifically, the electronic device can include, but is not limited to, mobile terminals such as mobile phones, in-vehicle devices, vehicle controllers, tablets, and wearable devices.
[0135] like Figure 5 As shown, the data processing device 500 may include a first processing module 510, a second processing module 520, a third processing module 530, and a result integration module 540.
[0136] The first processing module 510 can be used to preprocess the image data to be processed through the data preprocessing node to obtain the target image data.
[0137] The second processing module 520 can be used to perform target extraction processing on the target image data through the image data processing node to obtain the data of each target object in the target image data.
[0138] The third processing module 530 can be used to perform post-processing on the target object data through the first post-processing node corresponding to the target object data for each target object data, and obtain the first post-processing result corresponding to the first post-processing node.
[0139] The result integration module 540 can be used to integrate the first post-processing results corresponding to each first post-processing node through the data output node to obtain the data processing result corresponding to the image data to be processed.
[0140] Therefore, in this embodiment of the present disclosure, the image data to be processed can be preprocessed by the data preprocessing node to obtain the target image data. Then, the target image data can be extracted by the image data processing node to obtain the target object data in the target image data. For each target object data, the target object data can be postprocessed by the first postprocessing node corresponding to the target object data to obtain the first postprocessing result corresponding to the first postprocessing node. Finally, the results of the first postprocessing results corresponding to each first postprocessing node can be integrated by the data output node to obtain the data processing result corresponding to the image data to be processed. Thus, the image data to be processed can be processed and the corresponding data processing result can be obtained through data interaction between the nodes. By interacting with the nodes, the data processing latency can be reduced, the data throughput can be improved, and the development efficiency can be improved.
[0141] In some embodiments of this disclosure, the data processing apparatus 500 may further include a fourth processing module.
[0142] The fourth processing module can be used to perform further post-processing on the first post-processing results corresponding to each first post-processing node based on the second post-processing node, before integrating the results of the first post-processing results corresponding to each first post-processing node through the data output node to obtain the data processing result corresponding to the image data to be processed, in the case where the first post-processing node includes a first type of post-processing node, to obtain the second post-processing result corresponding to the second post-processing node.
[0143] In some embodiments of this disclosure, the result integration module 540 may include a first integration unit.
[0144] The result integration unit can be used to integrate the first post-processing results corresponding to each first post-processing node and the second post-processing results corresponding to each second post-processing node through the data output node to obtain the data processing result corresponding to the image data to be processed.
[0145] In some embodiments of this disclosure, the data processing apparatus 500 may further include a fifth processing module.
[0146] The fifth processing module can be used to obtain multiple image data acquired by the image acquisition device through the image data reading node before the image data to be processed is preprocessed through the data preprocessing node to obtain the target image data. The multiple image data are then aligned to obtain the image data to be processed. The alignment process includes timestamp alignment and frame rate filtering of the multiple images.
[0147] In some embodiments of this disclosure, the data processing apparatus 500 may further include a first acquisition module and a sixth processing module.
[0148] The first acquisition module can be used to acquire multiple sensor data collected by the sensor through the sensor data reading node.
[0149] The sixth processing module can be used to align multiple sensor data based on the image data to be processed through the data alignment node to obtain the sensor data to be processed; wherein, the alignment process includes time stamp alignment of multiple sensor data.
[0150] In some embodiments of this disclosure, the third processing module 530 can be specifically used to perform post-processing on the target object data and the sensor data to be processed through the first post-processing node corresponding to the target object data when the first post-processing node is a second type of post-processing node, so as to obtain the first post-processing result corresponding to the first post-processing node.
[0151] In some embodiments of this disclosure, data is transmitted between nodes via process communication, and each node includes a data sending node and a data receiving node.
[0152] In some embodiments of this disclosure, the data processing apparatus 500 may further include a second acquisition module, a seventh processing module, a node determination module, and a message transmission module.
[0153] The second acquisition module can be used to acquire data sent by the data sending node through the scheduling node.
[0154] This seventh processing module can be used to fill the data sent by the data sending node into the corresponding message object through the scheduling node to obtain the target message.
[0155] This node determination module can be used to determine the data receiving node associated with the data sending node by scheduling the node based on the node topology relationship.
[0156] This messaging module can be used to pass target messages to data receiving nodes via scheduling nodes.
[0157] In some embodiments of this disclosure, the seventh processing module may include an object lookup unit and a data entry unit.
[0158] This object lookup unit can be used to find the message object corresponding to the data sending node in the message object pool through the scheduling node.
[0159] This data filling unit can be used to fill the message object with the data sent by the data sending node through the scheduling node, so as to obtain the target message.
[0160] In some embodiments of this disclosure, the data processing apparatus 500 may further include a message storage module.
[0161] This message storage module can be used to store the target message in a message queue after the data sent by the data sending node is filled into the corresponding message object through the scheduling node and before the data receiving node associated with the data sending node is determined by the scheduling node according to the node topology relationship.
[0162] In some embodiments of this disclosure, the messaging module may include a message acquisition unit and a message delivery unit.
[0163] This message retrieval unit can be used to retrieve target messages from the message queue via a scheduling node.
[0164] This message passing unit can be used to pass target messages to data receiving nodes through scheduling nodes.
[0165] It should be noted that, Figure 5 The data processing device 500 shown can perform... Figures 1 to 4 The various steps in the method embodiment shown are implemented. Figures 1 to 4 The processes and effects in the method embodiments shown are not described in detail here.
[0166] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown.
[0167] In some embodiments of this disclosure, Figure 6 The electronic device shown can be a device with data processing or voice interaction capabilities. Specifically, the electronic device can include, but is not limited to, mobile terminals such as mobile phones, in-vehicle devices, vehicle controllers, tablets, and wearable devices.
[0168] like Figure 6 As shown, the electronic device may include a processor 601 and a memory 602 storing computer program instructions.
[0169] Specifically, the processor 601 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.
[0170] Memory 602 may include a large-capacity storage for information or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 602 may include removable or non-removable (or fixed) media. Where appropriate, memory 602 may be internal or external to the integrated gateway device. In a particular embodiment, memory 602 is a non-volatile solid-state memory. In a particular embodiment, memory 602 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (Electrically Programmable ROM, EPROM), an electrically erasable programmable PROM (EEPROM), an electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.
[0171] The processor 601 reads and executes computer program instructions stored in the memory 602 to perform the steps of the data processing method provided in the embodiments of this disclosure.
[0172] In one example, the electronic device may also include a transceiver 603 and a bus 604. Wherein, as... Figure 6 As shown, the processor 601, memory 602 and transceiver 603 are connected via bus 604 and communicate with each other.
[0173] Bus 604 includes hardware, software, or both. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 604 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.
[0174] This disclosure also provides a computer-readable storage medium that can store a computer program, which, when executed by a processor, causes the processor to implement the data processing method provided in this disclosure.
[0175] The aforementioned storage medium may, for example, include a memory 602 containing computer program instructions, which can be executed by a processor 601 of an electronic device to complete the data processing method provided in the embodiments of this disclosure. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), compact disc ROM (CD-ROM), magnetic tape, floppy disk, and optical data storage device.
[0176] This disclosure also provides a vehicle including the data processing device described above. It is understood that the vehicle may also include a processor, a memory, and a computer program. The computer program is stored in the memory and configured to be executed by the processor to implement the data processing method provided in this disclosure. The processor and memory are already... Figure 6 The parts of the illustrated embodiments will not be repeated here.
[0177] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.
[0178] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A data processing method, characterized by, include: The data preprocessing node preprocesses the image data to be processed to obtain the target image data. The target image data is processed by the image data processing node to extract the target object data; For each target object data, the target object data is post-processed through the first post-processing node corresponding to the target object data to obtain the first post-processing result corresponding to the first post-processing node; The data output node integrates the first post-processing results corresponding to each of the first post-processing nodes to obtain the data processing result corresponding to the image data to be processed. The method further includes: The sensor data is acquired from multiple sensors through the sensor data reading node; The data alignment node aligns the multiple sensor data based on the image data to be processed to obtain the sensor data to be processed; wherein, the alignment process includes timestamp alignment of the multiple sensor data. The step of post-processing the target object data through the first post-processing node corresponding to the target object data to obtain the first post-processing result corresponding to the first post-processing node includes: When the first post-processing node is a second type of post-processing node, the target object data and the sensor data to be processed are post-processed by the first post-processing node corresponding to the target object data to obtain the first post-processing result corresponding to the first post-processing node. The second type of post-processing node is the node that performs post-processing on the sensor data to be processed.
2. The method of claim 1, wherein, Before integrating the first post-processing results corresponding to each of the first post-processing nodes through the data output node to obtain the data processing result corresponding to the image data to be processed, the method further includes: In the case where the first post-processing node includes a first type of post-processing node, the first post-processing result corresponding to the first type of post-processing node is post-processed again based on the second post-processing node to obtain the second post-processing result corresponding to the second post-processing node. The first type of post-processing node is the node that needs to be post-processed again. The step of integrating the first post-processing results corresponding to each of the first post-processing nodes through the data output node to obtain the data processing result corresponding to the image data to be processed includes: The data output node integrates the first post-processing results corresponding to each of the first post-processing nodes and the second post-processing results corresponding to each of the second post-processing nodes to obtain the data processing result corresponding to the image data to be processed.
3. The method of claim 1, wherein, Before preprocessing the image data to be processed through the data preprocessing node to obtain the target image data, the method further includes: Multiple image data collected by the image acquisition device are acquired through the image data reading node, and the multiple image data are aligned to obtain the image data to be processed; wherein, the alignment process includes timestamp alignment and frame rate filtering of the multiple images.
4. The method of claim 1, wherein, The nodes transmit data through process communication, and each node includes a data sending node and a data receiving node. The communication process corresponding to the aforementioned process communication method includes: The data sent by the data sending node is obtained through the scheduling node; The scheduling node fills the data sent by the data sending node into the corresponding message object to obtain the target message; The scheduling node determines the data receiving node associated with the data sending node based on the node topology. The target message is transmitted to the data receiving node through the scheduling node.
5. The method of claim 4, wherein, The step of filling the data sent by the data sending node into the corresponding message object through the scheduling node to obtain the target message includes: The scheduling node searches for the message object corresponding to the data sending node in the message object pool. The scheduling node fills the message object with the data sent by the data sending node to obtain the target message.
6. The method according to claim 4 or 5, characterized in that, After the data sent by the data sending node is filled into the corresponding message object through the scheduling node to obtain the target message, and before the data receiving node associated with the data sending node is determined by the scheduling node according to the node topology, the method further includes: The target message is stored in a message queue through the scheduling node; The step of transmitting the target message to the data receiving node through the scheduling node includes: The target message is retrieved from the message queue by the scheduling node; The target message is transmitted to the data receiving node through the scheduling node.
7. A data processing apparatus, characterized by, include: The first processing module is used to preprocess the image data to be processed through the data preprocessing node to obtain the target image data; The second processing module is used to perform target extraction processing on the target image data through the image data processing node to obtain the data of each target object in the target image data. The third processing module is used to perform post-processing on each target object data through the first post-processing node corresponding to the target object data to obtain the first post-processing result corresponding to the first post-processing node. The result integration module is used to integrate the first post-processing results corresponding to each of the first post-processing nodes through the data output node to obtain the data processing result corresponding to the image data to be processed. The device further includes: The first acquisition module is used to acquire multiple sensor data collected by the sensor through the sensor data reading node; The sixth processing module is used to perform alignment processing on the multiple sensor data based on the image data to be processed through the data alignment node to obtain the sensor data to be processed; wherein, the alignment processing includes timestamp alignment processing on the multiple sensor data; Specifically, the third processing module is used to perform post-processing on the target object data and the sensor data to be processed through the first post-processing node corresponding to the target object data when the first post-processing node is a second type of post-processing node, to obtain a first post-processing result corresponding to the first post-processing node, wherein the second type of post-processing node is the node that performs post-processing on the sensor data to be processed.
8. An electronic device, comprising: include: processor; Memory, used to store executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the data processing method according to any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, causes the processor to implement the data processing method according to any one of claims 1-6.
10. A vehicle, characterized in that, Includes the data processing apparatus as described in claim 7.