Method, apparatus, electronic device, medium and vehicle for heterogeneous data processing

Abstract

The present disclosure relates to a method, device, electronic equipment, medium and vehicle for heterogeneous data processing. The method comprises: constructing a plurality of hierarchical data buffers according to heterogeneous related information; the heterogeneous related information is used to represent processing link information or data object information involved in heterogeneous data processing; constructing a target data structure for organizing the hierarchical data buffers; in the process of executing a data processing workflow, accessing a target hierarchical data buffer based on the target data structure, and performing a data reading operation or a data writing operation on the target hierarchical data buffer. For various online systems or offline systems, the present disclosure realizes unified modeling of heterogeneous data, facilitates data transfer and real-time flow, saves the cost of massive data copying, and facilitates efficient access of the hierarchical data buffer by data processing workflow or algorithm workflow.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, electronic device, medium, and vehicle for heterogeneous data processing. Background Technology

[0002] In related technologies, there are many scenarios that require data processing from heterogeneous data sources. Taking the data processing of autonomous driving systems or driver assistance systems in vehicles as an example, it is often necessary to determine the location, destination, navigation path, surrounding environment, and road conditions of the autonomous vehicle. This corresponds to the implementation of vehicle mapping and localization functions, path planning functions, and perception functions. To achieve these functions, various sensors are typically installed on the autonomous vehicle, and the data from these multiple sensors needs to be processed.

[0003] For example, to achieve mapping and positioning functions and path planning functions, motion sensing sensors need to be installed, including one or more of the following: Global Positioning System (GPS) module, Real-time Dynamic Carrier Phase Differential (RTK) module, inertial sensor, wheel speedometer, LiDAR, and camera. To achieve perception functions, environmental sensing sensors need to be installed, such as one or more of the following: LiDAR, camera, millimeter-wave radar, and ultrasonic radar. These sensors generate heterogeneous data sources. The following technical problems arise when processing heterogeneous data sources: Because the data structures and semantic information expressed by heterogeneous data sources are different, data processing workflows for heterogeneous data sources often require data copying when using a particular heterogeneous data source before processing. In scenarios involving large amounts of data processing and frequent copying, this results in significant memory resource consumption and low processing efficiency. Summary of the Invention

[0004] To overcome the problems existing in the related technologies, embodiments of this disclosure provide a method, apparatus, electronic device, medium, and vehicle for heterogeneous data processing.

[0005] According to a first aspect of the present disclosure, a method for heterogeneous data processing is provided. The method includes: constructing a hierarchical data buffer at multiple levels based on heterogeneous related information; the heterogeneous related information representing processing stage information or data object information involved in the heterogeneous data processing; constructing a target data structure for organizing the hierarchical data buffers; and, during the execution of a data processing workflow, accessing the target-level hierarchical data buffer based on the target data structure, and performing data read operations or data write operations on the target-level hierarchical data buffer.

[0006] In some embodiments, constructing a multi-level hierarchical data buffer based on heterogeneous information includes: obtaining information on the processing stages involved in heterogeneous data processing; the processing stage information includes at least one of the following: first information about data storage and retrieval in each processing stage of the processing flow, and second information about data storage and retrieval in each algorithm execution stage of the processing flow; generating definition information of the hierarchical data buffer to be constructed based on the processing stage information; the definition information includes: the set number and level information of the hierarchical data buffer, and the storage object information of each level of the hierarchical data buffer; and constructing a multi-level hierarchical data buffer based on the definition information.

[0007] In some embodiments, the aforementioned processing stage information refers to the first metadata of heterogeneous data to be stored in each processing stage of the processing flow; the first metadata includes: associated processing operations and corresponding operation call relationships of the data to be stored, and storage-related information of the data to be stored. Based on the aforementioned processing stage information, definition information for the hierarchical data buffer to be constructed is generated, including: determining a first number of hierarchical data buffers based on the total number of storage objects corresponding to the data to be stored; allocating corresponding data to be stored and level information to the hierarchical data buffers under the first number based on the associated processing operations and corresponding operation call relationships; and generating storage object information for each level of hierarchical data buffer based on the aforementioned storage-related information.

[0008] In some embodiments, the aforementioned processing stage information is the second metadata of heterogeneous data to be stored in each algorithm execution stage of the processing flow; the aforementioned second metadata includes: the associated algorithm of the data to be stored and the corresponding algorithm call relationship, and the storage-related information of the data to be stored. Based on the aforementioned processing stage information, the definition information of the hierarchical data buffer to be constructed is generated, including: determining the second number of hierarchical data buffers based on the total number of storage objects corresponding to the aforementioned data to be stored; allocating corresponding data to be stored and level information to the hierarchical data buffers under the aforementioned second number based on the aforementioned associated algorithm and the corresponding algorithm call relationship; and generating the storage object information of the hierarchical data buffers at each level based on the aforementioned storage-related information.

[0009] In some embodiments, the aforementioned processing stage information refers to the target metadata of heterogeneous data to be stored in each processing stage and each algorithm execution stage of the processing flow. The target metadata includes: associated processing operations and corresponding operation call relationships for the data to be stored, associated algorithms and corresponding algorithm call relationships for the data to be stored, and storage-related information for the data to be stored. Based on the total number of storage objects corresponding to the data to be stored, a target number of hierarchical data buffers is determined. Based on the associated processing operations, operation call relationships, associated algorithms, and algorithm call relationships, corresponding data to be stored and level information are allocated to the hierarchical data buffers at the target number. Based on the storage-related information, storage object information for each level of hierarchical data buffer is generated.

[0010] In some embodiments, the above-mentioned construction of multiple levels of hierarchical data buffers based on heterogeneous information includes: obtaining data object information involved in heterogeneous data processing; the data object information includes: the sensor type corresponding to the sensing time series data; generating definition information of the hierarchical data buffers to be constructed based on the sensor type; the definition information includes: the set number and level information of the hierarchical data buffers, and the storage object information of each level of the hierarchical data buffers; and constructing multiple levels of hierarchical data buffers based on the definition information.

[0011] In some embodiments, constructing a target data structure for organizing the hierarchical data buffers includes: defining and configuring a first content body of the target data structure to be constructed, wherein the first content body carries mapping relationship information for each level of hierarchical data buffers; defining and configuring a second content body of the target data structure to be constructed, wherein the second content body carries organizational relationship information for describing the association between multiple levels of hierarchical data buffers; and constructing the target data structure based on the configured first content body and the second content body.

[0012] In some embodiments, where the heterogeneous related information is used to represent information about the processing steps involved in heterogeneous data processing, the organizational relationship information is generated based on the processing step information.

[0013] In some embodiments, the method further includes: updating the first content body according to the updated configuration information of the mapping relationship information to obtain an updated target data structure; or updating the second content body according to the updated configuration information of the organizational relationship information to obtain an updated target data structure; or updating the first content body and the second content body according to the updated configuration information of the mapping relationship information and the updated configuration information of the organizational relationship information to obtain an updated target data structure.

[0014] In some embodiments, the hierarchical data buffer is a bar queue or a circular queue; the data stored in the hierarchical data buffer is sensor time-series data; the target data structure is a linked list structure, which contains multiple nodes; wherein the first content body in each node includes a data field, the data in the data field contains first address information of the bar queue or circular queue corresponding to the current node, and the first address information is used to represent the mapping relationship information; the second content body in each node includes a pointer field, the pointer in the pointer field contains second address information of the next node pointed to by the current node, and the second address information is used to represent the organizational relationship information.

[0015] In some embodiments, the above-described process of accessing a target-level hierarchical data buffer based on the target data structure and performing data read or write operations on the target-level hierarchical data buffer during the execution of the data processing workflow includes: determining the target-level hierarchical data buffer to be accessed based on the operation data object in the target processing stage or target algorithm execution stage of the heterogeneous data processing workflow; locating and accessing the target-level hierarchical data buffer based on the target data structure; and performing data read or write operations on the target storage unit within the target-level hierarchical data buffer based on the operation instructions for the operation object in the target processing stage or target algorithm execution stage.

[0016] In some embodiments, after performing a data read operation on the target storage unit within the hierarchical data buffer at the target level, the read result is synchronized to multiple algorithms or multiple processing threads that need to invoke the read result.

[0017] According to a second aspect of the present disclosure, an apparatus for heterogeneous data processing is provided. The apparatus includes: a buffer construction module, a data organization module, and a data access and operation module. The buffer construction module is used to construct hierarchical data buffers at multiple levels based on heterogeneous information; the heterogeneous information is used to represent processing stage information or data object information involved in heterogeneous data processing. The data organization module is used to construct a target data structure for organizing the hierarchical data buffers. The data access and operation module is used to access the target-level hierarchical data buffers based on the target data structure during the execution of a data processing workflow, and to perform data read operations or data write operations on the target-level hierarchical data buffers.

[0018] According to a third aspect of the present disclosure, a vehicle is provided that stores a set of instructions, which are executed by the vehicle's autonomous driving system to implement the heterogeneous data processing method provided in the first aspect of the present disclosure.

[0019] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; the processor being configured to read the executable instructions from the memory and execute the executable instructions to implement the heterogeneous data processing method provided in the first aspect of the present disclosure.

[0020] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the heterogeneous data processing method provided in the first aspect of the present disclosure.

[0021] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0022] By constructing multi-level hierarchical data buffers based on the processing steps or data object information involved in heterogeneous data processing, these hierarchical data buffers are used to store heterogeneous data. Each level of the hierarchical data buffer is matched to different processing steps or data objects, realizing unified modeling of heterogeneous data and facilitating data transfer and real-time flow, saving the cost of copying massive amounts of data. At the same time, by constructing a target data structure, the hierarchical data buffers of multiple levels are organized into a target data structure. By accessing the target data structure, the target hierarchical data buffer to be accessed can be located and data reading or writing operations can be performed. For various online or offline systems, this facilitates efficient access to hierarchical data buffers in data processing workflows or algorithm workflows, avoiding the problem of low search efficiency caused by discretely setting hierarchical data buffers.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0025] Figure 1 This is a flowchart illustrating a method for processing heterogeneous data according to an exemplary embodiment.

[0026] Figure 2A This is a schematic diagram illustrating the processing flow involved in heterogeneous data processing according to an exemplary embodiment.

[0027] Figure 2B This is a schematic diagram illustrating the algorithm execution process involved in heterogeneous data processing according to an exemplary embodiment.

[0028] Figure 3 This is a schematic diagram illustrating the correspondence between a target data structure and a hierarchical data buffer according to an exemplary embodiment.

[0029] Figure 4 This is a schematic diagram illustrating the implementation process of step S130 according to an exemplary embodiment.

[0030] Figure 5A A detailed implementation flowchart illustrating the generation of definition information for a hierarchical data buffer to be constructed based on processing stage information, according to an exemplary embodiment.

[0031] Figure 5B A detailed implementation flowchart illustrating, according to another exemplary embodiment, the generation of definition information for a hierarchical data buffer to be constructed based on processing stage information.

[0032] Figure 5C A detailed implementation flowchart illustrating, according to yet another exemplary embodiment, the generation of definition information for a hierarchical data buffer to be constructed based on processing stage information.

[0033] Figure 6 This is a block diagram illustrating a heterogeneous data processing apparatus according to an exemplary embodiment.

[0034] Figure 7 This is a block diagram of a vehicle according to an exemplary embodiment.

[0035] Figure 8 This is a block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0036] The exemplary embodiments will now be described in detail with reference to the accompanying drawings.

[0037] It should be noted that the relevant embodiments and accompanying drawings are only for describing and illustrating exemplary embodiments provided by this disclosure, and not all embodiments of this disclosure, nor should this disclosure be understood to be limited to the relevant exemplary embodiments.

[0038] It should be noted that the terms "first," "second," etc., used in this disclosure are only used to distinguish different steps, devices, or modules. These terms do not represent any specific technical meaning, nor do they indicate any order or interdependence between them.

[0039] It should be noted that the terms “a,” “a plurality of,” and “at least one” used in this disclosure are illustrative rather than restrictive. Unless otherwise expressly indicated in the context, they should be understood as “one or more.”

[0040] It should be noted that the term "and / or" used in this disclosure is used to describe the relationship between related objects, and generally indicates that at least three relationships exist. For example, A and / or B can at least indicate: A exists alone, A and B exist simultaneously, and B exists alone. The symbol " / " indicates the meaning of "or".

[0041] It should be noted that the various steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Unless otherwise specified, the scope of this disclosure is not limited by the order in which the steps are described in the relevant embodiments.

[0042] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0043] In related technologies, there are many scenarios requiring heterogeneous data processing. For example, vehicle data processing systems need to process data from various types of sensors (including, but not limited to, camera data, radar sensor data, and inertial navigation device data); drone data processing systems need to process data from various types of sensors (including, for example, accelerometer, gyroscope, magnetic compass, and barometric pressure sensors); and intelligent robot data processing systems need to process data from various types of sensors (including, but not limited to, object recognition sensors, object flaw detection sensors, distance sensors, force sensors, and auditory sensors). Furthermore, while different types of sensor data are used as examples of heterogeneous data here, it is understood that heterogeneous data is not limited to sensor data and can take other forms as well.

[0044] Taking autonomous driving systems or driver assistance systems as examples, from the perspective of sensor functionality, building map functions, positioning functions, navigation functions, or perception functions all involve the process of fusion processing of data from multiple types of sensors. Furthermore, from the user's demand side, autonomous driving systems face very stringent requirements. They need to be applicable to various weather conditions (such as sunny, rainy, foggy, snowy, and cloudy days) and lighting conditions (such as daytime, dusk, nighttime, direct sunlight, and backlighting); they need to meet detection requirements at various distances (e.g., close-range detection of 2m-40m, medium-range detection of 40m-80m, long-range detection of 80m-200m, and ultra-long-range detection of over 200 meters); they need to meet requirements for perception dimensions and accuracy (e.g., six-degree-of-freedom observation, with accuracy requirements for velocity, acceleration, and angular velocity); and they need to meet the requirements of the autonomous driving system level (e.g., no missed detections, no false detections, and time and position errors meeting L2 / L4 level requirements). To achieve these requirements, in most cases, it is necessary to combine the advantages of multiple sensors, which also involves heterogeneous data processing.

[0045] Because heterogeneous data sources have different data structures and express different semantic information, the data processing workflow for heterogeneous data sources in various online or offline systems often requires copying data when using a particular heterogeneous data source before processing it. In scenarios involving large amounts of data processing and frequent copying, this results in significant memory resource overhead and very low processing efficiency.

[0046] In view of this, embodiments of the present disclosure provide a method, apparatus, electronic device, medium, and vehicle for heterogeneous data processing. By constructing a multi-level hierarchical data buffer, which is used to store heterogeneous data and each level of the hierarchical data buffer is matched to different processing stages or data objects, a unified modeling of heterogeneous data is achieved, facilitating data transfer and real-time flow, and saving the cost of copying massive amounts of data. At the same time, by constructing a target data structure, the hierarchical data buffers of multiple levels are organized in a target data structure. By accessing the target data structure, the target hierarchical data buffer to be accessed can be located and data reading or writing operations can be performed. For various online or offline systems, this facilitates efficient access to the hierarchical data buffers in data processing workflows or algorithm workflows, avoiding the problem of low search efficiency caused by discretely setting hierarchical data buffers.

[0047] Exemplary methods

[0048] Figure 1 This is a flowchart illustrating a method for determining the state of a data queue according to an exemplary embodiment.

[0049] Reference Figure 1 As shown, the method provided in this embodiment includes the following steps: S110, S120 and S130.

[0050] The steps S110 to S130 described above can be performed by electronic devices with computing and storage capabilities. Examples include, but are not limited to, vehicles, desktop computers, laptops, tablets, mobile phones, smartwatches, wristbands, and smart robots.

[0051] In step S110, a hierarchical data buffer with multiple levels is constructed based on heterogeneous related information; the aforementioned heterogeneous related information is used to represent the processing stage information or data object information involved in heterogeneous data processing.

[0052] The aforementioned hierarchical data buffers are used to store heterogeneous data, and each level of hierarchical data buffer is matched to different processing stages or data objects.

[0053] In some embodiments, step S110 above, constructing a multi-level hierarchical data buffer based on heterogeneous information, includes: obtaining information on the processing steps involved in heterogeneous data processing; generating definition information for the hierarchical data buffer to be constructed based on the processing step information; and constructing the multi-level hierarchical data buffer based on the definition information. For example, this could involve obtaining information on the processing steps involved in heterogeneous data processing in a data processing system (referring to various systems involving data processing operations). Such data processing systems include, but are not limited to, autonomous driving systems, drone flight systems, and various intelligent robot systems (such as logistics robot systems, voice-activated intelligent robot systems, and intelligent rescue and detection robot systems).

[0054] In some embodiments, the above-mentioned processing stage information includes at least one of the following: first information regarding data storage and retrieval in each processing stage of the processing flow, and second information regarding data storage and retrieval in each algorithm execution stage of the processing flow. The definition information of the above-mentioned hierarchical data buffer includes: the set number and level information of the hierarchical data buffer, and the storage object information of each level of the hierarchical data buffer.

[0055] Figure 2A This is a schematic diagram illustrating the processing flow involved in heterogeneous data processing according to an exemplary embodiment.

[0056] For example, refer to Figure 2AAs shown, the heterogeneous data processing flow includes four processing stages: processing stages H1 to H4. In processing stage H1, raw data X0 from various sensors (e.g., radar and camera) is acquired within a first time window (e.g., including 5 time points from t(N) to t(N+4)). In processing stage H2, the raw data X0 is segmented according to pixels to obtain segmented data X1. In processing stage H3, target detection is performed on the raw data X0 to obtain target detection data X2. In processing stage H4, the segmented data X1 and the target detection data X2 are fused to obtain the processing result X3.

[0057] In this embodiment, the first information regarding data storage and retrieval in each processing stage of the process flow includes:

[0058] The information to be stored includes: raw data X0, segmented data X1, target detection data X2, and processing result X3.

[0059] The information to be invoked is as follows: the original data X0 is invoked by processing stage H2 and processing stage H3; the segmented data X1 and the target detection data X2 are invoked by processing stage H4.

[0060] Figure 2B This is a schematic diagram illustrating the algorithm execution process involved in heterogeneous data processing according to an exemplary embodiment.

[0061] For example, refer to Figure 2B As shown, the algorithm execution steps in the heterogeneous data processing flow include: For the raw data X0 from various sensors (e.g., radar and camera) within each acquired time window (e.g., including 5 time points from t(N) to t(N+4)), the raw data X0 is segmented according to pixels using a segmentation algorithm to obtain segmented data X1. Target detection is performed on the raw data X0 using a detection algorithm to obtain target detection data X2. Finally, the segmented data X1 and the target detection data X2 are fused using a fusion algorithm to obtain the processing result X3.

[0062] In this embodiment, the second information regarding data storage and retrieval in each algorithm execution stage of the processing flow includes:

[0063] The information to be stored includes: raw data X0, segmented data X1, target detection data X2, and processing result X3.

[0064] The information to be invoked is as follows: the original data X0 is invoked by the segmentation algorithm and the detection algorithm; the segmented data X1 and the target detection data X2 are invoked by the fusion algorithm.

[0065] Based on the above embodiments, the hierarchical data buffer determined according to the above processing information includes a 4-level hierarchical data buffer, which is used to store the original data X0, the segmented data X1, the target detection data X2, and the processing result X3.

[0066] For example, the correspondence between the hierarchical data buffers and the data they store is as follows: the first-level hierarchical data buffer is used to store the original data X0, the second-level hierarchical data buffer is used to store the segmented data X1, the third-level hierarchical data buffer is used to store the target detection data X2, and the fourth-level hierarchical data buffer is used to store the processing result X3.

[0067] In some embodiments, the hierarchical data buffer described above can be a strip queue or a circular queue. The hierarchical data buffer can be increased or decreased according to the needs of the processing logic or algorithm logic, and can also be decomposed and refined at a finer granularity.

[0068] In step S120, a target data structure for organizing the above-mentioned hierarchical data buffer is constructed.

[0069] In some embodiments, step S130 above, constructing a target data structure for organizing the hierarchical data buffers, includes: defining and configuring a first content body of the target data structure to be constructed, wherein the first content body carries mapping relationship information for each level of hierarchical data buffers; defining and configuring a second content body of the target data structure to be constructed, wherein the second content body carries organizational relationship information for describing the association relationship between multiple levels of hierarchical data buffers; and constructing the target data structure based on the configured first content body and the second content body.

[0070] For example, the target data structure is a linked list structure containing multiple nodes; each node's first content body includes a data field, the data in which contains the first address information of the strip queue or circular queue corresponding to the current node, the first address information being used to represent the mapping relationship information; each node's second content body includes a pointer field, the pointer in which contains the second address information of the next node pointed to by the current node, the second address information being used to represent the organizational relationship information.

[0071] In some embodiments, where the heterogeneous related information is used to represent information about the processing steps involved in heterogeneous data processing, the organizational relationship information is generated based on the processing step information.

[0072] For multi-level data buffers that correspond to multiple levels according to data processing stages, the connection order of the target data structure (e.g., a linked list pointer) can be consistent with the processing order.

[0073] For example, by setting the pointers of the linked list to point in the same order as the algorithm calls during workflow processing, it can also adapt to data processing scenarios and make the calls convenient.

[0074] Figure 3 This is a schematic diagram illustrating the correspondence between a target data structure and a hierarchical data buffer according to an exemplary embodiment.

[0075] Reference Figure 3 As shown, the target data structure can be a linked list. Figure 3 The diagram uses a linked list 310 containing 4 nodes as an example. The linked list includes 4 nodes, namely the first node 311, the second node 312, the third node 313, and the fourth node 314.

[0076] Each node in the linked list includes a data field and a pointer field. The data field stores the address (the overall address of the buffer) A1 of the hierarchical data buffer corresponding to the current node. This address A1 allows location of the hierarchical data buffer and, consequently, of each memory location within that buffer. The pointer field stores the address A2 of the next node that the current node will point to. Based on the pointer relationships between nodes, any node in the linked list can be located from the overall address of the linked list (e.g., the starting address). (See reference...) Figure 3 As shown, the first node 311 points to the second node 312, the second node 312 points to the third node 313, and the third node 313 points to the fourth node 314.

[0077] The correspondence between linked list nodes and hierarchical data buffers can be predefined or updated, and pointers between the nodes in the linked list are used to associate them. For example, refer to... Figure 3 As shown, the first node 311 corresponds to the first-level hierarchical data buffer, the second node 312 corresponds to the second-level hierarchical data buffer, the third node 313 corresponds to the third-level hierarchical data buffer, and the fourth node 314 corresponds to the fourth-level hierarchical data buffer. The first-level hierarchical data buffer is used to store the raw data corresponding to multiple sensors within each time window. Figure 3 The diagram shows the original data buffer 321; the second-level hierarchical data buffer is used to store segmented data, therefore in Figure 3 The diagram shows a segmented data buffer 322; the third-level hierarchical data buffer is used to store target detection data, therefore in Figure 3 The diagram shows target detection data buffer 323; the fourth-level hierarchical data buffer is used to store the processing results obtained after fusion processing, therefore in Figure 3 The diagram shows the processing result buffer 324.

[0078] In this embodiment, a linked list data structure is used to organize multiple levels of hierarchical data buffers. These hierarchical data buffers are adapted to one or more data processing workflows. During the execution of one or more data processing workflows, the linked list is used to find the target level hierarchical data buffer to be accessed (for example, when performing fusion processing, it is necessary to access the level 2 and level 3 hierarchical data buffers for data reading operations and write the result of the fusion processing to the level 4 hierarchical data buffer). In business logic (for example, executing a target detection workflow for an autonomous driving function module, corresponding to...),... Figure 2A As shown in the algorithm logic, during the process of accessing the hierarchical data buffer, it is only necessary to find the first address of the linked list to quickly locate the address of any node in the linked list, and then find the address of the target level hierarchical data buffer corresponding to the target node. Based on the address, the target level hierarchical data buffer is accessed, and data reading or writing operations are performed in the target level hierarchical data buffer.

[0079] Therefore, the advantage of organizing multi-level hierarchical data buffers based on linked lists is that it is easy to locate. After locating a linked list (e.g., the first address of the linked list), it is easy to locate any node in the linked list. Then, based on the address information of the corresponding hierarchical data buffer contained in the data field of the node, the target level hierarchical data buffer can be accurately and quickly located.

[0080] It should be noted that the linked list data structure can also be replaced by other structures that can implement the function of organizing hierarchical data buffers. In the embodiments of this disclosure, the target data structure is required to have two characteristics: First, it has a corresponding mapping relationship with the buffer, such as carrying the address of the corresponding buffer (e.g., the starting address); Second, it has an association relationship that can organize the various buffers.

[0081] In some embodiments, the connection order of the target data structure (e.g., a pointer to a linked list) can be consistent with the processing order.

[0082] For example, in some implementations, the pointers of the linked list can be set to point in the same order as the algorithm calls during workflow processing. This can also adapt to data processing scenarios and make the calls convenient.

[0083] In step S130, during the execution of the data processing workflow, the target-level hierarchical data buffer is accessed based on the aforementioned target data structure, and data read or write operations are performed on the aforementioned target-level hierarchical data buffer.

[0084] Figure 4This is a schematic diagram illustrating the implementation process of step S130 according to an exemplary embodiment.

[0085] In some embodiments, refer to Figure 4 As shown, in step S130 above, during the execution of the data processing workflow, the target-level hierarchical data buffer is accessed based on the target data structure (e.g., linked list 310), and data read or write operations are performed on the target-level hierarchical data buffer. This includes: during the execution of the data processing workflow, according to the target processing stage or target algorithm execution stage in the heterogeneous data processing flow (e.g., in... Figure 4 The operation data object in the radar segmentation algorithm (illustrated in the diagram) (the corresponding operation data object includes: the input object is the original data X0, and the output object is the segmented data X1) determines the hierarchical data buffer of the target level to be accessed (for example, including: level 1 and level 2 hierarchical data buffers: original data buffer 321 and segmented data buffer 322); according to the above target data structure (for example, linked list 310), the hierarchical data buffer of the target level is located and accessed; within the hierarchical data buffer of the target level, according to the operation instructions for the above operation object in the above target processing link or the above target algorithm execution link, data reading operations or data writing operations are performed on the target storage unit in the hierarchical data buffer of the target level; for example, data reading operations are performed on the original data buffer 321, and the read original data X0 can be used as the input of the radar segmentation algorithm, camera segmentation algorithm, radar detection algorithm and camera detection algorithm. Based on the radar segmentation algorithm and camera segmentation algorithm, the original data X0 is processed to output the segmented data X1, and the segmented data X1 is written to the segmented data buffer 322.

[0086] In some embodiments, after performing a data read operation on the target storage unit within the hierarchical data buffer at the target level, the read result is synchronized to multiple algorithms or multiple processing threads that need to invoke the read result.

[0087] For online processing, the current time window represents the iteration cycle of an algorithm. The results of the iterations within the current time window can be stored in a hierarchical data buffer of the corresponding level. The buffer data within a time window can be read and then used by one or more algorithms. For example, it can be published (published) so that after a thread reads the target data, it can publish the read target data to one or more algorithms that need it, achieving data reuse and saving the cost of repeatedly copying the same data.

[0088] In the embodiments of this disclosure, the data processing workflow can be one or more processing flows. As long as at least some processing steps or storage objects of the above processing flow match at least one hierarchical data buffer within the target organizational structure, data can be read or written by searching the target organizational structure and locating the corresponding hierarchical data buffer.

[0089] For reading and writing data within the hierarchical data buffer, based on the inherent read / write characteristics of striped or circular queues, and following the first-in, first-out (FIFO) principle, data reading and writing are implemented through internal read and write pointers. During the read / write process, either a single-producer, single-consumer pattern or a multi-producer, multi-consumer pattern can be used.

[0090] In embodiments including steps S110 to S130, multiple levels of hierarchical data buffers are constructed based on the processing stage information or data object information involved in heterogeneous data processing. These hierarchical data buffers are used to store heterogeneous data, and each level of hierarchical data buffer is matched with different processing stages or data objects. This achieves unified modeling of heterogeneous data and facilitates data transfer and real-time flow, saving the cost of copying massive amounts of data. At the same time, by constructing a target data structure, the hierarchical data buffers of multiple levels are organized in a target data structure. By accessing the target data structure, the target hierarchical data buffer to be accessed can be located and data reading or writing operations can be performed. For various online or offline systems, this facilitates efficient access to hierarchical data buffers in data processing workflows or algorithm workflows, avoiding the problem of low search efficiency caused by discretely setting hierarchical data buffers.

[0091] The aforementioned heterogeneous data processing method can be applied to autonomous driving systems or other data processing systems, such as drone flight systems and various intelligent robot systems (e.g., logistics robot systems, voice-activated intelligent robot systems, intelligent rescue and detection robot systems). Taking its application to autonomous driving systems as an example, this method designs different hierarchical data buffers (e.g., raw data circular buffers, segmented data circular buffers, detection data circular buffers, and fused data buffers) based on the different stages / steps of the data processing pipeline and algorithm workflow involved in sensor data fusion in autonomous driving systems. Various circular buffer units are used to store different types of sensor data from the same stage / step, and the hierarchical data buffers are organically organized using a target data structure (e.g., linked lists or other data structures) to adapt to various data processing and algorithm workflows of autonomous driving systems. These buffers can then be freely combined according to the needs of data processing logic or calling logic to adapt to the data flow framework workflow and algorithm workflow involved in multi-type sensor data fusion, reducing the cost of copying massive amounts of sensor data and effectively improving the efficiency of autonomous driving system operation. For example, it can be applied to online systems of domain control platforms (such as domain control platforms based on Orin chips) and offline simulation systems based on X86_64 (a 64-bit extension of the X86 architecture, backward compatible with 16-bit and 32-bit X86 architectures).

[0092] In some embodiments, the data stored in the hierarchical data buffers are sensor time-series data. Sensor data from various sensors, intermediate data generated by various processing workflows, and final result data are written to the corresponding hierarchical data buffers in chronological order. The data in these hierarchical data buffers is read out by the corresponding algorithms or workflows, realizing unified modeling of heterogeneous data and facilitating data transfer and real-time flow, saving the cost of copying massive amounts of data. The types of sensors depend on the corresponding data processing scenario. For autonomous driving systems, sensor data includes, but is not limited to, camera capture data, radar sensor data, inertial navigation device data, and other types of sensor data. For UAV data processing systems, sensor data includes sensor data from accelerometers, gyroscopes, magnetic compasses, and barometric pressure sensors. For intelligent robot data processing systems, sensor data includes, but is not limited to, sensor data from object recognition sensors, object flaw detection sensors, distance sensors, force sensors, and hearing sensors.

[0093] Taking radar and camera, two sensors in an autonomous driving system, as examples of heterogeneous data sources, the original data X0 consists of point cloud time-series data and image time-series data from the radar and camera within the time window t(0) to t(4). The point cloud time-series data is represented as {[t(0),P(0)]、[t(1),P(1)]、[t(2),P(2)]、[t(3),P(3)]、[t(4),P(4)]}, and the image time-series data is represented as {[t(0),I(0)]、[t(1),I(1)]、[ t(2),I(2)]、[t(3),I(3)]、[t(4),I(4)]};where P(0),P(1),P(2),P(3),P(4) represent the point cloud data corresponding to each time t(0), t(1), t(2), t(3), t(4) (the numbers in parentheses are used to distinguish different times); I(0),I(1),I(2),I(3),I(4) represent the image data corresponding to each time t(0), t(1), t(2), t(3), t(4).

[0094] The segmented data X1 is obtained by segmenting the point cloud data and image data at various time points. For example, the point cloud data [t(0), P(0)] can be segmented to obtain the data of each spatial point at time t(0), such as point 1, point 2, point 3, etc., [t(0), P1(0)], [t(0), P2(0)], [t(0), P3(0)], etc., containing three-dimensional position information, intensity information, and point cloud data labels (such as labels belonging to P(0)). Similarly, the image data [t(0), I(0)] can be segmented to obtain the data of each pixel at time t(0), such as pixel 1, pixel 2, pixel 3, etc., [t(0), I1(0)], [t(0), I2(0)], [t(0), I3(0)], containing pixel values ​​and image data labels (such as labels belonging to I(0)).

[0095] Target detection data X2 is the result data of one or more targets obtained by passing the original data at each time point through target detection. For example, radar point cloud data [t(0), P(0)] is processed by target detection to obtain information about one or more targets, including but not limited to some or all of the following: target position, velocity, acceleration, size, heading angle, heading angular velocity, target category, etc.; for example, camera image data [t(0), I(0)] is processed by target detection to obtain information about one or more targets, including but not limited to: target position in the image coordinate system, target bounding box size, target category, target position in the camera coordinate system, etc.

[0096] The processing result X3 is the target data and scene data obtained by fusing the segmented data X1 and the target detection data X2 at each time step. The target data includes, but is not limited to, some or all of the following: the target's position, velocity, acceleration, the size of the target's bounding box, the pose of the target's bounding box, the target's heading angular velocity, the target's category, etc. The scene data includes, but is not limited to: the scene's two-dimensional polygons, three-dimensional polygons, scene category, etc.

[0097] Figure 5A A detailed implementation flowchart illustrating the generation of definition information for a hierarchical data buffer to be constructed based on processing stage information, according to an exemplary embodiment.

[0098] In some embodiments, the aforementioned processing stage information refers to the first element information of heterogeneous data to be stored in each processing stage of the processing flow; the first element information includes: associated processing operations of the data to be stored and the corresponding operation call relationships, and storage-related information of the data to be stored. The heterogeneous data to be stored can, for example, be sensing time-series data corresponding to various types of sensors.

[0099] The associated processing operations of the data to be stored and the corresponding operation call relationships, specifically means: what operation corresponds to the output data or input data of the data to be stored.

[0100] The operation call relationship here can be the input object of the operation, the output object of the operation, or the intermediate processing object of the operation, etc.

[0101] The data to be stored here can be raw data, intermediate calculation results, or final calculation results, etc.

[0102] The relevant information to be stored can be the data type of the data to be stored, the amount of space occupied by the storage, etc.

[0103] Reference Figure 5A As shown, based on the information from the above processing steps, the definition information of the hierarchical data buffer to be constructed is generated, including the following steps: S511, S512 and S513.

[0104] In step S511, the first number of hierarchical data buffers is determined based on the total number of storage objects corresponding to the data to be stored.

[0105] For example, refer to Figure 2A As shown, the data to be stored in each processing stage includes: original data X0, segmented data X1, target detection data X2, and processing result X3. The total number of corresponding storage objects is 4, thus the first number of the hierarchical data buffer can be determined to be 4.

[0106] In step S512, based on the above-mentioned associated processing operation and the corresponding operation call relationship, the corresponding data to be stored and level information are allocated to the hierarchical data buffer under the first quantity.

[0107] Reference Figure 2A As shown, the following associated processing operations and their corresponding operation call relationships exist: Original data X0 is the output data corresponding to the acquisition operation; original data X0 is the input data corresponding to the segmentation operation, and segmented data X1 is the output data corresponding to the segmentation operation; original data X0 is the input data corresponding to the target detection operation, and target detection data X2 is the output data corresponding to the target detection operation; segmented data X1 and target detection data X2 are the input data corresponding to the fusion operation, and processing result X3 is the output data corresponding to the fusion operation. Based on the above associated processing operations and their corresponding operation call relationships, corresponding level information and data to be stored can be allocated to the four hierarchical data buffers. For example, the first-level hierarchical data buffer is used to store original data X0, the second-level hierarchical data buffer is used to store segmented data X1, the third-level hierarchical data buffer is used to store target detection data X2, and the fourth-level hierarchical data buffer is used to store processing result X3.

[0108] In step S513, storage object information for hierarchical data buffers at each level is generated based on the aforementioned storage-related information.

[0109] Based on the data type of the data to be stored, the size of the storage space, and other storage-related information, storage object information for hierarchical data buffers at various levels can be generated.

[0110] In the embodiment including steps S511 to S513, different processing stages / processing links involved in heterogeneous data processing can be decoupled according to one or more data processing workflows, and a hierarchical data buffer corresponding to the storage and retrieval in the processing link can be generated accordingly, which is adapted to the data transfer and real-time flow of these data processing work, and a unified heterogeneous data modeling architecture is constructed.

[0111] Figure 5B A detailed implementation flowchart illustrating, according to another exemplary embodiment, the generation of definition information for a hierarchical data buffer to be constructed based on processing stage information.

[0112] In some embodiments, the above-mentioned processing stage information is the second element information of heterogeneous data to be stored in each algorithm execution stage of the processing flow; the above-mentioned second element information includes: the associated algorithm of the data to be stored and the corresponding algorithm call relationship, and the storage-related information of the data to be stored.

[0113] The association algorithm of the data to be stored and the corresponding algorithm call relationship, specifically means: what algorithm's output data or input data the data to be stored is.

[0114] The algorithm call relationship here can be the input object of the algorithm, the output object of the operation, or the intermediate processing object of the algorithm, etc.

[0115] The data to be stored here can be raw data, intermediate calculation results, or final calculation results, etc.

[0116] The relevant information to be stored can be the data type of the data to be stored, the amount of space occupied by the storage, etc.

[0117] Reference Figure 5B As shown, based on the information from the above processing steps, the definition information of the hierarchical data buffer to be constructed is generated, including the following steps: S521, S522 and S523.

[0118] In step S521, the second number of hierarchical data buffers is determined based on the total number of storage objects corresponding to the data to be stored.

[0119] For example, refer to Figure 2B As shown, the data to be stored in each algorithm execution stage includes: original data X0, segmentation data X1, target detection data X2, and processing result X3. The total number of corresponding storage objects is 4, which determines that the second number of the hierarchical data buffer is 4.

[0120] In step S522, based on the above-mentioned association algorithm and the corresponding algorithm call relationship, the corresponding data to be stored and level information are allocated to the hierarchical data buffer under the second quantity.

[0121] Reference Figure 2B As shown, the following associated algorithms and their corresponding algorithm call relationships exist: Original data X0 is the input data for the segmentation algorithm, and segmented data X1 is the output data for the segmentation algorithm; original data X0 is the input data for the detection algorithm, and target detection data X2 is the output data for the detection algorithm; segmented data X1 and target detection data X2 are the input data for the fusion algorithm, and processing result X3 is the output data for the fusion algorithm. Based on the above associated processing operations and their corresponding operation call relationships, corresponding level information and data to be stored can be allocated to the four hierarchical data buffers. For example, the first-level hierarchical data buffer is used to store the original data X0, the second-level hierarchical data buffer is used to store the segmented data X1, the third-level hierarchical data buffer is used to store the target detection data X2, and the fourth-level hierarchical data buffer is used to store the processing result X3.

[0122] In step S523, storage object information for hierarchical data buffers at each level is generated based on the aforementioned storage-related information.

[0123] In the embodiment including steps S521 to S523, different algorithm execution stages involved in heterogeneous data processing can be decoupled according to one or more data processing workflows, and a hierarchical data buffer corresponding to the storage and calling in each algorithm execution stage can be generated accordingly. This adapts to the data transfer and real-time flow of these algorithm workflows, and constructs a unified heterogeneous data modeling architecture.

[0124] Figure 5C A detailed implementation flowchart illustrating, according to yet another exemplary embodiment, the generation of definition information for a hierarchical data buffer to be constructed based on processing stage information.

[0125] In some embodiments, the above-mentioned processing stage information refers to the target metadata of heterogeneous data to be stored in each processing stage of the processing flow and each algorithm execution stage; the target metadata includes: the associated processing operations of the data to be stored and the corresponding operation call relationships, the associated algorithms of the data to be stored and the corresponding algorithm call relationships, and the storage-related information of the data to be stored.

[0126] Reference Figure 5C As shown, based on the information from the above processing steps, the definition information of the hierarchical data buffer to be constructed is generated, including the following steps: S531, S532 and S533.

[0127] In step S531, the target number of hierarchical data buffers is determined based on the total number of storage objects corresponding to the data to be stored.

[0128] In step S532, based on the above-mentioned association processing operation, the above-mentioned operation call relationship, the above-mentioned association algorithm and the above-mentioned algorithm call relationship, the corresponding data to be stored and level information are allocated to the hierarchical data buffer under the above-mentioned target quantity.

[0129] In step S533, storage object information for hierarchical data buffers at each level is generated based on the aforementioned storage-related information.

[0130] In some embodiments, the association processing operation and the association algorithm can have a hierarchical relationship. For example, for a certain data to be stored, the association processing operation uses the association algorithm to process the data to be stored.

[0131] Association processing operations and association algorithms can also have a parallel relationship; a certain piece of data to be stored can be called by an algorithm and also by a certain processing operation.

[0132] In the embodiment including steps S531 to S533, different algorithm execution stages and different processing stages involved in heterogeneous data processing can be decoupled according to one or more data processing workflows, and a hierarchical data buffer corresponding to the storage and calling in each algorithm execution stage and processing stage can be generated accordingly. This adapts to the data transfer and real-time flow of these data processing workflows and algorithm workflows, and constructs a unified heterogeneous data modeling architecture.

[0133] Based on the above embodiments, in scenarios where multi-sensor fusion processing is applied, heterogeneous sensor data of multiple types of sensors in the same processing stage can be stored in the same level of hierarchical data buffer, with multiple processing stages / algorithms corresponding to multiple levels of hierarchical data buffers.

[0134] In other embodiments, a hierarchical data buffer with multiple levels is constructed based on heterogeneous information, including:

[0135] Obtain information about the data objects involved in heterogeneous data processing; the information about the data objects includes: the sensor type corresponding to the sensing time series data (e.g., radar type (or more specific lidar type, ordinary radar type, etc.), camera type, inertial navigation device type, etc.);

[0136] Based on the sensor type, generate definition information for the hierarchical data buffer to be constructed; the definition information includes: the set number and level information of the hierarchical data buffer, and the storage object information of each level of hierarchical data buffer;

[0137] Based on the defined information, a hierarchical data buffer with multiple levels is constructed.

[0138] In this embodiment, different hierarchical data buffers are generated according to different sensor types. For example, the first-level hierarchical data buffer is used to store radar timing data, which covers data from multiple stages in the processing flow; the second-level hierarchical data buffer is used to store camera timing data, which covers data from multiple stages in the processing flow; the hierarchical data buffers at multiple levels can also be organized using a target data structure.

[0139] In some embodiments, the above-described heterogeneous data processing method further includes:

[0140] Based on the updated configuration information of the above mapping relationship, update the first content body to obtain the updated target data structure; or...

[0141] Based on the updated configuration information of the aforementioned organizational relationships, update the second content body to obtain the updated target data structure; or...

[0142] Based on the updated configuration information of the above mapping relationship information and the above organizational relationship information, the above first content body and the above second content body are updated accordingly to obtain the updated target data structure.

[0143] In this embodiment, given a defined data processing workflow architecture, hierarchical data buffers and target organizational structures can be defined in the generated configuration information. Furthermore, if the processing logic or algorithm logic changes, the solution provided in this embodiment can adapt to these changes and adjust the target data structure accordingly.

[0144] Exemplary device

[0145] Figure 6 This is a block diagram illustrating a heterogeneous data processing apparatus according to an exemplary embodiment.

[0146] Reference Figure 6 As shown, the heterogeneous data processing apparatus 600 provided in this embodiment includes: a buffer construction module 610, a data organization construction module 620, and a data access and operation module 630.

[0147] The aforementioned buffer construction module 610 is used to construct hierarchical data buffers at multiple levels based on heterogeneous information. The aforementioned heterogeneous information is used to represent information about processing stages or data objects involved in heterogeneous data processing. The aforementioned processing stage information includes at least one of the following: first information regarding data storage and retrieval in each processing stage of the processing flow, and second information regarding data storage and retrieval in each algorithm execution stage of the processing flow.

[0148] The aforementioned data organization module 620 is used to construct the target data structure for organizing the aforementioned hierarchical data buffer.

[0149] The aforementioned data access and operation module 630 is used to access the target-level hierarchical data buffer based on the aforementioned target data structure during the execution of the data processing workflow, and to perform data read or write operations on the target-level hierarchical data buffer. Constructing the target data structure for organizing the aforementioned hierarchical data buffer includes: defining and configuring a first content body of the target data structure to be constructed, the first content body carrying mapping relationship information for each level of hierarchical data buffer; defining and configuring a second content body of the target data structure to be constructed, the second content body carrying organizational relationship information describing the association between multiple levels of hierarchical data buffers; and constructing the target data structure based on the configured first content body and the second content body.

[0150] In some embodiments, the above-described apparatus 600 further includes a data organization update module.

[0151] The aforementioned data organization update module is used to: update the first content body according to the update configuration information of the mapping relationship information to obtain the updated target data structure; or, update the second content body according to the update configuration information of the organization relationship information to obtain the updated target data structure; or, update the first content body and the second content body according to the update configuration information of the mapping relationship information and the organization relationship information to obtain the updated target data structure.

[0152] The specific details, beneficial effects, and other embodiments of the first embodiment can all be incorporated into this embodiment, and will not be repeated here.

[0153] Exemplary vehicle

[0154] Figure 7 This is a block diagram of a vehicle according to an exemplary embodiment.

[0155] Reference Figure 7 As shown, the vehicle 700 can be a gasoline-powered vehicle, a hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles. The vehicle 700 may include multiple subsystems, such as a drive system 710, a control system 720, a sensing system 730, a communication system 740, an information display system 750, and a computing processing system 760. The vehicle 700 may also include more or fewer subsystems, and each subsystem may include multiple components, which will not be detailed here.

[0156] The drive system 710 includes components that provide power to the vehicle 700. These include, for example, an engine, an energy source, and a transmission.

[0157] The control system 720 includes components that provide control for the vehicle 700. These include, for example, vehicle control, cockpit equipment control, and driver assistance control.

[0158] The perception system 730 includes components that provide the vehicle 700 with perception of its surroundings. These include, for example, a vehicle positioning system, a laser sensor, a voice sensor, an ultrasonic sensor, and camera equipment.

[0159] The communication system 740 includes components that provide communication connectivity for the vehicle 700. These may include, for example, mobile communication networks (e.g., 3G, 4G, 5G networks), WiFi, Bluetooth, and vehicle-to-everything (V2X) connectivity.

[0160] The information display system 750 includes components that provide various information displays for the vehicle 700. These include, for example, vehicle information displays, navigation information displays, and entertainment information displays.

[0161] The computing processing system 760 includes components that provide data computing and processing capabilities for the vehicle 700. The computing processing system 760 may include at least one processor 761 and a memory 762. The processor 761 can execute instructions stored in the memory 762.

[0162] The processor 761 can be any conventional processor, such as a commercially available CPU. The processor may also include, for example, a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application Specific Integrated Circuit (ASIC), or a combination thereof.

[0163] The memory 762 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0164] In this embodiment of the disclosure, a set of instructions is stored in the memory 762, and the processor 761 can execute the set of instructions to implement all or part of the steps of the heterogeneous data processing method described in any of the exemplary embodiments above.

[0165] Exemplary electronic devices

[0166] Figure 8 This is a block diagram of an electronic device according to an exemplary embodiment.

[0167] Reference Figure 8 As shown, the electronic device 800 can be a vehicle controller, an in-vehicle terminal, an in-vehicle computer, or other types of electronic devices. The electronic device 800 may include at least one processor 810 and a memory 820. The processor 810 can execute instructions stored in the memory 820. The processor 810 is communicatively connected to the memory 820 via a data bus. In addition to the memory 820, the processor 810 can also be communicatively connected to an input device 830, an output device 840, and a communication device 850 via a data bus.

[0168] The processor 810 can be any conventional processor, such as a commercially available CPU. The processor may also include, for example, a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application Specific Integrated Circuit (ASIC), or a combination thereof.

[0169] The memory 820 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0170] In this embodiment of the present disclosure, the memory 820 stores executable instructions, and the processor 810 can read the executable instructions from the memory 820 and execute the instructions to implement all or part of the steps of the heterogeneous data processing method described in any of the exemplary embodiments above.

[0171] Exemplary computer-readable storage media

[0172] In addition to the methods and apparatus described above, exemplary embodiments of this disclosure may also be a computer program product or a computer-readable storage medium storing the computer program product. The computer product includes computer program instructions that can be executed by a processor to perform all or part of the steps described in any of the methods in the exemplary embodiments described above.

[0173] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages, and scripting languages ​​(e.g., Python). The program code can be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0174] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: static random access memory (SRAM) having one or more electrically connected wires, electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk, or any suitable combination thereof.

[0175] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0176] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for heterogeneous data processing, characterized in that, include: Based on heterogeneous information, a hierarchical data buffer with multiple levels is constructed; the heterogeneous information is used to represent the processing steps or data object information involved in heterogeneous data processing. Construct a target data structure for organizing the hierarchical data buffer; During the execution of the data processing workflow, the target-level hierarchical data buffer is accessed based on the target data structure, and data read or write operations are performed on the target-level hierarchical data buffer. The process of constructing a multi-level hierarchical data buffer based on heterogeneous information includes: Obtain information on the processing steps involved in heterogeneous data processing; The processing information includes at least one of the following: first information about data storage and retrieval in each processing stage of the processing flow, and second information about data storage and retrieval in each algorithm execution stage of the processing flow; Based on the processing information, definition information for the hierarchical data buffer to be constructed is generated; the definition information includes: the set number and level information of the hierarchical data buffer, and the storage object information of each level of the hierarchical data buffer. Based on the defined information, a hierarchical data buffer with multiple levels is constructed.

2. The method according to claim 1, characterized in that, The processing stage information refers to the first element information of the heterogeneous data to be stored in each processing stage of the processing flow; the first element information includes: the associated processing operations of the data to be stored and the corresponding operation call relationships, and the storage-related information of the data to be stored. Based on the processing information, the definition information of the hierarchical data buffer to be constructed is generated, including: The first number of hierarchical data buffers is determined based on the total number of storage objects corresponding to the data to be stored. Based on the associated processing operation and the corresponding operation call relationship, allocate the corresponding data to be stored and level information to the hierarchical data buffer under the first quantity; Based on the storage-related information, storage object information for hierarchical data buffers at each level is generated.

3. The method according to claim 1, characterized in that, The process of constructing a multi-level hierarchical data buffer based on heterogeneous information includes: Acquire information about the data objects involved in heterogeneous data processing; the data object information includes: the sensor type corresponding to the sensing time series data; Based on the sensor type, generate definition information for the hierarchical data buffer to be constructed; the definition information includes: the set number and level information of the hierarchical data buffer, and the storage object information of each level of hierarchical data buffer; Based on the defined information, a hierarchical data buffer with multiple levels is constructed.

4. The method according to claim 1, characterized in that, The processing stage information refers to the target metadata of heterogeneous data to be stored in each processing stage of the processing flow and each algorithm execution stage. The target metadata includes: the associated processing operations of the data to be stored and the corresponding operation call relationships, the associated algorithms of the data to be stored and the corresponding algorithm call relationships, and the storage-related information of the data to be stored. The target number of hierarchical data buffers is determined based on the total number of storage objects corresponding to the data to be stored. Based on the association processing operation, the operation call relationship, the association algorithm, and the algorithm call relationship, allocate corresponding data to be stored and level information to the hierarchical data buffer under the target quantity; Based on the storage-related information, storage object information for hierarchical data buffers at each level is generated.

5. The method according to claim 1, characterized in that, The process of constructing a multi-level hierarchical data buffer based on heterogeneous information includes: Acquire information about the data objects involved in heterogeneous data processing; the data object information includes: the sensor type corresponding to the sensing time series data; Based on the sensor type, generate definition information for the hierarchical data buffer to be constructed; the definition information includes: the set number and level information of the hierarchical data buffer, and the storage object information of each level of hierarchical data buffer; Based on the defined information, a hierarchical data buffer with multiple levels is constructed.

6. The method according to any one of claims 1-5, characterized in that, Constructing the target data structure for organizing the hierarchical data buffer includes: Define and configure the first content body of the target data structure to be constructed. The first content body carries mapping relationship information for hierarchical data buffers at each level. Define and configure the second content body of the target data structure to be constructed, the second content body carrying organizational relationship information for describing the relationship between hierarchical data buffers at multiple levels; Based on the configured first content body and second content body, the target data structure is constructed.

7. The method according to claim 6, characterized in that, When the heterogeneous related information is used to represent the processing steps involved in heterogeneous data processing, the organizational relationship information is generated based on the processing step information.

8. The method according to claim 6, characterized in that, Also includes: Based on the updated configuration information of the mapping relationship information, the first content body is updated to obtain the updated target data structure; or, Based on the updated configuration information of the organizational relationship information, update the second content body to obtain the updated target data structure; or... Based on the updated configuration information of the mapping relationship information and the organizational relationship information, the first content body and the second content body are updated accordingly to obtain the updated target data structure.

9. The method according to claim 6, characterized in that, The hierarchical data buffer is a strip queue or a circular queue; The data stored in the hierarchical data buffer is sensor time-series data; The target data structure is a linked list structure, which contains multiple nodes; wherein the first content body of each node includes a data field, the data in the data field contains the first address information of the bar queue or circular queue corresponding to the current node, and the first address information is used to represent the mapping relationship information; the second content body of each node includes a pointer field, the pointer in the pointer field contains the second address information of the next node pointed to by the current node, and the second address information is used to represent the organizational relationship information.

10. The method according to any one of claims 1-5, characterized in that, During the execution of the data processing workflow, the process of accessing the target-level hierarchical data buffer based on the target data structure and performing data read or write operations on the target-level hierarchical data buffer includes: During the execution of the data processing workflow, the hierarchical data buffer corresponding to the target level to be accessed is determined based on the operational data objects in the target processing stage or target algorithm execution stage of the heterogeneous data processing process. Based on the target data structure, locate and access the hierarchical data buffer at the target level; Within the hierarchical data buffer at the target level, data read or write operations are performed on the target storage unit within the hierarchical data buffer at the target level according to the operation instructions for the operation data object in the target processing stage or the target algorithm execution stage.

11. The method according to claim 10, characterized in that, After performing a data read operation on the target storage unit within the hierarchical data buffer at the target level, the read result is synchronized to multiple algorithms or multiple processing threads that need to call the read result.

12. An apparatus for heterogeneous data processing, characterized in that, include: The buffer construction module is used to construct hierarchical data buffers at multiple levels based on heterogeneous related information; the heterogeneous related information is used to represent the processing steps or data object information involved in heterogeneous data processing. A data organization module is used to construct a target data structure for organizing the hierarchical data buffer. The data access and operation module is used to access the target-level hierarchical data buffer based on the target data structure during the execution of the data processing workflow, and to perform data read or data write operations on the target-level hierarchical data buffer. The process of constructing a multi-level hierarchical data buffer based on heterogeneous information includes: Obtain information on the processing steps involved in heterogeneous data processing; The processing information includes at least one of the following: first information about data storage and retrieval in each processing stage of the processing flow, and second information about data storage and retrieval in each algorithm execution stage of the processing flow; Based on the processing information, definition information for the hierarchical data buffer to be constructed is generated; the definition information includes: the set number and level information of the hierarchical data buffer, and the storage object information of each level of the hierarchical data buffer. Based on the defined information, a hierarchical data buffer with multiple levels is constructed.

13. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method of any one of claims 1-11.

14. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the method described in any one of claims 1-11.

15. A vehicle, characterized in that, The vehicle stores a set of instructions that are executed by the vehicle's autonomous driving system to implement the method described in any one of claims 1-11.

Images