Visualization method, device, vehicle and storage medium for posture time synchronization

Abstract

The present application relates to the field of vehicle technology, and in particular to a method, device, vehicle, and storage medium for visualization of posture and time synchronization, wherein the method comprises: obtaining raw data collected by any sensor or device in the vehicle; matching the coordinate conversion path between the current coordinate system and the target coordinate system according to the current coordinate system of any sensor or device and the target coordinate system where the visualization operation is located, and inserting a preset posture interpolation into the coordinate conversion path at a preset time to obtain a preset conversion relationship between the current coordinate system and the target coordinate system; using the preset conversion relationship to convert all raw data to the target coordinate system, obtaining at least one conversion data synchronized with time and posture, rendering the at least one conversion data into visualization data, and displaying the visualization data on a preset display terminal of the vehicle. Thus, the method solves the problems of data waste caused by not saving data during the visualization process and is only applicable to the visualization of vehicle status information.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a method, device, vehicle, and storage medium for posture and time synchronization visualization. Background Art

[0002] During the driving of an autonomous vehicle, the safety officer needs to keep abreast of the vehicle's status information and intervene in any problems that arise during the autonomous driving process in a timely manner to avoid accidents.

[0003] Related technologies obtain status information from autonomous vehicles and control displays installed within them to display this information, allowing safety officers to obtain real-time status information. However, these technologies only provide for the visualization of vehicle information, resulting in limited visualization data and a poor user experience. Summary of the Invention

[0004] The present application provides a posture and time synchronized visualization method, device, vehicle and storage medium to solve the problems in related technologies such as data waste caused by failure to save data during the visualization process, and only applicable to the visualization of vehicle information, single visualization data, and poor user visualization experience.

[0005] The first aspect of the present application provides a posture and time synchronized visualization method, comprising the following steps: acquiring raw data collected by any sensor or device in a vehicle; matching a coordinate conversion path between the current coordinate system and the target coordinate system according to the current coordinate system where the arbitrary sensor or device is located and the target coordinate system where the visualization operation is located, and inserting a preset posture interpolation into the coordinate conversion path at a preset time to obtain a preset conversion relationship between the current coordinate system and the target coordinate system; utilizing the preset conversion relationship to convert all raw data to the target coordinate system to obtain at least one conversion data synchronized in time and posture, rendering the at least one conversion data into visualization data, and displaying the visualization data on a preset display terminal of the vehicle.

[0006] According to the above-mentioned technical means, the embodiments of the present application can synchronize different types of data to the same time and the same coordinate system, and can realize the visualization of the time and posture synchronization of different types of data, thereby synchronizing the coordinate system and time by constructing a tree structure and posture interpolation, effectively solving the problem of difficulty in data visualization caused by different coordinate systems of different sensors or devices, making it easier for safety officers to monitor the vehicle status and improve safety during autonomous driving.

[0007] Furthermore, the coordinate conversion path between the current coordinate system and the target coordinate system is matched according to the current coordinate system of any sensor or device and the target coordinate system of the visualization operation, including: using the current coordinate system of any sensor or device and the target coordinate system of the visualization operation as indexes, querying a pre-constructed tree structure database to obtain the coordinate conversion path between the current coordinate system and the target coordinate system.

[0008] According to the above technical means, the embodiment of the present application can accurately index the required coordinate conversion path through a pre-built tree structure database.

[0009] Furthermore, each coordinate system in the tree structure in the tree structure database is a node, the upper layer node is the parent node, and the lower layer node is the child node. The query of the pre-constructed tree structure database obtains the coordinate conversion path between the current coordinate system and the target coordinate system, including: starting from the corresponding node of the current coordinate system where the arbitrary sensor or device is located, recursively searching all the parent nodes until the root node of the tree structure to obtain a first path; starting from the corresponding node of the target coordinate system, recursively searching all the parent nodes until the root node of the tree structure to obtain a second path; fusing the first path and the second path to obtain the coordinate conversion path between the current coordinate system and the target coordinate system.

[0010] According to the above technical means, the embodiment of the present application can obtain the coordinate conversion path through the tree structure database by recursion, which can facilitate the preservation of data in the visualization process and avoid data waste.

[0011] Furthermore, the interpolation of the preset posture into the coordinate transformation path at a preset moment to obtain a preset transformation relationship between the current coordinate system and the target coordinate system includes: using the preset moment as an index, querying a preset cache area to obtain a first moment and a second moment adjacent to the preset moment, and a first transformation relationship and a second transformation relationship corresponding to each segment of the coordinate transformation path at the first moment and the second moment; using a preset interpolation algorithm to interpolate the first transformation relationship and the second transformation relationship to obtain the transformation relationship of each segment of the path, and generating the preset transformation relationship based on the transformation relationships of all segments of the path.

[0012] According to the above technical means, the embodiment of the present application obtains the conversion relationship of all paths through a preset interpolation algorithm to synchronize the coordinate system and time, thereby accelerating the development of intelligent driving algorithms.

[0013] Furthermore, before matching the coordinate conversion path between the current coordinate system and the target coordinate system according to the current coordinate system where the arbitrary sensor or device is located and the target coordinate system where the visualization operation is located, it also includes: constructing a preset cache area for storing the conversion relationships of different coordinate systems at different times, and updating the data in the preset cache area according to the original data collected by any sensor or device in the vehicle, wherein, when the timestamp of the newly added data in the cache area is not within the preset storage time range of the cache area, the newly added data is discarded.

[0014] According to the above technical means, the embodiment of the present application constructs a preset cache area to store the posture transformation relationship between different coordinate systems and at different times and can clear useless data, thereby facilitating the preservation of all intermediate variables, avoiding data waste, and making it easier for staff to reproduce problems.

[0015] The second aspect of the present application provides a posture and time synchronized visualization device, including: an acquisition module for acquiring raw data collected by any sensor or device in a vehicle; a processing module for matching the coordinate conversion path between the current coordinate system and the target coordinate system according to the current coordinate system where the arbitrary sensor or device is located and the target coordinate system where the visualization operation is located, and inserting a preset posture interpolation into the coordinate conversion path at a preset time to obtain a preset conversion relationship between the current coordinate system and the target coordinate system; a visualization module for using the preset conversion relationship to convert all raw data to the target coordinate system, obtain at least one conversion data synchronized with time and posture, render the at least one conversion data into visualization data, and display the visualization data on a preset display terminal of the vehicle.

[0016] Furthermore, the processing module is further used to: use the current coordinate system of any sensor or device and the target coordinate system of the visualization operation as indexes to query a pre-built tree structure database to obtain a coordinate conversion path between the current coordinate system and the target coordinate system.

[0017] Furthermore, each coordinate system in the tree structure in the tree structure database is a node, the upper layer node is the parent node, and the lower layer node is the child node. The processing module is further used to: start from the corresponding node of the current coordinate system where the arbitrary sensor or device is located, recursively search all parent nodes until the root node of the tree structure to obtain a first path; start from the corresponding node of the target coordinate system, recursively search all parent nodes until the root node of the tree structure to obtain a second path; fuse the first path and the second path to obtain a coordinate conversion path between the current coordinate system and the target coordinate system.

[0018] The third aspect of the present application provides a vehicle, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the posture time synchronization visualization method as described in the above embodiment.

[0019] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the posture time synchronization visualization method as described in the above embodiment.

[0020] Therefore, this application has at least the following beneficial effects:

[0021] (1) The embodiments of the present application can synchronize different types of data to the same time and coordinate system, and can realize the visualization of the time and posture synchronization of different types of data. Thus, by constructing a tree structure and posture interpolation to synchronize the coordinate system and time, the problem of data visualization difficulties caused by different coordinate systems of different sensors or devices is effectively solved, which facilitates the safety officer to monitor the vehicle status and improves the safety during the autonomous driving process.

[0022] (2) The embodiment of the present application can accurately index the required coordinate conversion path through a pre-built tree structure database;

[0023] (3) The embodiment of the present application can obtain the coordinate conversion path through the tree structure database by recursion, which can facilitate the preservation of data in the visualization process and avoid data waste;

[0024] (4) The embodiment of the present application obtains the transformation relationship of all paths through a preset interpolation algorithm to synchronize the coordinate system and time, thereby accelerating the development of intelligent driving algorithms;

[0025] (5) The embodiment of the present application constructs a preset cache area to store the posture transformation relationship between different coordinate systems and at different times and can clear useless data, thereby facilitating the preservation of all intermediate variables, avoiding data waste, and making it easier for staff to reproduce problems.

[0026] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0028] Figure 1 A flowchart of a method for visualizing posture and time synchronization according to an embodiment of the present application;

[0029] Figure 2 A schematic diagram of the conversion path between D and F provided according to an embodiment of the present application;

[0030] Figure 3 A schematic diagram of the conversion path between D and J provided according to an embodiment of the present application;

[0031] Figure 4 A coordinate system association tree provided according to an embodiment of the present application;

[0032] Figure 5 A diagram illustrating the structure of a visualization system for posture and time synchronization according to an embodiment of the present application;

[0033] Figure 6 This is an example diagram of a device for visualizing posture and time synchronization according to an embodiment of the present application;

[0034] Figure 7 Schematic diagram of the structure of a vehicle according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0036] The following describes the posture and time synchronization visualization method, device, vehicle and storage medium of the embodiment of the present application with reference to the accompanying drawings. In response to the problem mentioned in the above background technology that the data in the visualization process of vehicle status information is not saved during the visualization process, which causes data waste, and refers to the problem that it is applicable to vehicles and has a single purpose, the present application provides a posture and time synchronization visualization method. In this method, the coordinate system and time are synchronized by constructing a tree structure and posture interpolation. The visualization of data is not limited to vehicles, and posture and time synchronization between multiple different sensors is supported. In addition, all intermediate variables are serialized and saved to disk during the visualization process to avoid data waste. As a result, the problem of data waste caused by not saving data during the visualization process in the related technology is solved, and it is only applicable to the visualization of vehicle information, the visualization data is single, and the user's visualization experience is poor is solved.

[0037] Specifically, Figure 1 A flowchart of a method for visualizing posture and time synchronization provided in an embodiment of the present application.

[0038] like Figure 1 As shown, the visualization method for posture time synchronization includes the following steps:

[0039] In step S101 , raw data collected by any sensor or device in the vehicle is obtained.

[0040] Among them, the original data can come from point cloud data of various sensors, vehicle driving trajectory, etc.

[0041] In step S102, the coordinate conversion path between the current coordinate system and the target coordinate system is matched according to the current coordinate system of any sensor or device and the target coordinate system where the visualization operation is located, and the preset posture interpolation is inserted into the coordinate conversion path at a preset time to obtain a preset conversion relationship between the current coordinate system and the target coordinate system.

[0042] The target coordinate system refers to a fixed coordinate system set in the main rendering window during visualization. Preset pose interpolation refers to the interpolation used to interpolate the coordinate transformation path to obtain the transformation relationship between the current coordinate system and the target coordinate system. Interpolation algorithms can include rotation vector interpolation, current interpolation, spherical linear interpolation, and others.

[0043] It is understandable that the data perceived by sensors comes from different coordinate systems, such as the world frame, vehicle frame, camera frame, and lidar frame. The data perceived by sensors needs to be converted to a fixed target coordinate system before correct visualization.

[0044] For example, the target coordinate system is set to A and the current coordinate system is B. It is necessary to obtain the transformation path from the B coordinate system to the A coordinate system, insert the pose interpolation into the coordinate transformation path from B to A at the preset time, and obtain the transformation relationship between B and A.

[0045] In an embodiment of the present application, the coordinate conversion path between the current coordinate system and the target coordinate system is matched according to the current coordinate system of any sensor or device and the target coordinate system where the visualization operation is located, including: using the current coordinate system of any sensor or device and the target coordinate system where the visualization operation is located as indexes, querying a pre-built tree structure database to obtain the coordinate conversion path between the current coordinate system and the target coordinate system.

[0046] It can be understood that the coordinate conversion path can be obtained in the tree structure database based on the current coordinate system and the target coordinate system. Therefore, the embodiment of the present application can quickly and accurately determine the coordinate conversion path by querying the pre-built tree structure database.

[0047] In an embodiment of the present application, each coordinate system in the tree structure in the tree structure database is a node, the upper layer node is the parent node, and the lower layer node is the child node. The pre-built tree structure database is queried to obtain the coordinate conversion path between the current coordinate system and the target coordinate system, including: starting from the corresponding node of the current coordinate system where any sensor or device is located, recursively searching all parent nodes until the root node of the tree structure to obtain a first path; starting from the corresponding node of the target coordinate system, recursively searching all parent nodes until the root node of the tree structure to obtain a second path; fusing the first path and the second path to obtain the coordinate conversion path between the current coordinate system and the target coordinate system.

[0048] The first path refers to the path starting from the node corresponding to the current coordinate system, recursively following all parent nodes, and ending at the root node. For example, if the current coordinate system is A and the root node is B, the first path is A->B. The second path refers to the path starting from the node corresponding to the target coordinate system, recursively following all parent nodes, and ending at the root node. For example, if the current target coordinate system is C and the root node is B, the second path is C->B.

[0049] It is understandable that each coordinate system in the tree structure database is a node, and different coordinate systems constitute the tree structure database. The coordinate conversion path between the current coordinate system and the target coordinate system is obtained by integrating the first path and the second path.

[0050] For example, if Figure 2 As shown in the figure, assuming that the original coordinate system is D and the target coordinate system is F, the specific method is explained by obtaining the transformation path from D to F. Each coordinate system is defined as a node, with the previous layer node as the parent node and the next layer node as the child node.

[0051] 1. Starting from D, recursively search all parent nodes until the root node, and get path 1: D->B->A.

[0052] 2. Starting from F, recursively search all parent nodes until the root node, and obtain path 2: F->E->A.

[0053] 3. The overlapping part of path 1 and path 2 is the intersection of the two paths, and the final path 3 is obtained by merging: D->B->A->E->F. If there is no intersection, it means that the conversion failed and the current error message is fed back, such as Figure 3 shown.

[0054] In an embodiment of the present application, a preset posture interpolation is inserted into the coordinate transformation path at a preset moment to obtain a preset transformation relationship between the current coordinate system and the target coordinate system, including: using the preset moment as an index, querying the preset cache area to obtain the first moment and the second moment adjacent to the preset moment, and the first transformation relationship and the second transformation relationship corresponding to the first moment and the second moment of each segment of the coordinate transformation path; using a preset interpolation algorithm to interpolate the first transformation relationship and the second transformation relationship to obtain the transformation relationship of each segment of the path, and generating a preset transformation relationship based on the transformation relationship of all segments of the path.

[0055] The preset cache area refers to an area where input data is stored according to certain rules, including the conversion relationship of each segment of the coordinate conversion path at different times.

[0056] For example, after obtaining the conversion path, based on the provided time t, taking D->B as an example, the two closest times t1 and t2 and the corresponding conversion relationship Tbd are searched in the cache, where t1≤t≤t2.

[0057] When t1 < t < t2 , Tbd_1 and Tbd_2 need to be interpolated to obtain Tbd ​​. Here, a transformation matrix is ​​used to represent rotation and translation. In practice, rotation matrices, unit quaternions, and other rotation expressions can also be used. Interpolation algorithms can also include rotation vector interpolation, linear interpolation, spherical linear interpolation, and other methods, which are not detailed here.

[0058] In an embodiment of the present application, before matching the coordinate conversion path between the current coordinate system and the target coordinate system according to the current coordinate system of any sensor or device and the target coordinate system where the visualization operation is located, it also includes: constructing a preset cache area for storing conversion relationships of different coordinate systems at different times, and updating the data in the preset cache area according to the original data collected by any sensor or device in the vehicle, wherein when the timestamp of the newly added data in the cache area is not within the preset storage time range of the cache area, the newly added data is discarded.

[0059] It can be understood that the preset cache area stores the posture transformation relationship between two coordinate systems at different times, such as A->B, A->D, and Figure 4 As shown, if a coordinate transformation is provided, it is represented by a connecting line in the figure.

[0060] It should be noted that if the timestamp of newly added data differs significantly from the time range of the entire cache, the newly added data will be discarded. In addition, to prevent the cache from becoming too large, useless data will be cleared according to certain rules, such as saving data within a preset time period, such as 5 or 6 minutes, and saving data within a preset distance traveled by the vehicle, such as 500 or 600 meters.

[0061] In step S103, all original data are converted to the target coordinate system using a preset conversion relationship to obtain at least one conversion data synchronized in time and posture, the at least one conversion data is rendered into visual data, and the visual data is displayed on a preset display terminal of the vehicle.

[0062] It can be understood that all raw data are transferred to the target coordinate system through a preset transformation relationship, and the converted data is obtained and visually rendered, which can achieve the synchronization of time and coordinate systems of different types of data, thereby synchronizing the coordinate system and time by constructing a tree structure and pose interpolation, accelerating the development of intelligent driving algorithms and problem location, and effectively solving the problem of difficulty in data visualization caused by different coordinate systems of different sensors or devices.

[0063] Specifically, the embodiments of the present application can be Figure 5 The data receiving module, visualization module and data storage shown in the figure realize the visualization method of posture time synchronization, as follows:

[0064] The data receiving module receives data that requires visualization, such as point cloud data from various sensors and vehicle trajectories. It utilizes mainstream distributed real-time communication frameworks such as DDS and ZeroMQ to achieve high-performance network communication. Furthermore, this module provides validation of received data, such as verifying data integrity by ensuring that all data contains the coordinate system in which it resides and the pose transformation relationship between that coordinate system and the associated coordinate system; and verifying data validity, such as ensuring that the quaternion representing the rotation is a unit quaternion and that the data frequency meets requirements.

[0065] During the visualization process, all intermediate variables are serialized and saved to disk. Algorithm developers can deserialize the data to reproduce and analyze problematic data, accelerating iterative algorithm development. Numerous serialization methods are available, including Google Protocol Buffers and Boost.Serialization. Each has its own advantages and disadvantages, and the choice is based on specific needs. I will not elaborate on these here.

[0066] It should be noted that the embodiment of the present application serializes and stores the data in the visualization process, which can be further used for algorithm development and test verification. The visualization of data in the embodiment of the present application is not limited to vehicles, and supports posture and time synchronization between a variety of different sensors. The raw data stored in the embodiment of the present application can be further used for hardware-in-the-loop, software-in-the-loop, and model-in-the-loop algorithm development, test verification. The present invention can perform multi-dimensional visualization of the parsed data results, including but not limited to point clouds, labels, drivable areas, lane structures, images, motion curves, positioning information, etc.

[0067] The following is a further description of a method for visualizing posture time synchronization using a specific embodiment, including the following steps:

[0068] 1. Obtain raw data collected by any sensor or device in the vehicle;

[0069] 2. Visualize the data;

[0070] Different data, due to their different sources, are scattered in different coordinate systems, such as the earth coordinate system, vehicle coordinate system, camera coordinate system, and lidar coordinate system. During the visualization process, the main rendering window will set a fixed coordinate system. Let's assume that the main window's current coordinate system is A. At this time, the data perceived by other sensors are in other coordinate systems such as B, C, and D. This data needs to be converted to coordinate system A for correct visualization. The entire conversion process requires the following steps:

[0071] 2.1 Construct and update the cache of different coordinate system transformation relationships at different times in real time

[0072] First, input data will provide the pose conversion relationship between two coordinate systems at different times, such as: A->B, A->D, as shown in the figure Figure 4 As shown, if the coordinate transformation is provided, it is represented by a connecting line in the figure. The main responsibility of this module is to maintain a data buffer and store the input data according to certain rules.

[0073] At the same time, if the timestamp of the newly added data differs too much from the time range of the entire cache, consider discarding the current cache data. To prevent the cache from becoming too large, this module will purge useless data according to certain rules, for example, only saving data within the last 5 minutes or only saving data for the last 500 meters of vehicle travel.

[0074] 2.2 Obtain the coordinate transformation path between the target coordinate system and the original coordinate system

[0075] like Figure 2 As shown in the figure, assuming that the original coordinate system is D and the target coordinate system is F, the specific method is explained by obtaining the transformation path from D to F. Each coordinate system is defined as a node, with the previous layer node as the parent node and the next layer node as the child node.

[0076] 1) Starting from D, recursively search all parent nodes until the root node, and obtain path 1: D->B->A.

[0077] 2) Starting from F, recursively search all parent nodes until the root node, and obtain path 2: F->E->A.

[0078] 3) The overlapping part of path 1 and path 2 is the intersection of the two paths, and the final path 3 is obtained by merging: D->B->A->E->F. If there is no intersection, it means that the conversion failed and the current error message is fed back, such as Figure 3 shown.

[0079] 2.3 Moment-based pose interpolation

[0080] After obtaining the conversion path, based on the provided time t, taking D->B as an example, the two closest times t1 and t2 and the corresponding conversion relationship Tbd are searched in the cache, where t1≤t≤t2.

[0081] When t1<t<t2, it is necessary to interpolate Tbd_1 and Tbd_2 to obtain Tbd.

[0082] 2.4 Get the converted data and render it

[0083] In step 3, we can obtain Tdb, Tba, Tae, and Tef, so the conversion relationship from D to F is: Tfd = inverse(Tdb * Tba * Tae * Tef). All points Pdn in the D coordinate system can be converted to points Fdn in the F coordinate system as Fdn = Tfd * Pdn.

[0084] Finally, all the data are rendered in 3D visualization and a user interaction interface is provided.

[0085] 3. Data storage.

[0086] According to the posture and time synchronization visualization method proposed in the embodiment of the present application, different types of data can be synchronized to the same time and the same coordinate system, and visualization of the time and posture synchronization of different types of data can be realized, so that the coordinate system and time are synchronized by constructing a tree structure and posture interpolation, effectively solving the problem of difficulty in data visualization caused by different coordinate systems of different sensors or devices, facilitating safety personnel to monitor vehicle status and improve safety during autonomous driving; the required coordinate conversion path can be accurately indexed through a pre-constructed tree structure database; the coordinate conversion path can be obtained recursively through the tree structure database, which can facilitate the preservation of data in the visualization process and avoid data waste; the conversion relationship of all paths is obtained through a preset interpolation algorithm to synchronize the coordinate system and time, accelerating the development of intelligent driving algorithms; by constructing a preset cache area to store the posture conversion relationship between different coordinate systems and at different times and to clear useless data, all intermediate variables can be saved, thus avoiding data waste and facilitating staff to reproduce problems.

[0087] Next, a posture time-synchronized visualization device proposed in accordance with an embodiment of the present application will be described with reference to the accompanying drawings.

[0088] Figure 6 It is a block diagram of a visualization device for posture time synchronization according to an embodiment of the present application.

[0089] like Figure 6 As shown, the posture time-synchronized visualization device 10 includes: an acquisition module 100 , a processing module 200 and a visualization module 300 .

[0090] Among them, the acquisition module 100 is used to acquire the original data collected by any sensor or device in the vehicle; the processing module 200 is used to match the coordinate conversion path between the current coordinate system and the target coordinate system according to the current coordinate system where any sensor or device is located and the target coordinate system where the visualization operation is located, and insert the preset posture interpolation into the coordinate conversion path at a preset time to obtain a preset conversion relationship between the current coordinate system and the target coordinate system; the visualization module 300 is used to use the preset conversion relationship to convert all original data to the target coordinate system, obtain at least one conversion data synchronized with time and posture, render at least one conversion data into visualization data, and display the visualization data on the preset display terminal of the vehicle.

[0091] In an embodiment of the present application, the processing module 200 is further used to: use the current coordinate system of any sensor or device and the target coordinate system of the visualization operation as indexes to query a pre-built tree structure database to obtain a coordinate conversion path between the current coordinate system and the target coordinate system.

[0092] In an embodiment of the present application, each coordinate system in the tree structure in the tree structure database is a node, the upper layer node is the parent node, and the lower layer node is the child node. The processing module is further used to: start from the corresponding node of the current coordinate system where any sensor or device is located, recursively search all parent nodes until the root node of the tree structure to obtain a first path; start from the corresponding node of the target coordinate system, recursively search all parent nodes until the root node of the tree structure to obtain a second path; fuse the first path and the second path to obtain a coordinate conversion path between the current coordinate system and the target coordinate system.

[0093] It should be noted that the above explanation of the embodiment of the visualization method for posture time synchronization is also applicable to the visualization device for posture time synchronization of this embodiment, and will not be repeated here.

[0094] According to the posture and time synchronization visualization device proposed in the embodiment of the present application, different types of data can be synchronized to the same time and the same coordinate system, and visualization of time and posture synchronization of different types of data can be realized, thereby synchronizing the coordinate system and time by constructing a tree structure and posture interpolation, effectively solving the problem of difficulty in data visualization caused by different coordinate systems of different sensors or devices, facilitating safety personnel to monitor vehicle status and improve safety during autonomous driving; the required coordinate conversion path can be accurately indexed through a pre-constructed tree structure database; the coordinate conversion path can be obtained recursively through the tree structure database, which can facilitate the preservation of data in the visualization process and avoid data waste; the conversion relationship of all paths is obtained through a preset interpolation algorithm to synchronize the coordinate system and time, accelerating the development of intelligent driving algorithms; by constructing a preset cache area to store the posture conversion relationship between different coordinate systems and at different times and to clear useless data, all intermediate variables can be saved, thus avoiding data waste and facilitating staff to reproduce problems.

[0095] Figure 7 A schematic diagram of the structure of a vehicle provided in an embodiment of the present application. The vehicle may include:

[0096] Memory 701 , processor 702 , and computer programs stored in the memory 701 and executable on the processor 702 .

[0097] When the processor 702 executes the program, the visualization method for posture time synchronization provided in the above embodiment is implemented.

[0098] Furthermore, the vehicle further comprises:

[0099] The communication interface 703 is used for communication between the memory 701 and the processor 702 .

[0100] The memory 701 is used to store computer programs that can be run on the processor 702 .

[0101] The memory 701 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0102] If the memory 701, processor 702, and communication interface 703 are implemented independently, the communication interface 703, memory 701, and processor 702 can be connected to each other via a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0103] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0104] The processor 702 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0105] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned posture and time synchronization visualization method.

[0106] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0107] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0108] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0109] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0110] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0111] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for visualization of posture time synchronization, characterized in that: The following steps are involved: Obtain raw data collected by any sensor or device in the vehicle; Matching a coordinate transformation path between the current coordinate system and the target coordinate system according to the current coordinate system of the arbitrary sensor or device and the target coordinate system of the visualization operation, and inserting a preset pose interpolation into the coordinate transformation path at a preset time to obtain a preset transformation relationship between the current coordinate system and the target coordinate system; Converting all raw data into the target coordinate system using the preset conversion relationship to obtain at least one conversion data synchronized in time and posture, rendering the at least one conversion data into visual data, and displaying the visual data on a preset display terminal of the vehicle; The interpolating the preset posture into the coordinate transformation path at a preset time to obtain a preset transformation relationship between the current coordinate system and the target coordinate system includes: Using the preset time as an index, querying a preset cache area to obtain a first time and a second time adjacent to the preset time, and a first conversion relationship and a second conversion relationship corresponding to each segment of the coordinate conversion path at the first time and the second time; interpolating the first transformation relationship and the second transformation relationship using a preset interpolation algorithm to obtain the transformation relationship of each path segment, and generating the preset transformation relationship based on the transformation relationships of all path segments, before matching the coordinate transformation path between the current coordinate system and the target coordinate system according to the current coordinate system of any sensor or device and the target coordinate system of the visualization operation, further comprising: Construct a preset cache area for storing different moments and different coordinate system conversion relationships, and update the data in the preset cache area based on the original data collected by any sensor or device in the vehicle, wherein when the timestamp of the newly added data in the cache area is not within the preset storage time range of the cache area, the newly added data is discarded.

2. The method according to claim 1, characterized in that The matching of a coordinate conversion path between the current coordinate system and the target coordinate system according to the current coordinate system of the arbitrary sensor or device and the target coordinate system of the visualization operation includes: The current coordinate system of the arbitrary sensor or device and the target coordinate system of the visualization operation are used as indexes to query a pre-built tree structure database to obtain a coordinate conversion path between the current coordinate system and the target coordinate system.

3. The method according to claim 2, characterized in that Each coordinate system in the tree structure in the tree structure database is a node, the upper layer node is a parent node, and the lower layer node is a child node. The querying of the pre-built tree structure database to obtain the coordinate conversion path between the current coordinate system and the target coordinate system includes: Starting from the node corresponding to the current coordinate system where the arbitrary sensor or device is located, recursively search all parent nodes until the root node of the tree structure to obtain a first path; Starting from the node corresponding to the target coordinate system, recursively search all parent nodes until the root node of the tree structure to obtain a second path; The first path and the second path are fused to obtain a coordinate transformation path between the current coordinate system and the target coordinate system.

4. A posture and time synchronization visualization device, characterized in that: include: The acquisition module is used to obtain the raw data collected by any sensor or device in the vehicle; a processing module, configured to match a coordinate transformation path between the current coordinate system and the target coordinate system according to the current coordinate system of the arbitrary sensor or device and the target coordinate system of the visualization operation, and insert a preset pose interpolation into the coordinate transformation path at a preset time to obtain a preset transformation relationship between the current coordinate system and the target coordinate system; a visualization module, configured to transform all raw data into the target coordinate system using the preset transformation relationship to obtain at least one transformed data synchronized in time and posture, render the at least one transformed data into visualization data, and display the visualization data on a preset display terminal of the vehicle; The interpolating the preset posture into the coordinate transformation path at a preset time to obtain a preset transformation relationship between the current coordinate system and the target coordinate system includes: Using the preset time as an index, querying a preset cache area to obtain a first time and a second time adjacent to the preset time, and a first conversion relationship and a second conversion relationship corresponding to each segment of the coordinate conversion path at the first time and the second time; interpolating the first transformation relationship and the second transformation relationship using a preset interpolation algorithm to obtain the transformation relationship of each path segment, and generating the preset transformation relationship based on the transformation relationships of all path segments, before matching the coordinate transformation path between the current coordinate system and the target coordinate system according to the current coordinate system of any sensor or device and the target coordinate system of the visualization operation, further comprising: Construct a preset cache area for storing different moments and different coordinate system conversion relationships, and update the data in the preset cache area based on the original data collected by any sensor or device in the vehicle, wherein when the timestamp of the newly added data in the cache area is not within the preset storage time range of the cache area, the newly added data is discarded.

5. The device according to claim 4, characterized in that The processing module is further configured to: The current coordinate system of the arbitrary sensor or device and the target coordinate system of the visualization operation are used as indexes to query a pre-built tree structure database to obtain a coordinate conversion path between the current coordinate system and the target coordinate system.

6. The device according to claim 5, characterized in that Each coordinate system in the tree structure in the tree structure database is a node, the upper layer node is the parent node, and the lower layer node is the child node. The processing module is further used to: Starting from the node corresponding to the current coordinate system where the arbitrary sensor or device is located, recursively search all parent nodes until the root node of the tree structure to obtain a first path; Starting from the node corresponding to the target coordinate system, recursively search all parent nodes until the root node of the tree structure to obtain a second path; The first path and the second path are fused to obtain a coordinate transformation path between the current coordinate system and the target coordinate system.

7. A vehicle, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the posture time synchronization visualization method according to any one of claims 1 to 3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the posture time synchronization visualization method as described in any one of claims 1 to 3.

Images