A data processing method, apparatus, and storage medium

Abstract

The present application provides a data processing method, apparatus, and storage medium. The data processing method includes obtaining acquisition data packets sent by multiple data acquisition nodes; the acquisition data packets include target acquisition data and a time offset of the target acquisition data; the time offset is used to represent the time offset information of the acquisition time of the target acquisition data relative to a target time base; determining, as the acquisition data at the target moment, the target acquisition data sent by multiple data acquisition nodes that have the same time offset; and performing data fusion on the acquisition data at the target moment to obtain object state information corresponding to the target moment. By determining the acquisition data at the target moment from the target acquisition data based on the target time base and the same time offset information, and performing fusion processing on the target acquisition data generated at the same moment, the present application can avoid the influence of the time delay in the data unpacking process during data synchronization on data fusion analysis, and obtain a more accurate scenario analysis result.

Description

Technical Field

[0001] The present invention relates to the technical field of time synchronization, and in particular, to a data processing method, apparatus, and storage medium. Background Art

[0002] In the architecture of automotive distributed electronics and electricals, data generation and consumption are often distributed in different ECUs (Electronic Control Units) or even different vehicle body domains. The delay from the data generation end to the consumption end caused by bus transmission can no longer meet the increasingly high data transmission requirements. Due to its strong real-time performance, low cost, and far transmission capacity, the CAN bus is still the most widely used one in automotive buses.

[0003] In the classical automotive open system architecture (CP AUTOSAR), an architecture for CAN network time synchronization is proposed. A node with accurate timing is used as the master node, and messages with global time are sent in the CAN network to the remaining nodes for global time synchronization. This design can solve the problem of inconsistent time bases of different nodes in the CAN network topology. However, although this time synchronization architecture can synchronize the time base, there is still a time delay in the data unpacking process during the data synchronization process. Therefore, the acquired acquisition data under this time synchronization architecture contains this part of the time delay, and the data fusion analysis based on the acquisition data cannot accurately obtain the scenario analysis result at the moment when the acquisition data is generated, thereby affecting the reliability of the scenario analysis result. Summary of the Invention

[0004] In view of this, the present application proposes a data processing method, apparatus, and storage medium. Based on the time synchronization architecture of the CAN network in the original automotive open system architecture (CP AUTOSAR), the data generation time of the data acquisition node is obtained, and data fusion analysis is performed based on the data generation time. Therefore, at least the problem that the influence of the delay time superposition cannot be removed during the data processing process from the data generation end to the consumption end of the current autonomous driving system can be solved.

[0005] According to one aspect of the present application, a data processing method is provided, and the method includes:

[0006] Obtain acquisition data packets sent by multiple data acquisition nodes; the acquisition data packets include target acquisition data and a time offset of the target acquisition data; the time offset is used to represent the time offset information of the acquisition time of the target acquisition data relative to a target time base;

[0007] Determine the target acquisition data with the same time offset sent by the multiple data acquisition nodes as the acquisition data at the target moment;

[0008] Perform data fusion on the collected data at the target time to obtain object state information corresponding to the target time.

[0009] In a possible implementation, before obtaining the collection data packets sent by multiple data collection nodes, the method further includes:

[0010] Obtain the current time;

[0011] Generate time synchronization information based on the current time, and send the time synchronization information to the multiple data collection nodes; so that the multiple data collection nodes generate the target time base based on the time synchronization information.

[0012] Further, the time synchronization information includes first synchronization information and second synchronization information;

[0013] The generating time synchronization information based on the current time and sending the time synchronization information to the multiple data collection nodes includes:

[0014] Generate the first synchronization information based on the current time;

[0015] Send the first synchronization information to the multiple data collection nodes, and record the first value of the first timer; so that the multiple data collection nodes obtain the current time;

[0016] When receiving the information that the first synchronization information is successfully sent, record the second value of the first timer;

[0017] Determine the target time delay based on the first value and the second value;

[0018] Generate the second synchronization information based on the target time delay;

[0019] Send the second synchronization information to the multiple data collection nodes; so that the multiple data collection nodes generate the target time base based on the current time and the target time delay.

[0020] In a possible implementation, the method further includes:

[0021] When the multiple data collection nodes receive the first synchronization information, start the second timers corresponding to the multiple data collection nodes based on the multiple data collection nodes;

[0022] When the multiple data collection nodes collect the target collection data, determine the time offset of the target collection data based on the third value of the second timer.

[0023] According to another aspect of the present application, a data processing method is provided, and the method includes:

[0024] Generating an acquisition data packet based on the acquired target acquisition data; the acquisition data packet includes the target acquisition data and the time offset of the target acquisition data; the time offset is used to represent the time offset information of the acquisition time of the target acquisition data relative to the target time base;

[0025] Sending the acquisition data packet to a target node, so that the target node determines the target acquisition data with the same time offset as the acquisition data at the target time, and performs data fusion on the acquisition data at the target time to obtain object state information corresponding to the target time.

[0026] In a possible implementation manner, before generating the acquisition data packet based on the acquired data, the method further includes:

[0027] Receiving time synchronization information sent by the target node; the time synchronization information includes the current time obtained by the target node when generating the time synchronization information;

[0028] Generating the target time base based on the current time.

[0029] Further, the time synchronization information includes first synchronization information and second synchronization information;

[0030] The receiving the time synchronization information sent by the target node includes:

[0031] Receiving the first synchronization information sent by the target node;

[0032] Obtaining the current time based on the first synchronization information;

[0033] Receiving the second synchronization information sent by the target node; the second synchronization information includes a target time delay determined based on a first value and a second value; the first value is the value recorded by a first timer when the target node sends the first synchronization information, and the second value is the value recorded by the first timer when the target node successfully sends the information after receiving the first synchronization information;

[0034] The generating the target time base based on the current time includes:

[0035] Generating the target time base based on the current time and the target time delay.

[0036] In a possible implementation manner, the method further includes:

[0037] When receiving the first synchronization information, start a second timer;

[0038] When collecting the target acquisition data, determine the time offset of the target acquisition data based on the third value of the second timer.

[0039] According to another aspect of the present application, there is provided a data processing device, including:

[0040] A data acquisition module, configured to acquire acquisition data packets sent by multiple data acquisition nodes; the acquisition data packets include target acquisition data and the time offset of the target acquisition data; the time offset is used to represent the time offset information of the acquisition time of the target acquisition data relative to a target time base;

[0041] A data alignment module, configured to determine the target acquisition data with the same time offset sent by the multiple data acquisition nodes as the acquisition data at a target moment;

[0042] A data analysis module, configured to perform data fusion on the acquisition data at the target moment to obtain object state information corresponding to the target moment.

[0043] According to another aspect of the present application, there is provided a data processing device, including:

[0044] A data generation module, configured to generate an acquisition data packet based on the acquired target acquisition data; the acquisition data packet includes the target acquisition data and the time offset of the target acquisition data; the time offset is used to represent the time offset information of the acquisition time of the target acquisition data relative to a target time base;

[0045] A data sending module, configured to send the acquisition data packet to a target node, so that the target node determines the target acquisition data with the same time offset as the acquisition data at a target moment, and performs data fusion on the acquisition data at the target moment to obtain object state information corresponding to the target moment.

[0046] According to another aspect of the present application, there is provided a storage medium, in which at least one instruction or at least one program segment is stored, and the at least one instruction or the at least one program segment is loaded and executed by a processor to implement any one of the above feasible data processing methods.

[0047] The data processing method, device and storage medium provided by the present application have the following beneficial effects:

[0048] By obtaining acquisition data packets sent by multiple data acquisition nodes; the acquisition data packet includes target acquisition data and the time offset of the target acquisition data; the time offset is used to represent the time offset information of the acquisition time of the target acquisition data relative to the target time base; determining the target acquisition data with the same time offset sent by multiple data acquisition nodes as the acquisition data at the target moment; performing data fusion on the acquisition data at the target moment to obtain the object state information corresponding to the target moment; it is possible to determine the acquisition data at the target moment from the target acquisition data based on the target time base and the same time offset information, and perform fusion processing on the target acquisition data generated at the same moment, avoiding the influence of the time delay in the data unpacking process during data synchronization on data fusion analysis, and obtaining a more accurate scenario analysis result.

[0049] Other features and aspects of the present application will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the present invention, the accompanying drawings required for use in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0051] Figure 1 It is a schematic diagram of an application environment provided by an embodiment of the present application;

[0052] Figure 2 It is an interaction schematic diagram for time synchronization based on the Autosar CAN time synchronization mechanism;

[0053] Figure 3 It is a schematic flowchart of a data processing method provided by an embodiment of the present application;

[0054] Figure 4 It is a schematic flowchart of another data processing method provided by an embodiment of the present application;

[0055] Figure 5 It is a schematic flowchart of another data processing method provided by an embodiment of the present application;

[0056] Figure 6 It is an interaction schematic diagram of a master data acquisition node synchronizing time with a slave data acquisition node in an example provided by an embodiment of the present application;

[0057] Figure 7 It is a schematic flowchart of yet another data processing method provided by an embodiment of the present application;

[0058] Figure 8 A schematic block diagram of a data processing device provided by an embodiment of the present application;

[0059] Figure 9 A schematic block diagram of another data processing device provided by an embodiment of the present application;

[0060] Figure 10 A schematic block diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0061] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0062] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above accompanying drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment including a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or equipment.

[0063] In addition, in order to better illustrate the present application, numerous specific details are given in the following detailed implementation manners. Those skilled in the art should understand that the present application can still be implemented without some specific details. In some instances, methods, means, elements and circuits well-known to those skilled in the art are not described in detail so as to highlight the gist of the present application.

[0064] The present application provides a data processing method, which can be applied to Figure 1 the application environment shown. As Figure 1As shown, in this application environment, the vehicle 100 can be an autonomous vehicle equipped with an autonomous driving system, and there are also multiple data acquisition nodes formed by a plurality of sensors on the vehicle. For the sake of simplicity of description, the first data acquisition node 110 and the second data acquisition node 120 are exemplified in this application environment. However, in actual applications, the number of data acquisition nodes on the vehicle 100 is not limited, and more data acquisition nodes can be set as needed. For example, the multiple data acquisition nodes include a first data acquisition node 110 and at least one second data acquisition node 120.

[0065] In one embodiment, the data acquisition node can include multiple sensors for assisting the autonomous driving function, such as camera devices (including monocular, binocular stereo vision, panoramic vision, and infrared cameras) and ranging sensors (including radar, lidar, etc.).

[0066] In the embodiments of the present application, each sensor is used as a data acquisition node of the perception system of the autonomous vehicle, and each data acquisition node can correspond to an ECU control unit of a sensor. The multiple data acquisition nodes communicate through the CAN network in the Autosar architecture to transmit signals and acquisition data packets. However, there will be a certain time delay from the generation of data by each ECU of the CAN network node to the broadcast of the data to the bus. Without data time synchronization, the CAN network node often defaults the received data as immediate data when processing the received data, that is, ignores the delay from data generation to data consumption. To ensure the reliability of the data of the data acquisition nodes used in the autonomous driving function, it is usually required to synchronize the time of multiple data acquisition nodes and start the data acquisition of the data acquisition nodes after time synchronization.

[0067] As a feasible implementation manner, the Autosar CAN time synchronization mechanism provides a method for synchronizing the time of different data acquisition nodes on the CAN network. According to the Autosar CAN time synchronization mechanism, there is a master data acquisition node in the CAN network that sends time synchronization information in a broadcast form. When each slave data acquisition node receives the corresponding time synchronization information, it will synchronize its own global clock to maintain the consistency of the global clock.

[0068] Under the Autosar CAN time synchronization mechanism architecture, the global clock of the master data acquisition node serves as the clock source of the master data acquisition node, and the global clocks of each slave data acquisition node serve as their respective clock sources. The master data acquisition node and the slave data acquisition nodes perform time synchronization by transmitting time synchronization messages via the CAN network. Thus, the master data acquisition node can perform data fusion processing on the data collected by the slave data acquisition nodes after synchronization through a preset fusion analysis algorithm to obtain the state information of the vehicle driving state and / or other objects in the vehicle driving environment.

[0069] Specifically, please refer to Figure 2 , an example of a time synchronization process based on the Autosar CAN time synchronization mechanism is provided in an embodiment of the present application, and the specific description is as follows:

[0070] In an embodiment of the present application, the first data acquisition node 100 in the above application environment is used as the master data acquisition node, and the second data acquisition node 120 is used as the slave data acquisition node. When a time synchronization event occurs and triggers the master data acquisition node to synchronize the time of the slave data acquisition node, the master data acquisition node attaches the global time Tm_Base in seconds to the SYNC message, and attaches the delay time Tm_Delay in nanoseconds to the subsequent FUP (Follow-up) message. After receiving the SYNC message sent by the master data acquisition node, the slave data acquisition node unpacks it to obtain Tm_Base, and after receiving the FUP message sent by the master data acquisition node, it unpacks it to obtain Tm_Delay. During the time synchronization process, the slave data acquisition node also calculates the difference between the first timer at the moment of processing synchronization and the moment of receiving the SYNC message (Ts_Counter - Ts_CounterBase), converts the difference into a delay time, and obtains the delay time Ts_Delay of the slave data acquisition node during the time synchronization process. Through the time information parsed from the SYNC and FUP messages, and the value recorded by the first timer of the slave data acquisition node, after the time synchronization is completed, the global time Ts of the slave data acquisition node after time synchronization can be calculated on the time axis of the slave data acquisition node (Ts = Tm_Base + Tm_Delay + Ts_Delay).

[0071] After the time synchronization is completed, the slave data acquisition node packs the collected target acquisition data with the global time to obtain an acquisition data packet with global time information and sends it to the master data acquisition node for data fusion analysis to obtain a scenario analysis result for the driving decision of the autonomous driving system.

[0072] It should be noted that the generation and sending of the acquisition data packet are not limited to once and can include multiple times. After generating the acquisition data packet, the slave data acquisition node sequentially sends the acquisition data packet to the master data acquisition node for data fusion analysis.

[0073] In the above data processing process based on the Autosar CAN time synchronization mechanism, the acquisition data packets are generated by the data acquisition node based on the synchronized global time, and it is impossible to avoid the influence of the delay time between the moment when the data acquisition node calculates and processes the synchronization and the moment when the SYNC message is received. The delay time Ts_Delay is superimposed on each acquisition data packet generated by the data acquisition node. Therefore, during data fusion processing, a scenario analysis result consistent with the data acquisition time cannot be obtained, which to a certain extent affects the reliability of the data fusion analysis result of the autonomous driving system, and further limits the decision-making ability of the autonomous driving system.

[0074] To solve the above problems, in the implementation of this application, a second timer is added to the master data acquisition node and the slave data acquisition nodes, and an attempt is made to synchronize the time when data is generated in the slave data acquisition nodes to the time axis of the master data acquisition node. Thus, the master data acquisition node obtains the time information of the acquisition data packets generated by the slave data acquisition nodes for data fusion analysis of the vehicle autonomous driving system, so as to obtain a scenario analysis result that can conform to the data generation time and improve the decision-making ability of the autonomous driving system.

[0075] In a specific embodiment, after the vehicle 100 is powered on and communicates and the network is activated, the master data acquisition node and all slave data acquisition nodes, as network nodes for CAN network time synchronization, initialize their respective global clocks, first timers, and second timers. Subsequently, the global clocks and first timers of the master data acquisition node and all slave data acquisition nodes are activated, and then the data processing process is started. The following provides corresponding embodiments of the data processing method, device, and storage medium of this application embodiment.

[0076] Please refer to Figure 3 , Figure 3 FIG. shows a schematic flowchart of a data processing method according to an embodiment of this application. The implementation subject of this data processing method is the above-mentioned master data acquisition node. This data processing method includes:

[0077] S210, obtaining acquisition data packets sent by multiple data acquisition nodes; the acquisition data packets include target acquisition data and the time offset of the target acquisition data; the time offset is used to represent the time offset information of the acquisition time of the target acquisition data relative to the target time base;

[0078] In this embodiment, the data acquisition node can be the above-mentioned slave data acquisition node 120 or the above-mentioned master data acquisition node 110, and specifically can include but is not limited to radar sensors, camera devices, vehicle speed detection sensors, collision detection sensors, etc. For example, in an autonomous driving scenario, ultrasonic radar sensors, lidar sensors, and cameras are used as multiple data acquisition nodes to collect vehicle driving environment data based on a set period and generate acquisition data packets.

[0079] In particular, in the present application, by reasonably setting the target time base, it is possible to make the slave data acquisition node obtain the same time reference as the master data acquisition node. That is, on the basis that the master data acquisition node and the slave data acquisition node have the same target time base, by recording the time offset information of the acquisition time of the acquired data relative to the target time base, the time offset amount information of the acquisition time of the target acquired data of the slave data acquisition node relative to the target time base can be converted into the time offset amount between the target acquired data and the target time base on the time axis of the master data acquisition node.

[0080] In the embodiment of the present application, the acquisition data packet acquired by the master data acquisition node includes the target acquired data, and also includes the time offset amount information of the acquisition time of the target acquired data relative to the target time base. For the slave data acquisition node, the slave data acquisition node acquires the target acquired data based on the sensor, and obtains the time offset information through the second timer, packs the target acquired data and the time offset information into an acquisition data packet, and sends it to the master data acquisition node. The acquisition data packet acquired by the master data acquisition node may also include the target acquired data acquired based on the sensor of its own node.

[0081] S220. Determine the acquired data at the target moment from the target acquired data sent by multiple data acquisition nodes with the same time offset amount;

[0082] The target moment is obtained by superimposing the time offset amount on the basis of the target time base, and each data acquisition node has the same target time base.

[0083] S230. Perform data fusion on the acquired data at the target moment to obtain the object state information corresponding to the target moment.

[0084] In the embodiment of the present application, the object state information may include, but is not limited to, vehicle driving state information, state information of obstacles or other vehicles in the driving environment.

[0085] The data processing method in the embodiment of the present application can obtain the acquisition moment of the target acquired data based on its own time base by setting the target time base and obtaining the time offset information in the acquisition data packet. This acquisition moment is not affected by the time deviation existing in the data processing process of the slave data acquisition node, and can truly correspond to the acquisition time of the target acquired data. Therefore, by performing data fusion on the acquired data at the target moment, a scene analysis result that conforms to the data generation time is obtained, and more accurate object state information is obtained, thereby improving the decision-making ability of the vehicle automatic driving system.

[0086] In one embodiment, referring to Figure 4 , before step S210 of the above data processing method, it further includes:

[0087] S110, Obtain the current time;

[0088] The current time is the global time obtained by the master data acquisition node based on its own global clock. This global time can be directly obtained from the display time based on its own global clock, or indirectly obtained after arithmetic operations or processing on the display time based on its own global clock.

[0089] S120, Generate time synchronization information based on the current time, and send the time synchronization information to multiple data acquisition nodes; so that the multiple data acquisition nodes generate a target time base based on the time synchronization information.

[0090] In the embodiment of the present application, the master data acquisition node generates the time synchronization information based on its own global time.

[0091] For the slave data acquisition node, receive the time synchronization information sent by the master data acquisition node, obtain the global time information of the master data acquisition node, and generate its own target time base based on the global time in the time synchronization information.

[0092] In this way, by sending the time synchronization information from the master data acquisition node to the slave data acquisition node, the slave data acquisition node can generate its own target time base based on the global time of the master data acquisition node carried in the synchronization information. In particular, the master data acquisition node can obtain its own target time base based on its own global clock. Thus, according to the data processing method of the present application, during the process of the master data acquisition node obtaining the acquisition data packet, the master data acquisition node can obtain the target acquisition data generated at the same moment based on its own target time base and the time offset information of the target acquisition data.

[0093] In a specific embodiment, the time synchronization information includes first synchronization information and second synchronization information;

[0094] Refer to Figure 5 and Figure 6 , the above step S120 may include:

[0095] S1210, Generate first synchronization information based on the current time;

[0096] Preferably, adopting the design specification for time synchronization with each data acquisition node in the CAN network in the Autosar CAN time synchronization mechanism, the master data acquisition node generates the first synchronization information in the same way as generating the SYNC message. The generation of the first synchronization message is triggered and started based on a preset time synchronization event. The preset event synchronization event can be a fixed time period or a preset time deviation. When the time setting is met, the master data acquisition node is triggered to generate the first synchronization message.

[0097] S1220. Send the first synchronization message to multiple data acquisition nodes and record the first value of the first timer, so that the multiple data acquisition nodes can obtain the current time.

[0098] In the embodiments of the present application, when a time synchronization event is triggered and the master data acquisition node attempts to send the first synchronization message, the master data acquisition node will store the global time Tm_Base corresponding to the current moment in seconds and the first value Tm_CounterBase of the first timer according to its global clock, encapsulate Tm_Base into the first synchronization message, and trigger the sending of the first synchronization message.

[0099] For the slave data acquisition nodes, when receiving the first synchronization message, they can unpack the first synchronization message to obtain the global time Tm_Base corresponding to the time when the master data acquisition node sends the first synchronization message.

[0100] S1230. When receiving the information that the first synchronization message has been successfully sent, record the second value of the first timer.

[0101] When it is confirmed that the CAN network has successfully sent the first synchronization message, at this time, the master data acquisition node will store the second value Tm_CounterTx of the current moment according to the value of its first timer for calculating the target delay.

[0102] It should be noted that when it is confirmed that the CAN network has successfully sent the first synchronization message, the master data acquisition node also obtains its own global time and records it as the target time base of its own node.

[0103] S1240. Determine the target delay based on the first value and the second value.

[0104] The target delay Tm_Delay can be calculated by formula (1):

[0105] Tm_Dealy = Tm_Counter Tx - Tm_CounterBase (1)

[0106] Where, Tm_Delay is the target delay;

[0107] Tm_CounterTx is the second value of the first timer;

[0108] Tm_CounterBase is the first value of the first timer.

[0109] S1250. Generate the second synchronization message based on the target delay.

[0110] When the target is delayed, the process of sending the second synchronization message is triggered. The second synchronization message encapsulates the target delay Tm_Delay in nanoseconds, which is used to notify the slave data acquisition nodes of the target delay of the first synchronization message transmitted on the CAN bus.

[0111] S1260, send the second synchronization information to multiple data acquisition nodes; so that multiple data acquisition nodes generate a target time base based on the current time and the target delay.

[0112] After the master data acquisition node sends the second synchronization message to the data acquisition nodes, for the slave data acquisition nodes, when receiving the second synchronization message, the second synchronization message can be unpacked to obtain the target delay Tm_Delay of the first synchronization message transmitted in the CAN network when the master data acquisition node sends the first synchronization message, and generate the target time base T0 of the slave data acquisition node itself. The target time base T0 can be calculated by the formula (2):

[0113] T0 = Tm_Base + Tm_Delay (2)

[0114] Wherein, T0 is the target time base T0 of the slave data acquisition node itself;

[0115] Tm_Base is the current time corresponding to when the master data acquisition node sends the first synchronization message;

[0116] Tm_Delay is the target delay of the first synchronization message transmitted on the CAN bus.

[0117] In one embodiment, the data processing method may further include:

[0118] S101, when multiple data acquisition nodes receive the first synchronization information, start the second timers corresponding to the multiple data acquisition nodes based on the multiple data acquisition nodes;

[0119] In the embodiment of the present application, a second timer is set in each data acquisition node to obtain the time offset T_offset of the target acquisition data relative to the target time base.

[0120] Preferably, when starting the second timer, the count value of the second timer is cleared at the same time.

[0121] S103, when multiple data acquisition nodes acquire the target acquisition data, determine the time offset of the target acquisition data based on the third value of the second timer.

[0122] In step S101, when starting the second timer and clearing the count value of the second timer at the same time, the time offset T_offset is the time interval directly converted from the third value of the second timer.

[0123] In this way, based on the count value of the second timer, the time offset of the target acquisition data relative to the target time base can be obtained in each data acquisition node.

[0124] In particular, the master data acquisition node in the embodiment of the present application synchronizes data with the slave data acquisition nodes, achieving the alignment of the target time bases T0 of different data acquisition nodes. Only by capturing the time offset of the target acquisition data relative to the target time base through the second timer can the data generation time be obtained on the time axes of each data acquisition node.

[0125] The embodiment of the present application further provides a data processing method, and the implementation subject of this method is any slave data acquisition node. For example, it can run in the above ultrasonic radar sensor and lidar sensor. The ultrasonic radar sensor and lidar sensor act as slave data acquisition nodes, generate acquisition data packets based on the set logic for collecting vehicle driving environment data, and send them to the data processing module for data fusion analysis.

[0126] Please refer to Figure 7 , Figure 7 , which shows a schematic flowchart of a data processing method, and realizes the interaction with the target node through one end of the slave data acquisition node. In the embodiment of the present application, the target node is set as the master data acquisition node for implementing the above method. As Figure 7 shown, this data processing method includes:

[0127] S610, generating an acquisition data packet based on the acquired target acquisition data; the acquisition data packet includes the target acquisition data and the time offset of the target acquisition data; the time offset is used to represent the time offset information of the acquisition time of the target acquisition data relative to the target time base;

[0128] The target acquisition data can be the detection data obtained from the sensors of the slave data acquisition node, and the time offset is the time offset information of the acquisition time of the target acquisition data recorded by the slave data acquisition node relative to the target time base. By reasonably setting the target time base, for example, aligning the target time base with the target time base in the above method embodiment, the slave data acquisition node records the time offset information of the acquisition time of the acquisition data relative to the target time base, and the time offset of the acquisition time of the target acquisition data on its own time axis relative to the target time base can be obtained. Since the target time bases of the master data acquisition node and the slave data acquisition nodes are aligned, for the master data acquisition node, through the time offset information recorded in the acquisition data, the time offset can be obtained, and the target acquisition data at the same moment can be determined accordingly.

[0129] In one embodiment, it may be to set a corresponding data address in the CAN data segment, fill the time offset information into the corresponding data address, and obtain a collection data packet including the target collection data and the time offset information.

[0130] S620. Send the collection data packet to the target node, so that the target node determines the target collection data with the same time offset as the target moment's collection data, and performs data fusion on the target moment's collection data to obtain the object state information corresponding to the target moment.

[0131] The data processing method of the embodiment of the present application packs the target collection data collected from the data collection node and its corresponding time offset and sends them to the main data collection node. The main data collection node can determine the target moment's collection data according to the time offset, and obtain the scene analysis result that conforms to the data generation time after data fusion processing, so as to obtain more accurate object state information, and further improve the decision-making ability of the vehicle automatic driving system.

[0132] In one implementation, before step S610, the data processing method further includes:

[0133] S601. Receive the time synchronization information sent by the target node; the time synchronization information includes the current time obtained when the target node generates the time synchronization information.

[0134] The main data collection node, as the target node, generates the time synchronization information based on the current time of its own global clock, and sends the time synchronization information to the slave data collection node. The current time of this global clock can be directly obtained based on the display time of its own global clock, or indirectly obtained through operations or processing on the display time of its own global clock.

[0135] S603. Generate a target time base based on the current time.

[0136] Based on the received time synchronization information, the slave data collection node can unpack and obtain the current time obtained by the main data collection node carried in the time synchronization information, so that it can generate its own target time base based on the current time of the main data collection node, and realize the alignment of the target time bases of the main data collection node and the slave data collection node.

[0137] In one embodiment, the above time synchronization information includes first synchronization information and second synchronization information; step S601 may include:

[0138] S6011. Receive the first synchronization information sent by the target node.

[0139] S6012. Obtain the current time based on the first synchronization information.

[0140] Upon receiving the first synchronization message, the slave data acquisition node can unpack the first synchronization message to obtain the current time Tm_Base acquired by the master data acquisition node when sending the first synchronization message.

[0141] S6013, receive the second synchronization information sent by the target node; the second synchronization information includes a target delay determined based on a first value and a second value; the first value is the value recorded by the first timer when the target node sends the first synchronization information, and the second value is the value recorded by the first timer when the target node successfully sends the information after receiving the first synchronization information;

[0142] Upon receiving the second synchronization message, the slave data acquisition node can unpack the second synchronization message to obtain the target delay Tm_Delay transmitted in the CAN network by the master data acquisition node when sending the first synchronization message.

[0143] Step S603 may include:

[0144] S6031, generate a target time base based on the current time and the target delay.

[0145] The target time base T0 of the slave node can be calculated through the above formula (2).

[0146] In one embodiment, the data processing method further includes:

[0147] S701, upon receiving the first synchronization information, start the second timer;

[0148] The second timer is used to obtain the time offset T_offset of the target acquisition data relative to the target time base. Preferably, when starting the second timer, the count value of the second timer is cleared simultaneously.

[0149] S703, when the target acquisition data is acquired, determine the time offset of the target acquisition data based on the third value of the second timer.

[0150] On the premise of clearing the count value of the second timer as mentioned above, the time offset of the target acquisition data relative to the target time base can be obtained only through the third count value of the second timer. On the premise that the count value of the second timer is not cleared, the time offset of the target acquisition data relative to the target time base in the slave data acquisition node can also be obtained through the third value of the second timer and its initial count value.

[0151] For a better understanding of the data processing method of this embodiment, Figure 6 shows an interactive schematic diagram of time synchronization of the master data acquisition node to the slave data acquisition node in an example. Combining Figure 6, the implementation process of the above method is described from the perspective of the interaction between the master data acquisition node and the slave data acquisition node:

[0152] After the time synchronization event is triggered, the master data acquisition node synchronizes the time of the slave data acquisition node.

[0153] The master data acquisition node sends a first synchronization message including its own global time Tm_Base to the slave data acquisition node, and records the first value of the first timer when sending the first synchronization message.

[0154] The slave data acquisition node receives the first synchronization message, unpacks it to obtain Tm_Base, and at the same time, clears and starts its own second timer.

[0155] When the master data acquisition node determines that the first synchronization message is successfully sent, it records the second value Tm_CounterTx of the first timer, calculates the target delay Tm_Delay, generates a second synchronization message and starts the sending of the second synchronization message; at the same time, the master data acquisition node also records its own global time and uses it as its own target time base T0.

[0156] The slave data acquisition node receives the second synchronization message, unpacks it to obtain Tm_Delay; adds Tm_Delay to the previously obtained Tm_Base to get its own target time base T0. After this step, the time alignment at the position of the target time base T0 on the time axis between the slave data acquisition node and the master data acquisition node is achieved. Since the second timers of the master data acquisition node and the slave data acquisition node are cleared and started at the position of T0 on the time axis, during the data fusion process, the time offset of the data generation time relative to T0 can be recorded. By packing this time offset with the collected target data and sending it to the data receiving node, and unpacking it at the receiving end to obtain the time offset, the generation time of the target data acquisition packet can be obtained by calculating the sum of T0 and the time offset T_offset on the time axes of each data acquisition node.

[0157] Correspondingly, as Figure 6 shown, after the slave data acquisition node calculates the target time base T0, when the slave data acquisition node generates data, it records the count value of the second timer to obtain the time offset of the time point when the data is generated relative to the target time base T0, packs the collected target data with this time offset to generate a collection data packet, and sends the collection data packet to the master data acquisition node for data fusion analysis.

[0158] In some application scenarios, when data is generated on the master data acquisition node and needs to be sent to other data acquisition nodes for data fusion analysis, the time offset of the time point when the data is generated relative to the target time base T0 can also be obtained through its second timer and the target time base, and the collected target data is packaged with the time offset to generate a collection data packet, which is then sent to the receiving node. Thus, the data generation time on the master data acquisition node can also be obtained at the receiving node. As Figure 6 shown by the corresponding part of the dashed line in

[0159] Therefore, after the master data acquisition node and the slave data acquisition node of the embodiments of the present application complete the alignment of the above-mentioned target time base T0, both can obtain the generation time of the acquisition data of other data acquisition nodes through the received acquisition data packets, without being limited to performing data fusion processing in the master data acquisition node. Those skilled in the art can, according to needs, after the slave data acquisition node completes the update of the target time base T0, use the slave data acquisition node as the receiving node of the data acquisition packet and perform data fusion analysis at the slave data acquisition node. Thus, the data processing method of the present application can provide a more flexible solution for the execution entity of data fusion processing in the perception system of autonomous driving vehicles.

[0160] Based on the above method embodiments, the embodiments of the present application further provide a data processing device.

[0161] In one embodiment, as Figure 8 shown, the data processing device includes:[[]]END]]

[0162] A data acquisition module 110, configured to acquire acquisition data packets sent by multiple data acquisition nodes; the acquisition data packet includes target acquisition data and the time offset of the target acquisition data; the time offset is used to represent the time offset information of the acquisition time of the target acquisition data relative to the target time base.

[0163] A data alignment module 120, configured to determine the target acquisition data with the same time offset sent by multiple data acquisition nodes as the acquisition data at the target moment.

[0164] A data analysis module 130, configured to perform data fusion on the acquisition data at the target moment to obtain object state information corresponding to the target moment.

[0165] In one embodiment, the data processing device further includes a time synchronization module, configured to acquire the current time; and generate time synchronization information based on the current time, and send the time synchronization information to multiple data acquisition nodes; so that multiple data acquisition nodes generate a target time base based on the time synchronization information.

[0166] In one embodiment, the time synchronization information includes first synchronization information and second synchronization information.

[0167] The time synchronization module of the data processing device further includes a first synchronization unit and a second synchronization unit. Among them, the first synchronization unit is used to generate first synchronization information based on the current time; send the first synchronization information to multiple data acquisition nodes, and record the first value of the first timer; so that the multiple data acquisition nodes can obtain the current time. The second synchronization unit is used to record the second value of the first timer when receiving the information that the first synchronization information is successfully sent; determine the target delay based on the first value and the second value; generate second synchronization information based on the target delay; send the second synchronization information to multiple data acquisition nodes; so that the multiple data acquisition nodes can generate a target time base based on the current time and the target delay.

[0168] In one embodiment, the data processing device further includes a time offset acquisition module, which is used to, when multiple data acquisition nodes receive the first synchronization information, start a second timer corresponding to each of the multiple data acquisition nodes based on the multiple data acquisition nodes; and, when the multiple data acquisition nodes acquire target acquisition data, determine the time offset of the target acquisition data based on the third value of the second timer.

[0169] The embodiment of the present application also provides another data processing device.

[0170] In one embodiment, as Figure 9 shown, the data processing device includes:

[0171] A data generation module 210, which is used to generate an acquisition data packet based on the acquired target acquisition data; the acquisition data packet includes the target acquisition data and the time offset of the target acquisition data; the time offset is used to represent the time offset information of the acquisition time of the target acquisition data relative to the target time base.

[0172] A data sending module 220, which is used to send the acquisition data packet to a target node, so that the target node determines the target acquisition data with the same time offset as the acquisition data at the target moment, and performs data fusion on the acquisition data at the target moment to obtain object state information corresponding to the target moment.

[0173] In one embodiment, the device further includes:

[0174] A time alignment module, which is used to receive the time synchronization information sent by the target node; the time synchronization information includes the current time obtained by the target node when generating the time synchronization information; and, generate a target time base based on the current time.

[0175] In one embodiment, the time synchronization information includes first synchronization information and second synchronization information.

[0176] The time alignment module further includes a first alignment unit, a second alignment unit, and a time base alignment unit. Among them, the first alignment unit is configured to receive the first synchronization information sent by the target node; and obtain the current time based on the first synchronization information.

[0177] The second alignment unit is configured to receive the second synchronization information sent by the target node; the second synchronization information includes a target time delay determined based on a first value and a second value; the first value is the value recorded by the first timer when the target node sends the first synchronization information, and the second value is the value recorded by the first timer when the target node successfully sends the information after receiving the first synchronization information.

[0178] The time base alignment unit is configured to generate a target time base based on the current time and the target time delay.

[0179] In one embodiment, the data generation module 210 of the data processing device is further configured to start a second timer when receiving the first synchronization information; and determine the time offset of the target acquisition data based on a third value of the second timer when the target acquisition data is collected.

[0180] For the specific limitations of the data processing device, reference can be made to the limitations of the data processing method in the above text respectively, which will not be elaborated here. Each module of the above data processing device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above respective modules.

[0181] The data processing method of the present application can be a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to implement various aspects of the present application.

[0182] Embodiments of the present application also provide an electronic device, which includes a processor and a memory. At least one instruction or at least one program segment is stored in the memory, and the at least one instruction or the at least one program segment is loaded and executed by the processor to implement the data processing method provided in the above control method embodiments. The electronic device 800 may vary greatly due to different configurations or performances, and may include one or more central processing units (CPUs) 810 (the processor 810 may include, but is not limited to, a processing device such as a microprocessor MCU or a field programmable gate array FPGA), a memory 830 for storing data, and one or more storage media 820 (such as one or more mass storage devices) for storing application programs 823 or data 822. Among them, the memory 830 and the storage media 820 may be transient storage or persistent storage. The program stored in the storage media 820 may include one or more modules, and each module may include a series of instruction operations on the electronic device. Further, the central processor 810 may be configured to communicate with the storage media 820 and execute a series of instruction operations in the storage media 820 on the electronic device 800. The electronic device 800 may also include one or more power supplies 860, one or more wired or wireless network interfaces 850, one or more input / output interfaces 840, and / or one or more operating systems 821, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, and so on.

[0183] The input / output interface 840 can be used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of the electronic device 800. In one example, the input / output interface 840 includes a network interface controller (NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one example, the input / output interface 840 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0184] Those of ordinary skill in the art can understand that Figure 10 the structure shown is only schematic and does not limit the structure of the above electronic device. For example, the electronic device 800 may also include more or fewer components than those shown Figure 10 in the figure, or have a different configuration from that shown Figure 10 in the figure.

[0185] The memory can be used to store software programs and modules. By running the software programs and modules stored in the memory, the processor can execute various functional applications and data processing. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include high-speed random access memory and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device or other volatile solid-state storage devices. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0186] The message processing method provided by the embodiments of the present application can be executed in a mobile terminal, a computer terminal, a server or a similar computing device.

[0187] The embodiments of the present application also provide a computer-readable storage medium. The storage medium can be arranged in the server to store at least one instruction or at least one segment of program related to the vehicle information processing method or the vehicle remote control method in the method embodiments. The at least one instruction or the at least one segment of program is loaded and executed by the processor to implement the above data processing method.

[0188] Optionally, in this embodiment, the above storage medium can be located in at least one of multiple network servers in a computer network. Optionally, in this embodiment, the above storage medium can include, but is not limited to: USB flash drives, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disks, magnetic disks or optical discs and other media that can store program codes.

[0189] It should be noted that: the above sequence of the embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the above specific embodiments of the present application have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be executed in a different order from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0190] Each embodiment in this application is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of devices, equipment, and storage media, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, reference can be made to the corresponding descriptions in the method embodiments.

[0191] Those of ordinary skill in the art can understand that all or part of the steps to implement the above embodiments can be completed by hardware or can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a magnetic disk, an optical disc, or the like.

[0192] The above are only the preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included within the protection scope of this application.

[0193] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any reference signs in the claims should not be regarded as limiting the claims involved.

[0194] In addition, it should be understood that although this specification is described according to embodiments, not every embodiment only contains an independent technical solution. This narrative way of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A data processing method, characterized in that, the method includes: obtaining acquisition data packets sent by multiple data acquisition nodes; the acquisition data packet includes target acquisition data and a time offset of the target acquisition data; the time offset is used to represent time offset information of the acquisition time of the target acquisition data relative to a target time base; determining the target acquisition data with the same time offset sent by the multiple data acquisition nodes as the acquisition data at a target moment; performing data fusion on the acquisition data at the target moment to obtain object state information corresponding to the target moment; before obtaining the acquisition data packets sent by the multiple data acquisition nodes, the method further includes: obtaining the current time; generating time synchronization information based on the current time, and sending the time synchronization information to the multiple data acquisition nodes; the time synchronization information includes first synchronization information and second synchronization information; generating the first synchronization information based on the current time; sending the first synchronization information to the multiple data acquisition nodes, and recording a first value of a first timer; when receiving the information that the first synchronization information is successfully sent, recording a second value of the first timer; determining a target time delay based on the first value and the second value; generating the second synchronization information based on the target time delay; sending the second synchronization information to the multiple data acquisition nodes; so that the multiple data acquisition nodes generate the target time base based on the current time encapsulated in the first synchronization information and the target time delay encapsulated in the second synchronization information.

2. The method according to claim 1, characterized in that, the method further includes: when the multiple data acquisition nodes receive the first synchronization information, starting second timers corresponding to the multiple data acquisition nodes based on the multiple data acquisition nodes; when the multiple data acquisition nodes acquire the target acquisition data, determining the time offset of the target acquisition data based on a third value of the second timer.

3. A data processing method, characterized in that, the method includes: generating an acquisition data packet based on the acquired target acquisition data; the acquisition data packet includes the target acquisition data and a time offset of the target acquisition data; the time offset is used to represent time offset information of the acquisition time of the target acquisition data relative to a target time base; sending the acquisition data packet to a target node, so that the target node determines the target acquisition data with the same time offset as the acquisition data at a target moment, and performs data fusion on the acquisition data at the target moment to obtain object state information corresponding to the target moment; before generating the acquisition data packet based on the acquired data, the method further includes: receiving first synchronization information sent by the target node, where the current time is encapsulated in the first synchronization information; obtaining the current time based on the first synchronization information; Receive the second synchronization information sent by the target node; the second synchronization information includes a target time delay determined based on a first value and a second value; the first value is the value recorded by a first timer when the target node sends the first synchronization information, and the second value is the value recorded by the first timer when the target node receives the information successfully sent for the first synchronization information; Generate the target time base based on the current time and the target time delay.

4. The method according to claim 3, characterized in that, the method further includes: When receiving the first synchronization information, start a second timer; When collecting the target acquisition data, determine the time offset of the target acquisition data based on a third value of the second timer.

5. A data processing device, characterized in that, it includes: A data acquisition module, configured to acquire acquisition data packets sent by multiple data acquisition nodes; The acquisition data packet includes target acquisition data and a time offset of the target acquisition data; the time offset is used to represent the time offset information of the acquisition time of the target acquisition data relative to the target time base; A data alignment module, configured to determine the acquisition data at the target moment from the target acquisition data sent by the multiple data acquisition nodes and having the same time offset; A data analysis module, configured to perform data fusion on the acquisition data at the target moment to obtain object state information corresponding to the target moment; A time synchronization module, configured to acquire the current time; and generate time synchronization information based on the current time, and send the time synchronization information to multiple data acquisition nodes; so that the multiple data acquisition nodes generate a target time base based on the time synchronization information; the time synchronization information includes first synchronization information and second synchronization information; The time synchronization module further includes a first synchronization unit and a second synchronization unit, wherein the first synchronization unit is configured to generate first synchronization information based on the current time; send the first synchronization information to multiple data acquisition nodes, and record a first value of a first timer; The second synchronization unit is configured to record a second value of the first timer when receiving the information successfully sent for the first synchronization information; determine a target time delay based on the first value and the second value; generate second synchronization information based on the target time delay; send the second synchronization information to multiple data acquisition nodes; so that the multiple data acquisition nodes generate the target time base based on the current time encapsulated in the first synchronization information and the target time delay encapsulated in the second synchronization information.

6. A data processing device, characterized in that, it includes: A data generation module, configured to generate an acquisition data packet based on the acquired target acquisition data; The acquisition data packet includes the target acquisition data and a time offset of the target acquisition data; the time offset is used to represent the time offset information of the acquisition time of the target acquisition data relative to the target time base; A data sending module, configured to send the collected data packet to a target node, so that the target node determines the target collected data with the same time offset as the collected data at the target moment, and performs data fusion on the collected data at the target moment to obtain object state information corresponding to the target moment; A time alignment module, configured to receive time synchronization information sent by the target node; the time synchronization information includes the current time obtained when the target node generates the time synchronization information; the time synchronization information includes first synchronization information and second synchronization information; The time alignment module further includes a first alignment unit, a second alignment unit, and a time base alignment unit. Among them, the first alignment unit is configured to receive the first synchronization information sent by the target node; obtain the current time based on the first synchronization information; The second alignment unit is configured to receive the second synchronization information sent by the target node; the second synchronization information includes a target time delay determined based on a first value and a second value; the first value is the value recorded by the first timer when the target node sends the first synchronization information, and the second value is the value recorded by the first timer when the target node receives the information successfully sent for the first synchronization information; The time base alignment unit is configured to generate a target time base based on the current time and the target time delay.

7. A storage medium, characterized in that, at least one instruction or at least one program segment is stored in the storage medium, and the at least one instruction or the at least one program segment is loaded and executed by a processor to implement the data processing method according to any one of claims 1 to 4.

Images