Time synchronization method and device for multi-sensor data, electronic device and storage medium

Abstract

The application relates to a time synchronization method and device for multi-sensor data, electronic equipment, a storage medium and a computer program product. The method comprises the following steps: after receiving sensor data sent by any sensor (target sensor) in a plurality of sensors, updating a receiving count value of the target sensor; after the updating, detecting whether a preset time synchronization condition is met based on the receiving count values of the sensors in the plurality of sensors; and when the condition is met, performing time synchronization processing on a plurality of target sensor data according to the receiving time of the target sensor data, so as to divide the batches to which the target sensor data belong. Through the above method, the batches to which the target sensor data belong can be divided, a master sensor does not need to be set, time synchronization errors caused by uncertain data reaching sequences of the master sensor are avoided, and the accuracy of time synchronization is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, electronic device, storage medium, and computer program product for time synchronization of multi-sensor data. Background Technology

[0002] In the field of autonomous driving, vehicles are typically equipped with multiple sensors. These sensors periodically send the collected sensor data to the vehicle, which then uses the received sensor data to perceive the external environment and achieve safe driving based on the perception results.

[0003] In traditional technology, a master sensor is set among multiple sensors. After the vehicle receives the sensing data sent by the master sensor, it determines that it has received the sensing data collected by all sensors in the same period, realizing the time synchronization of the sensing data collected by multiple sensors, thereby triggering the perception task, fusing all the currently received sensing data and performing perception processing to obtain the perception result.

[0004] However, the synchronization accuracy of the above time synchronization methods is poor. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, electronic device, storage medium, and computer program product for time synchronization of multi-sensor data to address the aforementioned technical problems.

[0006] In a first aspect, embodiments of this disclosure provide a method for time synchronization of multi-sensor data, including:

[0007] After receiving sensing data from the target sensor among multiple sensors, the receive count value of the target sensor is updated. The target sensor is any one of the multiple sensors.

[0008] After the update process, the preset time synchronization conditions are checked based on the received count values ​​of each sensor among multiple sensors.

[0009] If the conditions are met, then time synchronization processing is performed on the multiple target sensor data based on the reception time of the multiple target sensor data that have not been time synchronized, so as to classify the batch to which the multiple target sensor data belong.

[0010] Secondly, embodiments of this disclosure provide a multi-sensor data synchronization device, comprising:

[0011] The counting update module is used to update the received count value of the target sensor after receiving the sensing data sent by the target sensor among multiple sensors. The target sensor is any one of the multiple sensors.

[0012] The synchronization determination module is used to detect whether the preset time synchronization conditions are met based on the received count values ​​of each sensor among multiple sensors after the update process.

[0013] The data synchronization module is used to perform time synchronization processing on multiple target sensor data based on the reception time of multiple target sensor data that have not been time synchronized, under the premise of meeting the preset time synchronization conditions, so as to classify the batch to which the multiple target sensor data belong.

[0014] Thirdly, embodiments of this disclosure provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0015] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0016] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0017] The aforementioned method, apparatus, electronic device, storage medium, and computer program product for time synchronization of multi-sensor data, after receiving sensing data from any of the multiple sensors (i.e., the target sensor), updates the receive count value of the target sensor. After the update, it checks whether a preset time synchronization condition is met based on the receive count values ​​of each sensor. If the condition is met, it performs time synchronization processing on the multiple target sensor data based on the reception time of the unsynchronized data, thereby classifying the multiple target sensor data into batches. This method can achieve batch classification of multiple target sensor data without the need for a master sensor, avoiding time synchronization errors caused by uncertain arrival order of master sensor data, and improving the accuracy of time synchronization. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a time synchronization method for multi-sensor data in one embodiment;

[0019] Figure 2 This is a flowchart illustrating the process of detecting whether a preset time synchronization condition is met in one embodiment.

[0020] Figure 3 This is a flowchart illustrating the process of detecting whether a preset time synchronization condition is met in another embodiment.

[0021] Figure 4 This is a flowchart illustrating time synchronization processing in one embodiment.

[0022] Figure 5 This is a flowchart illustrating a time synchronization method for multi-sensor data in another embodiment;

[0023] Figure 6 This is a structural block diagram of a time synchronization device for multi-sensor data in one embodiment;

[0024] Figure 7 This is a diagram of the internal structure of an electronic device in one embodiment. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0026] First, before detailing the technical solutions of the embodiments of this disclosure, we will first introduce the technical background or evolution of the technologies upon which these embodiments are based. In the field of autonomous driving, vehicles are typically equipped with multiple sensors. These sensors periodically transmit the collected sensor data to the vehicle, which then perceives the external environment based on the received sensor data to achieve safe driving. Throughout this process, the vehicle also needs to synchronize the sensor data periodically transmitted by the multiple sensors in time and space, so as to process the perception data and obtain the perception results. The accuracy of the synchronization directly affects the accuracy of the perception results.

[0027] For time synchronization of multi-sensor data, traditional technology uses a master sensor among multiple sensors. When the vehicle receives data from the master sensor, it determines that it has received data from all sensors in the same period. It then identifies currently received, unprocessed data as part of the same batch and triggers a perception task. This unprocessed data is then fused and processed to obtain the perception result. However, during the periodic transmission of data from multiple sensors, the order in which the vehicle receives each batch of data is not the same. For example, in the first batch, the vehicle receives data from sensors A, B, and C in that order; in the second batch, it receives data from sensors C, A, and B in that order. In this case, if the traditional method of using a master sensor for data synchronization is employed, such as using sensor C as the master sensor, synchronization errors will occur, leading to perception errors. Therefore, improving the accuracy of multi-sensor data time synchronization is a pressing problem. Furthermore, it should be noted that the applicant has invested significant creative effort in the technical solutions described in the following embodiments.

[0028] The technical solutions involved in the embodiments of this disclosure will be described below in conjunction with the scenarios in which they are applied.

[0029] In one embodiment, such as Figure 1 As shown, a method for time synchronization of multi-sensor data is provided. This embodiment illustrates the application of this method to an electronic device. It is understood that the electronic device can be installed in a vehicle or on the roadside, interacting with the vehicle to achieve time synchronization of multi-sensor data. In this embodiment, the method includes the following steps:

[0030] S110. After receiving the sensing data sent by the target sensor among multiple sensors, update the receive count value of the target sensor. The target sensor is any one of the multiple sensors.

[0031] The vehicle is equipped with multiple sensors, each with a corresponding counter. The sensors collect data from the driving environment, and each sensor periodically collects and sends the data to the electronic equipment. The counter records the number of times the electronic equipment receives the data.

[0032] Specifically, once any of the multiple sensors mounted on the vehicle collects sensing data, it is identified as the target sensor, and the sensing data collected by the target sensor is sent to the electronic device. After receiving the sensing data sent by the target sensor, the electronic device updates the receive count value of the counter corresponding to the target sensor, for example, incrementing the receive count value by 1 each time sensing data is received.

[0033] S120. After the update process, the preset time synchronization conditions are checked based on the received count values ​​of each sensor among multiple sensors.

[0034] The time synchronization condition is used to determine whether to perform time synchronization processing, which is the process of identifying sensor data belonging to the same batch. Multiple sensors have the same acquisition and transmission cycle, meaning they acquire and transmit sensor data once every time interval T. Sensor data from the same cycle among multiple sensors belong to the same batch. For example, if a vehicle has three sensors that periodically acquire and transmit sensor data to electronic devices, the sensor data transmitted in the same cycle belongs to the same batch.

[0035] Optionally, after updating the received count value of each counter, the electronic device obtains the received count value of the counter corresponding to each sensor, and checks whether the currently received sensor data meets a preset time synchronization condition based on each received count value. To ensure the integrity of the same batch of data, the preset time synchronization condition may be whether the sum of the received count values ​​of all sensors is equal to the number of sensors mounted on the vehicle.

[0036] S130. If satisfied, then based on the reception time of multiple target sensor data that have not been time synchronized, perform time synchronization processing on the multiple target sensor data to classify the batch to which the multiple target sensor data belong.

[0037] Among them, multiple target sensing data are the sensing data received by the electronic device.

[0038] Specifically, if the received count values ​​of each of the above multiple sensors meet the preset time synchronization conditions, for example, the sum of the received count values ​​of all sensors is equal to the number of multiple sensors mounted on the vehicle, the electronic device obtains the reception time of each received sensor data, and performs time synchronization on the target sensor data according to the reception time of the currently received target sensor data that has not been time synchronized, so as to divide the batch to which the multiple target sensor data belong, and obtain the sensor data belonging to the same batch.

[0039] In this embodiment, after receiving sensing data from any of the multiple sensors (i.e., the target sensor), the electronic device updates the receive count value of the target sensor. After the update, it checks whether a preset time synchronization condition is met based on the receive count values ​​of each sensor. If the condition is met, it performs time synchronization processing on the multiple target sensor data based on the reception time of the unsynchronized data, thereby classifying the multiple target sensor data into batches. This method achieves the classification of multiple target sensor data into batches without the need for a master sensor, avoiding time synchronization errors caused by the uncertain arrival order of master sensor data, and improving the accuracy of time synchronization.

[0040] Sensor disconnections frequently occur during the use of multiple sensors. Therefore, determining whether preset time synchronization conditions are met requires identifying whether any of the sensors have disconnected. In one embodiment, such as... Figure 2 As shown, the step S120 above, which detects whether a preset time synchronization condition is met based on the received count values ​​of each of the multiple sensors, includes:

[0041] S210. Detect whether there is a first receive count value and a second receive count value among the receive count values ​​of each sensor.

[0042] Among them, the difference between the first received count value and the second received count value is greater than the count threshold.

[0043] Specifically, each time the electronic device receives sensor data, it updates the corresponding sensor's receive count value and acquires the current receive count values ​​of all sensors. It then checks whether there exists a first receive count value and a second receive count value whose difference is greater than a counting threshold. For example, the electronic device can calculate the difference between the receive count values ​​of any two sensors to determine if there is a sensor whose difference is greater than the counting threshold, thus determining whether there is a first receive count value and a second receive count value among the receive count values ​​of all sensors. If there is a sensor whose difference is greater than the counting threshold, then the first and second receive count values ​​are confirmed to exist; if there is no sensor whose difference is greater than the counting threshold, then the first and second receive count values ​​are confirmed to not exist.

[0044] Optionally, the aforementioned counting threshold is 1. Taking a vehicle equipped with three sensors (A, B, C) as an example, the electronic device first receives the sensing data sent by sensor A. At this time, the received count value of sensor A's counter is updated to 1 (the initial received count value of all counters is 0), and the received count values ​​of the other sensors are all 0. The electronic device calculates the difference between any two received count values ​​and determines that there is no first or second received count value whose difference is greater than the counting threshold 1 at the current moment, then continues to wait for receiving sensing data. When the electronic device receives sensing data sent by sensor A again, the received count value of sensor A's counter is updated to 2. At this time, the received count values ​​of the other sensors are still 0. The electronic device calculates the difference between any two received count values ​​and determines that the difference between the received count value of sensor A and the received count values ​​of sensors B and C is greater than 1. The electronic device can then determine that there is a first and second received count value among the received count values ​​of each sensor at the current moment.

[0045] Optionally, if it is determined that there is a first receive count value and a second receive count value among the receive count values ​​of each sensor at the current time, the electronic device can further determine that the sensor corresponding to the smaller value of the first receive count value and the second receive count value is the offline sensor. Continuing the above example, the difference between the receive count value (2) of sensor A and the receive count value (0) of sensor B is 2, which is greater than the counting threshold 1. Therefore, sensor B, which corresponds to the smaller receive count value (0), is the offline sensor. At the same time, the difference between the receive count value (2) of sensor A and the receive count value (0) of sensor C is also 2, which is greater than the counting threshold 1. Therefore, sensor C, which corresponds to the smaller receive count value (0), is also the offline sensor.

[0046] S220. If a first receive count value and a second receive count value exist, then the time synchronization condition is determined to be met.

[0047] Specifically, if it is determined that there is a first received count value and a second received count value where the difference between the received count values ​​is greater than the count threshold, it means that the electronic device has received sensor data belonging to the next batch. At this time, the time synchronization condition is met, and the electronic device needs to classify the target sensor data currently received into its respective batch.

[0048] In this embodiment, the electronic device detects whether there is a first and a second received count value among the received count values ​​of each sensor whose difference is greater than the counting threshold. If such a difference exists, the time synchronization condition is determined to achieve active triggering of time synchronization. This avoids the failure of the entire time synchronization (i.e., single point failure) caused by the main sensor going offline, improves the stability of time synchronization, and enhances robustness.

[0049] In one embodiment, if it is determined that none of the multiple sensors are offline, such as Figure 3 As shown, the above method also includes:

[0050] S310. If there is no first receiving count value and no second receiving count value, then check whether the sum of the receiving count values ​​of each sensor meets the preset sum condition.

[0051] Specifically, if there is no first and second receive count values, the electronic device can determine that there are no offline sensors among the multiple sensors, and can then determine whether to trigger time synchronization processing, i.e. whether the time synchronization conditions are met, based on the sum of the receive count values ​​of each sensor.

[0052] The preset sum condition is that the sum of the received count values ​​of each sensor equals the number of sensors.

[0053] Optionally, if no sensor among the multiple sensors is offline, the electronic device can calculate the sum of the received count values ​​of each sensor and determine whether the sum of the received count values ​​is equal to the number of sensors. If the sum of the received count values ​​is equal to the number of sensors, a preset sum condition is satisfied; if the sum of the received count values ​​is not equal to the number of sensors, the preset sum condition is not satisfied.

[0054] S320. If satisfied, then the time synchronization condition is satisfied.

[0055] Specifically, if the sum of the received counts of each sensor meets the preset sum condition, it can be determined that the time synchronization condition is met and the time synchronization process is triggered. Conversely, if the sum of the received counts of each sensor does not meet the preset sum condition, it can be determined that the time synchronization condition is not met and the time synchronization process is not triggered.

[0056] Taking a vehicle equipped with three sensors (A, B, and C) as an example, the electronic device first receives sensor data from sensor A. At this time, the receive count value of sensor A's counter is updated to 1 (the initial receive count value of all counters is 0), while the receive count values ​​of the other sensors are 0. Based on the current receive count values ​​of each sensor, the electronic device determines that the time synchronization condition is not met and continues to receive sensor data. The electronic device then receives sensor data from sensor B. The receive count value of sensor A's counter remains 1, while the receive count value of sensor B's counter is updated to 1, and the receive count value of sensor C's counter remains 1. At this point, the time synchronization condition is still not met, and the electronic device continues to receive sensor data. The electronic device then receives sensor data from sensor C. The receive count values ​​of sensor A, B, and C remain 1, while the receive count value of sensor C is updated to 1. At this point, the sum of the receive count values ​​of sensors A, B, and C equals the number of sensors (3), thus satisfying the time synchronization condition. The electronic device then triggers time synchronization processing.

[0057] In this embodiment, if there is no difference between the received count values ​​of the individual sensors that is greater than the counting threshold, the electronic device further detects whether the sum of the received count values ​​of each sensor meets a preset sum condition. Specifically, it can detect whether the sum of the received count values ​​is equal to the number of multiple sensors. If it is equal, it is determined that the time synchronization condition is met, so as to perform subsequent time synchronization processing, which ensures the integrity of the sensor data in the same batch and improves the accuracy of time synchronization.

[0058] In practical applications, the synchronization and reception of data from multiple sensors are performed simultaneously. To achieve stable time synchronization of the entire multi-sensor data, in one embodiment, the update process for the reception count value of the target sensor in S120 above includes:

[0059] Increment the receive count value of the target sensor by 1.

[0060] Specifically, for each time the electronic device receives sensing data sent by a target sensor, it increments the receive count value of the counter corresponding to that target tactile sensor by 1.

[0061] At the same time, the receive count value of the counter corresponding to the target sensing data after time synchronization processing must also be updated.

[0062] Specifically, after performing time synchronization processing on the sensor data of multiple targets, the above method also includes:

[0063] Decrease the receive count value of the sensor corresponding to each target sensing data that is divided into the same receiving batch by 1.

[0064] Continuing with the example of a vehicle equipped with three sensors (A, B, C), the electronic device first receives sensor data from sensor A. At this point, the receive count values ​​for sensors A, B, and C are 1, 0, and 0 respectively, and time synchronization processing is not triggered. The electronic device then receives sensor data from sensor C. Again, the receive count values ​​for sensors A, B, and C are 1, 0, and 1 respectively, and time synchronization processing is not triggered. Finally, the electronic device receives sensor data from sensor A again. At this point, the receive count values ​​for sensors A, B, and C are 2, 0, and 1 respectively, triggering time synchronization processing (the difference in receive count values ​​is greater than a counting threshold, and the sum of the receive count values ​​equals the number of sensors). Having received three sets of sensor data, the electronic device, after time synchronization processing, determines that the first two sets of data belong to the same batch, and the last set belongs to the next batch. The electronic device then decrements the receive count value of the sensor corresponding to the target sensor data that underwent time synchronization processing (i.e., the sensor corresponding to the first two sets of data) by 1 for update processing. After the update, the receive count values ​​for sensors A, B, and C are 1, 0, and 0 respectively.

[0065] In one embodiment, to improve the accuracy of time synchronization processing, as shown in 4, the time synchronization processing of multiple target sensor data based on the reception time of multiple target sensor data that have not been time synchronized includes:

[0066] S410, Perform difference processing on the pairwise reception times of the reception times of multiple target sensor data.

[0067] Specifically, after determining that time synchronization processing is required, i.e., the time synchronization condition is met, the electronic device acquires the reception times of multiple target sensor data that are not currently being synchronized, and calculates the difference between any two reception times to obtain the difference value. For example, at the current moment, the electronic device receives three sensor data, namely sensor data S1, S2, and S3. After triggering time synchronization processing, the difference between S1 and S2 is calculated to obtain ΔS1, the difference between S2 and S3 is calculated to obtain ΔS2, and the difference between S3 and S1 is calculated to obtain ΔS3.

[0068] S420. The target sensing data corresponding to the receiving time with a difference less than the interval threshold are divided into the same batch.

[0069] Specifically, the electronic device determines target sensor data belonging to the same batch based on the differences obtained by pairwise subtraction. Specifically, target sensor data corresponding to reception times with differences less than an interval threshold are grouped into the same batch. Optionally, the aforementioned interval threshold can be 1 / 2T, where T is the period during which multiple touch sensors collect and transmit sensor data. Continuing the example above, if ΔS1 < 1 / 2T, ΔS2 < 1 / 2T, and ΔS3 < 1 / 2T, then sensor data S1, S2, and S3 are determined to belong to the same batch; if ΔS1 < 1 / 2T, and ΔS2 ≥ 1 / 2T, and S3 ≥ 1 / 2T, then sensor data S1 and S2 are determined to belong to the same batch, and S3 belongs to another batch; if ΔS2 < 1 / 2T, and ΔS1 ≥ 1 / 2T, and S3 ≥ 1 / 2T, then sensor data S2 and S3 are determined to belong to the same batch, and S1 belongs to another batch.

[0070] Optionally, the above method further includes performing perception processing on the time-synchronized target sensing data.

[0071] Among them, perception processing is the process of performing target perception and recognition on the sensing data to obtain the perception result.

[0072] Specifically, for target sensing data after time synchronization processing, the time order of the target sensing data to which it belongs is determined based on the reception time of the target sensing data. Excluding the batch with the latest time order, target sensing data belonging to the same batch are fused and then processed sequentially according to the time order of their respective batches. For example, continuing with the above distance, after determining that sensing data S1 and S2 belong to the same batch and S3 to another batch, the time order of their respective batches is determined based on the reception times of sensing data S1, S2, and S3 (S1 reception time is earlier than S2, and S2 reception time is earlier than S3). S1 and S2 belong to a batch earlier than S3. Excluding the batch with the latest time order, i.e., the batch to which S3 belongs, sensing data S1 and S2 are then fused and processed.

[0073] In this embodiment, the electronic device performs difference processing on the pairwise reception times of multiple target sensor data, and classifies the target sensor data corresponding to the reception times with differences less than the interval threshold into the same batch. This achieves accurate classification of the target sensor data into batches, improves the accuracy of time synchronization processing, improves the accuracy of the entire multi-sensor data time synchronization, and thus improves the accuracy of subsequent sensing processing.

[0074] To facilitate understanding by those skilled in the art, the time synchronization method for multi-sensor data provided in this application is described in detail below, such as... Figure 5 As shown, the method may include:

[0075] S510. After receiving the sensing data sent by the target sensor among multiple sensors, increment the receive count value of the target sensor by 1. The target sensor is any one of the multiple sensors.

[0076] S520. After the update process, detect whether there is a first receive count value and a second receive count value among the receive count values ​​of each sensor; wherein the difference between the first receive count value and the second receive count value is greater than the counting threshold.

[0077] S530. If a first receive count value and a second receive count value exist, then the time synchronization condition is determined to be met.

[0078] S540. If there is no first receive count value and no second receive count value, then check whether the sum of the receive count values ​​of each sensor is equal to the number of multiple sensors.

[0079] S550, if equal to, then the time synchronization condition is satisfied;

[0080] S560. Under the condition of meeting the time synchronization condition, the difference between the reception times of the pairwise reception times of multiple target sensor data that have not been time synchronized is processed.

[0081] S570. The target sensing data corresponding to the receiving time with a difference less than the interval threshold are divided into the same batch.

[0082] S580, Decrement the receive count value of the sensor corresponding to each target sensing data in the same receiving batch by 1.

[0083] It should be noted that the descriptions in S510-S580 above can be found in the relevant descriptions in the above embodiments, and their effects are similar, so they will not be repeated here.

[0084] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0085] In one embodiment, such as Figure 6 As shown, a time synchronization device for multi-sensor data is provided, comprising: a counting update module 601, a synchronization determination module 602, and a data synchronization module 603, wherein:

[0086] The counting update module 601 is used to update the received count value of the target sensor after receiving the sensing data sent by the target sensor among multiple sensors. The target sensor is any one of the multiple sensors.

[0087] The synchronization determination module 602 is used to detect whether the preset time synchronization conditions are met based on the received count values ​​of each sensor among multiple sensors after the update process.

[0088] The data synchronization module 603 is used to perform time synchronization processing on multiple target sensor data based on the reception time of multiple target sensor data that have not been time synchronized, under the condition that the preset time synchronization conditions are met, so as to classify the batch to which the multiple target sensor data belong.

[0089] In one embodiment, the synchronization determination module 602 is specifically used for:

[0090] The system detects whether there is a first receive count value and a second receive count value among the receive count values ​​of each sensor; wherein the difference between the first receive count value and the second receive count value is greater than a counting threshold; if there is a first receive count value and a second receive count value, it is determined that the time synchronization condition is met.

[0091] In one embodiment, the synchronization determination module 602 is further configured to:

[0092] If neither the first nor the second receive count value exists, then check whether the sum of the receive count values ​​of each sensor meets the preset sum condition; if it does, then it is determined that the time synchronization condition is met.

[0093] In one embodiment, the count update module 601 is specifically used for:

[0094] Increment the receive count value of the target sensor by 1; after time synchronization processing of multiple target sensor data, decrement the receive count value of the sensor corresponding to each target sensor data in the same receive batch by 1.

[0095] In one embodiment, the synchronization determination module 602 is specifically used for:

[0096] Check whether the sum of the received count values ​​of each sensor is equal to the number of sensors.

[0097] In one embodiment, the data synchronization module 603 is specifically used for:

[0098] The difference between the reception times of multiple target sensor data is calculated; the target sensor data corresponding to the reception times with a difference less than the interval threshold are grouped into the same batch.

[0099] Each module in the aforementioned multi-sensor data time synchronization device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0100] Figure 7 This is a block diagram illustrating an electronic device 1300 according to an exemplary embodiment. The electronic device 1300 may be a vehicle infotainment system, mobile phone, tablet device, personal digital assistant, etc. The electronic device 1300 may include one or more of the following components: a processing component 1302, a memory 1304, a power supply component 1306, a multimedia component 1308, an audio component 1310, an input / output (I / O) interface 1312, a sensor component 1314, and a communication component 1316. The memory stores computer programs or instructions that run on the processor.

[0101] Processing component 1302 typically controls the overall operation of electronic device 1300, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1302 may include one or more processors 1320 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1302 may include one or more modules to facilitate interaction between processing component 1302 and other components. For example, processing component 1302 may include a multimedia module to facilitate interaction between multimedia component 1308 and processing component 1302.

[0102] Memory 1304 is configured to store various types of data to support the operation of electronic device 1300. Examples of such data include instructions for any application or method operating on electronic device 1300, contact data, phonebook data, messages, pictures, videos, etc. Memory 1304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0103] Power supply component 1306 provides power to various components of electronic device 1300. Power supply component 1306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 1300.

[0104] Multimedia component 1308 includes a touch display screen that provides an output interface between the electronic device 1300 and the user. In some embodiments, the touch display screen may include a liquid crystal display (LCD) and a touch panel (TP). The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of a touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 1308 includes a front-facing camera and / or a rear-facing camera. When the electronic device 1300 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0105] Audio component 1310 is configured to output and / or input audio signals. For example, audio component 1310 includes a microphone (MIC) configured to receive external audio signals when electronic device 1300 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1304 or transmitted via communication component 1316. In some embodiments, audio component 1310 also includes a speaker for outputting audio signals.

[0106] I / O interface 1312 provides an interface between processing component 1302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0107] Sensor assembly 1314 includes one or more sensors for providing state assessments of various aspects of electronic device 1300. For example, sensor assembly 1314 may detect the on / off state of electronic device 1300, the relative positioning of components such as the display and keypad of electronic device 1300, changes in position of electronic device 1300 or a component of electronic device 1300, the presence or absence of user contact with electronic device 1300, the orientation or acceleration / deceleration of electronic device 1300, and temperature changes of electronic device 1300. Sensor assembly 1314 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1314 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1314 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0108] Communication component 1316 is configured to facilitate wired or wireless communication between electronic device 1300 and other devices. Electronic device 1300 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 1316 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1316 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0109] In an exemplary embodiment, the electronic device 1300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform any of the above-described multi-sensor data time synchronization methods.

[0110] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0111] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1304 including instructions, which can be executed by a processor 1320 of an electronic device 1300 to complete any of the aforementioned multi-sensor data time synchronization methods. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0112] In an exemplary embodiment, a computer program product is also provided. When executed by a processor, this computer program can implement any of the above-described methods for time synchronization of multi-sensor data. The computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, some or all of the methods described above can be implemented, in whole or in part, according to the processes or functions described in the embodiments of this disclosure.

[0113] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0114] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for time synchronization of multi-sensor data, characterized in that, The method includes: After receiving sensing data sent by a target sensor from a plurality of sensors, the receive count value of the target sensor is updated. The target sensor is any one of the plurality of sensors. After the update process, it is detected whether there is a first receive count value and a second receive count value among the receive count values ​​of each sensor; wherein the difference between the first receive count value and the second receive count value is greater than a counting threshold. If the first receive count value and the second receive count value exist, it is determined that the preset time synchronization condition is met; if the first receive count value and the second receive count value do not exist, it is detected whether the sum of the receive count values ​​of each sensor meets the preset sum value condition. If it does, it is determined that the preset time synchronization condition is met. If the preset time synchronization conditions are met, the multiple target sensor data that have not been time synchronized are time synchronized according to their reception time, so as to classify the batch to which the multiple target sensor data belong.

2. The method according to claim 1, characterized in that, The process of updating the received count value of the target sensor includes: Increment the receive count value of the target sensor by 1; After performing time synchronization processing on the multiple target sensing data, the method further includes: Decrement the receive count value of the sensor corresponding to each of the target sensing data that is divided into the same receiving batch by 1.

3. The method according to claim 1, characterized in that, The step of detecting whether the sum of the received count values ​​of each of the sensors meets a preset sum condition includes: The sum of the received counts of each of the sensors is checked to see if it equals the number of the plurality of sensors.

4. The method according to any one of claims 1 to 3, characterized in that, The step of performing time synchronization processing on the multiple target sensor data based on the reception time of the multiple target sensor data that have not been time synchronized includes: The difference between each pair of reception times of the multiple target sensor data is processed. Target sensing data corresponding to reception times with a difference less than the interval threshold are grouped into the same batch.

5. A multi-sensor data synchronization device, characterized in that, The device includes: The counting update module is used to update the received count value of the target sensor after receiving the sensing data sent by the target sensor among the multiple sensors. The target sensor is any one of the multiple sensors. The synchronization determination module is used to detect, after the update process, whether there is a first receiving count value and a second receiving count value among the receiving count values ​​of each sensor; wherein the difference between the first receiving count value and the second receiving count value is greater than a counting threshold; if the first receiving count value and the second receiving count value exist, it is determined that a preset time synchronization condition is met; if the first receiving count value and the second receiving count value do not exist, it is detected whether the sum of the receiving count values ​​of each sensor meets a preset sum value condition; if it does, it is determined that the preset time synchronization condition is met. The data synchronization module is used to perform time synchronization processing on the multiple target sensor data that have not been time synchronized, based on the reception time of the multiple target sensor data that have not been time synchronized, under the premise that the preset time synchronization conditions are met, so as to classify the batch to which the multiple target sensor data belong.

6. The apparatus according to claim 5, characterized in that, The count update module is also used for: Increment the receive count value of the target sensor by 1; After performing time synchronization processing on the multiple target sensor data, Decrement the receive count value of the sensor corresponding to each of the target sensing data that is divided into the same receiving batch by 1.

7. The apparatus according to claim 5, characterized in that, The synchronization determination module is also used for: The sum of the received counts of each of the sensors is checked to see if it equals the number of the plurality of sensors.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images