Clock synchronization processing method and device, computer readable medium and electronic equipment

Abstract

The embodiment of the invention provides a clock synchronization processing method and device, a computer readable medium and electronic equipment. The clock synchronization processing method is applied to cascade equipment, the cascade equipment is connected between sensing equipment and a clock server, and the clock synchronization processing method comprises the following steps: performing clock synchronization processing with the clock server; obtaining time delay information between the first sensing equipment and the cascade equipment; when sensing data output by the first sensing device is received, acquiring a local timestamp of the cascade device; and according to the time delay information and the local timestamp, predicting an output timestamp of the sensing data, and associating the output timestamp with the sensing data. According to the technical scheme, even if the sensing device does not have a time synchronization function or does not support a hardware clock function of a PTP protocol, it can be guaranteed that the sensing data can be added with an accurate timestamp, and the requirement for low-cost clock synchronization is met.

Description

Technical Field

[0001] The present application relates to the field of computer and communication technology, and in particular to a clock synchronization processing method, device, computer-readable medium and electronic device. Background Art

[0002] TSN (Time Sensitive Network) is a network technology defined by IEEE (Institute of Electrical and Electronics Engineers) to achieve time synchronization and deterministic communication between nodes. It is mainly used for real-time communication and network control. And TSN's time-sensitive communication (TSC) is also introduced in the 5G (5th Generation) system to enable the 5G system to support industrial automation manufacturing applications with precise time control. However, in related technologies, additional hardware devices need to be introduced for clock synchronization processing, which undoubtedly increases the hardware cost and implementation complexity. Summary of the invention

[0003] The embodiments of the present application provide a clock synchronization processing method, apparatus, computer-readable medium and electronic device, so that even if the perception device does not have a time synchronization function or does not support the hardware clock function of the PTP protocol, it can ensure that the perception data can be added with an accurate timestamp, thereby meeting the demand for low-cost clock synchronization.

[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0005] In the first aspect, an embodiment of the present application provides a clock synchronization processing method, which is applied to a cascade device, and the cascade device is connected between a perception device and a clock server. The clock synchronization processing method includes: performing clock synchronization processing with the clock server; obtaining delay information between a first perception device and the cascade device; when receiving perception data output by the first perception device, obtaining a local timestamp of the cascade device; predicting an output timestamp of the perception data based on the delay information and the local timestamp, and associating the output timestamp with the perception data.

[0006] In the second aspect, an embodiment of the present application provides a clock synchronization processing method, which is applied to a perception device, and the clock synchronization processing method includes: establishing a communication connection with a cascade device, the cascade device is connected between the perception device and a clock server, and clock synchronization processing is performed between the cascade device and the clock server; collecting data of an object to be perceived to obtain perception data; sending the perception data to the cascade device, so that the cascade device predicts the output timestamp of the perception data based on the delay information between the perception device and the cascade device, and the local timestamp when the cascade device receives the perception data, and associates the output timestamp with the perception data.

[0007] In the third aspect, an embodiment of the present application provides a clock synchronization processing device, which is applied to a cascade device, and the cascade device is connected between a perception device and a clock server. The clock synchronization processing device includes: a synchronization unit, configured to perform clock synchronization processing with the clock server; an acquisition unit, configured to acquire delay information between a first perception device and the cascade device, and to acquire a local timestamp of the cascade device when receiving perception data output by the first perception device; a processing unit, configured to predict an output timestamp of the perception data based on the delay information and the local timestamp, and to associate the output timestamp with the perception data.

[0008] In the fourth aspect, an embodiment of the present application provides a clock synchronization processing device, which is applied to a perception device, and the clock synchronization processing device includes: a connection establishment unit, configured to establish a communication connection with a cascade device, the cascade device is connected between the perception device and a clock server, and clock synchronization processing is performed between the cascade device and the clock server; a collection unit, configured to collect data of an object to be perceived to obtain perception data; a sending unit, configured to send the perception data to the cascade device, so that the cascade device predicts the output timestamp of the perception data based on the delay information between the perception device and the cascade device, and the local timestamp when the cascade device receives the perception data, and associates the output timestamp with the perception data.

[0009] In a fifth aspect, an embodiment of the present application provides a computer-readable medium on which a computer program is stored. When the computer program is executed by a processor, the clock synchronization processing method described in the above embodiment is implemented.

[0010] In a sixth aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; a storage device for storing one or more computer programs, wherein when the one or more computer programs are executed by the one or more processors, the electronic device implements the clock synchronization processing method as described in the above embodiments.

[0011] In a seventh aspect, an embodiment of the present application provides a computer program product, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of an electronic device reads and executes the computer program from the computer-readable storage medium, so that the electronic device executes the clock synchronization processing method provided in the above-mentioned various optional embodiments.

[0012] In the technical solutions provided in some embodiments of the present application, the cascade device is connected between the perception device and the clock server, and the cascade device can perform clock synchronization processing with the clock server, and the cascade device can predict the output timestamp of the perception data based on the delay information between the cascade device and the first perception device, and the local timestamp when the cascade device receives the perception data when receiving the perception data output by the first perception device, and associate the output timestamp with the perception data. It can be seen that the technical solution of the embodiments of the present application enables the cascade device to perform clock synchronization on behalf of the perception device, so that even if the perception device does not have a time synchronization function or does not support the hardware clock function of the PTP protocol, it can also ensure that the perception data can add an accurate timestamp, avoiding the hardware cost and complexity of installing additional hardware devices on the perception device to implement clock synchronization processing, and meeting the demand for low-cost clock synchronization.

[0013] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Schematic diagram showing the architecture of the 5G system supporting time synchronization in the external time domain

[0015] Figure 2 A schematic diagram showing an exemplary system architecture to which the technical solution of the embodiments of the present application can be applied;

[0016] Figure 3 A flowchart of a clock synchronization processing method according to an embodiment of the present application is shown;

[0017] Figure 4 A flowchart of a clock synchronization processing method according to an embodiment of the present application is shown;

[0018] Figure 5A schematic diagram of a clock synchronization process according to an embodiment of the present application is shown;

[0019] Figure 6 A schematic diagram of a clock synchronization process according to an embodiment of the present application is shown;

[0020] Figure 7 A schematic diagram of a clock synchronization process according to an embodiment of the present application is shown;

[0021] Figure 8 A block diagram of a clock synchronization processing device according to an embodiment of the present application is shown;

[0022] Fig. 9 A block diagram of a clock synchronization processing device according to an embodiment of the present application is shown;

[0023] Fig.10 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] The exemplary embodiments are now described in a more comprehensive manner with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be understood as being limited to these examples; on the contrary, the purpose of providing these embodiments is to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0025] In addition, the features, structures or characteristics described in the present application may be combined in one or more embodiments in any suitable manner. In the following description, there are many specific details so that the embodiments of the present application can be fully understood. However, those skilled in the art will appreciate that when implementing the technical scheme of the present application, all the detailed features in the embodiments may not be needed, one or more specific details may be omitted, or other methods, elements, devices, steps, etc. may be adopted.

[0026] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0027] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0028] It should be noted that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0029] With the development of technology, TSN time-sensitive communication is introduced in 5G systems to enable 5G systems to support industrial automation manufacturing applications with precise time control. Figure 1 As shown, in a system architecture where the 5G system is integrated with TSN, the 5G system is integrated into the TSN system as a TSN bridge. This "logical" TSN bridge includes a TSN converter for user plane interaction between the TSN system and the 5G system. The 5GS (5G system) TSN converter function includes DS-TT (DeviceSide TSN Translator) and NW-TT (Network TSN Translator). The UE (User Equipment) in the 5G system is connected to one or more ES (End Station, which is a device connected to a local area network or a metropolitan area network, acting as the source and / or destination of traffic carried on the local area network or the metropolitan area network) in the TSN DN (Data Network) outside the 5G system through DS-TT. The UPF (User Plane Function) is connected to one or more ES in the TSN DN (Data Network) through NW-TT.

[0030] In order to implement the TSN synchronization mechanism, the entire end-to-end 5G system can be regarded as an IEEE 802.1AS time aware system. Figure 1 As shown in the figure, there are two time synchronization domains, namely the 5G time domain and the TSN time domain. The 5G system has its own time system (such as GPS (Global Positioning System) time). Figure 1 5G GM (5G Grand Master, 5G internal master clock) is used to represent the clock domain of the 5G system (i.e., the 5G time domain).

[0031] Figure 1The gNB in ​​the 5G system represents a 5G base station. All devices in the 5G system are synchronized to the 5G clock domain. The devices in the 5G system include: UPF, SMF (Session Management Function), UE (User Equipment), DS-TT, NW-TT, NG RAN (NG Radio Access Network), etc. The NG interface is the interface between the radio access network and the 5G core network.

[0032] like Figure 1 As shown in the figure, the clock source of the external TSN Time Domain is outside the UPF. The ES device on the UE side of TSN is connected to the UE through DS-TT, accessed to the 5G network, and then accessed to the external TSN network through the UPF and the NW-TT on the UPF to synchronize time with the TSN clock source.

[0033] Figure 1 In the 5G network, the time synchronization message is sent by TSN GM through downlink (DL) data, that is, it is sent through the user plane of the UE. The downlink data including the time synchronization message of TSN GM first reaches the NW-TT / UPF, enters the 5G system, then reaches the UE and its DS-TT, and finally reaches the ES on the UE side.

[0034] TSN GM marks its current time in the originTimestamp field of the time synchronization message it sends. When the user plane of UE transmits this time synchronization message, NW-TT marks the reception time of the received downlink data on this time synchronization message, and updates the CorrectionField field value in this data packet to the original CorrectionField field value plus the transmission delay value between NW-TT and the Ethernet Bridge Port (port) that sends this message to NW-TT. Before forwarding the downlink data to ES, DS-TT subtracts the reception time added by NW-TT on the downlink data from its current time to obtain the transmission delay value of this time synchronization message in the entire 5G system, and accumulates this delay value with the previous transmission time from TSN master clock to NW-TT (in the CorrectionField field of the received time synchronization message) to obtain the updated transmission delay value, and marks this updated transmission delay value in the CorrectionField field of the time synchronization message (i.e. the total transmission delay of this message), and deletes the reception time marked by NW-TT before. Then this modified time synchronization message is sent to ES.

[0035] ES adds the delay value on the time synchronization message directly to the transmission delay between DS-TT and ES based on the delay value marked on the time synchronization message by DS-TT (i.e., the CorrectionField field value on the time synchronization message), and the total transmission delay of the time synchronization message from the TSN master clock to the ES can be obtained. This total transmission delay plus the originTimestamp field value on the time synchronization message can obtain a calculated time value, and then its clock is set to this calculated time value, thereby realizing the time synchronization between the ES and the TSN GM on the UE side.

[0036] In the above-mentioned time synchronization process, it is necessary to add specific hardware and network equipment, such as TSN converters. This requirement for hardware or special equipment limits the flexibility of actual deployment and raises the cost. At the same time, IEEE also proposed the PTP (Precision Time Protocol) protocol for synchronizing time on the network. Although the accuracy of PTP can reach the sub-microsecond level, it is also necessary to deploy specific hardware equipment, such as PTP switches. Based on this, the technical solution of the embodiment of the present application proposes a new clock synchronization processing method, so that even if the perception device does not have the time synchronization function or does not support the hardware clock function of the PTP protocol, it can ensure that the perception data can add an accurate timestamp, meeting the demand for low-cost clock synchronization.

[0037] Specifically, in Figure 2 In the system architecture shown, the sensing device 201 and the sensing device 202 are in communication connection with the cascade device 203, and the cascade device 203 is connected with the clock server 204. Among them, the cascade device 203 and the clock server can perform clock synchronization processing based on the time synchronization protocol, and the cascade device 203 can obtain the delay information between the sensing device 201 (taking the sensing device 201 as an example for explanation). When the cascade device 203 receives the sensing data output by the sensing device 201, the local timestamp of the cascade device 203 can be obtained, and then the cascade device 203 can predict the output timestamp of the sensing data based on the delay information between the sensing device 201 and the local timestamp obtained, and then the predicted output timestamp can be associated with the received sensing data.

[0038] In some optional embodiments, any time synchronization protocol may be used for clock synchronization processing between the cascade device 203 and the clock server 204, such as the PTP protocol or the NTP (Network Time Protocol) protocol. Alternatively, the LW-PTP (Light-Weight PTP) protocol proposed in the embodiment of the present application may be used. It should be noted that the LW-PTP protocol proposed in the embodiment of the present application may be implemented in a pure software manner, does not need to rely on dedicated hardware devices, and can achieve clock synchronization by running on an IP data network, which will be described in detail in the following embodiments.

[0039] In some optional embodiments, the cascade device 203 can obtain the delay information between the cascade device 203 and the perception device 201 in a statistical manner. For example, the cascade device 203 can send a request data packet to the perception device 201, and then receive a response data packet returned by the perception device 201 based on the request data packet. Then, the cascade device 203 can calculate the delay information between the perception device 201 and the cascade device 203 according to the sending time of the request data packet and the receiving time of the response data packet. For example, the time difference between the receiving time of the response data packet and the sending time of the request data packet is calculated, and then half of the time difference is used as the delay information between the perception device 201 and the cascade device 203.

[0040] It should be noted that: the sensing device can be a smart phone, tablet computer, laptop computer, desktop computer, smart TV, smart home, vehicle-mounted terminal, aircraft, camera, etc.; the clock server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0041] It should be understood that Figure 2 The number of sensing devices, cascade devices and clock servers shown in the figure is only for illustration. Any number of sensing devices, cascade devices and clock servers may be provided according to implementation requirements.

[0042] The implementation details of the technical solution of the embodiment of the present application are described in detail below:

[0043] Figure 3A flowchart of a clock synchronization processing method according to an embodiment of the present application is shown. The clock synchronization processing method can be executed by a cascade device connected between a sensing device and a clock server. Figure 3 As shown, the clock synchronization processing method at least includes S310 to S340, which are described in detail as follows:

[0044] In S310, clock synchronization processing is performed with a clock server.

[0045] In some optional embodiments, any time synchronization protocol may be used when performing clock synchronization between the cascade device and the clock server, such as the PTP protocol or the NTP protocol. Alternatively, the LW-PTP protocol proposed in the embodiment of the present application may be used, for example, based on a lightweight application for clock synchronization installed on the cascade device, clock synchronization may be performed with the clock server through a lightweight time synchronization protocol (i.e., LW-PTP).

[0046] In some optional embodiments, the process of synchronization between the cascade device and the clock server through the LW-PTP protocol can be implemented through an IP data network. Specifically, assuming that the communication link between the cascade device and the clock server is a symmetrical link, the local timestamp when the cascade device sends data packet 1 is t1, and the local timestamp when the clock server receives data packet 1 is t2. The local timestamp when the clock server sends data packet 2 is t3, and the local timestamp when the cascade device receives data packet 2 is t4; at the same time, assuming that the clock deviation between the cascade device and the clock server is x, then the local timestamp when the clock server theoretically receives data packet 1 is t2'=t2+x, and the local timestamp when the clock server theoretically sends data packet 2 is t3'=t3+x. Considering that the cascade device and the clock server are symmetrical links, t2'-t1=t4-t3', substituting t2'=t2+x and t3'=t3+x into the equation, we can get: t2-t1+x=t4-t3-x, and then we can obtain the clock deviation x=1 / 2[(t4-t3)-(t2-t1)] between the cascade device and the clock server. After calculating the clock deviation between the cascade device and the clock server, the clock of the cascade device can be synchronized to the clock of the clock server. It can be seen that the LW-PTP protocol proposed in the embodiment of the present application can be implemented in a pure software manner, without relying on dedicated hardware devices, effectively reducing the cost and complexity of clock synchronization.

[0047] In S320, delay information between the first sensing device and the cascade device is obtained.

[0048] In some optional embodiments, the first perception device may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart TV, a smart home, a vehicle terminal, an aircraft, a camera, etc. For example, in a smart transportation or smart driving assistance scenario, the first perception device may be a camera device or a laser radar device installed beside the road.

[0049] In some optional embodiments, the process of the cascade device acquiring the delay information between the first perception device and the cascade device may be: sending a request data packet to the first perception device, receiving a response data packet returned by the first perception device based on the request data packet, and then calculating the delay information between the first perception device and the cascade device according to the sending time of the request data packet and the receiving time of the response data packet. For example, the time difference between the receiving time of the response data packet and the sending time of the request data packet may be calculated, and then half of the time difference may be used as the delay information between the perception device and the cascade device.

[0050] It should be noted that multiple calculations can be performed by sending request data packets multiple times and receiving response data packets, and then the results of the multiple calculations are averaged, and the averaged result is used as the delay information between the first perception device and the cascade device. This can avoid the problem of inaccurate delay information calculation caused by network jitter.

[0051] In some optional embodiments, the process of the cascade device obtaining the delay information between the first perception device and the cascade device may be: obtaining a mapping relationship between a network status parameter and the delay information, where the network status parameter is used to represent the network status between the cascade device and the first perception device; when receiving the perception data output by the first perception device, obtaining the current network status parameter between the first perception device and the cascade device; and then determining the delay information between the first perception device and the cascade device based on the current network status parameter and the mapping relationship.

[0052] In some optional embodiments, the network status parameter may be various parameters used to represent the network status between the first sensing device and the cascade device, such as the uplink transmission delay between the first sensing device and the cascade device, the downlink transmission delay between the first sensing device and the cascade device, the total uplink and downlink transmission delay between the first sensing device and the cascade device, and the delay jitter parameter between the first sensing device and the cascade device. Optionally, the delay jitter parameter may be, for example, the standard deviation corresponding to the delay parameter (such as the uplink transmission delay, the downlink transmission delay, or the total uplink and downlink transmission delay) obtained by multiple measurements.

[0053] In some optional embodiments, the process of obtaining the mapping relationship between the network state parameters and the delay information may be obtained by statistical means. Specifically, for example, the historical delay information of the sensing device and the cascade device under various historical network state parameters during the historical clock synchronization process may be obtained, and then the mapping relationship between the network state parameters and the delay information may be determined based on the historical delay information of the sensing device and the cascade device under various historical network state parameters.

[0054] In S330, when the perception data output by the first perception device is received, the local timestamp of the cascade device is obtained.

[0055] Optionally, the local timestamp of the cascade device is the time value of the local clock of the cascade device. The local timestamp when receiving the perception data output by the first perception device is the reception time of the perception data output by the cascade device to the first perception device.

[0056] In S340, the output timestamp of the perception data is predicted according to the delay information between the first perception device and the cascade device and the local timestamp when the cascade device receives the perception data, and the output timestamp is associated with the perception data.

[0057] In some optional embodiments, the process of predicting the output timestamp of the perception data according to the delay information and the local timestamp may specifically include: calculating the time difference between the local timestamp and the delay value represented by the delay information, and using the time difference as the output timestamp of the perception data. After obtaining the output timestamp of the perception data, the output timestamp is associated with the perception data, so that the perception data can carry accurate time information, so as to facilitate business analysis.

[0058] In some optional embodiments, for a perception device that can perform clock synchronization with a cascade device through a time synchronization protocol (referred to as a second perception device for ease of distinction), the cascade device and the second perception device can perform clock synchronization processing, and then after the second perception device performs clock synchronization processing, the timestamp of the output perception data can be set according to the clock after synchronization processing. In this case, the first clock deviation between the second perception device and the cascade device, and the second clock deviation between the cascade device and the clock server can be obtained, and then the clock deviation between the second perception device and the clock server is calculated based on the first clock deviation and the second clock deviation. For example, the clock deviation between the second perception device and the clock server is the sum of the first clock deviation and the second clock deviation, so when it is necessary to control the clock deviation, it is possible to ensure that the time deviation meets the requirements by controlling at least one of the first clock deviation and the second clock deviation.

[0059] Optionally, the process of obtaining the first clock deviation between the second sensing device and the cascade device may also adopt the technical solution of calculating the clock deviation between the cascade device and the clock server in the aforementioned embodiment, that is, estimating by counting the sending time and receiving time of the statistical data packet.

[0060] Figure 3 The embodiment shown is an explanation of the technical solution of the embodiment of the present application from the perspective of the cascade device. Figure 4 From the perspective of the sensing device, the implementation details of the technical solution of the embodiment of the present application are explained again:

[0061] Figure 4 A flowchart of a clock synchronization processing method according to an embodiment of the present application is shown, and the clock synchronization processing method can be executed by a sensing device. Figure 4 As shown, the clock synchronization processing method at least includes S410 to S430, which are described in detail as follows:

[0062] In S410, a communication connection is established with a cascade device, which is connected between the sensing device and the clock server, and clock synchronization processing is performed between the cascade device and the clock server.

[0063] In some optional embodiments, the cascade device may be a MEC (Mobile Edge Computing) server, or other device capable of implementing clock synchronization processing.

[0064] In an embodiment of the present application, after the perception device establishes a communication connection with the cascade device, the cascade device can determine the output timestamp of the perception data based on the delay information between the perception device and the cascade device. Therefore, in order to ensure that the delay information between the perception device and the cascade device tends to be stable and no large jitter occurs, a communication connection can be established between the perception device and the cascade device through a light-load link. The light-load link is a link in which the physical bandwidth of the link and the amount of data transmitted by the link are less than a set threshold.

[0065] In S420, data of the object to be sensed is collected to obtain sensed data.

[0066] In some optional embodiments, the perception object is the object that the perception device needs to perceive. For example, the perception device may be a camera installed next to the road, and the perception object may be a traffic participant, such as a pedestrian, a vehicle, etc.

[0067] In S430, the perception data is sent to the cascade device, so that the cascade device predicts the output timestamp of the perception data according to the delay information between the perception device and the cascade device and the local timestamp when the cascade device receives the perception data, and associates the output timestamp with the perception data.

[0068] In some optional embodiments, the cascade device may calculate the time difference between the local timestamp when the perception data is received and the delay value represented by the delay information, and use the time difference as the output timestamp of the perception data.

[0069] In some optional embodiments, if the sensing device can perform clock synchronization processing with the cascade device through a time synchronization protocol, the sensing device can perform clock synchronization processing with the cascade device and set the timestamp of the output sensing data according to the synchronized clock. In this case, the Figure 4 If the sensing device cannot directly synchronize the clock with the cascade device through the time synchronization protocol, then it can execute Figure 4 S430 shown.

[0070] In some optional embodiments, if an application for clock synchronization is installed on the perception device, the perception device can perform clock synchronization processing with the clock server based on the application through a lightweight time synchronization protocol, and then set the timestamp of the output perception data according to the synchronized clock. In this case, the execution of Figure 4 S430 shown. Optionally, the application may be a lightweight application based on the LW-PTP protocol.

[0071] The following reference Figures 5 to 7 , taking the perception device as a camera and the cascade device as an MEC server as an example, combined with specific application scenarios such as intelligent traffic perception and assisted driving, the implementation details of the technical solution of the embodiment of the present application are elaborated in detail:

[0072] like Figure 5 As shown, camera 501, camera 502 and camera 503 are all installed with LW-PTP client, that is, the application client supporting the lightweight time synchronization protocol in the embodiment of the present application. And MEC server 504 is also installed with LW-PTP client, and clock server 505 serves as the server of LW-PTP. In this scenario, the perception device (i.e. camera 501, camera 502 and camera 503) with LW-PTP client installed can be directly synchronized with clock server 505. Or the perception device (i.e. camera 501, camera 502 and camera 503) with LW-PTP client installed can be synchronized with MEC server 504, and MEC server 504 is synchronized with clock server 505.

[0073] like Figure 6As shown, the camera 601 and the MEC server 604 are installed with an LW-PTP client, that is, an application client that supports the lightweight time synchronization protocol in the embodiment of the present application, and the clock server 605 serves as the server of LW-PTP. At the same time, the camera 602 has NTP synchronization capability, and the camera 603 has PTP synchronization capability. In this scenario, the camera 601 with the LW-PTP client installed can directly synchronize the clock with the clock server 605. Or the camera 601 with the LW-PTP client installed can synchronize the clock with the MEC server 604, and the MEC server 604 synchronizes the clock with the clock server 605.

[0074] Cameras 602 and 603 that do not have the LW-PTP client installed can perform clock synchronization through cascade synchronization. Specifically, MEC server 604 first achieves clock synchronization with clock server 605 through the LW-PTP client to ensure that the synchronization accuracy of MEC server 604 and clock server 605 reaches the millisecond level, and then camera 602 and camera 603 use NTP and PTP to synchronize the clock with MEC server 604 respectively. In this case, the clock deviation (i.e., clock error) between MEC server 604 and clock server 605 can be ensured to be no greater than C1, and the clock error between camera 602 and camera 603 and MEC server 604 can be no greater than C2, and the clock error C3 between camera 602 and camera 603 and clock server 605 is the result of the superposition of C1 and C2. Therefore, as long as C1 and C2 are controlled within a certain synchronization error range, the synchronization error of C3 can be achieved to meet the needs of the business (such as smart transportation applications). It can be seen that Figure 6 The technical solution of the illustrated embodiment can realize clock synchronization processing by cascading with the aid of a sensing device that supports the NTP or PTP protocol. In this way, since the clock error between the MEC server 604 and the clock server 605 is ensured to be low through the LW-PTP protocol, the clock deviation between the sensing device and the clock server 605 can be reduced as a whole, and there is no need to perform software upgrades on various sensors.

[0075] like Figure 7As shown, MEC server 704 has installed LW-PTP client, that is, the application client supporting the lightweight time synchronization protocol in the embodiment of the present application, and clock server 705 serves as the server of LW-PTP. However, camera 701, camera 702 and camera 703 have neither installed LW-PTP client nor NTP and PTP synchronization capabilities. In this scenario, MEC server 704 first synchronizes the clock with clock server 705 through LW-PTP client to ensure that the synchronization accuracy of MEC server 704 and clock server 705 reaches millisecond level. The MEC server 704 can obtain the transmission delay between the camera and the camera 701. Taking the camera 701 as an example, the MEC server 704 can send a request data packet to the camera 701. After receiving the request data packet, the camera 701 can return a response data packet. Then the MEC server 704 can capture the response data packet and analyze the delay characteristics between the MEC server 704 and the camera 701 by calculating the time difference between the reception time of the response data packet and the transmission time of the request data packet, and then calculate the average delay between the MEC server 704 and the camera 701. Alternatively, the transmission delay at different times or in different network states can be obtained by statistically analyzing historical delay data.

[0076] After obtaining the transmission delay between the MEC server 704 and the camera 701, if the MEC server 704 receives the perception data packet (such as pictures, video frames, point clouds, etc.) sent by the camera 701, the MEC server 704 can receive the timestamp and the transmission delay to reversely obtain the sending timestamp of the perception data packet, and then configure the sending timestamp to the perception data packet. Assuming that the synchronization accuracy between the MEC server 704 and the camera 701 is d2, since d2 can reach the millisecond level, the accuracy of clock synchronization between the MEC server 704 and the clock server 705 through the LW-PTP protocol can reach the microsecond level, so the synchronization accuracy between the camera 701 and the clock server 705 is also in the millisecond level. Since the transmission delay is affected by the amount of data, in order to ensure that the transmission delay between the MEC server 704 and the camera 701 is stable, a light-load link can be configured between the MEC server 704 and the camera 701, or it can be configured when the MEC server 704 detects that the average network delay jitter between the camera 701 and the camera is too large. This can avoid excessive delay causing excessive clock back-calculation errors.

[0077] exist Figures 5 to 7In the technical solution of the embodiment shown, cascading can be implemented based on heterogeneous synchronization mechanisms to solve clock synchronization problems in scenarios such as large-scale intelligent traffic perception + communication fusion systems. And different synchronization methods can be adopted according to different types of sensors (i.e., whether LW-PTP client is installed, whether it has NTP capability, whether it has PTP capability, etc.), to solve the current problems of low NTP synchronization accuracy and non-support of synchronization protocols in sensors, and improve the overall performance of the fusion perception system.

[0078] It should be noted that in actual application scenarios, there may be perception devices with LW-PTP clients installed, perception devices with NTP or PTP capabilities, and perception devices that have neither LW-PTP clients installed nor NTP and PTP capabilities. In this case, when performing clock synchronization processing, the above can be combined. Figures 5 to 7 The technical solution of the illustrated embodiment selects a corresponding synchronization method for synchronization processing according to the actual capability of the sensing device.

[0079] The following describes an apparatus embodiment of the present application, which can be used to execute the clock synchronization processing method in the above-mentioned embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the above-mentioned embodiment of the clock synchronization processing method of the present application.

[0080] Figure 8 A block diagram of a clock synchronization processing device according to an embodiment of the present application is shown. The clock synchronization processing device is applied to a cascade device, which is connected between a sensing device and a clock server.

[0081] Reference Figure 8 As shown, a clock synchronization processing device 800 according to an embodiment of the present application includes: a synchronization unit 802 , an acquisition unit 804 and a processing unit 806 .

[0082] Among them, the synchronization unit 802 is configured to perform clock synchronization processing with the clock server; the acquisition unit 804 is configured to obtain the delay information between the first perception device and the cascade device, and when receiving the perception data output by the first perception device, obtain the local timestamp of the cascade device; the processing unit 806 is configured to predict the output timestamp of the perception data based on the delay information and the local timestamp, and associate the output timestamp with the perception data.

[0083] In some embodiments of the present application, based on the aforementioned scheme, the acquisition unit 804 is configured to: send a request data packet to the first perception device, and receive a response data packet returned by the first perception device based on the request data packet; calculate the delay information between the first perception device and the cascade device according to the sending time of the request data packet and the receiving time of the response data packet.

[0084] In some embodiments of the present application, based on the aforementioned scheme, the acquisition unit 804 is configured to: obtain a mapping relationship between network status parameters and delay information, wherein the network status parameters are used to represent the network status between the cascade device and the first perception device; when receiving the perception data output by the first perception device, obtain the current network status parameters between the first perception device and the cascade device; and determine the delay information between the first perception device and the cascade device based on the current network status parameters and the mapping relationship.

[0085] In some embodiments of the present application, based on the aforementioned scheme, the processing unit 806 is configured to: calculate the time difference between the local timestamp and the delay value represented by the delay information, and use the time difference as the output timestamp of the perception data.

[0086] In some embodiments of the present application, based on the aforementioned scheme, the synchronization unit 802 is also configured to: perform clock synchronization processing with a second perception device so that the second perception device sets a timestamp of the perception data output according to the synchronized clock; wherein the second perception device is a perception device capable of performing clock synchronization with the cascade device through a time synchronization protocol.

[0087] In some embodiments of the present application, based on the aforementioned scheme, the processing unit 806 is also configured to: obtain a first clock deviation between the second perception device and the cascade device, and a second clock deviation between the cascade device and the clock server; calculate the clock deviation between the second perception device and the clock server based on the first clock deviation and the second clock deviation.

[0088] In some embodiments of the present application, based on the aforementioned solution, the synchronization unit 802 is configured to: perform clock synchronization processing with the clock server through a lightweight time synchronization protocol based on an application for clock synchronization installed on the cascade device.

[0089] Fig. 9 A block diagram of a clock synchronization processing apparatus according to an embodiment of the present application is shown, where the clock synchronization processing apparatus is applied to a sensing device.

[0090] Reference Fig. 9As shown, a clock synchronization processing device 900 according to an embodiment of the present application includes: a connection establishing unit 902 , a collection unit 904 and a sending unit 906 .

[0091] Among them, the connection establishment unit 902 is configured to establish a communication connection with a cascade device, the cascade device is connected between the perception device and the clock server, and the cascade device and the clock server perform clock synchronization processing; the collection unit 904 is configured to collect data of the object to be perceived to obtain perception data; the sending unit 906 is configured to send the perception data to the cascade device, so that the cascade device predicts the output timestamp of the perception data based on the delay information between the perception device and the cascade device, and the local timestamp when the cascade device receives the perception data, and associates the output timestamp with the perception data.

[0092] In some embodiments of the present application, based on the aforementioned scheme, the clock synchronization processing device 900 also includes: a processing unit, configured to perform clock synchronization with the cascade device if the perception device can perform clock synchronization with the cascade device through a time synchronization protocol, and set the timestamp of the output perception data according to the synchronized clock; the sending unit 906 is configured to execute the process of sending the data to the cascade device if the perception device cannot directly perform clock synchronization with the cascade device through a time synchronization protocol.

[0093] In some embodiments of the present application, based on the aforementioned scheme, the clock synchronization processing device 900 also includes: a processing unit, configured to perform clock synchronization processing based on the application and the clock server through a lightweight time synchronization protocol if an application for clock synchronization is installed on the perception device; and set the timestamp of the output perception data according to the synchronized clock.

[0094] In some embodiments of the present application, based on the aforementioned scheme, the connection establishment unit 902 is configured to: establish a communication connection with the cascade device through a light-load link, and the light-load link is a link in which the physical bandwidth of the link and the amount of data transmitted by the link are less than a set threshold.

[0095] Fig.10 A schematic diagram of the structure of a computer system of an electronic device suitable for implementing an embodiment of the present application is shown. The electronic device may be the sensing device in the aforementioned embodiment, or may be a cascade device connected between the sensing device and a clock server.

[0096] It should be noted that Fig.10 The computer system 1000 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0097] like Fig.10 As shown, the computer system 1000 may include a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage part 1008 to the random access memory (RAM) 1003, such as executing the method described in the above embodiment. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, the ROM 1002 and the RAM 1003 are connected to each other through the bus 1004. The input / output (I / O) interface 1005 is also connected to the bus 1004.

[0098] The following components can be connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0099] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program is used to perform the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part 1009, and / or installed from a removable medium 1011. When the computer program is executed by a central processing unit (CPU) 1001, various functions defined in the system of the present application are executed.

[0100] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a computer program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0101] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and a computer program.

[0102] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0103] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more computer programs, and when the above one or more computer programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0104] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0105] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable an electronic device to execute the method according to the implementation methods of the present application.

[0106] For example, the electronic device may be a cascade device, and the cascade device may perform Figure 3 The clock synchronization processing method shown in FIG. 1; for example, the electronic device may be a sensing device, and the sensing device may perform Figure 4 The clock synchronization processing method shown.

[0107] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0108] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A clock synchronization processing method, characterized in that: The clock synchronization processing method is applied to a cascade device, wherein the cascade device is connected between a sensing device and a clock server, and the clock synchronization processing method includes: Perform clock synchronization with the clock server; Acquire delay information between the first sensing device and the cascade device; When receiving the perception data output by the first perception device, obtaining a local timestamp of the cascade device; An output timestamp of the perception data is predicted according to the delay information and the local timestamp, and the output timestamp is associated with the perception data.

2. The clock synchronization processing method according to claim 1, characterized in that: Acquiring delay information between the first sensing device and the cascade device, including: Sending a request data packet to the first sensing device, and receiving a response data packet returned by the first sensing device based on the request data packet; The delay information between the first sensing device and the cascade device is calculated according to the sending time of the request data packet and the receiving time of the response data packet.

3. The clock synchronization processing method according to claim 1, characterized in that: Acquiring delay information between the first sensing device and the cascade device, including: Acquire a mapping relationship between a network status parameter and delay information, wherein the network status parameter is used to represent a network status between the cascade device and the first sensing device; When receiving the perception data output by the first perception device, obtaining current network status parameters between the first perception device and the cascade device; Determine the delay information between the first sensing device and the cascade device according to the current network status parameters and the mapping relationship.

4. The clock synchronization processing method according to claim 1, characterized in that: Predicting the output timestamp of the sensing data according to the delay information and the local timestamp includes: A time difference between the local timestamp and the delay value represented by the delay information is calculated, and the time difference is used as an output timestamp of the perception data.

5. The clock synchronization processing method according to claim 1, characterized in that: The clock synchronization method further comprises: Performing clock synchronization processing with a second sensing device so that the second sensing device sets a timestamp of the output sensing data according to the synchronized clock; The second sensing device is a sensing device capable of performing clock synchronization with the cascade device through a time synchronization protocol.

6. The clock synchronization processing method according to claim 5, characterized in that: The clock synchronization method further comprises: Acquire a first clock deviation between the second sensing device and the cascade device, and a second clock deviation between the cascade device and the clock server; The clock deviation between the second sensing device and the clock server is calculated according to the first clock deviation and the second clock deviation.

7. The clock synchronization processing method according to any one of claims 1 to 6, characterized in that: Perform clock synchronization with the clock server, including: Based on the application for clock synchronization installed on the cascade device, clock synchronization processing is performed with the clock server through a lightweight time synchronization protocol.

8. A clock synchronization processing method, characterized in that: The clock synchronization processing method is applied to a sensing device, and the clock synchronization processing method includes: Establishing a communication connection with a cascade device, wherein the cascade device is connected between the sensing device and the clock server, and performing clock synchronization processing between the cascade device and the clock server; Collect data of the object to be sensed to obtain sensed data; The perception data is sent to the cascade device, so that the cascade device predicts the output timestamp of the perception data according to the delay information between the perception device and the cascade device and the local timestamp when the cascade device receives the perception data, and associates the output timestamp with the perception data.

9. The clock synchronization processing method according to claim 8, characterized in that: The clock synchronization processing method further includes: If the sensing device can perform clock synchronization processing with the cascade device through a time synchronization protocol, then perform clock synchronization processing with the cascade device, and set the timestamp of the output sensing data according to the synchronized clock; If the sensing device cannot directly perform clock synchronization processing with the cascade device through the time synchronization protocol, the step of sending the data to the cascade device is executed.

10. The clock synchronization processing method according to claim 8, characterized in that: The clock synchronization processing method further includes: If an application for clock synchronization is installed on the sensing device, clock synchronization processing is performed based on the application and the clock server through a lightweight time synchronization protocol; The timestamp of the output sensed data is set according to the synchronized clock.

11. The clock synchronization processing method according to any one of claims 8 to 10, characterized in that: Establish communication connection with cascaded devices, including: A communication connection is established with the cascade device via a light-load link, where the light-load link is a link whose physical bandwidth and the amount of data transmitted by the link are less than a set threshold.

12. A clock synchronization processing device, characterized in that: The clock synchronization processing device is applied to a cascade device, the cascade device is connected between a sensing device and a clock server, and the clock synchronization processing device includes: A synchronization unit, configured to perform clock synchronization processing with a clock server; an acquisition unit, configured to acquire delay information between the first perception device and the cascade device, and to acquire a local timestamp of the cascade device when receiving perception data output by the first perception device; The processing unit is configured to predict an output timestamp of the perception data according to the delay information and the local timestamp, and associate the output timestamp with the perception data.

13. A clock synchronization processing device, characterized in that: The clock synchronization processing device is applied to a sensing device, and the clock synchronization processing device includes: a connection establishing unit, configured to establish a communication connection with a cascade device, the cascade device being connected between the sensing device and the clock server, and performing clock synchronization processing between the cascade device and the clock server; A collection unit configured to collect data of the object to be sensed to obtain sensed data; A sending unit is configured to send the perception data to the cascade device, so that the cascade device predicts the output timestamp of the perception data based on the delay information between the perception device and the cascade device and the local timestamp when the cascade device receives the perception data, and associates the output timestamp with the perception data.

14. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the clock synchronization processing method according to any one of claims 1 to 11 is implemented.

15. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more computer programs, which, when executed by the one or more processors, enables the electronic device to implement the clock synchronization processing method as described in any one of claims 1 to 11.

Images