Data processing method, monitoring system, computing device and storage medium

Abstract

The embodiment of the invention provides a data processing method, a monitoring system, computing equipment and a storage medium. The method comprises the following steps: receiving a first type of data and a second type of data sent by a vehicle; according to the sending time of the first type of data, determining a first data frame with the latest sending time from the cached first type of data; according to the sending time of the second type of data, determining a second data frame with the latest sending time from the cached second type of data; and sending the first data frame and the second data frame to the monitoring equipment in response to the conditions that both the first data frame and the second data frame meet the time delay requirement and the difference value of the first timestamp and the second timestamp meets the synchronous sending requirement. Data frames with time delays not meeting requirements can be filtered, and the situation that the first type of data and the second type of data received by monitoring equipment are not synchronous is reduced.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and particularly to a data processing method, a monitoring system, a computing device, and a storage medium. Background Art

[0002] In the intelligent driving scenario, a vehicle sends more than one type of data to a remote monitoring device in real time, such as video frame data, point cloud frame data collected by vehicle sensors, vehicle operating state data, etc.

[0003] However, there is a time delay in data transmission, and different types of data may be sent separately, which may result in the asynchronous situation of different types of data received at the receiving end, affecting the accuracy of remotely monitoring the vehicle and posing a safety hazard. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a data processing method, a monitoring system, a computing device, and a storage medium to filter data frames whose time delay does not meet the requirements, and reduce the asynchronous situation between the first type of data and the second type of data received by the monitoring device, improve the accuracy of remotely monitoring the vehicle, and enhance safety.

[0005] The specific technical solutions are as follows:

[0006] According to the first aspect of the present disclosure, a data processing method is provided, which is applied to a cloud server. The cloud server is respectively connected to a vehicle and a monitoring device. The method includes:

[0007] Receiving and caching the first type of data and the second type of data sent by the vehicle;

[0008] Determining the first data frame with the latest sending time from the cached first type of data according to the sending time of the first type of data;

[0009] Determining the second data frame with the latest sending time from the cached second type of data according to the sending time of the second type of data;

[0010] In response to that both the first data frame and the second data frame meet the time delay requirements, and the difference between the first timestamp and the second timestamp meets the synchronous sending requirements, sending the first data frame and the second data frame to the monitoring device;

[0011] Wherein, the first timestamp represents the sending time of the first data frame, and the second timestamp represents the sending time of the second data frame.

[0012] According to the second aspect of the present disclosure, a data processing method is provided, which is applied to a cloud server. The cloud server is respectively connected to a vehicle and a monitoring device. The method includes:

[0013] Receive and cache the third type of data and the fourth type of data sent by the monitoring device;

[0014] According to the sending time of the third type of data, determine the third data frame with the latest sending time from the cached third type of data;

[0015] According to the sending time of the fourth type of data, determine the fourth data frame with the latest sending time from the cached third type of data;

[0016] In response to both the third data frame and the fourth data frame meeting the delay requirements, and the difference between the third timestamp and the fourth timestamp meeting the synchronous sending requirements, send the third data frame and the fourth data frame to the vehicle;

[0017] Wherein, the third timestamp represents the sending time of the third data frame, and the fourth timestamp represents the sending time of the fourth data frame.

[0018] According to the third aspect of the present disclosure, a monitoring system is provided, including a vehicle, a cloud server, and a monitoring device. The cloud server is respectively connected to the vehicle and the monitoring device. Among them,

[0019] The vehicle is used to send the first type of data and the second type of data to the cloud server; receive the third type of data and the fourth type of data sent by the cloud server;

[0020] The cloud server is used to receive the first type of data and the second type of data sent by the vehicle; according to the sending time of the first type of data, determine the first data frame with the latest sending time from the cached first type of data; according to the sending time of the second type of data, determine the second data frame with the latest sending time from the cached second type of data; in response to both the first data frame and the second data frame meeting the delay requirements, and the difference between the first timestamp and the second timestamp meeting the synchronous sending requirements, send the first data frame and the second data frame to the monitoring device; wherein, the first timestamp represents the sending time of the first data frame, and the second timestamp represents the sending time of the second data frame;

[0021] The cloud server is used to receive the third type of data and the fourth type of data sent by the monitoring device; according to the sending time of the third type of data, determine the third data frame with the latest sending time from the cached third type of data; according to the sending time of the fourth type of data, determine the fourth data frame with the latest sending time from the cached third type of data; in response to both the third data frame and the fourth data frame meeting the delay requirements, and the difference between the third timestamp and the fourth timestamp meeting the synchronous sending requirements, send the third data frame and the fourth data frame to the vehicle; wherein, the third timestamp represents the sending time of the third data frame, and the fourth timestamp represents the sending time of the fourth data frame;

[0022] The monitoring device is used to receive the first data frame and the second data frame sent by the cloud server; send the third type of data and the fourth type of data to the cloud server.

[0023] According to a fourth aspect of the present disclosure, there is provided a computing device including one or more processors and a memory storing a program, the program including instructions which, when executed by the processors, cause the processors to execute any of the above data processing methods.

[0024] According to a fifth aspect of the present disclosure, there is provided a computer-readable storage medium storing a program, the program including instructions which, when executed by one or more processors of a computing device, instruct the computing device to execute any of the above data processing methods.

[0025] Beneficial effects of the examples of the present application:

[0026] The embodiments of the present application provide a data processing method, a monitoring system, a computing device and a storage medium. The method is applied to a cloud server which is respectively connected to a vehicle and a monitoring device. The method includes: receiving and caching first type of data and second type of data sent by the vehicle; determining, according to the sending time of the first type of data, a first data frame with the latest sending time from the cached first type of data; determining, according to the sending time of the second type of data, a second data frame with the latest sending time from the cached second type of data; and sending the first data frame and the second data frame to the monitoring device in response to that both the first data frame and the second data frame meet the delay requirements and the difference between the first timestamp and the second timestamp meets the synchronous sending requirements.

[0027] It can be seen that the data frames of the first type of data and the second type of data sent by the vehicle carry timestamps representing the sending time. Further, the cloud server determines the first data frame with the latest sending time and the second data frame with the latest sending time currently cached, that is, it is ensured that the first data frame carries the latest first type of data and the second data frame carries the latest second type of data. Since the timestamp represents the sending time of the data frame, it can be judged whether the delay of the data frame meets the requirements according to the timestamp. When the delay requirements are met, according to the difference between the timestamps corresponding to the first data frame and the second data frame, it is further judged whether the synchronous sending requirements are met. If so, the first data frame and the second data frame are sent synchronously. Data frames with unmet delay requirements can be filtered out, avoiding large delays in the data frames received by the monitoring device. And when the first data frame and the second data frame meet the synchronous sending requirements, they are sent synchronously, which can reduce the situation of out-of-sync between the first type of data and the second type of data received by the monitoring device. Description of the Drawings

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other embodiments can also be obtained based on these drawings.

[0029] Figure 1 A schematic flowchart of a data processing method provided by an embodiment of the present application;

[0030] Figure 2 A schematic diagram of the process executed in actual operation provided by an embodiment of the present application;

[0031] Figure 3 A schematic flowchart of another data processing method provided by an embodiment of the present application;

[0032] Figure 4 A schematic structural diagram of a monitoring system provided by an embodiment of the present application;

[0033] Figure 5 A schematic diagram of a cockpit device provided by an embodiment of the present application;

[0034] Figure 6 A schematic structural diagram of a computing device provided by an embodiment of the present application. Detailed implementation manners

[0035] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art based on the present application belong to the scope of protection of the present invention.

[0036] Public Network: A public network refers to a network that can be globally accessed through the Internet. This is an open network that can be accessed through a public IP address. Devices on the public network can directly communicate with other devices on the public network. For example, accessing a website through an Internet browser. The IP address on the public network is globally unique, so devices on the public network can be directly identified and accessed through their IP addresses.

[0037] Private Network: A private network refers to a network in which devices communicate through private IP addresses in a closed network. This type of network is usually established within an organization and is not directly connected to the Internet. The purpose of a private network is to provide internal communication and resource sharing within the organization while protecting the internal network from direct access on the Internet.

[0038] In the scenarios of autonomous driving or remote driving, vehicles, cloud servers, and monitoring devices are involved. The cloud server is respectively connected to the vehicle and the monitoring device.

[0039] Among them, a vehicle-mounted communication device is integrated in the vehicle, and the vehicle can communicate with the cloud server through the vehicle-mounted communication device. For example, the vehicle-mounted communication device can be a T-BOX (Telematics BOX, in-vehicle electronic box), and the T-Box can be connected to the vehicle's network system and cloud services, and is used to provide vehicle remote monitoring, communication, and information services, and realize functions such as vehicle remote tracking, fault diagnosis, and navigation services.

[0040] Specifically, the vehicle sends different types of data to the cloud server in real time, such as video frame data and point cloud frame data collected by vehicle sensors, data characterizing the vehicle's running state, etc., and the cloud server forwards the data sent by the vehicle to the monitoring device.

[0041] In this process, there is a time delay in data transmission, and the time delay is inevitable. However, if the time delay of the data is too high, the data received by the monitoring device may have a negative impact on the monitoring of the vehicle. For example, the video frame received by the monitoring device shows the road condition picture collected by the vehicle 8 seconds ago. Due to the excessive time delay of 8 seconds, the video frame cannot reflect the real-time road conditions around the vehicle, so the video frame will instead have a negative impact.

[0042] On the other hand, the above different types of data are usually sent separately, which may result in out-of-sync situations for different types of data received by the monitoring device.

[0043] To solve the above technical problems, an embodiment of the present application provides a data processing method, which is applied to a cloud server. Refer to Figure 1 , which shows a schematic flowchart of a data processing method provided by an embodiment of the present application, including the following steps:

[0044] Step S101: Receive and cache the first type of data and the second type of data sent by the vehicle.

[0045] In an embodiment of the present application, the first type of data includes: video frame data collected by vehicle-mounted sensors; the second type of data includes data carrying the status information of the vehicle, and the status information includes at least one of the following: vehicle speed, gear position, steering wheel angle, hardware status information, or abnormal warning information.

[0046] Specifically, the video frame data can be road condition images collected by vehicle-mounted sensors. The hardware status information can include the status of each hardware in the vehicle system, such as the processor temperature and load condition of the T-BOX device. In addition, during vehicle driving, abnormal situations may occur, such as the failure of autonomous driving. At this time, the vehicle-mounted communication device generates an abnormal warning message, which is used to inform the monitoring device that the vehicle is in an abnormal state. The abnormal warning message can be carried in the next data frame of the second type of data.

[0047] It can be seen that when the data processing method provided by the embodiments of the present application is applied to intelligent driving and autonomous driving scenarios, the vehicle-mounted sensors collect video frame data in real time to generate the first type of data containing the video frame data; and the second type of data is generated according to various status information of the vehicle. The vehicle sends the first type of data and the second type of data to the cloud server, and the cloud server processes the first type of data and the second type of data by using the data processing method provided by the embodiments of the present application.

[0048] Step S102: Determine the first data frame with the latest sending time from the cached first type of data according to the sending time of the first type of data.

[0049] In the embodiments of the present application, after obtaining the first type of data, the vehicle-mounted communication device integrated in the vehicle generates a timestamp for the first type of data according to the current time and sends the first type of data to the cloud server. This timestamp is recorded as the first timestamp, and the first timestamp can represent the sending time of the first type of data.

[0050] For example, if the first type of data is video frame data, the vehicle communication device obtains the video frame data collected by the vehicle-mounted sensors, transcodes the video frame data, encapsulates the transcoded video frame data into a message, uses the current time as the timestamp, and at the same time sends the message carrying the timestamp.

[0051] In fact, the above timestamp does not strictly correspond to the time when the message is sent, but the time when the timestamp is generated and the time when the message is sent are almost the same. Therefore, the above timestamp can be used to represent the sending time of the message, that is, the sending time of the first type of data.

[0052] It should be noted that in the embodiments of the present application, the vehicle, the cloud server, and the monitoring device have completed clock synchronization in advance, and the embodiments of the present application do not limit the method of clock synchronization. For example, the vehicle, the cloud server, and the monitoring device can perform clock synchronization based on NTP (Network Time Protocol) in advance.

[0053] During the execution of the solution of this application, the cloud server continuously receives the first type of data and the second type of data sent by the vehicle. However, for steps S102 - S104, they can be triggered and executed periodically, and the trigger period can be preset, that is, steps S102 - S104 are re - executed every once in a while. The trigger period can be related to the frequency at which the vehicle sends the first type of data or the second type of data. Now, only the process of one execution of steps S102 - S104 is described.

[0054] In the embodiment of this application, the vehicle sends the first type of data in the form of data frames, and each data frame corresponds to a timestamp, indicating the sending time of the data frame. Correspondingly, the first type of data received by the cloud server is also cached in the form of data frames.

[0055] In step S102, for the first type of data, the cloud server determines the data frame with the latest sending time from the cached data frames, denoted as the first data frame. Specifically, since the timestamp represents the sending time of the data frame, the data frame with the latest sending time can be determined according to the timestamp of the data frame.

[0056] Since the vehicle sends data frames in the order of generating data frames, the first data frame can be understood as: the latest generated data frame among the data frames of the first type of data currently cached. Since the cloud server needs to forward the latest data to the monitoring device in real - time, in the embodiment of this application, each time step S102 is executed, the first data frame with the latest sending time is determined from the cached data frames, that is, it is ensured that the first data frame carries the latest first type of data.

[0057] Step S103: Determine the second data frame with the latest sending time from the cached second type of data according to the sending time of the second type of data.

[0058] For the second type of data, the data frames sent by the vehicle also carry timestamps representing the sending time. Therefore, step S103 is basically the same as step S102, and the only difference is that the data carried in the message is the second type of data instead of the first type of data, and the other processing processes are the same. Therefore, the specific implementation process will not be elaborated.

[0059] Step S104: In response to that both the first data frame and the second data frame meet the delay requirements, and the difference between the first timestamp and the second timestamp meets the synchronous sending requirements, send the first data frame and the second data frame to the monitoring device; where the first timestamp represents the sending time of the first data frame, and the second timestamp represents the sending time of the second data frame.

[0060] For the sake of easy description, for the data frames sent by the vehicle, the transmission delay between the vehicle and the cloud server is denoted as the first delay, and the transmission delay between the cloud server and the monitoring device is denoted as the second delay.

[0061] In the embodiments of the present application, the cloud server can be deployed in a private network. When the vehicle sends data to the cloud server, it needs to be transmitted through the public network. Moreover, the vehicle's location is not fixed, and the distance between the vehicle and the cloud server is usually very far. Therefore, the first delay is relatively large. Correspondingly, between the cloud server and the monitoring device, it is generally configured to transmit data through the private network, and the second delay is relatively small. Therefore, in the embodiments of the present application, the first delay is used to evaluate whether the first data frame and the second data frame meet the delay requirements.

[0062] Specifically, the vehicle and the cloud server have completed clock synchronization in advance. Therefore, the difference between the current time when the cloud server receives the data frame and the time when the data frame is sent can be regarded as the transmission delay of the data frame. That is, the difference between the current time and the time corresponding to the first timestamp is regarded as the transmission delay. Combining with the pre-set delay requirements, it can be judged whether the requirements are met. The judgment for the second video frame is the same.

[0063] When both the first data frame and the second data frame meet the delay requirements, it is further judged whether the synchronous sending requirements are met according to the difference between the first timestamp and the second timestamp.

[0064] Specifically, if the difference between the first timestamp and the second timestamp is large, it means that the first video frame and the second video frame do not meet the synchronous sending requirements; otherwise, they meet the synchronous sending requirements.

[0065] When the synchronous sending requirements are met, the first data frame and the second data frame are sent to the monitoring device.

[0066] The data processing method provided by the embodiments of the present application is applied to a cloud server. The cloud server is respectively connected to a vehicle and a monitoring device. The method includes: receiving the first type of data and the second type of data sent by the vehicle; determining the first data frame with the latest sending time from the cached first type of data according to the sending time of the first type of data; determining the second data frame with the latest sending time from the cached second type of data according to the sending time of the second type of data; and sending the first data frame and the second data frame to the monitoring device in response to that both the first data frame and the second data frame meet the delay requirements and the difference between the first timestamp and the second timestamp meets the synchronous sending requirements. Wherein, the first timestamp represents the sending time of the first data frame, and the second timestamp represents the sending time of the second data frame.

[0067] It can be seen that the data frames of the first type of data and the data frames of the second type of data sent by the vehicle carry time stamps representing the sending time. Furthermore, the cloud server determines the latest sent first data frame and the latest sent second data frame currently cached, that is, ensuring that the first data frame carries the latest first type of data and the second data frame carries the latest second type of data. Since the time stamp represents the sending time of the data frame, the delay of the data frame can be judged according to the time stamp to see if it meets the requirements. When the delay requirement is met, according to the difference between the time stamps corresponding to the first data frame and the second data frame respectively, it is further judged whether the synchronous sending requirement is met. If it is met, the first data frame and the second data frame are sent synchronously. This solution can filter out data frames with unmet delay requirements, avoid large delays in the data frames received by the monitoring device, and when the first data frame and the second data frame meet the synchronous sending requirement, send them synchronously, which can reduce the situation where the first type of data and the second type of data received by the monitoring device are out of sync.

[0068] In one embodiment of the present application, in response to the first data frame meeting the delay requirement and the second data frame not meeting the delay requirement, the first data frame is sent to the monitoring device; in response to the first data frame not meeting the delay requirement and the second data frame meeting the delay requirement, the second data frame is sent to the monitoring device; in response to both the first data frame and the second data frame not meeting the delay requirement, neither the first data frame nor the second data frame is sent.

[0069] Specifically, when the first data frame or the second data frame does not meet the delay requirement, the cloud server will not send the data frame that does not meet the delay requirement to the monitoring device, so as to avoid misleading the monitoring personnel at the monitoring device end by data with too large a delay, thereby improving safety. In addition, the cloud server can also be configured to discard data frames that do not meet the delay requirement to relieve the pressure on the cache.

[0070] It can be seen that when the first data frame or the second data frame does not meet the delay requirement, only the data frames that meet the delay requirement are sent. On the one hand, data frames with high delays are filtered out, avoiding large delays in the data received by the monitoring device and reducing potential safety hazards; on the other hand, if the data frames corresponding to a certain type of data meet the delay requirement, the corresponding data frames are sent to the monitoring device, thereby improving the coherence of the data and preventing the loss of valid data.

[0071] In one embodiment of the present application, in response to the difference between the current time and the time corresponding to the first time stamp not exceeding the first threshold, it is determined that the first data frame meets the delay requirement; in response to the difference between the current time and the time corresponding to the second time stamp not exceeding the first threshold, it is determined that the second data frame meets the delay requirement.

[0072] Specifically, in response to the difference between the current time and the time corresponding to the first timestamp not exceeding the first threshold, it means that after the cloud server subtracts the time corresponding to the first timestamp from the current time to obtain the difference, it determines that the difference does not exceed the first threshold. The first threshold can be preset by the user and can be adjusted according to the tolerance of the time delay in the application scenario. For example, if a higher requirement for time delay is set, a smaller first threshold, such as 50 ms, is set; on the contrary, a larger first threshold, such as 100 ms, is set.

[0073] In this embodiment, since the vehicle and the cloud server have been pre-synchronized in time, the difference between the current time and the time corresponding to the first timestamp can characterize the time delay of the first data frame, and the difference between the current time and the time corresponding to the second timestamp can characterize the time delay of the second data frame. Furthermore, it can be determined whether the time delay meets the requirements, and the time delay of the data frame can be conveniently and efficiently confirmed.

[0074] In an embodiment of the present application, in response to both the first data frame and the second data frame meeting the time delay requirements, and the difference between the first timestamp and the second timestamp not meeting the synchronous sending requirement, the order of the first timestamp and the second timestamp is determined; in response to the first timestamp being earlier than the second timestamp, the second data frame is sent to the monitoring device; in response to the second timestamp being earlier than the first timestamp, the first data frame is sent to the monitoring device.

[0075] Among them, in response to the first timestamp being earlier than the second timestamp, it means that the cloud server compares the order of the first timestamp and the second timestamp and determines that the time corresponding to the first timestamp is earlier than the time corresponding to the second timestamp. In response to the second timestamp being earlier than the first timestamp, it means that the cloud server compares the order of the first timestamp and the second timestamp and determines that the time corresponding to the second timestamp is earlier than the time corresponding to the first timestamp.

[0076] Specifically, the difference between the first timestamp and the second timestamp not meeting the synchronous sending requirement indicates that the first data frame and the second data frame do not match in time and are not suitable for synchronous sending.

[0077] Taking the first data frame carrying video frame data collected by in-vehicle sensors and the second data frame carrying vehicle status information as an example. If the first data frame and the second data frame that do not meet the synchronous sending requirement are synchronously sent, the monitoring device will also perform synchronous display after receiving the first data frame and the second data frame. Then, the video frame picture corresponding to the first data frame and the vehicle status information corresponding to the second data frame will not match in time, which may mislead the user on the monitoring device side.

[0078] In the embodiments of the present application, when the difference between the first timestamp and the second timestamp does not meet the synchronous transmission requirement, neither the first data frame nor the second data frame will be sent to the monitoring device. Instead, a reasonable discard is performed, that is, according to the order of the first timestamp and the second timestamp, the data frame with a later emission timestamp (i.e., the generation time is closer to the current time) is selected and sent to the monitoring device.

[0079] Specifically, if the first timestamp is earlier than the second timestamp, it means that the data corresponding to the first data frame is generated earlier, and the data corresponding to the second data frame is generated later. The data generated later can also be understood as relatively new data. In a monitoring scenario, new data has higher value than old data. Therefore, in such cases, the second data frame is sent to the monitoring device. Conversely, the first data frame is sent to the monitoring device.

[0080] It can be seen that in the embodiments of the present application, when both the first data frame and the second data frame meet the latency requirements but do not meet the synchronous transmission requirements, the relatively new data with higher value is sent to the monitoring device, which is beneficial to monitoring the latest status of the vehicle and improving the timeliness of vehicle monitoring. Moreover, it is beneficial to maintain the coherence of the data and avoid losing too much data.

[0081] In one embodiment of the present application, in response to the difference between the first timestamp and the second timestamp not exceeding the second threshold, it is determined that the difference between the first timestamp and the second timestamp meets the synchronous transmission requirement.

[0082] Specifically, the second threshold can be preset by the user and can be adjusted according to the severity of the data synchronization requirements in the application scenario. For example, if the requirements for data synchronization are relatively strict, a smaller second threshold, such as 50 ms, is set; conversely, a larger second threshold, such as 80 ms or 100 ms, can be set.

[0083] It can be seen that in this embodiment, since both the first data frame and the second data frame have corresponding timestamps representing the emission time, by comparing the timestamp difference with the preset threshold, it can be determined whether the first data frame and the second data frame meet the synchronous transmission requirements.

[0084] For ease of understanding, the following will be based on Figure 1 the method shown, and a more detailed description of the complete implementation in the data processing method mentioned above will be given. Refer to Figure 2 , which shows a schematic diagram of the process executed in actual operation, including the following steps:

[0085] S201: Obtain the first data frame and the second data frame with the latest emission timestamps cached in the cloud server. The first timestamp represents the emission timestamp of the first data frame, and the second timestamp represents the emission timestamp of the second data frame.

[0086] S202: Calculate the first difference between the current time and the time corresponding to the first timestamp, and the second difference between the current time and the time corresponding to the second timestamp respectively.

[0087] S203: Determine whether both the first difference and the second difference do not exceed a set first threshold. If so, execute step S205; if not, execute step S204.

[0088] S204: If the first difference exceeds the first threshold and the second difference does not exceed the first threshold, only send the second data frame; if the second difference exceeds the first threshold and the first difference does not exceed the first threshold, only send the first data frame; if both the first difference and the second difference exceed the first threshold, continue to wait for receiving the first type of data and the second type of data with a later sending time.

[0089] S205: Determine whether the difference between the first timestamp and the second timestamp does not exceed a predetermined second threshold. If so, execute step S206; if not, execute step S207.

[0090] S206: Send the first data frame and the second data frame.

[0091] S207: Determine whether the first timestamp is earlier than the second timestamp. If so, execute step S209; if not, execute step S208.

[0092] S208: Send the first data frame.

[0093] S209: Send the second data frame.

[0094] It can be seen that by applying the data processing method provided in the embodiments of the present application, data frames with unsatisfactory time delays can be filtered, avoiding large time delays in the data frames received by the monitoring device. And when the first data frame and the second data frame meet the synchronous sending requirements, they are sent synchronously, which can reduce the situation of the first type of data and the second type of data received by the monitoring device being out of sync. When the first data frame or the second data frame does not meet the time delay requirements, only the data frames that meet the time delay requirements are sent. On the one hand, data frames with high time delays are filtered out, preventing the data received by the monitoring device from having a large time delay and misleading the users at the monitoring device end; on the other hand, sending the data frames that meet the time delay requirements to the monitoring device can improve the coherence of the data and prevent the loss of valid data. When both the first data frame and the second data frame meet the time delay requirements but do not meet the synchronous sending requirements, the more valuable and newer data is sent to the monitoring device, which is beneficial to monitoring the latest status of the vehicle, improving the timeliness of vehicle monitoring, and is also conducive to maintaining the coherence of the data and avoiding the loss of more data.

[0095] See Figure 3, which shows a schematic flowchart of another data processing method provided by an embodiment of the present application, applied to a cloud server. The method includes the following steps:

[0096] S301: Receive and cache the third type of data and the fourth type of data sent by the monitoring device;

[0097] S302: Determine the third data frame with the latest sending time from the cached third type of data according to the sending time of the third type of data.

[0098] S303: Determine the fourth data frame with the latest sending time from the cached fourth type of data according to the sending time of the fourth type of data;

[0099] S304: In response to both the third data frame and the fourth data frame meeting the latency requirements, and the difference between the third timestamp and the fourth timestamp meeting the synchronous sending requirements, send the third data frame and the fourth data frame to the vehicle; where the third timestamp represents the sending time of the third data frame, and the fourth timestamp represents the sending time of the fourth data frame.

[0100] In the embodiments of the present application, the monitoring device may also need to send different types of data to the vehicle, such as remote control instructions, status acquisition instructions, or other communication data. Therefore, the cloud server can also perform corresponding processing on two different types of data sent by the monitoring device.

[0101] For the specific implementation of steps S301 - S304, reference can be made to steps S101 - S104. The difference lies only in the data transmission direction and the data content carried in the data frame, which will not be elaborated here.

[0102] It can be seen that the data frames of the third type of data and the fourth type of data sent by the monitoring device carry timestamps representing the sending time. Furthermore, the cloud server determines the third data frame with the latest sending time and the fourth data frame with the latest sending time currently cached, ensuring that the third data frame carries the latest third type of data and the fourth data frame carries the latest fourth type of data. Since the timestamp represents the sending time of the data frame, it is possible to determine whether the latency of the data frame meets the requirements based on the timestamp. When the latency requirements are met, based on the difference between the timestamps corresponding to the third data frame and the fourth data frame, it is further determined whether the synchronous sending requirements are met. If so, the third data frame and the fourth data frame are sent synchronously. The above solution can filter out data frames with latency not meeting the requirements, avoid large latency in the data frames received by the vehicle, and when the third data frame and the third data frame meet the synchronous sending requirements, send them synchronously, which can reduce the situation where the third type of data and the fourth type of data received by the vehicle are out of sync.

[0103] See Figure 4, which shows a schematic structural diagram of the monitoring system provided by the embodiments of the present application, including a vehicle 401, a cloud server 402, and a monitoring device 403. The cloud server 402 is respectively connected to the vehicle 401 and the monitoring device 403.

[0104] The vehicle is used to send the first type of data and the second type of data to the cloud server; receive the third type of data and the fourth type of data sent by the cloud server;

[0105] The cloud server is used to receive and cache the first type of data and the second type of data sent by the vehicle; determine the first data frame with the latest sending time from the cached first type of data according to the sending time of the first type of data; determine the second data frame with the latest sending time from the cached second type of data according to the sending time of the second type of data; in response to both the first data frame and the second data frame meeting the delay requirements, and the difference between the first timestamp and the second timestamp meeting the synchronous sending requirements, send the first data frame and the second data frame to the monitoring device; wherein, the first timestamp represents the sending time of the first data frame, and the second timestamp represents the sending time of the second data frame;

[0106] The cloud server is used to receive and cache the third type of data and the fourth type of data sent by the monitoring device; determine the third data frame with the latest sending time from the cached third type of data according to the sending time of the third type of data; determine the fourth data frame with the latest sending time from the cached third type of data according to the sending time of the fourth type of data; in response to both the third data frame and the fourth data frame meeting the delay requirements, and the difference between the third timestamp and the fourth timestamp meeting the synchronous sending requirements, send the third data frame and the fourth data frame to the vehicle; wherein, the third timestamp represents the sending time of the third data frame, and the fourth timestamp represents the sending time of the fourth data frame;

[0107] The monitoring device is used to receive the first data frame and the second data frame sent by the cloud server; send the third type of data and the fourth type of data to the cloud server.

[0108] The above implementation process can be referred to steps S101 - S104 and steps S301 - S304, which will not be elaborated here.

[0109] The following further illustrates the monitoring system in combination with specific scenarios. The monitoring system provided by the embodiments of the present application can be applied to scenarios of intelligent driving and driverless driving. In this application scenario, the vehicle is an intelligent driving vehicle or a driverless vehicle, and is equipped with vehicle - end communication devices such as T - BOX, which can provide network for the vehicle, collect vehicle CAN data, and perform power supply management control.

[0110] The cloud server is responsible for receiving and parsing the video stream data and vehicle status information reported by the vehicle - end, and adopts Figure 1The data processing method shown is used to process the data and forward it to the monitoring device. In addition, the cloud server can also be responsible for the storage and playback of the entire driving data, including the storage of key information such as network status, link delay, vehicle-side anomalies, and instruction data.

[0111] In one embodiment of the present application, the monitoring device may be a cockpit device, see Figure 5 , Figure 5 A schematic diagram of a cockpit device provided in an embodiment of the present application, such as Figure 5 As shown, the cockpit equipment includes a cockpit 501 and multiple display screens 502, wherein the cockpit 501 includes a steering wheel 501a and a control panel 501b, and may also include controls such as an accelerator pedal and a brake pedal, which are not shown in the figure.

[0112] The display screen can display the video frame data and vehicle status information forwarded by the cloud server. Figure 5 For example, three channels of video frame data and one channel of vehicle status information can be displayed. Therefore, the safety officer at the remote end can understand the real road conditions around the vehicle and the vehicle operation status information based on the images displayed on the display screen.

[0113] In addition, the safety officer can operate the steering wheel, accelerator pedal, brake pedal, control panel, etc. in the cockpit equipment to issue vehicle control commands.

[0114] The following is an example of a specific application scenario to illustrate the implementation principle of the monitoring system and monitoring solution of this solution.

[0115] During the driving process, autonomous vehicles will inevitably encounter different environments and various system problems, some of which may even cause the autonomous driving system to fail. Since autonomous vehicles may not be equipped with safety officers, the operator needs to have the ability to remotely control the vehicle to help the vehicle escape or avoid danger in an emergency, minimize the risk of accidents, and ensure the safety of people and property.

[0116] However, in the existing solutions, the image data showing the road conditions and the operating status of the vehicle received by the monitoring equipment may have a large time delay and may not be synchronized. The safety officer may issue unreasonable control instructions when performing remote driving based on the road condition images and vehicle operating parameters with large delays and non-synchronization, which will fail to help the vehicle get out of trouble and may even cause a traffic accident, resulting in serious safety problems.

[0117] The following is an example for illustration. If the road condition image received by the monitoring device shows that a vehicle is about to pass through an intersection and there are no pedestrians at the intersection, and the vehicle speed in the vehicle operation parameters is 30 km / h. At this time, the monitoring device receives an alarm message, which indicates that the automatic driving of the vehicle fails and the safety officer needs to remotely control the vehicle. The safety officer combines the road condition image and the vehicle operation status, makes a decision to decelerate, turn right and enter the intersection, and then stop, and issues corresponding control instructions through the cockpit device. After receiving the control instructions, the vehicle sequentially decelerates, turns right, and stops. However, if the delay of the road condition image is large, it is very likely that the vehicle has already passed through the intersection. At this time, if the vehicle turns right again, there is a possibility of a traffic accident. Or, although the delays of the road condition image and the vehicle operation parameters are not high, but they are not synchronized, it may also cause a traffic accident. For example, under this road condition image, the corresponding actual vehicle speed is 45 km / h, rather than 30 km / h. The safety officer issues a deceleration instruction based on the wrong vehicle speed, and the deceleration amplitude may be unreasonable. The speed after deceleration may still be large, and turning right at a large speed is likely to cause safety problems.

[0118] The above only takes the vehicle speed in the vehicle operation parameters as an example for illustration. In fact, if the information such as the steering wheel angle and gear of the vehicle does not correspond to the video frame image, it may also cause the safety officer to issue unreasonable control instructions, thus causing more serious safety problems.

[0119] In the embodiment of the present application, the cloud server adopts Figure 1 the data processing method shown to process the video frame data and operation status information sent by the vehicle. On the one hand, the above solution can filter out the data frames with large delays, ensuring that the data received and displayed by the cockpit device meets the time delay requirements; on the other hand, the cloud server synchronizes the video frame data and operation status information, which can ensure that the video frame data and operation status information displayed by the cockpit device meet the synchronization requirements, so that when the safety officer performs remote control, reasonable remote control instructions can be issued according to the video frame data and vehicle operation status with small time delay and meeting the synchronization requirements.

[0120] Correspondingly, the cloud server can also be used to receive the remote control instructions sent by the cockpit device and send the remote control instructions to the vehicle-side communication device; among them, the remote control instructions can be generated by the cockpit device according to the manipulation instructions acting on the cockpit device.

[0121] It can be seen that the data processing method provided by the embodiment of the present application is applied in the fields of intelligent driving and driverless driving, which can improve the timeliness and accuracy of remotely monitoring vehicles. It solves the technical problem in the prior art that the time delays of the road condition image and the vehicle operation status received by the monitoring device are too high and not synchronized, resulting in the safety officer issuing unreasonable control instructions and easily causing potential safety hazards.

[0122] An embodiment of the present invention further provides a computing device, such as Figure 6 shown, including a processor 601, a communication interface 602, a memory 603, and a communication bus 604. Among them, the processor 601, the communication interface 602, and the memory 603 complete communication with each other through the communication bus 604.

[0123] The memory 603 is used to store computer programs;

[0124] When the processor 601 is used to execute the program stored on the memory 603, the following steps are implemented:

[0125] Receive the first type of data and the second type of data sent by the vehicle;

[0126] According to the sending time of the first type of data, determine the first data frame with the latest sending time from the cached first type of data;

[0127] According to the sending time of the second type of data, determine the second data frame with the latest sending time from the cached second type of data;

[0128] In response to the first data frame and the second data frame both meeting the delay requirements, and the difference between the first timestamp and the second timestamp meeting the synchronous sending requirements, send the first data frame and the second data frame to the monitoring device;

[0129] Among them, the first timestamp represents the sending time of the first data frame, and the second timestamp represents the sending time of the second data frame.

[0130] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is used in the figure to represent it, but it does not mean that there is only one bus or one type of bus.

[0131] The communication interface is used for communication between the above electronic device and other devices.

[0132] The memory may include a Random Access Memory (RAM), or may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0133] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0134] When applying the computing device provided by the embodiments of the present application, the data frames of the first type of data and the data frames of the second type of data sent by the vehicle carry timestamps representing the sending time. Furthermore, the cloud server determines the latest sent first data frame and the latest sent second data frame currently cached, that is, it ensures that the first data frame carries the latest first type of data, and the second data frame carries the latest second type of data. Since the timestamp represents the sending time of the data frame, it is possible to determine whether the delay of the data frame meets the requirements according to the timestamp. When the delay requirement is met, according to the difference between the timestamps corresponding to the first data frame and the second data frame, it is further determined whether the synchronous sending requirement is met. If it is met, the first data frame and the second data frame are synchronously sent. It is possible to filter out the data frames whose delays do not meet the requirements, avoid large delays in the data frames received by the monitoring device, and when the first data frame and the second data frame meet the synchronous sending requirement, send them synchronously, which can reduce the situation where the first type of data and the second type of data received by the monitoring device are out of sync.

[0135] In another embodiment provided by the present invention, a computer-readable storage medium is further provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above data processing methods are implemented.

[0136] In another embodiment provided by the present invention, a computer program product containing instructions is further provided. When it runs on a computer, it causes the computer to execute any of the data processing methods in the above embodiments.

[0137] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0138] It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0139] Each embodiment in this specification is described in a related manner. The same or similar parts among the embodiments can be referred to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

[0140] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are all included in the protection scope of the present invention.

Claims

1. A data processing method, characterized in that, Applied to a cloud server, the cloud server is respectively connected to a vehicle and a monitoring device, and the method includes: Receiving and caching the first type of data and the second type of data sent by the vehicle; Determining, according to the sending time of the first type of data, the first data frame with the latest sending time from the cached first type of data; Determining, according to the sending time of the second type of data, the second data frame with the latest sending time from the cached second type of data; Responding to that both the first data frame and the second data frame meet the latency requirements, and the difference between the first timestamp and the second timestamp meets the synchronous sending requirements, and sending the first data frame and the second data frame to the monitoring device; Wherein, the first timestamp represents the sending time of the first data frame, and the second timestamp represents the sending time of the second data frame.

2. The method according to claim 1, characterized in that It further includes: Responding to that the first data frame meets the latency requirements and the second data frame does not meet the latency requirements, and sending the first data frame to the monitoring device; Responding to that the first data frame does not meet the latency requirements and the second data frame meets the latency requirements, and sending the second data frame to the monitoring device; Responding to that both the first data frame and the second data frame do not meet the latency requirements, and not sending the first data frame and the second data frame.

3. The method according to claim 1 or 2, characterized in that, It further includes: Responding to that the difference between the current time and the time corresponding to the first timestamp does not exceed a first threshold, and determining that the first data frame meets the latency requirements; Responding to that the difference between the current time and the time corresponding to the second timestamp does not exceed the first threshold, and determining that the second data frame meets the latency requirements.

4. The method according to claim 1, wherein It further includes: Responding to that both the first data frame and the second data frame meet the latency requirements, and the difference between the first timestamp and the second timestamp does not meet the synchronous sending requirements, and determining the sequence of the first timestamp and the second timestamp; Responding to that the first timestamp is earlier than the second timestamp, and sending the second data frame to the monitoring device; Responding to that the second timestamp is earlier than the first timestamp, and sending the first data frame to the monitoring device.

5. The method according to claim 1 or 4, characterized in that, It further includes: Responding to that the difference between the first timestamp and the second timestamp does not exceed a second threshold, and determining that the difference between the first timestamp and the second timestamp meets the synchronous sending requirements.

6. The method according to claim 1, wherein: The first type of data includes at least one of the following: video frame data collected by in-vehicle sensors, point cloud frame data collected by in-vehicle sensors; The second type of data includes: data carrying the status information of the vehicle; the status information includes at least one of the following: vehicle speed, gear position, steering wheel angle, hardware status information, or abnormal alarm information.

7. A data processing method, characterized in that, Applied to a cloud server, the cloud server is respectively connected to a vehicle and a monitoring device, and the method includes: Receiving and caching the third type of data and the fourth type of data sent by the monitoring device; Determining, according to the sending time of the third type of data, the third data frame with the latest sending time from the cached third type of data; Determine the fourth data frame with the latest emission time from the cached fourth type of data according to the emission time of the fourth type of data; In response to both the third data frame and the fourth data frame meeting the latency requirements, and the difference between the third timestamp and the fourth timestamp meeting the synchronous transmission requirements, send the third data frame and the fourth data frame to the vehicle; Wherein, the third timestamp represents the emission time of the third data frame, and the fourth timestamp represents the emission time of the fourth data frame.

8. A monitoring system, characterized in that, It includes a vehicle, a cloud server, and a monitoring device. The cloud server is respectively connected to the vehicle and the monitoring device, where The vehicle is configured to send the first type of data and the second type of data to the cloud server; receive the third type of data and the fourth type of data sent by the cloud server; The cloud server is configured to receive and cache the first type of data and the second type of data sent by the vehicle; determine the first data frame with the latest emission time from the cached first type of data according to the emission time of the first type of data; determine the second data frame with the latest emission time from the cached second type of data according to the emission time of the second type of data; in response to both the first data frame and the second data frame meeting the latency requirements, and the difference between the first timestamp and the second timestamp meeting the synchronous transmission requirements, send the first data frame and the second data frame to the monitoring device; wherein, the first timestamp represents the emission time of the first data frame, and the second timestamp represents the emission time of the second data frame; The cloud server is configured to receive and cache the third type of data and the fourth type of data sent by the monitoring device; determine the third data frame with the latest emission time from the cached third type of data according to the emission time of the third type of data; determine the fourth data frame with the latest emission time from the cached third type of data according to the emission time of the fourth type of data; in response to both the third data frame and the fourth data frame meeting the latency requirements, and the difference between the third timestamp and the fourth timestamp meeting the synchronous transmission requirements, send the third data frame and the fourth data frame to the vehicle; wherein, the third timestamp represents the emission time of the third data frame, and the fourth timestamp represents the emission time of the fourth data frame; The monitoring device is configured to receive the first data frame and the second data frame sent by the cloud server; send the third type of data and the fourth type of data to the cloud server.

9. A computing device, characterized in that, It includes one or more processors and a memory storing a program. The program includes instructions that, when executed by the processor, cause the processor to execute the method according to any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that, The program includes instructions that, when executed by one or more processors of a computing device, direct the computing device to execute the method according to any one of claims 1-7.

Images