Data processing method, apparatus, and storage medium

Abstract

The application relates to a data processing method, device and storage medium, the method comprising: acquiring first data collected by a sensor; determining a first frame rate according to a frame rate interval and an index value, the index value being used for indicating a state of a target vehicle, the first frame rate being within the frame rate interval; determining second data according to the first frame rate and the first data, information of the second data being used for instructing a server to perform a first operation. According to the embodiment of the application, the frame rate of uploaded data can be dynamically changed according to different states indicated by the index value of the target vehicle, so that redundant data can be eliminated, and the speed of data uploading can be improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method, apparatus and storage medium. Background Technology

[0002] In the field of autonomous driving, vehicles need to collect data through their own sensors. The sensors in an autonomous vehicle are expected to generate 80GB of data per hour, but the proportion of high-value data is as low as 10%. There is a lot of redundant data, which has little effect on analyzing the vehicle's driving status, but increases the cost of storage and playback.

[0003] Currently, when processing sensor data, related technologies typically collect all data in real time or only when the algorithm fails, resulting in a large amount of redundant data and low data processing efficiency. Summary of the Invention

[0004] In view of this, a data processing method, apparatus and storage medium are proposed.

[0005] In a first aspect, embodiments of this application provide a data processing method, the method comprising: acquiring first data collected by a sensor; determining a first frame rate based on a frame rate range and an indicator value, wherein the indicator value is used to indicate the state of a target vehicle, and the first frame rate is within the frame rate range; and determining second data based on the first frame rate and the first data, wherein information in the second data is used to instruct a server to perform a first operation.

[0006] According to the embodiments of this application, for the first data collected by the sensor, the first frame rate of the uploaded data can be dynamically determined according to the different states indicated by the index values ​​of the target vehicle. Based on the first frame rate and the first data, the second data used to instruct the server to perform the first operation can be determined, thereby eliminating redundant data in the first data, improving the value of the uploaded second data, and speeding up the data upload speed. By keeping the first frame rate within the frame rate range, data redundancy can be reduced while meeting operational requirements.

[0007] According to the first aspect, in one possible implementation, determining a first frame rate based on a frame rate range and indicator values ​​includes: if all indicator values ​​in the indicator values ​​exceed the threshold corresponding to each indicator value, the first frame rate is the upper limit of the frame rate range; or if some indicator values ​​in the indicator values ​​exceed the threshold corresponding to each indicator value, the first frame rate is greater than the lower limit of the frame rate range and less than the upper limit of the frame rate range; or if all indicator values ​​in the indicator values ​​do not exceed the threshold corresponding to each indicator value, the first frame rate is the lower limit of the frame rate range.

[0008] According to the embodiments of this application, the indicator value is used to indicate the state of the target vehicle. The more indicator values ​​exceed the threshold, the more complex the driving scenario is, requiring a larger amount of data to complete the first operation, i.e., requiring a larger first frame rate. Conversely, the fewer indicator values ​​exceed the threshold, the simpler or more conventional the driving scenario is, requiring a smaller amount of data to complete the first operation, i.e. requiring a smaller first frame rate.

[0009] By associating the first frame rate with the number of indicator values ​​exceeding a threshold, and setting the first frame rate as the upper and lower limits of the frame rate range for both cases where all indicator values ​​exceed the threshold and cases where none exceed the threshold, and by ensuring that the first frame rate lies between the upper and lower limits when some indicator values ​​exceed the threshold, the system guarantees that the first frame rate remains within the frame rate range while adapting to the number of indicator values ​​exceeding the threshold. This allows the first frame rate to adapt to changes in the target vehicle's state, reducing data redundancy while meeting operational requirements.

[0010] According to the first aspect, in one possible implementation, the method further includes: obtaining a first message; processing the second data according to the first message to obtain third data, the size of the third data being less than or equal to the size of the second data, and the information of the third data being used to instruct the server to perform the first operation.

[0011] According to the embodiments of this application, by obtaining the first message, the vehicle can further process the data according to the instructions of the first message, thereby further reducing the amount of data uploaded to the server, thus ensuring the smoothness and timeliness of the first operation.

[0012] According to the first aspect, in one possible implementation, the first message indicates that the time difference between the second moment and the first moment exceeds a predetermined threshold, wherein the first moment is the moment when the server receives the second data or the third data, and the second moment is the moment when the server starts to execute the first operation based on the received second data or the third data.

[0013] According to the embodiments of this application, the server can maintain synchronization in receiving data and performing the first operation, thereby improving the timeliness of the data.

[0014] According to the first aspect, in one possible implementation, the first operation includes one or more of the following: data playback, data annotation, scene mining, and data simulation.

[0015] According to the embodiments of this application, by including multiple types of first operations, the type of first operation to be performed on the uploaded data can be flexibly selected as needed.

[0016] Secondly, embodiments of this application provide a data processing method, the method comprising: acquiring second data sent by a target vehicle, the second data being determined by the target vehicle based on a first frame rate and first data collected by a sensor, wherein the first frame rate is determined based on a frame rate range and an indicator value, the indicator value being used to indicate the state of the target vehicle, and the first frame rate being within the frame rate range; and performing a first operation based on the indication of the information in the second data.

[0017] According to the embodiments of this application, by obtaining the second data which is determined by the target vehicle based on the first frame rate and the first data collected by the sensor, the frame rate of the second data obtained by the server can be adjusted according to the state of the target vehicle, thereby increasing the value of the data obtained by the server. By keeping the first frame rate within the frame rate range, data redundancy can be reduced while meeting the operational requirements of the server when performing the first operation, and the speed at which the server obtains data can be adapted to the speed at which the server performs the operation.

[0018] According to the second aspect, in one possible implementation, the first frame rate is determined based on a frame rate range and indicator values, including: when all indicator values ​​in the indicator range exceed the threshold corresponding to each indicator value, the first frame rate is the upper limit of the frame rate range; or when some indicator values ​​in the indicator range exceed the threshold corresponding to each indicator value, the first frame rate is greater than the lower limit of the frame rate range and less than the upper limit of the frame rate range; or when all indicator values ​​in the indicator range do not exceed the threshold corresponding to each indicator value, the first frame rate is the lower limit of the frame rate range.

[0019] According to the embodiments of this application, the indicator values ​​are used to indicate the state of the target vehicle. The more indicator values ​​exceed the threshold, the more complex the driving scenario, requiring a larger amount of data to complete the first operation, i.e., a larger first frame rate. Conversely, the fewer indicator values ​​exceed the threshold, the simpler or more conventional the driving scenario, requiring a smaller amount of data to complete the first operation, i.e., a smaller first frame rate. By associating the size of the first frame rate with the number of indicator values ​​exceeding the threshold, and making the first frame rate the upper and lower limits of the frame rate range in the two cases where all indicator values ​​exceed the threshold and none exceed the threshold, and ensuring that the first frame rate is between the upper and lower limits when some indicator values ​​exceed the threshold, it is guaranteed that the first frame rate is within the frame rate range and can adapt to the number of indicator values ​​exceeding the threshold. This allows the first frame rate to adapt to changes in the state of the target vehicle, reducing data redundancy while meeting operational requirements.

[0020] According to the second aspect, in one possible implementation, performing a first operation based on the indication of information in the second data includes: obtaining third data, the size of which is less than or equal to the size of the second data, based on the indication of information in the second data; and performing the first operation based on the indication of information in the third data.

[0021] According to the embodiments of this application, the server can further process the data, thereby further reducing the amount of data acquired by the server, thus ensuring the smoothness and timeliness of the server when performing the first operation.

[0022] According to the second aspect, in one possible implementation, the method further includes: generating a first message indicating that the time difference between the second moment and the first moment exceeds a predetermined threshold, the first moment being the moment when the second data is received, and the second moment being the moment when the first operation is started based on the second data.

[0023] According to the embodiments of this application, by generating a first message, when the time difference between the moment the server starts executing the first operation and the moment the server receives the data is too large, the first message can instruct the vehicle to process the second data to reduce the amount of the second data, thereby enabling the server to maintain the synchronization of receiving data and executing the first operation, and improving the smoothness and timeliness of the server performing the first operation.

[0024] According to the second aspect, in one possible implementation, the method further includes: sending the first message to the target vehicle, the first message being used to instruct the target vehicle to process the first data.

[0025] According to the embodiments of this application, by sending a first message, the server can notify the target vehicle to process the data, thereby improving the smoothness and timeliness of the server performing the first operation.

[0026] According to the second aspect, in one possible implementation, the first operation includes one or more of the following: data playback, data annotation, scene mining, and data simulation.

[0027] According to the embodiments of this application, by including multiple types of first operations, the server can flexibly select the type of first operation to be performed on the second data as needed.

[0028] Thirdly, embodiments of this application provide a data processing apparatus, which includes: a first acquisition module for acquiring first data collected by a sensor; a first determination module for determining a first frame rate based on a frame rate range and an indicator value, wherein the indicator value is used to indicate the state of a target vehicle, and the first frame rate is within the frame rate range; and a second determination module for determining second data based on the first frame rate and the first data, wherein information in the second data is used to instruct a server to perform a first operation.

[0029] According to the third aspect, in one possible implementation, determining the first frame rate based on the frame rate range and indicator values ​​includes: if all indicator values ​​in the indicator values ​​exceed the threshold corresponding to each indicator value, the first frame rate is the upper limit of the frame rate range; or if some indicator values ​​in the indicator values ​​exceed the threshold corresponding to each indicator value, the first frame rate is greater than the lower limit of the frame rate range and less than the upper limit of the frame rate range; or if all indicator values ​​in the indicator values ​​do not exceed the threshold corresponding to each indicator value, the first frame rate is the lower limit of the frame rate range.

[0030] According to a third aspect, in one possible implementation, the apparatus further includes: a third acquisition module for acquiring a first message; and a processing module for processing the second data according to the first message to obtain third data, wherein the size of the third data is less than or equal to the size of the second data, and the information in the third data is used to instruct the server to perform the first operation.

[0031] According to the third aspect, in one possible implementation, the first message indicates that the time difference between the second moment and the first moment exceeds a predetermined threshold, wherein the first moment is the moment when the server receives the second data or the third data, and the second moment is the moment when the server starts to execute the first operation based on the received second data or the third data.

[0032] According to the third aspect, in one possible implementation, the first operation includes one or more of the following: data playback, data annotation, scene mining, and data simulation.

[0033] Fourthly, embodiments of this application provide a data processing apparatus, comprising: a second acquisition module, configured to acquire second data sent by a target vehicle, the second data being determined by the target vehicle based on a first frame rate and first data collected by a sensor, wherein the first frame rate is determined based on a frame rate range and an indicator value, the indicator value being used to indicate the state of the target vehicle, and the first frame rate being within the frame rate range; and an operation module, configured to perform a first operation based on the indication of information in the second data.

[0034] According to the fourth aspect, in one possible implementation, the first frame rate is determined based on a frame rate range and indicator values, including: when all indicator values ​​in the indicator range exceed the threshold corresponding to each indicator value, the first frame rate is the upper limit of the frame rate range; or when some indicator values ​​in the indicator range exceed the threshold corresponding to each indicator value, the first frame rate is greater than the lower limit of the frame rate range and less than the upper limit of the frame rate range; or when all indicator values ​​in the indicator range do not exceed the threshold corresponding to each indicator value, the first frame rate is the lower limit of the frame rate range.

[0035] According to the fourth aspect, in one possible implementation, performing a first operation based on the indication of the information in the second data includes: obtaining third data, the size of which is less than or equal to the size of the second data, based on the indication of the information in the second data; and performing the first operation based on the indication of the information in the third data.

[0036] According to the fourth aspect, in one possible implementation, the apparatus further includes: a generation module for generating a first message, the first message indicating that the time difference between the second moment and the first moment exceeds a predetermined threshold, the first moment being the moment when the second data is received, and the second moment being the moment when the first operation is started based on the second data.

[0037] According to the fourth aspect, in one possible implementation, the apparatus further includes: a sending module for sending the first message to the target vehicle, the first message being used to instruct the target vehicle to process the first data.

[0038] According to the fourth aspect, in one possible implementation, the first operation includes one or more of the following: data playback, data annotation, scene mining, and data simulation.

[0039] Fifthly, embodiments of this application provide a data processing apparatus, the apparatus comprising: a processor and a memory; the memory being used to store a program; the processor being used to execute the program stored in the memory, so that the apparatus implements one or more of the data processing methods of the first aspect or multiple possible implementations of the first aspect, or implements one or more of the data processing methods of the second aspect or multiple possible implementations of the second aspect.

[0040] Sixthly, embodiments of this application provide a computer-readable storage medium, including: computer instructions, which, when executed, implement one or more data processing methods of the first aspect or multiple possible implementations of the first aspect, or implement one or more data processing methods of the second aspect or multiple possible implementations of the second aspect.

[0041] Seventhly, embodiments of this application provide a terminal device that can execute one or more data processing methods from the first aspect or various possible implementations of the first aspect, or execute one or more data processing methods from the second aspect or various possible implementations of the second aspect.

[0042] Eighthly, embodiments of this application provide a computer program product including instructions that, when executed on a computer, cause the computer to perform one or more data processing methods of the first aspect or multiple possible implementations thereof, or to perform one or more data processing methods of the second aspect or multiple possible implementations thereof.

[0043] Ninthly, embodiments of this application provide an in-vehicle computing device, which includes a processor for executing one or more data processing methods from the first aspect or various possible implementations of the first aspect.

[0044] In a tenth aspect, embodiments of this application provide a server including a processor for performing one or more data processing methods according to the second aspect or various possible implementations of the second aspect.

[0045] Eleventhly, embodiments of this application provide a chip system including at least one processor for supporting one or more data processing methods for implementing the first aspect or multiple possible implementations of the first aspect, or for implementing one or more data processing methods for implementing the second aspect or multiple possible implementations of the second aspect, such as receiving or processing data and / or information involved in the above methods.

[0046] According to the eleventh aspect, in one possible implementation, the chip system further includes a memory for storing program instructions and data, the memory being located within or outside the processor, and the chip system may be composed of chips or may contain chips and other discrete devices.

[0047] In a twelfth aspect, embodiments of this application provide a data processing apparatus, the data processing apparatus including at least one processor and a communication interface, the communication interface being used to send and / or receive data, the at least one processor being used to invoke a computer program stored in at least one memory, so that the data processing apparatus implements one or more of the data processing methods of the first aspect or multiple possible implementations of the first aspect, or implements one or more of the data processing methods of the second aspect or multiple possible implementations of the second aspect. Attached Figure Description

[0048] Figure 1 A schematic diagram of the architecture of a data processing system according to an embodiment of this application is shown.

[0049] Figure 2 A flowchart illustrating a data processing method according to an embodiment of this application is shown.

[0050] Figure 3 A flowchart illustrating a data processing method according to an embodiment of this application is shown.

[0051] Figure 4 A flowchart illustrating the acquisition phase of a data processing method according to an embodiment of this application is shown.

[0052] Figure 5 A schematic diagram of the acquisition phase according to an embodiment of this application is shown.

[0053] Figure 6 A flowchart illustrating a data processing method according to an embodiment of this application is shown.

[0054] Figure 7 A flowchart illustrating the execution phase of a data processing method according to an embodiment of this application is provided.

[0055] Figure 8 A flowchart illustrating a data processing method according to an embodiment of this application is shown.

[0056] Figure 9 A schematic diagram illustrating a data playback operation according to an embodiment of this application is shown.

[0057] Figure 10 A structural diagram of a data processing apparatus according to an embodiment of this application is shown.

[0058] Figure 11 A structural diagram of a data processing apparatus according to an embodiment of this application is shown.

[0059] Figure 12 A structural diagram of a data processing apparatus according to an embodiment of this application is shown. Detailed Implementation

[0060] Various exemplary embodiments, features, and aspects of this application will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0061] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0062] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented without certain specific details. In some instances, methods, means, components, and circuits well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0063] To address the aforementioned technical problems, this application provides a data processing method. The data processing method of this application embodiment enables vehicles to efficiently and efficiently upload data to a server, and the server to perform a first operation on the received data in real time. This method can be applied to a data processing system.

[0064] Figure 1 This diagram illustrates the architecture of a data processing system according to an embodiment of this application. To facilitate understanding of the embodiments of this application, we will first use... Figure 1 The data processing system shown is used as an example to illustrate the data processing system applicable to the embodiments of this application. It should be noted that the solutions in the embodiments of this application can also be applied to other data processing systems, and the corresponding names can be replaced by the names of the corresponding functions in other data processing systems. The data processing system may include a communication system, such as a vehicle-to-everything (V2X) communication system, a device-to-device (D2D) communication system, or a vehicle-to-everything (V2X) communication system. The data processing system can be based on one or more of the following communication standards: 4th generation (4G) mobile communication systems, such as Long Term Evolution (LTE) systems; 5th generation (5G) mobile communication systems, such as New Radio (NR) systems; and future mobile communication systems, such as 6th generation (6G) mobile communication systems.

[0065] like Figure 1As shown, the data processing system may include a vehicle and a server. The server can be a device located in the cloud or locally within the data processing system, possessing data processing capabilities. It can be a physical device such as a host or server, or a virtual device such as a virtual machine or container. It should be noted that for ease of description, it is referred to as a server here; in the actual implementation, it can be a server, other devices with data processing capabilities, or a module (such as a chip or integrated circuit) within a device. The server can be a server that collects data such as the vehicle's geographical location information, vehicle trajectory, and surrounding information, and has wired or wireless transceiver capabilities. These wired or wireless transceiver capabilities can be configured within the server's chip (system) or other components or parts.

[0066] The aforementioned vehicle is a vehicle capable of accessing the aforementioned data processing system and possessing wired or wireless transceiver capabilities. These wired or wireless transceiver capabilities can be located in the vehicle's onboard terminal, onboard module, onboard unit, chip (system), or other components or assemblies. The vehicle may also be referred to as a user device, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, or user agent. In the embodiments of this application, the vehicle may be a wireless terminal in self-driving, a wireless terminal in transportation safety, a wireless terminal in a smart city, an onboard terminal, or an RSU with terminal functionality, etc. This application does not limit the scope of these claims.

[0067] like Figure 1 As shown, the aforementioned vehicle may be equipped with at least one sensor, such as vehicle-mounted radar (e.g., millimeter-wave radar, infrared radar, lidar, Doppler radar, etc.), light sensor, rain sensor, audio / video sensor (e.g., camera, dashcam), vehicle attitude sensor (e.g., gyroscope), or speed sensor (e.g., Doppler radar), inertial measurement unit (IMU), etc. The aforementioned sensors may be installed on one vehicle or multiple vehicles. Through at least one sensor installed on the vehicle, image data, point cloud data, perception data, prediction data, planning data, control data, positioning data, chassis data, etc., stored frame by frame can be collected.

[0068] In this application embodiment, the server and the vehicle can communicate wirelessly, such as via Wi-Fi or Bluetooth. Alternatively, the server and the vehicle can communicate via wired connections, such as via fiber optic cables or electrical cables. For example, in testing, the server can be mounted on the vehicle and communicate with it via a wired connection. This application does not impose any restrictions on this, as long as it enables the exchange of data and information.

[0069] In one possible implementation, the data processing method of this application embodiment can be used in a scenario where a vehicle is performing autonomous driving. The vehicle can send all or part of the data collected by the sensor indicating the state of the target vehicle to the server according to a dynamically determined first frame rate. The server can use the data to perform at least one operation such as data playback, data annotation, scene mining, and data simulation. The vehicle and / or the server can further process the data to improve the efficiency of performing the above-mentioned data playback, data annotation, scene mining, and data simulation operations. The implementation of the above process can be referred to the following method embodiments, which will not be repeated here.

[0070] It should be understood that Figure 1 This is a simplified illustration for ease of understanding only. The data processing system may also include other servers, and / or other vehicles. Figure 1 The data processing method described in this application embodiment is not shown in the figure. It can also be used in other scenarios, and this application does not limit it.

[0071] Figure 2 A flowchart illustrating a data processing method according to an embodiment of this application is provided. This method can be executed by the aforementioned vehicle, such as... Figure 2 As shown, the following steps S201-S203 may be included:

[0072] Step S201: Acquire the first data collected by the sensor;

[0073] The sensor can be one or more, and this application does not limit the number of sensors or their installation method. For example, the sensor can be installed only on the target vehicle among multiple vehicles, or on other vehicles besides the target vehicle, or on other facilities besides vehicles, such as fixed facilities on both sides of the road. The first data collected by the sensor can be data specific to the target vehicle, such as the target vehicle's speed, acceleration, etc. For example, taking vehicle A as the target vehicle, the first data of vehicle A can be collected by the sensor on vehicle A; the first data of vehicle A can also be collected by the sensor on vehicle B and then sent to vehicle A or to a server; or the first data of vehicle A can be collected by the sensor on the fixed facility and then sent to vehicle A; or at least two of the sensors on vehicle A, the sensor on vehicle B, and the sensor on the fixed facility can collect the first data of vehicle A and summarize it to vehicle A. This application does not limit this. The vehicle executing the method of this application embodiment can be the target vehicle or other vehicles besides the target vehicle.

[0074] For example, the first data collected by the sensor can be one or more of the data mentioned above, such as image data, point cloud data, perception data, prediction data, planning data, control data, positioning data, and chassis data. The above data can be collected by one or more sensors. For example, image data can be collected by an audio-visual sensor (such as a camera), point cloud data can be collected by a lidar, and perception data can be collected by one or more of the sensors such as lidar, camera, and millimeter-wave radar. The frame rate of the sensor data collection can be determined according to the sensor's rated acquisition frame rate, which can be the frame rate determined when the sensor leaves the factory.

[0075] Step S202: Determine a first frame rate based on the frame rate range and the indicator value, wherein the indicator value is used to indicate the state of the target vehicle, and the first frame rate is within the frame rate range;

[0076] The indicator value represents the value of an indicator, which includes various types of indicators that can be used to represent the status of the target vehicle. The first frame rate refers to the frequency of data uploaded by the vehicle to the server. The frame rate range represents the range of the set first frame rate. According to the determined frame rate range, the maximum value of the first frame rate of the vehicle's uploaded data does not exceed the upper limit of the frame rate range, and the minimum value is not lower than the lower limit of the frame rate range.

[0077] Step S203: Determine second data based on the first frame rate and the first data, wherein the information in the second data is used to instruct the server to perform the first operation.

[0078] The information in the second data can refer to the data content of the second data or the information carried in the second data. Under the guidance of the information in the second data, a first operation can be performed. For example, when the second data is image data, the information in the second data can include pixel values, image acquisition time, etc., and under the guidance of the information in the second data, a first operation such as image playback can be performed. When the second data is positioning data, the information in the second data can include coordinate values, etc., and under the guidance of the information in the second data, a first operation such as location marking can be performed. This application does not limit the specific content of the information in the second data.

[0079] As can be seen, since the first frame rate corresponding to the uploaded second data can be different at different times, the amount of second data uploaded by the vehicle at different times can be different. For example, in complex driving scenarios (such as when the target vehicle is making a sharp turn or at an intersection), the amount of second data uploaded is larger, while in normal driving scenarios (such as when the target vehicle is moving at a constant speed on a straight road), the amount of second data uploaded is smaller.

[0080] According to the embodiments of this application, for the first data collected by the sensor, the first frame rate of the uploaded data can be dynamically determined according to the different states indicated by the index values ​​of the target vehicle. Based on the first frame rate and the first data, the second data used to instruct the server to perform the first operation can be determined, thereby eliminating redundant data in the first data, improving the value of the uploaded second data, and speeding up the data upload speed. By keeping the first frame rate within the frame rate range, data redundancy can be reduced while meeting operational requirements.

[0081] Figure 3 A flowchart illustrating a data processing method according to an embodiment of this application is provided. This method can be executed via the aforementioned server, such as... Figure 3 As shown, the following steps S301-S302 may be included:

[0082] Step S301: Obtain second data sent by the target vehicle. The second data is determined by the target vehicle based on the first frame rate and the first data collected by the sensor. The first frame rate is determined based on the frame rate range and the index value. The index value is used to indicate the state of the target vehicle. The first frame rate is within the frame rate range.

[0083] Step S302: Perform the first operation according to the instructions of the information in the second data.

[0084] According to the embodiments of this application, by obtaining the second data which is determined by the target vehicle based on the first frame rate and the first data collected by the sensor, the frame rate of the second data obtained by the server can be adjusted according to the state of the target vehicle, thereby increasing the value of the data obtained by the server. By keeping the first frame rate within the frame rate range, data redundancy can be reduced while meeting the operational requirements of the server when performing the first operation, and the speed at which the server obtains data can be adapted to the speed at which the server performs the operation.

[0085] In one possible implementation, the first operation mentioned above includes one or more of the following: data playback, data annotation, scene mining, and data simulation.

[0086] According to the embodiments of this application, by including multiple types of first operations, the server can flexibly select the type of first operation to be performed on the second data as needed.

[0087] In one possible implementation, determining the first frame rate in step S202 based on the frame rate range and the indicator values ​​may include: if all indicator values ​​in the indicator values ​​exceed the threshold corresponding to each indicator value, the first frame rate is the upper limit of the frame rate range; or if some indicator values ​​in the indicator values ​​exceed the threshold corresponding to each indicator value, the first frame rate is greater than the lower limit of the frame rate range and less than the upper limit of the frame rate range; or if all indicator values ​​in the indicator values ​​do not exceed the threshold corresponding to each indicator value, the first frame rate is the lower limit of the frame rate range.

[0088] According to the embodiments of this application, the indicator values ​​are used to indicate the state of the target vehicle. The more indicator values ​​exceed the threshold, the more complex the driving scenario, requiring a larger amount of data to complete the first operation, i.e., a larger first frame rate. Conversely, the fewer indicator values ​​exceed the threshold, the simpler or more conventional the driving scenario, requiring a smaller amount of data to complete the first operation, i.e., a smaller first frame rate. By associating the size of the first frame rate with the number of indicator values ​​exceeding the threshold, and making the first frame rate the upper and lower limits of the frame rate range in the two cases where all indicator values ​​exceed the threshold and none exceed the threshold, and ensuring that the first frame rate is between the upper and lower limits when some indicator values ​​exceed the threshold, it is guaranteed that the first frame rate is within the frame rate range and can adapt to the number of indicator values ​​exceeding the threshold. This allows the first frame rate to adapt to changes in the state of the target vehicle, reducing data redundancy while meeting operational requirements.

[0089] The method for determining the first frame rate can be referred to in the following formulas (1)-(5). Those skilled in the art should understand that the method for determining the first frame rate is not limited to the examples provided below. As long as the first frame rate changes with the state of the target vehicle, the amount of data of the second data determined according to the first frame rate can meet the requirements of the first operation in the state of the target vehicle. For example, other methods can also be used to make the first frame rate increase as the number of indicators whose index values ​​exceed the threshold increases.

[0090] The following is combined Figures 4 to 9 The flow of the data processing method provided in this application embodiment will be further illustrated using vehicles and servers as examples.

[0091] Figure 4 A flowchart illustrating the acquisition phase of the data processing method according to an embodiment of this application is provided. An exemplary description of step S401 can be found in step S201 above, and steps S402-S404 can be used as examples of step S202 above. An exemplary description of step S406 can be found in step S203 above.

[0092] like Figure 4 As shown, the data acquisition phase may include the following steps:

[0093] Step S401: The vehicle acquires the first data collected by the sensors.

[0094] Step S402: The vehicle determines the index value and frame rate range.

[0095] By analyzing the relationship between indicator values ​​and thresholds, it can be determined whether the target vehicle is in the state indicated by the indicator. The target vehicle's state can include its own state, the environmental state it is in, and the road state it is in. Indicators corresponding to the target vehicle's own state can include rapid acceleration / deceleration, large rate of change of acceleration, sharp turns, U-turns, reversing, merging into a lane, and leaving a lane. Indicators corresponding to the environmental state of the target vehicle can include poor weather (rain, snow, sandstorms, smog), low light intensity (night, tunnels), slippery road surfaces (ice, water accumulation), traffic jams, and many pedestrians. Indicators corresponding to the road state of the target vehicle can include complex intersections (crossroads, Y-shaped intersections), uphill, downhill, curves, zebra crossings, rural roads, mountain roads, closed parks, and abnormal scenarios. In this embodiment, the indicators determined for the vehicle can include one or more of the above. By combining various indicators, any customized complex scenario can be achieved. For example, if the values ​​of the three indicators—sharp turn, slippery road surface, and rural road—meet a predetermined relationship with their corresponding thresholds (e.g., are greater than the threshold), it can indicate a scenario where the target vehicle makes a sharp turn on a slippery rural road. Table 1 shows an example of the correspondence between indicators, the target vehicle state indicated by the indicators, and the basis for determining the indicator values ​​and thresholds.

[0096] Table 1

[0097]

[0098] It should be noted that Table 1 only shows a portion of the indicators. The number of indicators determined for a vehicle can be more or less, and may include a portion of the indicators shown in Table 1, or may include other indicators besides those shown in Table 1. This application embodiment does not limit this.

[0099] For example, as shown in Table 1, the "basis for determining indicator values ​​and thresholds" in Table 1 can be a method for determining the target vehicle's state as indicated by the indicator. For instance, corresponding to the indicator "rapid acceleration / rapid deceleration," the indicator value can be the acceleration of the target vehicle. An acceleration threshold can be set. If the current indicator value is greater than the acceleration threshold, the target vehicle is determined to be in a "rapid acceleration / rapid deceleration" state. Furthermore, different acceleration thresholds can be set for different speeds. If the current indicator value is greater than the acceleration threshold at the current speed, the target vehicle is determined to be in a "rapid acceleration / rapid deceleration" state. Similarly, corresponding to the indicator "large rate of change of acceleration," the indicator value can be the rate of change of acceleration of the target vehicle. The rate of change of acceleration can be determined by acceleration. A threshold for the rate of change of acceleration can be set. If the current indicator value is greater than the rate of change of acceleration threshold, the target vehicle is determined to be in a "rapid acceleration / rapid deceleration" state. Under the condition of (or the corresponding acceleration change rate threshold at the current speed), the current target vehicle is determined to be in a state of "large acceleration change rate"; corresponding to the indicator "sharp turn", the indicator value can be the lateral speed of the target vehicle, and a threshold for lateral speed can be set. If the current indicator value is greater than the lateral speed threshold, the current target vehicle is determined to be in a state of "sharp turn"; corresponding to the indicator "curve", the indicator value can be the curve wheel axle angle of the target vehicle, and a threshold for curve wheel axle angle can be set. If the current indicator value is greater than the curve wheel axle angle threshold, the current target vehicle is determined to be in a state of "curve"; corresponding to the indicator "many pedestrians", the indicator value can be the number of perceived targets (pedestrians) of the target vehicle, and a threshold for the number of perceived targets (pedestrians) can be set. If the current indicator value is greater than the number threshold, the current target vehicle is determined to be in a state of "many pedestrians".

[0100] The above-mentioned index values ​​can be determined based on the first data collected by the sensors. For example, the index values ​​(acceleration values) for "rapid acceleration / rapid deceleration" and "large rate of change of acceleration" can be determined based on the speed collected by the speed sensor; the index value (lateral speed value) for "sharp turn" can be determined based on the lateral speed collected by the speed sensor; the index value (curving wheel axle angle) for "curving" can be determined based on the turning wheel axle angle collected by the vehicle posture sensor; and the index value (number of pedestrians) for "many pedestrians" can be determined based on the image data collected by the camera, and the number of pedestrians can be identified based on the image data.

[0101] One or more of the aforementioned metrics, frame rate ranges, and thresholds can be pre-set in the vehicle; they can also be pre-set in other devices and obtained by the vehicle from those devices; they can be pre-set on a server and then deployed to the vehicle via the network; or they can be input into the vehicle by the user as needed. This application does not impose any restrictions on this. Correspondingly, one or more of the metrics, frame rate ranges, and thresholds can also be modified. For example, if they are pre-set in the vehicle, the metrics, frame rate ranges, or thresholds in the vehicle can be reset; if they are pre-set in other devices, the metrics, frame rate ranges, or thresholds in those other devices can be reset, and the vehicle can obtain the modified metrics, frame rate ranges, or thresholds from those other devices; if they are pre-set on a server, the metrics, frame rate ranges, or thresholds in the server can be reset and redeployed to the vehicle via the network; if they are directly input into the vehicle by the user, the user can directly input new metrics, frame rate ranges, or thresholds for modification.

[0102] In step S403, the vehicle determines the weights corresponding to each indicator based on the dynamic data collection strategy.

[0103] Among them, the dynamic acquisition strategy can refer to the method of determining the first frame rate.

[0104] In one possible implementation, the weights corresponding to each indicator can be determined first, and then the first frame rate can be determined based on the weights corresponding to each indicator. An exemplary way to determine the weights corresponding to each indicator value can be referred to the following formulas (1)-(3).

[0105]

[0106]

[0107]

[0108] Formula (1) can be used to obtain the information content C of the corresponding indicator. The larger the value of C, the greater the objective role of the corresponding indicator among all indicators. r can represent the standardized matrix of the judgment matrix A. ij This can represent the element in the i-th row and j-th column of the standardized matrix r, where n is the total number of indicators. In the judgment matrix A, each row can correspond to one indicator, and each column can also correspond to one indicator. ij The judgment matrix A can represent the importance of the indicator corresponding to the i-th row relative to the indicator corresponding to the j-th column. The judgment matrix A can be set when setting the indicators. This application does not restrict the method of obtaining the judgment matrix A. j δ can represent the information content of the indicator corresponding to the j-th column in A. j This represents the standard deviation of column j in A.

[0109] Formula (2) can be used to obtain the objective weight W of the corresponding indicator. critic W criticj Based on the information content C corresponding to column j in A j The proportion of the total information in all columns (1-n) can be used to represent the objective weight of the indicator corresponding to column j in A.

[0110] Formula (3) can be used to obtain the weight W of the corresponding indicator. i W can represent the weight of the indicator corresponding to the i-th column in A, and the weight values ​​W of indicators 1-n. i The sum of W is 1. ahp W can represent the weight vector of A. ahpi W can represent the weight of the i-th index in the weight vector. critici The objective weight of the indicator corresponding to the i-th column in A can be represented (which can be obtained according to formulas (1) and (2)). Table 2 shows an example of the judgment matrix A and the weights of each indicator calculated using formulas (1) to (3).

[0111] Table 2

[0112] Indicator 1 1 1 / 2 4 3 3 0.26 Indicator 2 2 1 7 5 5 0.48 Indicator 3 1 / 4 1 / 7 1 1 / 2 1 / 3 0.06 Indicator 4 1 / 3 1 / 5 2 1 1 0.09 Indicator 5 1 / 3 1 / 5 3 1 1 0.11

[0113] As shown in Table 2, Indicators 1-5 can each correspond to 5 different indicators (e.g., the 5 indicators in Table 1). The matrix formed by the values ​​in columns 2-6 of Table 2 can be the judgment matrix A corresponding to Indicators 1-5. For example, the value of (Indicator 1, Indicator 2) is 1 / 2, which can indicate the importance of Indicator 1 relative to Indicator 2. The larger the value of (Indicator 1, Indicator 2), the more important Indicator 1 is relative to Indicator 2. For example, in the examples of Indicators 1-5 in Table 2, the value of (Indicator 2, Indicator 3) is 7, which can indicate that Indicator 2 is relatively important relative to Indicator 3. For example, Indicator 2 is "high rate of change of acceleration," and Indicator 3 is "sharp turn," so the correlation between the two is relatively strong, and therefore Indicator 2 is relatively important relative to Indicator 3. The value of (Indicator 4, Indicator 2) is 1 / 5, which can indicate that Indicator 4 is not very important relative to Indicator 2. For example, Indicator 4 is "curve," and Indicator 2 is "high rate of change of acceleration," so the correlation between the two is not strong, and therefore Indicator 4 is not very important relative to Indicator 2. The meanings of the other values ​​in columns 2-6 of Table 2 can be deduced similarly. The last column of Table 2 shows the weights of each indicator calculated using formulas (1)-(3). According to the judgment matrix shown in Table 2, the weight of indicator 1 is 0.26, the weight of indicator 2 is 0.48, the weight of indicator 3 is 0.06, the weight of indicator 4 is 0.09, and the weight of indicator 5 is 0.11.

[0114] It should be noted that when calculating the weights corresponding to the indicators, the weights corresponding to each indicator can be calculated using the analytic hierarchy process (AHP) based on the judgment matrix A, or the weights corresponding to each indicator can be calculated directly using the analytic network process (ANP) without using the judgment matrix A. This application does not restrict the method for determining the weights corresponding to the indicators.

[0115] In step S404, the vehicle determines the first frame rate based on the weights and frame rate ranges corresponding to each indicator.

[0116] In this process, after the weights of each indicator are determined for the vehicle, it can be determined whether the value of each indicator satisfies the condition that the target vehicle is in the state indicated by the indicator. The weights of the indicators that satisfy the above conditions are added together and scored on a percentage basis, as shown in formula (4).

[0117]

[0118] Where Score can represent a score indicating the current state of the target vehicle, p i This can represent whether the value of the i-th indicator satisfies the condition that the target vehicle is in the target vehicle state indicated by the indicator. i When the value of p is 0, it can indicate that the i-th indicator does not meet the condition. i When the value is 1, it indicates that the i-th indicator meets the condition. Meeting the condition can include the indicator value exceeding the corresponding threshold. For example, in the case of the indicator "curve," if the current indicator value is greater than the set curve wheel axle angle threshold, it can be determined that the above condition is met, and p... i The value is 1.

[0119] After obtaining the score, the first frame rate can be obtained according to the set frame rate range and the mean method, see formula (5):

[0120] FR = Score * (f max -f min ) / 100+f min Formula (5)

[0121] Where FR can represent the first frame rate, f max It can represent the upper limit of the frame rate range, f min It can represent the lower limit of the frame rate range.

[0122] Without changing the weights of the indicators, since the indicator values ​​can be determined based on the data collected by the sensor in real time, the indicators that meet the above conditions can also be different at different times. Therefore, the score can change accordingly at different times, so the first frame rate can be dynamically updated.

[0123] Where all index values ​​do not exceed the corresponding threshold, i.e., all p i If the value is 0, then the score is 0, and the first frame rate FR is the lower limit of the frame rate range. min If all indicator values ​​exceed the corresponding threshold, that is, all p... i If the value is 1, then the score is 100, and the first frame rate FR is the upper limit of the frame rate range f. max If some metric values ​​exceed the corresponding threshold, the Score will be a value between 0 and 100. The first frame rate (FR) is greater than the lower limit and less than the upper limit. The value of the Score depends on the value of p (which is 1). i The number of frames, the first frame rate (FR) value depends on the Score value, a value of 1 for p i The larger the number, the more indicator values ​​exceed the threshold, indicating that the target vehicle is in the state indicated by the indicator. The higher the first frame rate, the more data is transmitted when the vehicle is in a state with many indicator indicators (such as rapid acceleration, multiple indicator indicators with many pedestrians, etc.), so that the data volume meets the needs of complex scenarios. When the vehicle is in a state with fewer indicator indicators, less data is transmitted according to the lower first frame rate, so that the transmitted data meets the requirements and reduces redundancy.

[0124] In one possible implementation, after the weights of each indicator for the vehicle are determined, the score can be skipped, and the first frame rate can be obtained directly using the mean method, as shown in formula (6):

[0125]

[0126] In one possible implementation, the vehicle can determine the first frame rate directly based on the indicator values ​​and frame rate ranges, without needing to determine the weights corresponding to each indicator. For example, the p value corresponding to each indicator can be determined based on the indicator values. i According to each indicator, the corresponding p i The ratio of the number of values ​​of 1 to the total number of indicators determines the first frame rate, see formula (7):

[0127]

[0128] Referring to the above analysis, similarly, the first frame rate determined based on formula (6) or (7) also satisfies that it is within the frame rate range and increases as the number of indicators whose values ​​exceed the threshold increases.

[0129] It should be noted that the above method for determining the first frame rate is only an example for reference. This application does not limit the method for determining the first frame rate, as long as the first frame rate can be adjusted accordingly based on the changes in the state of the target vehicle.

[0130] In step S405, the vehicle determines the second data based on the first data and the first frame rate.

[0131] The vehicle can use the data corresponding to the first frame rate in the first data as the second data, and the second data can be all or part of the data in the first data.

[0132] For example, when the frame rate of the sensor acquiring the first data is 30Hz (i.e., 30 frames / second) and the first frame rate is 15Hz (i.e., 15 frames / second), for the first data within each second, 15 frames of data can be selected as the corresponding second data. Specifically, one frame of data can be selected as the corresponding second data every other frame of data, or the first 15 frames of data within the 30 frames of the first data can be selected as the corresponding second data. As long as the second data within each second is 15 frames less than the first data, this application does not impose any restrictions on this.

[0133] In step S406, the vehicle uploads the second data to the server.

[0134] In one possible implementation, the vehicle can upload the second data to a storage device in the server for the server to perform relevant operations.

[0135] Therefore, the amount of second data uploaded by the vehicle can be dynamically changed according to the change of the target vehicle's status. This can eliminate redundant data, so that the second data obtained by the server is of higher value, which can meet the current needs and avoid data redundancy. At the same time, by dynamically adjusting the amount of second data, the cost of server data storage can also be reduced and the data upload speed can be accelerated.

[0136] Figure 5 A schematic diagram of the data acquisition phase according to an embodiment of this application is shown. The indicators for determining the target vehicle can be as shown in Table 1.

[0137] like Figure 5As shown, for the target vehicle "Hu A XXXXX", corresponding to each indicator of the target vehicle, the indicator values detected by the current vehicle that exceed the set threshold are respectively: acceleration: 0.4g, corresponding to the indicator "rapid acceleration / rapid deceleration"; corner axle angle: 15°, corresponding to the indicator "cornering". When it is determined that the weight corresponding to the indicator "rapid acceleration / rapid deceleration" is 0.26, the weight corresponding to the indicator "cornering" is 0.48, and the set frame rate interval is [0.1, 5], the first frame rate at this time can be calculated as 3.7Hz according to formula (4) to formula (5). At this time, the vehicle can upload the second data corresponding to the first frame rate of 3.7Hz in the first data to the server. For example, 3.7Hz can be rounded off, and the corresponding second data in the first data can be uploaded to the server at a frame rate of 4Hz.

[0138] In a possible implementation, the server or the vehicle may further adjust the bandwidth when a preset condition is satisfied. The preset condition may be a condition set for the time delay of the server performing the first operation based on the second data relative to the server receiving the second data.

[0139] For example, the second data received by the server may be stored in a memory of the server. When the server performs the first operation based on the second data, the second data can be obtained from the memory in the server. At this time, the bandwidth of data transmission between the memory and the module configured to perform the first operation may fluctuate. Such fluctuation may be reflected in the time delay existing when the server performs the first operation based on the received second data. In order to make the receiving of the second data and the performing of the first operation based on the second data proceed as synchronously as possible and reduce the time delay, the server may perform processing based on the received second data. In a possible implementation, the server may also notify the vehicle, and the vehicle processes the second data. For example, the manner of processing the second data may further reduce the data volume of the data to adapt to the current transmission bandwidth.

[0140] Wherein, after the second data is processed, the execution speed of the server performing operations based on the processed data can be increased, so as to shorten the time delay between performing the first operation based on the data and receiving the data thereafter, so that the server performing operations based on the data can be as synchronous as possible with the server receiving the data, thereby improving the timeliness of the data.

[0141] For the above example of processing the second data, reference may be made to the following description in conjunction with Figures 6-9 below.

[0142] Figure 6 shows a flowchart of a data processing method according to an embodiment of the present application. Figure 6 illustrates exemplarily the above scenario where the server notifies the vehicle and the vehicle processes the second data.

[0143] like Figure 6 As shown, the process may include the following steps S601-S602, wherein steps S601-S602 can be performed by the aforementioned vehicle:

[0144] Step S601: Obtain the first message;

[0145] Step S602: Based on the first message, process the second data to obtain third data. The size of the third data is less than or equal to the size of the second data. The information in the third data is used to instruct the server to perform the first operation.

[0146] According to the embodiments of this application, by obtaining the first message, the vehicle can further process the data according to the instructions of the first message, thereby further reducing the amount of data uploaded to the server, thus ensuring the smoothness and timeliness of the first operation.

[0147] In one possible implementation, the first message indicates that the time difference between the second moment and the first moment exceeds a predetermined threshold, wherein the first moment is the moment when the server receives the second data or the third data, and the second moment is the moment when the server starts to execute the first operation based on the received second data or the third data.

[0148] The first message may be generated by the server, meaning the server may also perform the following steps: generating the first message and sending the first message to the target vehicle, wherein the first message is used to instruct the target vehicle to process the first data.

[0149] According to the embodiments of this application, by generating a first message, when the time difference between the moment the server starts executing the first operation and the moment the server receives the data is too large, the first message can instruct the vehicle to process the second data to reduce the amount of the second data, thereby enabling the server to maintain the synchronization of receiving data and executing the first operation, and improving the smoothness and timeliness of the server performing the first operation.

[0150] For example, in a scenario where a vehicle processes the second data to obtain the third data, the server notifies the vehicle, for instance, by generating a first message indicating that the time difference between the second and first moments exceeds a threshold, and then sending this first message to the vehicle. Upon receiving this first message, the vehicle can process the second data according to its instructions to obtain the third data, and then send the third data to the server. Data transmission between the vehicle and the server can occur in real time; that is, before receiving the third data, the vehicle can still send unprocessed second data to the server until the third data is received.

[0151] Figure 7 A flowchart illustrating a data processing method according to an embodiment of this application is shown. Figure 7 This example illustrates the scenario described above where the server processes the received second data. For instance... Figure 7 As shown, when the server performs step S302 above and performs the first operation according to the information in the second data, it may include the following steps:

[0152] Step S701: Obtain third data according to the information of the second data, wherein the size of the third data is less than or equal to the size of the second data;

[0153] Step S702: Perform the first operation according to the instructions of the information in the third data.

[0154] According to the embodiments of this application, the server can further process the data, thereby further reducing the amount of the second data, thus ensuring the smoothness and timeliness of the server when performing the first operation.

[0155] For example, when the server processes the second data to obtain the third data, it could be that the second data received by the server's interface for receiving external data is processed to obtain the third data, and then the third data is sent to the module for performing the first operation. Alternatively, if the server stores the received second data in its memory, the server could also process the second data in memory to obtain the third data and then send the third data to the module for performing the first operation. Before the module for performing the first operation obtains the third data, the module may still be receiving unprocessed second data (either from the server's interface for receiving external data or from memory) until the module for performing the first operation receives the third data.

[0156] The information in the third data can refer to the data content of the third data or the information carried by the third data.

[0157] The above Figure 6 and Figure 7 Examples of the process can be found below. Figure 8 The relevant steps in the process.

[0158] Figure 8 A flowchart illustrating the execution phases of a data processing method according to an embodiment of this application is provided. Figure 8 As shown, the process for performing the operation phase may include:

[0159] Step S801: The server performs the first operation based on the received second data.

[0160] The first operation can be one or more of data playback, data annotation, scene mining, and data simulation. Related operations can also be any other operations that process the second data, and this application does not limit them.

[0161] In one possible implementation, the second data received from the vehicle can be stored in the server's memory, and the second data can be retrieved from the memory when the server performs the first operation based on the second data.

[0162] An example of this step can be found here. Figure 3 Step S302.

[0163] In step S802, the server determines the first and second time points.

[0164] The first moment can be the moment when the server receives the second data, and the second moment can be the moment when the server begins to execute the first operation based on the second data.

[0165] In one possible implementation, the first moment could be the moment when the interface on the server used to receive external data receives the second data. Alternatively, if the server stores the second data received from the vehicle in its memory, the first moment could also be the moment when the server's memory receives the second data. The second moment could be the moment when the module on the server responsible for performing the first operation retrieves the second data from its memory and begins performing the first operation based on the second data.

[0166] For example, if the first operation is data playback, the second moment could be the time when the server starts playing back the second data.

[0167] In the case where the first operation is data playback and the server stores the second data received from the vehicle in the server's memory, the first moment can be the moment when the server's memory receives the second data, the second moment can be the moment when the module in the server responsible for data playback retrieves the second data from the memory and starts a series of steps to initiate data playback based on the second data, or the second moment can be the moment when the server and the module responsible for data playback start displaying the playback content on the interface based on the second data.

[0168] The method for determining the first and second moments can be selected as needed, as long as the delay between the second and first moments can reflect the delay between the server receiving the second data and executing the first operation based on the second data.

[0169] It should be noted that the server can determine the first and second moments corresponding to each frame of data in the second data, or the server can determine the first and second moments corresponding to each frame of data at fixed intervals. The server can choose how many frames to interval to determine the time interval between each first and second moment as needed, and this application does not limit this.

[0170] Step S803: The server determines whether the time difference between the second moment and the first moment exceeds the threshold.

[0171] The threshold can be set as needed. A smaller threshold can be set for higher synchronization requirements, and a larger threshold can be set for lower requirements. It can be preset in the server or determined by the user. This application does not limit the way to set and modify the threshold.

[0172] In step S804, if the time difference between the second time and the first time exceeds a threshold, the second data is processed to obtain the third data; otherwise, the process returns to step S801 and subsequent steps.

[0173] If the time difference between the second moment and the first moment exceeds the threshold, it can be considered that the server's delay in performing the first operation on the second data is too long. In order to ensure that receiving the second data and performing the first operation on the second data are as synchronous as possible, the second data can be processed to further reduce the amount of the second data and reduce the delay when performing the first operation.

[0174] The server can process the second data to obtain the third data, for example, through... Figure 7 The process shown can also be modified so that the server notifies the vehicle, which then processes the second data to obtain the third data, for example, through... Figure 6 The process is shown.

[0175] The second data can include various types of data. For different types of second data, the processing method can be flexibly selected as needed to obtain the third data. In one possible implementation, the second data includes image data. Processing the second data includes processing one or more of the following: the resolution of the image data, the frame rate of the image data, and the second data corresponding to the first priority sensor. In another possible implementation, the second data includes point cloud data. Processing the second data includes processing one or more of the following: the density of the point cloud data, the frame rate of the point cloud data, and the second data corresponding to the first priority sensor.

[0176] This application does not limit the method of processing the second data, as long as it reduces the amount of second data. In one possible implementation, the clarity of the second data can be reduced first. For example, the clarity of related data in the second data can be reduced sequentially according to a predetermined order. For instance, the resolution of the image data in the second data can be reduced first. If the resolution is reduced to the corresponding resolution threshold and the time difference still exceeds the threshold, the density of the point cloud data in the second data can be reduced to the corresponding density threshold. Setting a minimum value to which the second data can be reduced (e.g., the aforementioned resolution threshold and density threshold) ensures that key information in the second data is not lost. For example, for image data, reducing its resolution to above the resolution threshold will not affect the ability to identify pedestrians in the image. In another possible implementation, the clarity of the second data and related data can be reduced simultaneously during data processing, and this application does not limit this approach.

[0177] Secondly, the frame rate corresponding to the second data can be reduced to decrease the amount of data. For example, the first frame rate corresponding to the second data can be reduced to a corresponding threshold (which can be called the second frame rate). This second frame rate can be determined as needed, for example, it can be the lower limit of the frame rate range. The data in the second data that corresponds to the second frame rate can be used as the third data.

[0178] Furthermore, the second data can be filtered. For example, data other than the first priority data can be selected as the third data, so that the amount of the third data is less than or equal to that of the second data.

[0179] The data corresponding to the first priority can be one or more data corresponding to the lowest priority in a pre-set priority sequence. The order in the priority sequence can be determined according to the needs of executing the first operation. The order in the priority sequence can be different for different types of first operations.

[0180] For example, in a data playback operation, the data corresponding to the priority sequence can be chassis data, positioning data, control data, planning data, prediction data, perception data, point cloud data, and image data, with the priority increasing in that order. The data corresponding to the current first priority can be chassis data. The chassis data in the second data can be filtered out first. If the chassis data has been filtered out and the time difference between the second time and the first time still exceeds the threshold, the data corresponding to the current first priority can be filtered out again, which can be positioning data.

[0181] It should be noted that in the process of reducing the amount of the second data, the amount of the second data can be reduced to the threshold all at once when the time difference between the second time and the first time exceeds the threshold (for example, the resolution of the image data in the second data can be reduced to the resolution threshold all at once). Alternatively, a portion can be reduced each time the first message is received (which can be set as needed) until the amount of the second data is reduced to the threshold. Alternatively, the amount of the second data that needs to be reduced so that the time difference between the second time and the first time does not exceed the threshold can be calculated first, and then the second data can be processed according to the calculated result. This application does not impose any restrictions on this.

[0182] For example, the server can calculate the time difference between the second moment and the first moment according to a preset frame interval. If the time difference exceeds a threshold, the server sends a first message to the vehicle. When the vehicle receives the first message for the first time, it can reduce the clarity of all second data to a predetermined threshold all at once. When the first message is received for the second time, i.e., the time difference still exceeds the threshold, the server can reduce the frame rate corresponding to the second data to a predetermined threshold (i.e., the second frame rate) all at once. When the first message is received for the third time, i.e. the time difference still exceeds the threshold, the server can first filter out the data corresponding to the first priority in the second data. When the first message is received in subsequent times, the server can filter out the data corresponding to the current first priority in the second data in turn, until it is confirmed that the time difference does not exceed the threshold (i.e., no more first messages are received).

[0183] For example, when the vehicle receives the first message for the first time, the resolution of a portion of the image data in the second data can be reduced, for example, by reducing the initial resolution size by 1 / 3 of the difference between the initial resolution size and the corresponding resolution threshold. When the first message is received a second time, i.e., the time difference still exceeds the threshold, the resolution can be further reduced by 1 / 3. When the first message is received a third time, i.e., the time difference still exceeds the threshold, the resolution is reduced to the corresponding resolution threshold. Secondly, when the first message is received a fourth time, i.e., the time difference still exceeds the threshold, the density of a portion of the point cloud data in the second data can be reduced. The specific reduction method can refer to the above description of reducing image data density. The resolution method will not be elaborated here. If the resolution of all second data is reduced to a predetermined threshold and the time difference still exceeds the threshold, a portion of the frame rate corresponding to the second data can be reduced, for example, by reducing the difference between the current frame rate and the corresponding frame rate threshold by 1 / 2. If the time difference still exceeds the threshold, the frame rate can be further reduced to the corresponding frame rate threshold. If the frame rate is reduced and the time difference still exceeds the threshold, the data corresponding to the first priority in the second data can be filtered out first. When the first message is received subsequently (i.e., the time difference still exceeds the threshold), the data corresponding to the current first priority in the second data can be filtered out in turn until it is confirmed that the time difference does not exceed the threshold (i.e., the first message is not received).

[0184] For example, if the server processes the second data to obtain the third data, the process can be similar to the two examples above, where the server itself processes the second data based on the time difference to obtain the third data.

[0185] See again Figure 8 In step S805, the server performs the first operation based on the third data.

[0186] In the case where the vehicle processes the second data to obtain the third data, the third data can be sent to the server. The server can retrieve the third data from its memory when performing the first operation. The server can also retrieve the third data received by the interface used to receive external data and perform the first operation based on the third data. In the case where the server processes the second data to obtain the third data, the third data does not need to be retrieved from the server's memory or from the interface. The server can directly perform the first operation on the processed third data.

[0187] After completing step S805, step S802 and subsequent steps can be executed. At this time, the third data can be used as the new second data to determine whether the time difference exceeds the threshold, so as to proceed with the subsequent steps.

[0188] pass Figure 8 The steps shown enable the server to perform the first operation on the data as synchronously as possible with the data upload process, achieving real-time performance of the first operation on the data.

[0189] The following is Figure 9 For example, for Figure 8 The process of the execution operation phase shown is illustrated by way of example.

[0190] Figure 9 A schematic diagram illustrating a data playback operation according to an embodiment of this application is shown. For example... Figure 9 As shown, the latest data moment can be the first moment, the replay data moment can be the second moment, the collection speed V0 can represent the speed at which data is uploaded from the vehicle to the server (i.e., the first frame rate mentioned above), and the replay speed V t It can represent the speed at which the server replays the uploaded data. The replay speed can be determined based on the moment when the second frame of data begins to be replayed. For example, the number of frames replayed per unit time can be used as the replay speed. When the collection speed is greater than the replay speed, the time difference between the second moment and the first moment will be larger. When the collection speed is close to the replay speed, the time difference will be smaller.

[0191] like Figure 9 As shown in (a), at the start of data playback, tdelay This can represent the time difference between the second moment and the first moment, where t represents the time difference between the two moments. delay Less than the threshold The playback speed V1 and the collection speed V0 are basically equal, which means that the server's data playback operation is basically synchronous with the data upload to the server.

[0192] like Figure 9 As shown in (b), during data playback, fluctuations in the network bandwidth for transmitting data from the server's storage to the module used for data playback may result in significant latency, t' delay This can represent the time difference between the second moment and the first moment, where t' is the time difference. delay Greater than the threshold Since the playback speed V'1 is less than the collection speed V0, it can be assumed that there is a delay when the server performs data playback operations, and the data needs to be processed to reduce the amount of data.

[0193] like Figure 9 As shown in (c), image data, point cloud data, perception data, prediction data, planning data, and other data in the dataset can be processed by vehicles or servers to reduce the data volume and improve the speed of data playback operations. Methods for reducing data volume can be found in [reference needed]. Figure 8 As shown in step S804, it will not be repeated here.

[0194] like Figure 9 As shown in (d), reducing the amount of data can improve the data playback speed, t” delay This can represent the time difference between the second moment and the first moment, where t” is the time difference. delay Less than the threshold The playback speed V1 and the collection speed V0 are basically equal, which means that the server's data playback operation is basically synchronized with the data upload to the server. This reduces the cost of event reshaping and makes the playback process smoother and more timely.

[0195] Figure 10 A structural diagram of a data processing apparatus according to an embodiment of this application is shown. This apparatus can be deployed in the aforementioned vehicle, such as... Figure 10 As shown, the device 1000 includes:

[0196] The first acquisition module 1001 is used to acquire the first data collected by the sensor;

[0197] The first determining module 1002 is used to determine a first frame rate based on a frame rate range and an indicator value, wherein the indicator value is used to indicate the state of the target vehicle, and the first frame rate is within the frame rate range.

[0198] The second determining module 1003 is used to determine second data based on the first frame rate and the first data, wherein the information in the second data is used to instruct the server to perform the first operation.

[0199] According to the embodiments of this application, for the first data collected by the sensor, the first frame rate of the uploaded data can be dynamically determined according to the different states indicated by the index values ​​of the target vehicle. Based on the first frame rate and the first data, the second data used to instruct the server to perform the first operation can be determined, thereby eliminating redundant data in the first data, improving the value of the uploaded second data, and speeding up the data upload speed. By keeping the first frame rate within the frame rate range, data redundancy can be reduced while meeting operational requirements.

[0200] In one possible implementation, the first operation includes one or more of the following: data playback, data annotation, scene mining, and data simulation.

[0201] According to the embodiments of this application, by including multiple types of first operations, the type of first operation to be performed on the uploaded data can be flexibly selected as needed.

[0202] In one possible implementation, determining a first frame rate based on a frame rate range and indicator values ​​includes: if all indicator values ​​in the indicator range exceed the threshold corresponding to each indicator value, the first frame rate is the upper limit of the frame rate range; or if some indicator values ​​in the indicator range exceed the threshold corresponding to each indicator value, the first frame rate is greater than the lower limit of the frame rate range and less than the upper limit of the frame rate range; or if all indicator values ​​in the indicator range do not exceed the threshold corresponding to each indicator value, the first frame rate is the lower limit of the frame rate range.

[0203] According to the embodiments of this application, the indicator values ​​are used to indicate the state of the target vehicle. The more indicator values ​​exceed the threshold, the more complex the driving scenario, requiring a larger amount of data to complete the first operation, i.e., a larger first frame rate. Conversely, the fewer indicator values ​​exceed the threshold, the simpler or more conventional the driving scenario, requiring a smaller amount of data to complete the first operation, i.e., a smaller first frame rate. By associating the size of the first frame rate with the number of indicator values ​​exceeding the threshold, and making the first frame rate the upper and lower limits of the frame rate range in the two cases where all indicator values ​​exceed the threshold and none exceed the threshold, and ensuring that the first frame rate is between the upper and lower limits when some indicator values ​​exceed the threshold, it is guaranteed that the first frame rate is within the frame rate range and can adapt to the number of indicator values ​​exceeding the threshold. This allows the first frame rate to adapt to changes in the state of the target vehicle, reducing data redundancy while meeting operational requirements.

[0204] In one possible implementation, the device further includes: a third acquisition module for acquiring a first message; and a processing module for processing the second data according to the first message to obtain third data, wherein the size of the third data is less than or equal to the size of the second data, and the information in the third data is used to instruct the server to perform the first operation.

[0205] According to the embodiments of this application, by obtaining the first message, the vehicle can further process the data according to the instructions of the first message, thereby further reducing the amount of data uploaded to the server, thus ensuring the smoothness and timeliness of the first operation.

[0206] In one possible implementation, the first message indicates that the time difference between the second moment and the first moment exceeds a predetermined threshold, wherein the first moment is the moment when the server receives the second data or the third data, and the second moment is the moment when the server starts to execute the first operation based on the received second data or the third data.

[0207] According to the embodiments of this application, the server can maintain synchronization in receiving data and performing the first operation, thereby improving the timeliness of the data.

[0208] Figure 11 A structural diagram of a data processing apparatus according to an embodiment of this application is shown. This apparatus can be deployed in the aforementioned server, such as... Figure 11 As shown, the device 1100 includes:

[0209] The second acquisition module 1101 is used to acquire second data sent by the target vehicle. The second data is determined by the target vehicle based on a first frame rate and first data collected by a sensor. The first frame rate is determined based on a frame rate range and an indicator value. The indicator value is used to indicate the state of the target vehicle. The first frame rate is within the frame rate range.

[0210] Operation module 1102 is used to perform a first operation according to the instructions of the information in the second data.

[0211] According to the embodiments of this application, by obtaining the second data which is determined by the target vehicle based on the first frame rate and the first data collected by the sensor, the frame rate of the second data obtained by the server can be adjusted according to the state of the target vehicle, thereby increasing the value of the data obtained by the server. By keeping the first frame rate within the frame rate range, data redundancy can be reduced while meeting the operational requirements of the server when performing the first operation, and the speed at which the server obtains data can be adapted to the speed at which the server performs the operation.

[0212] In one possible implementation, the first operation includes one or more of the following: data playback, data annotation, scene mining, and data simulation.

[0213] According to the embodiments of this application, by including multiple types of first operations, the server can flexibly select the type of first operation to be performed on the second data as needed.

[0214] In one possible implementation, the first frame rate is determined based on a frame rate range and indicator values, including: if all indicator values ​​in the indicator range exceed the threshold corresponding to each indicator value, the first frame rate is the upper limit of the frame rate range; or if some indicator values ​​in the indicator range exceed the threshold corresponding to each indicator value, the first frame rate is greater than the lower limit of the frame rate range and less than the upper limit of the frame rate range; or if all indicator values ​​in the indicator range do not exceed the threshold corresponding to each indicator value, the first frame rate is the lower limit of the frame rate range.

[0215] According to the embodiments of this application, the indicator values ​​are used to indicate the state of the target vehicle. The more indicator values ​​exceed the threshold, the more complex the driving scenario, requiring a larger amount of data to complete the first operation, i.e., a larger first frame rate. Conversely, the fewer indicator values ​​exceed the threshold, the simpler or more conventional the driving scenario, requiring a smaller amount of data to complete the first operation, i.e., a smaller first frame rate. By associating the size of the first frame rate with the number of indicator values ​​exceeding the threshold, and making the first frame rate the upper and lower limits of the frame rate range in the two cases where all indicator values ​​exceed the threshold and none exceed the threshold, and ensuring that the first frame rate is between the upper and lower limits when some indicator values ​​exceed the threshold, it is guaranteed that the first frame rate is within the frame rate range and can adapt to the number of indicator values ​​exceeding the threshold. This allows the first frame rate to adapt to changes in the state of the target vehicle, reducing data redundancy while meeting operational requirements.

[0216] In one possible implementation, performing a first operation based on the information of the second data includes: obtaining third data, the size of which is less than or equal to the size of the second data, based on the information of the second data; and performing the first operation based on the information of the third data.

[0217] According to the embodiments of this application, the server can further process the data, thereby further reducing the amount of data acquired by the server, thus ensuring the smoothness and timeliness of the server when performing the first operation.

[0218] In one possible implementation, the device further includes: a generation module for generating a first message, the first message indicating that the time difference between the second moment and the first moment exceeds a predetermined threshold, the first moment being the moment when the second data is received, and the second moment being the moment when the first operation is started based on the second data.

[0219] According to the embodiments of this application, by generating a first message, when the time difference between the moment the server starts executing the first operation and the moment the server receives the data is too large, the first message can instruct the vehicle to process the second data to reduce the amount of the second data, thereby enabling the server to maintain the synchronization of receiving data and executing the first operation, and improving the smoothness and timeliness of the server performing the first operation.

[0220] In one possible implementation, the device further includes a sending module for sending the first message to the target vehicle, the first message being used to instruct the target vehicle to process the first data.

[0221] According to the embodiments of this application, by sending a first message, the server can notify the target vehicle to process the data, thereby improving the smoothness and timeliness of the server performing the first operation.

[0222] Figure 12 A structural diagram of a data processing apparatus according to an embodiment of this application is shown. This data processing apparatus is applicable to… Figure 1 In the data processing system shown, the above is performed. Figures 2-9 The data processing device can be the function of a vehicle or server in any of the data processing methods shown. For example, the data processing device can be the aforementioned server or vehicle, or it can be a chip (system) or other component or assembly that can be disposed inside the server or vehicle. As another example, the data processing device can also be the aforementioned data processing device 1000 or data processing device 1100. This application does not limit the scope of the embodiments.

[0223] like Figure 12 As shown, the data processing device 700 may include a processor 701 and a transceiver 702. Optionally, the data processing device 700 may include a memory 703. The processor 701 is coupled to the transceiver 702 and the memory 703, for example, they can be connected via a communication bus.

[0224] The following is combined Figure 12 The various components of the data processing device 700 will be described in detail.

[0225] The processor 701 described above is the control center of the data processing device 700. It can be a single processor or a collective term for multiple processing elements. For example, the processor 701 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement the embodiments of this application, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0226] Optionally, the processor 701 can perform various functions of the data processing device 700 by running or executing software programs stored in the memory 703 and calling data stored in the memory 703.

[0227] In a specific implementation, as one example, the processor 701 may include one or more CPUs, for example... Figure 12 CPU0 and CPU1 are shown in the diagram.

[0228] In one possible implementation, the data processing device 700 may also include multiple processors, for example... Figure 12 The processors 701 and 704 are shown. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, "processor" can refer to one or more communication devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0229] The transceiver 702 described above is used for communication with other data processing devices. For example, see reference... Figure 1 For example, if the data processing device 700 is a vehicle, the transceiver 702 can be used to communicate with a server or with another vehicle. Alternatively, if the data processing device 700 is a server, the transceiver 702 can be used to communicate with a vehicle or with another server.

[0230] Optionally, transceiver 702 may include a receiver and a transmitter. Figure 12 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function.

[0231] Alternatively, the transceiver 702 can be integrated with the processor 701 or exist independently, and can be connected via the input / output port of the data processing device 700. Figure 12(Not shown in the image) is coupled to the processor 701, but this application embodiment does not limit this.

[0232] The memory 703 described above can be used to store software programs that execute the scheme of this application, and the processor 701 controls the execution. For specific implementation methods, please refer to the above method embodiments, which will not be repeated here.

[0233] The memory 703 can be a read-only memory (ROM) or other type of static storage communication device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage communication device capable of storing information and instructions, or it can be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage communication device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited to these. It should be noted that the memory 703 can be integrated with the processor 701 or exist independently, and can be accessed through the input / output ports of the data processing device 700. Figure 12 (Not shown in the image) is coupled to the processor 701, but this application embodiment does not limit this.

[0234] It should be noted that, Figure 12 The structure of the data processing device 700 shown does not constitute a limitation on the implementation of the data processing device. The actual data processing device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0235] An embodiment of this application provides a data processing apparatus, comprising: a processor and a memory; the memory for storing a program; and the processor for executing the program stored in the memory to enable the apparatus to implement the above-described method.

[0236] Embodiments of this application provide a terminal device that can perform the above-described actions. Figures 2 to 9 The method shown in any of the embodiments.

[0237] Embodiments of this application provide a computer-readable storage medium, including: computer instructions, which, when executed, implement the above-described... Figures 2 to 9The method shown in any of the embodiments.

[0238] Embodiments of this application provide instructions that, when executed on a computer, cause the computer to perform the above-described... Figures 2 to 9 The method shown in any of the embodiments.

[0239] An embodiment of this application provides an in-vehicle computing device, which includes a processor for performing the above-described... Figures 2 to 9 The method shown in any of the embodiments.

[0240] An embodiment of this application provides a server, which includes a processor for performing the above-described actions. Figures 2 to 9 The method shown in any of the embodiments.

[0241] An embodiment of this application provides a chip system including at least one processor for supporting the implementation of the above. Figures 2 to 9 The method shown in any of the embodiments.

[0242] In one possible implementation, the chip system also includes a memory for storing program instructions and data, which may be located within or outside the processor. The chip system may be composed of chips or may contain chips and other discrete devices.

[0243] An embodiment of this application provides a data processing apparatus, which includes at least one processor and a communication interface. The communication interface is used to send and / or receive data, and the at least one processor is used to call a computer program stored in at least one memory to enable the data processing apparatus to implement the above-described data processing method.

[0244] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), electrically programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital video disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing.

[0245] The computer-readable program instructions or code described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0246] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from computer-readable program instructions. These electronic circuits can execute computer-readable program instructions to implement various aspects of this application.

[0247] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0248] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0249] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0250] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.

[0251] It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented using hardware (such as circuits or ASICs (Application Specific Integrated Circuits)) that performs the corresponding function or action, or using a combination of hardware and software, such as firmware.

[0252] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, disclosure, and appended claims in carrying out the claimed invention. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0253] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A data processing method, characterized in that, The method includes: Acquire the first data collected by the sensor; A first frame rate is determined based on a frame rate range and an indicator value, wherein the indicator value is used to indicate the state of the target vehicle, and the first frame rate is within the frame rate range. Based on the first frame rate and the first data, second data is determined, and the information in the second data is used to instruct the server to perform the first operation; Get the first message; Based on the first message, the second data is processed to obtain third data, the size of the third data being less than or equal to the size of the second data, and the information in the third data is used to instruct the server to perform the first operation.

2. The method according to claim 1, characterized in that, Determining the first frame rate based on the frame rate range and the indicator value includes: If all the indicator values ​​exceed the threshold corresponding to each indicator value, the first frame rate is the upper limit of the frame rate range; or If some of the indicator values ​​exceed the threshold corresponding to each indicator value, the first frame rate is greater than the lower limit of the frame rate range and less than the upper limit of the frame rate range; or If all the indicator values ​​in the indicator value do not exceed the threshold corresponding to each indicator value, the first frame rate is the lower limit of the frame rate range.

3. The method according to claim 1, characterized in that, The first message indicates that the time difference between the second moment and the first moment exceeds a predetermined threshold, wherein the first moment is the moment when the server receives the second data or the third data, and the second moment is the moment when the server starts to execute the first operation based on the received second data or the third data.

4. The method according to any one of claims 1-3, characterized in that, The first operation includes one or more of the following: data playback, data annotation, scene mining, and data simulation.

5. A data processing method, characterized in that, The method includes: Acquire second data sent by the target vehicle, the second data being determined by the target vehicle based on a first frame rate and first data collected by a sensor, wherein the first frame rate is determined based on a frame rate range and an indicator value, the indicator value being used to indicate the state of the target vehicle, and the first frame rate being within the frame rate range; Performing a first operation based on the information in the second data includes: obtaining third data, the size of which is less than or equal to the size of the second data, based on the information in the second data; and performing the first operation based on the information in the third data.

6. The method according to claim 5, characterized in that, The first frame rate is determined based on a frame rate range and an indicator value, including: If all the indicator values ​​exceed the threshold corresponding to each indicator value, the first frame rate is the upper limit of the frame rate range; or If some of the indicator values ​​exceed the threshold corresponding to each indicator value, the first frame rate is greater than the lower limit of the frame rate range and less than the upper limit of the frame rate range; or If all the indicator values ​​in the indicator value do not exceed the threshold corresponding to each indicator value, the first frame rate is the lower limit of the frame rate range.

7. The method according to claim 5 or 6, characterized in that, The method further includes: A first message is generated, indicating that the time difference between the second moment and the first moment exceeds a predetermined threshold. The first moment is the moment when the second data is received, and the second moment is the moment when the first operation is started based on the second data.

8. The method according to claim 7, characterized in that, The method further includes: The first message is sent to the target vehicle, and the first message is used to instruct the target vehicle to process the second data.

9. The method according to any one of claims 5-8, characterized in that, The first operation includes one or more of the following: data playback, data annotation, scene mining, and data simulation.

10. A data processing apparatus, characterized in that, The device includes: The first acquisition module is used to acquire the first data collected by the sensor; The first determining module is used to determine a first frame rate based on a frame rate range and an indicator value, wherein the indicator value is used to indicate the state of the target vehicle, and the first frame rate is within the frame rate range. The second determining module is used to determine second data based on the first frame rate and the first data, wherein the information in the second data is used to instruct the server to perform the first operation; The third acquisition module is used to acquire the first message; The processing module is configured to process the second data according to the first message to obtain third data, wherein the size of the third data is less than or equal to the size of the second data, and the information of the third data is used to instruct the server to perform the first operation.

11. The apparatus according to claim 10, characterized in that, Determining the first frame rate based on the frame rate range and the indicator value includes: If all the indicator values ​​exceed the threshold corresponding to each indicator value, the first frame rate is the upper limit of the frame rate range; or If some of the indicator values ​​exceed the threshold corresponding to each indicator value, the first frame rate is greater than the lower limit of the frame rate range and less than the upper limit of the frame rate range; or If all the indicator values ​​in the indicator value do not exceed the threshold corresponding to each indicator value, the first frame rate is the lower limit of the frame rate range.

12. The apparatus according to claim 10, characterized in that, The first message indicates that the time difference between the second moment and the first moment exceeds a predetermined threshold, wherein the first moment is the moment when the server receives the second data or the third data, and the second moment is the moment when the server starts to execute the first operation based on the received second data or the third data.

13. The apparatus according to any one of claims 10-12, characterized in that, The first operation includes one or more of the following: data playback, data annotation, scene mining, and data simulation.

14. A data processing apparatus, characterized in that, The device includes: The second acquisition module is used to acquire second data sent by the target vehicle. The second data is determined by the target vehicle based on the first frame rate and the first data collected by the sensor. The first frame rate is determined based on the frame rate range and the index value. The index value is used to indicate the state of the target vehicle. The first frame rate is within the frame rate range. An operation module is configured to perform a first operation based on the indication of the information in the second data, including: obtaining third data according to the indication of the information in the second data, wherein the size of the third data is less than or equal to the size of the second data; and performing the first operation according to the indication of the information in the third data.

15. The apparatus according to claim 14, characterized in that, The first frame rate is determined based on a frame rate range and an indicator value, including: If all the indicator values ​​exceed the threshold corresponding to each indicator value, the first frame rate is the upper limit of the frame rate range; or If some of the indicator values ​​exceed the threshold corresponding to each indicator value, the first frame rate is greater than the lower limit of the frame rate range and less than the upper limit of the frame rate range; or If all the indicator values ​​in the indicator value do not exceed the threshold corresponding to each indicator value, the first frame rate is the lower limit of the frame rate range.

16. The apparatus according to claim 14 or 15, characterized in that, The device further includes: The generation module is used to generate a first message, which indicates that the time difference between the second moment and the first moment exceeds a predetermined threshold. The first moment is the moment when the second data is received, and the second moment is the moment when the first operation is started based on the second data.

17. The apparatus according to claim 16, characterized in that, The device further includes: The sending module is used to send the first message to the target vehicle, wherein the first message is used to instruct the target vehicle to process the second data.

18. The apparatus according to any one of claims 14-17, characterized in that, The first operation includes one or more of the following: data playback, data annotation, scene mining, and data simulation.

19. A data processing apparatus, characterized in that, include: Processor and memory; The memory is used to store programs; The processor is configured to execute the program stored in the memory to enable the device to implement the method according to any one of claims 1-4, or to implement the method according to any one of claims 5-9.

20. A computer-readable storage medium, characterized in that, include: Computer instructions, when executed, implement the method of any one of claims 1-4, or implement the method of any one of claims 5-9.

21. A computer program product comprising instructions that, when executed on a computer, cause the computer to perform the method of any one of claims 1-4, or to perform the method of any one of claims 5-9.

22. An in-vehicle computing device, characterized in that, The in-vehicle computing device includes a processor for performing the data processing method as described in any one of claims 1-4.

23. A server, characterized in that, The server includes a processor for performing the data processing method as described in any one of claims 5-9.

Images