A data processing method and apparatus

Abstract

The application provides a data processing method and device. In the application, identification information of target rosbag data is obtained. The target rosbag data is located in a rosbag file stored in the cloud. According to the identification information of the target rosbag data and a first correspondence relationship stored in advance, an offset position of the target rosbag data in the rosbag file is obtained, and the first correspondence relationship includes a correspondence relationship between identification information of rosbag data in the rosbag file and an offset position of the rosbag data in the rosbag file. The target rosbag data is obtained from the rosbag file stored in the cloud according to the offset position of the target rosbag data in the rosbag file. According to the application, time can be saved, and the efficiency of the target rosbag data can be improved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a data processing method and apparatus. Background Technology

[0002] With the rapid development of technology, the ROS (Robot Operating System) has been widely used in the fields of robotics and autonomous driving. Currently, autonomous driving platforms based on the ROS operating system can record information collected by sensors on autonomous driving equipment into ROSBAT format files (hereinafter referred to as ROSBAT files) and upload them to cloud storage.

[0003] Sensors can include cameras and radar, etc. Cameras can collect image data, and radar can include lidar and millimeter-wave radar, etc. Radar is used to collect radar data, which can be used to measure distance or generate maps, etc.

[0004] A rosbag file can include point cloud data, image data, log data, and annotation data. Point cloud data can include radar data, log data can include logs of actions performed by autonomous driving equipment and related algorithms, and annotation data includes objects identified in point cloud data or image data.

[0005] Sometimes, depending on the actual situation, it is necessary to replay a specific scenario actually involved by the autonomous driving equipment. For example, in the laboratory, a terminal can be used to download a rosbag file from the cloud and replay the data in the rosbag file to simulate outdoor scenarios to test the autonomous driving function. This helps R&D personnel to quickly reproduce the actual scenario based on offline data, so as to achieve repeatable and low-cost analysis and debugging.

[0006] However, in actual use, a single rosbag file occupies a very large space, often reaching tens or even hundreds of GB. When playback is required, the entire rosbag file needs to be downloaded from the cloud to the terminal first, and then the complete rosbag file needs to be parsed on the terminal before the parsed data can be played back to enable playback of the actual scenario involved by the autonomous driving device.

[0007] However, downloading the entire rosbag file from the cloud to the local terminal is time-consuming, inefficient, and consumes a lot of network resources. Summary of the Invention

[0008] This application discloses a data processing method and apparatus.

[0009] In a first aspect, this application discloses a data processing method applied to an electronic device. The method includes: obtaining identification information of target rosbag data, the target rosbag data being located in a rosbag file stored in the cloud; obtaining the offset position of the target rosbag data in the rosbag file based on the identification information of the target rosbag data and a pre-stored first correspondence relationship; the first correspondence relationship including the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file; and obtaining the target rosbag data from the rosbag file stored in the cloud based on the offset position.

[0010] Secondly, this application discloses a data processing method, the method comprising: in a scenario where an autonomous driving device uploads a rosbag file to the cloud, obtaining the identification information of each rosbag data and the offset position of each rosbag data in the rosbag file from the rosbag file; storing the identification information of each rosbag data and the offset position of each rosbag data in the rosbag file in a first correspondence relationship, the first correspondence relationship including the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file.

[0011] Thirdly, this application discloses a data processing apparatus applied to an electronic device, the apparatus comprising: a first acquisition module for acquiring identification information of target rosbag data, the target rosbag data being located in a rosbag file stored in the cloud; a second acquisition module for acquiring the offset position of the target rosbag data in the rosbag file based on the identification information of the target rosbag data and a pre-stored first correspondence relationship; the first correspondence relationship including the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file; and a third acquisition module for acquiring the target rosbag data from the rosbag file stored in the cloud based on the offset position.

[0012] Fourthly, this application discloses a data processing apparatus, the apparatus comprising: a fifth acquisition module, configured to acquire, in a scenario where an autonomous driving device uploads a rosbag file to the cloud, the identification information of each rosbag data and the offset position of each rosbag data in the rosbag file; and a storage module, configured to store, in a first correspondence relationship, the identification information of each rosbag data and the offset position of each rosbag data in the rosbag file, the first correspondence relationship including the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file.

[0013] Fifthly, this application discloses an electronic device comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the methods shown in any of the foregoing aspects.

[0014] Sixthly, this application discloses a non-transitory computer-readable storage medium that, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the methods shown in any of the foregoing aspects.

[0015] In a seventh aspect, this application discloses a computer program product in which, when the instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to perform the methods shown in any of the foregoing aspects.

[0016] Compared with the prior art, this application has the following advantages:

[0017] In this application, the identification information of the target rosbag data is obtained. The target rosbag data is located in a rosbag file stored in the cloud. Based on the identification information of the target rosbag data and a pre-stored first correspondence, the offset position of the target rosbag data within the rosbag file is obtained. The first correspondence includes the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data within the rosbag file. The target rosbag data is then retrieved from the rosbag file stored in the cloud based on its offset position within the rosbag file.

[0018] By utilizing this application, since a first correspondence between the identifier information of rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file is stored in advance, when the electronic device needs to obtain the offset position of the target rosbag data in the rosbag file based on the identifier information of the target rosbag data, it can directly obtain the offset position of the target rosbag data in the rosbag file by means of the first correspondence, without having to obtain the offset position of the target rosbag data in the rosbag file through data interaction with the cloud. In this way, it is not necessary to trigger a large number of random read and write operations to the cloud, nor is it necessary to establish and then disconnect connections for a large number of acquisition requests, nor is it necessary to perform multiple handshake actions and other connection configuration actions for establishing and disconnecting connections for a large number of acquisition requests. Thus, time can be saved and efficiency can be improved.

[0019] Furthermore, since the offset positions of each rosbag data item within the rosbag file are stored in advance using the identification information of each rosbag data item, the offset position of the target rosbag data within the rosbag file can be directly obtained based on the identification information of the target rosbag data. This achieves the goal of obtaining the offset position of the target rosbag data within the rosbag file without downloading the rosbag file from the cloud to the local device. It is evident that obtaining the offset position of the target rosbag data within the rosbag file does not depend on "the offset position of the rosbag data in the rosbag file that precedes the target rosbag data." In other words, the offset position of the target rosbag data within the rosbag file can be obtained without knowing "the offset position of the rosbag data in the rosbag file that precedes the target rosbag data." This enables streaming acquisition and streaming parsing of the target rosbag data, improving the acquisition and parsing speed, especially when there is a large amount of target rosbag data. Attached Figure Description

[0020] Figure 1 This is a flowchart of the steps of a data processing method according to this application.

[0021] Figure 2 This is a scenario illustration of this application.

[0022] Figure 3 This is a scenario illustration of this application.

[0023] Figure 4This is a scenario illustration of this application.

[0024] Figure 5 This is a scenario illustration of this application.

[0025] Figure 6 This is a flowchart of the steps of a data processing method according to this application.

[0026] Figure 7 This is a flowchart of the steps of a data processing method according to this application.

[0027] Figure 8 This is a flowchart of the steps of a data processing method according to this application.

[0028] Figure 9 This is a structural block diagram of a data processing device according to this application.

[0029] Figure 10 This is a structural block diagram of a data processing device according to this application.

[0030] Figure 11 This is a structural block diagram of a device according to this application. Detailed Implementation

[0031] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0032] Before introducing the solution of this application, the technical terms that may be involved in this application will be explained.

[0033] rosbag: rosbag is the data storage format of the ROS operating system, widely used in autonomous driving and robotics systems. The data files contain images, audio / video, radar data, and location information, and the parsed data is used for data playback and analysis.

[0034] See Figure 1 This application illustrates a data processing method applied to an electronic device, which may include a terminal, etc. The terminal may include a terminal used by developers or testers, and the terminal may include a desktop computer, a laptop computer, a tablet computer, and an industrial computer, etc.

[0035] The method may include:

[0036] In step S101, the identification information of the target rosbag data is obtained. The target rosbag data is located in a rosbag file stored in the cloud.

[0037] A rosbag file is a record of information collected by sensors on autonomous driving devices during historical processes, generated by an autonomous driving platform based on the ROS operating system, and uploaded to cloud storage. The cloud storage may include systems such as OSS (Object Storage Service). Electronic devices can access the storage system via protocols such as S3.

[0038] Sometimes, depending on the actual situation, it is necessary to replay a certain scenario actually involved by the autonomous driving device. However, the inventors discovered that the playback of the scenario only involves a part of the rosbag data in the rosbag file, not the complete rosbag file. For example, playing only a part of the rosbag data in the rosbag file can achieve the playback of the scenario.

[0039] For example, some scenarios are based on radar data, such as reversing scenarios. For the playback of these scenarios, only point cloud data is involved, not image data. Therefore, only the point cloud data in the rosbag file can be obtained, and the complete rosbag file does not need to be obtained.

[0040] For example, some scenarios are based on image data, such as traffic light recognition. For the playback of these scenarios, only image data is involved, not point cloud data. Therefore, only the image data in the rosbag file can be obtained, and the complete rosbag file can be omitted.

[0041] For example, some scenarios involve a specific time period. For the playback of these scenarios, only the rosbag data corresponding to that time period is needed, and not the rosbag data corresponding to other time periods. Therefore, only the rosbag data corresponding to that time period needs to be obtained, and the complete rosbag file does not need to be obtained.

[0042] Thus, when it is necessary to replay the scene, the target rosbag data (partial rosbag data in the rosbag file) related to the scene can be downloaded from the cloud via an electronic device, the downloaded target rosbag data can be parsed, and then the parsed data can be played to replay the scene.

[0043] In order for the terminal to download the target rosbag data from the cloud, the identification information of the target rosbag data can be entered into the electronic device, and the electronic device can receive the entered identification information of the target rosbag data.

[0044] The identification information of the target rosbag data may include: the time when the rosbag data was generated (time stamp), the time period when the rosbag data was generated (time interval), the topic of the rosbag data, the type of the rosbag data, the ID (Identity Document) of the rosbag data, the block identifier of the file block in which the rosbag data is located, the definition of each field in the rosbag data, and the field value of each field, etc. Of course, it is understood that the identification information of the rosbag data may also include other content depending on the actual situation, and this application does not limit it in this regard.

[0045] Different file blocks have different block identifiers, and different rosbag data have different IDs.

[0046] The types of rosbag data include image data, point cloud data, log data, and labeled data.

[0047] In step S102, the offset position of the target rosbag data in the rosbag file is obtained according to the identification information of the target rosbag data and the first correspondence relationship stored in advance. The first correspondence relationship includes the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file.

[0048] In order for an electronic device to obtain the target rosbag data in a rosbag file without downloading the complete rosbag file from the cloud, the electronic device can first obtain the offset position of the target rosbag data in the rosbag file, and then obtain the target rosbag data in the rosbag file based on the offset position of the target rosbag data in the rosbag file.

[0049] To obtain the offset position of the target rosbag data within the rosbag file, one method involves having the offset positions of each rosbag data item within the rosbag file, with these offset positions evenly distributed throughout the rosbag file. For example, the rosbag file might be arranged sequentially as follows: header of rosbag data 1, rosbag data 1, header of rosbag data 2, rosbag data 2, header of rosbag data 3, rosbag data 3... header of rosbag data N, and rosbag data N. N is a positive integer.

[0050] If you need to get the offset position of the target rosbag data in the rosbag file, it will often trigger a lot of random accesses, which is time-consuming and inefficient.

[0051] For example, the header of rosbag data 1 includes: the space occupied by the header of rosbag data 1, the starting address of rosbag data 1 in the rasbag file, and the space occupied by rosbag data 1; the header of rosbag data 2 includes: the space occupied by the header of rosbag data 2, the starting address of rosbag data 2 in the rasbag file, and the space occupied by rosbag data 2... and so on, the header of rosbag data N includes: the space occupied by the header of rosbag data N, the starting address of rosbag data N in the rasbag file, and the space occupied by rosbag data N.

[0052] The target rosbag data is one of rosbag data 1 to rosbag data N. For an electronic device, in order to obtain the offset position of the target rosbag data in the rosbag file, if the target rosbag data is rosbag data M, where M is greater than or equal to 1 and less than or equal to N, the electronic device needs to obtain the data headers of rosbag data 1 to rosbag data M sequentially from the cloud.

[0053] For example, if the starting address of the header of rosbag data 1 in the rosbag file is the same as the starting address (base address) of the rosbag file, the electronic device can send a retrieval request 1 to the cloud. Retrieval request 1 can include HTTP (Hypertext Transfer Protocol) requests, etc. Retrieval request 1 carries the starting address of the header of rosbag data 1 in the rosbag file (or the starting address of the rosbag file itself), and is used to request the content of the header of rosbag data 1. After receiving retrieval request 1, the cloud can retrieve the content of the header of rosbag data 1 from the rosbag file according to retrieval request 1, and return the content of the header of rosbag data 1 to the electronic device. The electronic device can then receive the content of the header of rosbag data 1 returned by the cloud.

[0054] The electronic device determines the starting address of the header of rosbag data 2 in the rosbag file based on the space occupied by the header of rosbag data 1 and the space occupied by rosbag data 1.

[0055] Thus, the electronic device can send a retrieval request 2 to the cloud. Retrieval request 2 can include HTTP requests, and it carries the starting address of the header of rosbag data 2 within the rosbag file. Retrieval request 2 is used to request the content of the header of rosbag data 2. After receiving retrieval request 2, the cloud can retrieve the content of the header of rosbag data 2 from the rosbag file based on retrieval request 2 and return the content of the header of rosbag data 2 to the electronic device. The electronic device can then receive the content of the header of rosbag data 2 returned by the cloud.

[0056] The electronic device determines the starting address of the header of rosbag data 3 in the rosbag file based on the starting address of the header of rosbag data 2 in the rosbag file, the space occupied by the header of rosbag data 2 in the header of rosbag data 2, and the space occupied by rosbag data 2.

[0057] Thus, the electronic device can send a retrieval request 3 to the cloud. Retrieval request 3 can include HTTP requests, and it carries the starting address of the rosbag data 3 header in the rosbag file. Retrieval request 3 is used to request the content of the rosbag data 3 header. After receiving retrieval request 3, the cloud can retrieve the content of the rosbag data 3 header from the rosbag file based on retrieval request 3 and return the content of the rosbag data 3 header to the electronic device. The electronic device can then receive the content of the rosbag data 3 header returned by the cloud.

[0058] Similarly, the electronic device determines the starting address of the rosbag data M header in the rosbag file based on the starting address of the rosbag data M-1 header in the rosbag file, the space occupied by the rosbag data M-1 header, and the space occupied by the rosbag data M-1.

[0059] Thus, the electronic device can send a retrieval request M to the cloud. Request M can include HTTP requests, etc. Request M carries the starting address of the header of the rosbag data M within the rosbag file. Request M is used to request the content of the header of the rosbag data M. After receiving Request M, the cloud can retrieve the content of the header of the rosbag data M from the rosbag file based on Request M and return the content of the header of the rosbag data M to the electronic device. The electronic device can then receive the content of the header of the rosbag data M returned by the cloud.

[0060] The electronic device obtains the starting address of the rosbag data M in the rosbag file based on the starting address of the rosbag data M header in the rosbag file, the space occupied by the rosbag data M header, and the space occupied by the rosbag data M. It also obtains the offset position of the rosbag data M in the rosbag file based on the starting address of the rosbag data M in the rosbag file and the space occupied by the rosbag data M.

[0061] However, while the above method can obtain the offset position of the rosbag data M within the rosbag file, the electronic device needs to perform multiple data interactions with the cloud. This is equivalent to triggering a large number of random reads and writes to the cloud. For example, for each retrieval request, the electronic device and the cloud need to establish a connection and then disconnect. Establishing and disconnecting the connection involves multiple handshake actions and other connection configuration actions before the retrieval request and its response can be transmitted. This consumes a lot of time and reduces efficiency.

[0062] Thus, in order to save time and improve efficiency in scenarios where the offset position of the target rosbag data in the rosbag file is to be obtained, multiple data interactions between the electronic device and the cloud can be avoided. However, since the offset positions of each rosbag data in the rosbag file are evenly distributed in the rosbag file, if it is necessary to obtain the data header of the rosbag data from the rosbag file stored in the cloud and determine the offset position of the target rosbag data in the rosbag file based on the content of the data header of the rosbag data, a large number of random reads and writes will inevitably be triggered to the cloud.

[0063] In light of this, the inventors abandoned the approach of "obtaining the header of the rosbag data from the rosbag file stored in the cloud and determining the offset position of the target rosbag data in the rosbag file based on the content of the rosbag data header," and instead devised another method:

[0064] For example, since different rosbag data in a rosbag file have different offset positions in the rosbag file, in this application, a first correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file can be stored in advance, so as to record the offset position of the rosbag data in the rosbag file through the identification information of the rosbag data.

[0065] For example, in a scenario where an autonomous driving device uploads a rosbag file to the cloud, see one embodiment, Figure 2 The first correspondence can be located in the autonomous driving device. The autonomous driving device can obtain the identification information of each rosbag data and the offset position of each rosbag data in the rosbag file from the rosbag file, and store the identification information of each rosbag data and the offset position of each rosbag data in the rosbag file in the first correspondence.

[0066] Alternatively, in another embodiment, see Figure 3 The first correspondence can be located in the cloud. After obtaining the rosbag file, the cloud can retrieve the identifier information of each rosbag data item and its offset position within the rosbag file, and store these information in the first correspondence. Alternatively, in another embodiment, see... Figure 4 The autonomous driving device can transmit the identification information of each rosbag data point and its offset position within the rosbag file to the cloud. This allows the cloud to obtain the identification information and offset positions of each rosbag data point transmitted by the autonomous driving device. The cloud can then store the identification information and offset positions of each rosbag data point in a first correspondence relationship. For example, for any rosbag data point, the data identifier of that rosbag data point and its offset position within the rosbag file can be combined to form a corresponding entry, which is then stored in the first correspondence relationship in the cloud. The same applies to each other rosbag data point.

[0067] Alternatively, in yet another embodiment, see Figure 5The first correspondence is located in the electronic device. Thus, the autonomous driving device can transmit the identification information of each rosbag data point and its offset position within the rosbag file to the electronic device. This allows the electronic device to obtain the identification information and offset positions of each rosbag data point transmitted by the autonomous driving device. The electronic device can then store the identification information and offset positions of each rosbag data point in the first correspondence. For example, for any given rosbag data point, its data identifier and its offset position within the rosbag file can be combined to form a corresponding entry and stored in the first correspondence within the electronic device. The same applies to each other rosbag data point.

[0068] Thus, when it is necessary to obtain the offset position of the target rosbag data in the rosbag file, if the identification information of the target rosbag data has been obtained, the offset position corresponding to the identification information of the target rosbag data can be found in the first pre-stored correspondence and used as the offset position of the target rosbag data in the rosbag file.

[0069] In one embodiment, when the first correspondence is located locally on the electronic device, the electronic device can directly find the offset position corresponding to the identification information of the target rosbag data in the first correspondence stored locally on the electronic device, and use it as the offset position of the target rosbag data in the rosbag file.

[0070] In this embodiment, since the first correspondence is located locally on the electronic device, the electronic device does not need to interact with the cloud, does not need to trigger a large number of random reads and writes to the cloud, does not need to establish a connection and then disconnect for each of the aforementioned acquisition requests, and does not need to perform multiple handshake actions and other connection configuration actions for each acquisition request to establish and disconnect. This can save time and improve efficiency.

[0071] Alternatively, in another embodiment, if the first correspondence is located in the cloud, the electronic device can send a download request to the cloud to request the download of the first correspondence. After receiving the download request, the cloud can return the first correspondence to the electronic device according to the download request. The electronic device receives the first correspondence and stores it in the electronic device. Then, it can directly find the offset position corresponding to the identification information of the target rosbag data in the first correspondence stored locally in the electronic device, and use it as the offset position of the target rosbag data in the rosbag file.

[0072] In this embodiment, the first correspondence is located in the cloud. The first correspondence can be stored continuously in the cloud, unlike the previously mentioned case where the data headers of each rosbag data are stored non-contiguously (e.g., intermittently). Non-contiguously stored content cannot be obtained from the cloud in one retrieval request, while continuously stored content can be obtained from the cloud in one retrieval request. Thus, since the first correspondence is stored continuously in the cloud, the electronic device can obtain all the content in the first correspondence from the cloud in one retrieval request. It can be seen that the electronic device can obtain the first correspondence without multiple data interactions with the cloud.

[0073] Electronic devices can avoid sending multiple retrieval requests to the cloud, avoid triggering a large amount of random read and write operations on the cloud, and avoid establishing and then closing a connection for each retrieval request. They can also avoid performing multiple handshake actions and other connection configuration actions for each retrieval request. Instead, they can simply establish a connection for each retrieval request and then close it, and perform multiple handshake actions and other connection configuration actions for each retrieval request. This saves time and improves efficiency.

[0074] Alternatively, in another embodiment, if the first correspondence is located in the autonomous driving device, the electronic device can send a download request to the autonomous driving device. The download request is used to request the download of the first correspondence. After receiving the download request, the autonomous driving device can return the first correspondence to the electronic device according to the download request. The electronic device receives the first correspondence and stores it in the electronic device. Then, it can directly find the offset position corresponding to the identification information of the target rosbag data in the first correspondence stored locally in the electronic device, and use it as the offset position of the target rosbag data in the rosbag file.

[0075] In step S103, the target rosbag data is obtained from the rosbag file stored in the cloud according to the offset position of the target rosbag data in the rosbag file.

[0076] Then, the target rosbag data can be parsed and the parsed data can be played back to replay the scene.

[0077] In one embodiment of this application, the electronic device can send a download request to the cloud. The download request carries the offset position of the target rosbag data in the rosbag file. The download request is used to request the download of the target rosbag data. After receiving the download request, the cloud can find the target rosbag data in the rosbag file stored in the cloud according to the offset position of the target rosbag data in the rosbag file in the download request, and return the target rosbag data to the electronic device. The electronic device receives the target rosbag data.

[0078] In this application, the identification information of the target rosbag data is obtained. The target rosbag data is located in a rosbag file stored in the cloud. Based on the identification information of the target rosbag data and a pre-stored first correspondence, the offset position of the target rosbag data within the rosbag file is obtained. The first correspondence includes the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data within the rosbag file. The target rosbag data is then retrieved from the rosbag file stored in the cloud based on its offset position within the rosbag file.

[0079] By utilizing this application, since a first correspondence between the identifier information of rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file is stored in advance, when the electronic device needs to obtain the offset position of the target rosbag data in the rosbag file based on the identifier information of the target rosbag data, it can directly obtain the offset position of the target rosbag data in the rosbag file by means of the first correspondence, without having to obtain the offset position of the target rosbag data in the rosbag file through data interaction with the cloud. In this way, it is not necessary to trigger a large number of random read and write operations to the cloud, nor is it necessary to establish and then disconnect connections for a large number of acquisition requests, nor is it necessary to perform multiple handshake actions and other connection configuration actions for establishing and disconnecting connections for a large number of acquisition requests. Thus, time can be saved and efficiency can be improved.

[0080] Furthermore, since the offset positions of each rosbag data item within the rosbag file are stored in advance using the identification information of each rosbag data item, the offset position of the target rosbag data within the rosbag file can be directly obtained based on the identification information of the target rosbag data. This achieves the goal of obtaining the offset position of the target rosbag data within the rosbag file without downloading the rosbag file from the cloud to the local device. It is evident that obtaining the offset position of the target rosbag data within the rosbag file does not depend on "the offset position of the rosbag data in the rosbag file that precedes the target rosbag data." In other words, the offset position of the target rosbag data within the rosbag file can be obtained without knowing "the offset position of the rosbag data in the rosbag file that precedes the target rosbag data." This enables streaming acquisition and streaming parsing of the target rosbag data, improving the acquisition and parsing speed, especially when there is a large amount of target rosbag data.

[0081] In one embodiment of this application, the target rosbag data may have been cached in the electronic device beforehand. For example, the need to obtain the target rosbag data in the rosbag file this time is not the first time that the target rosbag data in the rosbag file needs to be obtained for playback. The electronic device has previously obtained the target rosbag data in the rosbag file for playback and cached the target rosbag data in the electronic device.

[0082] Alternatively, in one embodiment of this application, the target rosbag data may not have been cached in the electronic device beforehand. For example, this is the first time that the target rosbag data in the rosbag file needs to be obtained for playback. The electronic device has never obtained the target rosbag data in the rosbag file for playback before, and therefore the target rosbag data has not been cached in the electronic device before.

[0083] If the target rosbag data is already cached on the electronic device, it can be retrieved directly from the local device instead of from the rosbag file stored in the cloud. This saves network resources, time, and improves efficiency.

[0084] Whenever the electronic device obtains a rosbag data in the rosbag file, the electronic device can form a corresponding table entry with the offset position of the rosbag data in the rosbag file and store it in the second correspondence in the local storage of the electronic device. The second correspondence includes the correspondence between the offset position of the rosbag data in the rosbag file and the rosbag data, so as to mark the cached rosbag data by the offset position.

[0085] Thus, it is possible to determine whether the target rosbag data has been cached in the electronic device based on the offset position of the target rosbag data in the rosbag file. For example, the rosbag data corresponding to the offset position of the target rosbag data in the rosbag file can be found in the second correspondence stored locally on the electronic device.

[0086] If the target rosbag data is found in the second correspondence relationship and corresponds to the offset position of the target rosbag data in the rosbag file, it is determined that the target rosbag data has been cached in the electronic device, and the rosbag data found in the second correspondence relationship that corresponds to the offset position of the target rosbag data in the rosbag file is the target rosbag data.

[0087] Alternatively, if no rosbag data corresponding to the offset position of the target rosbag data in the rosbag file is found in the second correspondence, it is determined that the target rosbag data is not cached in the electronic device.

[0088] If the target rosbag data is already cached in the electronic device, it can be retrieved. Then, the target rosbag data can be parsed, and the parsed data can be played back to replay the scene.

[0089] If the target rosbag data is not cached in the electronic device, step S103 can be performed: retrieve the target rosbag data from the rosbag file stored in the cloud according to the offset position of the target rosbag data in the rosbag file.

[0090] Thus, in this application, if the target rosbag data is already cached in the electronic device, the cached target rosbag data can be obtained directly from the local electronic device, instead of obtaining the target rosbag data from the rosbag file stored in the cloud. This can save network resources, save time, and improve efficiency.

[0091] Furthermore, after retrieving the target rosbag data from the cloud-stored rosbag file based on its offset within the rosbag file, the target rosbag data can be cached on the electronic device based on its offset within the rosbag file. This allows the electronic device to directly retrieve the cached target rosbag data locally when needed, without retrieving it from the cloud-stored rosbag file, thus saving network resources, time, and improving efficiency.

[0092] Specifically, when caching target rosbag data in an electronic device based on the offset position of the target rosbag data in the rosbag file, the offset position of the target rosbag data in the rosbag file and the target rosbag data can be stored in a second correspondence stored locally on the electronic device.

[0093] In one possible scenario, there are multiple target rosbag data sets, each with a different offset within the rosbag file, and none of these target rosbag data sets are cached in the electronic device.

[0094] Therefore, it is necessary to retrieve the target rosbag data from the rosbag file stored in the cloud according to the offset position of each target rosbag data in the rosbag file.

[0095] In one possible scenario, the data for each target rosbag is not stored contiguously within the rosbag file.

[0096] The individual contents of non-contiguous storage cannot be retrieved from the cloud simultaneously with a single download request. For example, the individual target rosbag data in non-contiguous storage cannot be retrieved from the cloud simultaneously with a single download request, while the contents of contiguous storage can be retrieved from the cloud simultaneously with a single download request.

[0097] For example, when the target rosbag data are not stored contiguously in the rosbag file, in one embodiment of this application, for any one of the target rosbag data, the electronic device can send a download request to the cloud. The download request carries the offset position of the target rosbag data in the rosbag file. The download request is used to request the download of the target rosbag data. After receiving the download request, the cloud can find the target rosbag data in the rosbag file stored in the cloud according to the offset position of the target rosbag data in the rosbag file in the download request, and return the target rosbag data to the electronic device. The electronic device receives the target rosbag data.

[0098] The same applies to each of the other target rosbag data in multiple target rosbag datasets.

[0099] However, while the above method can obtain the target rosbag data, the electronic device needs to perform multiple data interactions with the cloud. This is equivalent to triggering a large number of random reads and writes to the cloud. For example, for each download request, the electronic device and the cloud need to establish a connection and then disconnect. Establishing and disconnecting the connection involves multiple handshake actions and other connection configuration actions before the download request and its response can be transmitted. This consumes a lot of time and reduces efficiency.

[0100] Thus, in order to save time and improve efficiency in scenarios where rosbag data is retrieved from rosbag files stored in the cloud, multiple data interactions between electronic devices and the cloud can be avoided. However, since the positions of each target rosbag data in the rosbag file are disconnected and the target rosbag data is not stored contiguously, if each target rosbag data is retrieved from the cloud separately according to download requests for each target rosbag data, a large number of random reads and writes will inevitably be triggered to the cloud.

[0101] In light of this, the inventors abandoned the approach of "retrieving individual target rosbag data from the cloud based on separate download requests for each target rosbag data," and instead devised an alternative method:

[0102] For example, when retrieving target rosbag data from a rosbag file stored in the cloud based on the target rosbag data's offset position within the rosbag file, see [link to relevant documentation]. Figure 6 This can be achieved through the following process, including:

[0103] In step S201, offset position intervals are generated based on the offset positions of multiple target rosbag data in the rosbag file.

[0104] In one embodiment of this application, this step can be implemented through the following process, including:

[0105] 11) Among the offset positions of multiple target rosbag data in the rosbag file, filter the first and last offset positions.

[0106] In this application, the offset of the rosbag data in the rosbag file includes: the starting address of the rosbag data in the rosbag file and the space occupied by the rosbag data.

[0107] The starting address of rosbag data within a rosbag file indicates the exact position within the rosbag file where that rosbag data begins. Multiple target rosbag data sets will have different starting addresses within their respective offset positions within the rosbag file.

[0108] Thus, filtering the earliest offset position among the offset positions of multiple target rosbag data in the rosbag file can be understood as filtering the offset position with the earliest starting address.

[0109] Furthermore, among the offset positions of multiple target rosbag data in the rosbag file, filtering the last offset position can be understood as: filtering the last offset position of the starting address.

[0110] 12) Generate an offset position range based on the foremost and last offset positions.

[0111] In one embodiment, the starting address of the foremost offset position can be used as the left endpoint of the offset position interval. The sum of the starting address of the last offset position and the space occupied by the target rosbag data corresponding to the last offset position is calculated to obtain an address, which is used as the right endpoint of the offset position interval. The left endpoint and the right endpoint together constitute the offset position interval.

[0112] In another embodiment of this application, the rosbag data in the rosbag file stored in the cloud is divided into file blocks. For example, the rosbag file stored in the cloud includes multiple file blocks, each of which contains rosbag data. The rosbag data in different file blocks is different, and the positions (e.g., offset position ranges) of different file blocks in the rosbag file stored in the cloud are different. In this case, each time rosbag data is retrieved from the rosbag data stored in the cloud, the rosbag data can be retrieved separately according to the dimensions of the file blocks.

[0113] Therefore, this step can be achieved through the following process:

[0114] 21) For any file block, obtain the offset position range of the file block in the rosbag file stored in the cloud.

[0115] In this application, the offset position ranges of different file blocks in the rosbag file stored in the cloud are different. After the autonomous driving device generates the rosbag file, each file block in the rosbag file has been generated, and the offset position range of each file block in the rosbag file has been determined. Thus, the autonomous driving device can obtain the offset position range of each file block in the rosbag file and transmit the offset position range of each file block in the rosbag file to the cloud for storage. Alternatively, it can transmit the offset position range of each file block in the rosbag file to an electronic device for storage.

[0116] Thus, in this application, the offset position range of the file block in the rosbag file stored in the cloud can be obtained locally from the electronic device, or the offset position range of the file block in the rosbag file stored in the cloud can be obtained from the cloud.

[0117] 22) Select the offset position within the offset position range of the file block from the offset positions of multiple target rosbag data in the rosbag file.

[0118] Multiple target rosbag data have different offset positions in the rosbag file. Some rosbag data may be offset within the offset position range of the file block, while others may not be offset within the offset position range of the file block.

[0119] The offset of the rosbag data within the rosbag file includes: the starting address of the rosbag data within the rosbag file and the space occupied by the rosbag data.

[0120] Thus, for any target rosbag data, the sum of the starting address of the target rosbag data in the rosbag file and the space occupied by the target rosbag data can be calculated to obtain an address, which is used as the ending address of the target rosbag data in the rosbag file. The starting address and the ending address of the target rosbag data in the rosbag file together form the offset position range of the target rosbag data in the rosbag file.

[0121] If the offset position of the target rosbag data in the rosbag file is within the offset position range of the file block, then it means that the offset position of the target rosbag data in the rosbag file is within the offset position range of the file block.

[0122] Alternatively, if the offset position range of the target rosbag data in the rosbag file is not within the offset position range of the file block, then it means that the offset position of the target rosbag data in the rosbag file is not within the offset position range of the file block.

[0123] For each other target rosbag data, the same operation is performed to obtain the offset position of each target rosbag data within the offset position range of the file block in the rosbag file.

[0124] 23) Among the offset positions within the offset position range of the file block, filter the first and last offset positions.

[0125] In this application, the offset of the rosbag data in the rosbag file includes: the starting address of the rosbag data in the rosbag file and the space occupied by the rosbag data.

[0126] The starting addresses of each offset position within the offset position range of this file block are different.

[0127] Thus, filtering the earliest offset position within the offset position range of this file block can be understood as filtering the earliest offset position from the starting address.

[0128] Furthermore, filtering the last offset position within the offset position range of the file block can be understood as filtering the last offset position of the starting address.

[0129] 24) Generate an offset position range based on the foremost and last offset positions.

[0130] In one embodiment, the starting address of the foremost offset position can be used as the left endpoint of the offset position interval. The sum of the starting address of the last offset position and the space occupied by the target rosbag data corresponding to the last offset position is calculated to obtain an address, which is used as the right endpoint of the offset position interval. The left endpoint and the right endpoint together constitute the offset position interval.

[0131] In step S202, a download request is generated, which carries at least an offset position range.

[0132] In step S203, a download request is sent to the cloud.

[0133] This enables the cloud to retrieve data within the offset position range from the rosbag file stored in the cloud based on the download request, and then return the data within the offset position range to the electronic device.

[0134] In step S204, data located in the offset position range returned by the cloud is received.

[0135] In step S205, based on the offset position of each target rosbag data in the rosbag file, the data of each target rosbag data is obtained from the data located in the offset position interval.

[0136] In this application, the offset of the rosbag data in the rosbag file includes: the starting address of the rosbag data in the rosbag file and the space occupied by the rosbag data.

[0137] Regarding the aforementioned embodiments involving file blocks (e.g., the embodiments shown in steps 21) to 24), for any given file block, the offset position range of that file block in the rosbag file is also known. Based on the offset position range of that file block in the rosbag file, the starting address of that file block in the rosbag file can be determined. The starting address of the target rosbag data in the rosbag file is equal to or located after the starting address of that file block in the rosbag file (within the offset position range of that file block in the rosbag file).

[0138] For any rosbag data whose offset position is within the offset range of the file block, the difference between the starting address of the target rosbag data in the rosbag file and the starting address of the file block in the rosbag file can be calculated to obtain the offset address of the target rosbag data in the file block. Then, the data occupied by the target rosbag data can be read from that offset address in the file block to obtain the target rosbag data. The same process is repeated for each other target rosbag data whose offset position is within the offset range of the file block, thus obtaining each target rosbag data whose offset position is within the offset range of the file block.

[0139] The same applies to every other file block.

[0140] This application reduces the number of download requests sent by electronic devices to the cloud. For example, the number of download requests sent can be less than the number of target rosbag data sets. Thus, the electronic device can obtain multiple target rosbag data sets from the cloud with fewer download requests than the number of target rosbag data sets. Therefore, the electronic device can obtain multiple target rosbag data sets with fewer data interactions with the cloud. The electronic device does not need to send download requests equal to the number of target rosbag data sets to the cloud. This avoids triggering a large amount of random read / write operations on the cloud, and also avoids establishing and closing connections separately for each download request, and avoiding performing multiple handshake actions and other connection configuration actions for each connection establishment and disconnection for each download request. Instead, it only needs to establish and close connections for a smaller number of download requests, and perform handshake actions and other connection configuration actions for each connection establishment and disconnection for a smaller number of download requests. This saves time and improves efficiency.

[0141] Although the data transmitted from the cloud to the electronic device based on the download request includes other rosbag data besides multiple target rosbag data, that is, although the amount of data transmission increases, the transmission process actually takes less time because "it is not necessary to trigger a large number of random reads and writes to the cloud, nor is it necessary to establish and then disconnect connections for multiple download requests, nor is it necessary to perform multiple handshake actions and other connection configuration actions for establishing and disconnecting connections for each download request." Furthermore, multiple experiments have proven that the time taken to download rosbag data (including multiple target rosbag data and other rosbag data) located within the generated offset position range from the cloud using fewer download requests is less than the time taken to serially download each target rosbag data from the cloud using download requests corresponding to each target rosbag data. This saves time and improves efficiency.

[0142] For the aforementioned embodiments involving file blocks (e.g., the embodiments shown in steps 21) to 24), see [link to relevant documentation]. Figure 7 The step S103, which determines whether the target rosbag data has been cached in the electronic device based on its offset position in the rosbag file, can be implemented using the following process:

[0143] In step S301, the offset position range of the target rosbag data in the rosbag file is determined within the offset position range of the rosbag files stored in the cloud for different file blocks.

[0144] The determined offset position range is the offset position range of one of the multiple file blocks in the rosbag file stored in the cloud.

[0145] In this application, the sum of the starting address of the target rosbag data in the rosbag file and the space occupied by the target rosbag data can be calculated to obtain an address, which is used as the ending address of the target rosbag data in the rosbag file. The starting address and the ending address of the target rosbag data in the rosbag file constitute the offset position range of the target rosbag data in the rosbag file.

[0146] If the offset position range of the target rosbag data in the rosbag file is within the offset position range of a certain file block, then it means that the offset position range of that file block is the offset position range of the target rosbag data in the rosbag file.

[0147] Alternatively, if the offset position range of the target rosbag data in the rosbag file is not located within the offset position range of a certain file block, it means that the offset position range of that file block is not the offset position range of the target rosbag data in the rosbag file.

[0148] In step S302, based on the determined offset position range, it is determined whether the file block containing the target rosbag data has been cached in the electronic device.

[0149] If the file block containing the target rosbag data is already cached in the electronic device, in step S303, it is determined that the target rosbag data is already cached in the electronic device.

[0150] If the file block containing the target rosbag data is not cached in the electronic device, in step S304, it is determined that the target rosbag data is not cached in the electronic device.

[0151] Specifically, whenever the electronic device obtains a file block from the rosbag file, it stores the offset range of that file block within the rosbag file and the file block itself in a third correspondence relationship in the electronic device's local storage. This third correspondence relationship marks the cached file block by its offset range.

[0152] Thus, when determining whether the file block containing the target rosbag data has been cached in the electronic device based on the determined offset position range, the file block corresponding to the determined offset position range can be found in the third correspondence stored locally in the electronic device.

[0153] If a file block corresponding to the determined offset position interval is found in the third correspondence, the file block corresponding to the determined offset position interval is the file block where the target rosbag data is located. In this way, the file block where the target rosbag data is cached in the electronic device can be determined.

[0154] Alternatively, if no file block corresponding to the determined offset position interval is found in the third correspondence, the file block where the target rosbag data is not cached in the electronic device can be determined.

[0155] Alternatively, in another embodiment, the offset position range of the target rosbag data within the rosbag file can be determined from the offset position ranges of different file blocks stored in the cloud. Then, based on the determined offset position range, the block identifier of the file block containing the target rosbag data can be determined. Then, based on the block identifier of the file block containing the target rosbag data, it can be determined whether the file block containing the target rosbag data is cached in the electronic device. If the file block containing the target rosbag data is cached in the electronic device, it is determined that the target rosbag data is cached in the electronic device. Alternatively, if the file block containing the target rosbag data is not cached in the electronic device, it is determined that the target rosbag data is not cached in the electronic device.

[0156] In this application, the offset position ranges of different file blocks in the rosbag file stored in the cloud are different. After the autonomous driving device generates the rosbag file, each file block in the rosbag file has been generated, and the offset position range of each file block in the rosbag file has been determined. Thus, the autonomous driving device can obtain the offset position range of each file block in the rosbag file, as well as the block identifier of each file block. The block identifiers of different file blocks are different.

[0157] The offset position range of each file block in the rosbag file and the block identifier of each file block are transmitted to the cloud. The cloud can then store the offset position range of each file block in the rosbag file and the block identifier of each file block in the fourth correspondence relationship between the offset position range of the file block in the rosbag file and the block identifier of the file block. For example, for any file block, the cloud can form a corresponding table entry with the offset position range of the file block in the rosbag file and the block identifier of the file block and store it in the fourth correspondence relationship. The same applies to each other file block.

[0158] Alternatively, the offset position range of each file block in the rosbag file and the block identifier of each file block can be transmitted to the electronic device. The electronic device can then store the offset position range of each file block in the rosbag file and the block identifier of each file block in the fourth correspondence between the offset position range of the file block in the rosbag file and the block identifier of the file block. For example, for any file block, the electronic device can form a corresponding entry with the offset position range of the file block in the rosbag file and the block identifier of the file block and store it in the fourth correspondence. The same applies to each other file block.

[0159] Thus, in this application, the block identifier corresponding to the determined offset position range can be found from the fourth correspondence stored locally on the electronic device, and used as the block identifier of the file block where the target rosbag data is located.

[0160] Alternatively, the electronic device can request the cloud to find the block identifier corresponding to the determined offset position range from the fourth correspondence stored in the cloud, and use it as the block identifier of the file block where the target rosbag data is located. Then, the cloud returns the block identifier of the file block where the target rosbag data is located to the electronic device, which can receive the block identifier of the file block where the target rosbag data is located returned by the cloud.

[0161] See Figure 8 The following example illustrates the solution proposed in this application.

[0162] exist Figure 8 In this process, the user's input information can be obtained first, and the identification information of the target rosbag data can be parsed from the input information.

[0163] Load the first correspondence in the electronic device / autonomous driving device / cloud, the first correspondence including the correspondence between the identification information of rosbag data in the rosbag file and the offset position of rosbag data in the rosbag file.

[0164] Based on the identifier information of the target rosbag data, find the corresponding offset position in the first correspondence and use it as the offset position of the target rosbag data in the rosbag file.

[0165] Based on the offset position of the target rosbag data in the rosbag file, determine whether the file block containing the target rosbag data has been cached in the electronic device.

[0166] If the file block containing the target rosbag data is already cached in the electronic device, retrieve the file block containing the target rosbag data cached in the electronic device. Then, parse the file block containing the target rosbag data to obtain the target rosbag data.

[0167] If the file block containing the target rosbag data is not cached in the electronic device, obtain the offset range of the file block containing the target rosbag data within the rosbag file.

[0168] Based on the offset range of the file block containing the target rosbag data within the rosbag file, the file block containing the target rosbag data is retrieved from the rosbag file stored in the cloud, and the file block containing the target rosbag data is cached. Additionally, the file block containing the target rosbag data is parsed to obtain the target rosbag data.

[0169] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions involved are not necessarily required by this application.

[0170] Reference Figure 9 This diagram illustrates a structural block diagram of a data processing apparatus according to this application, applied to an electronic device, the apparatus comprising:

[0171] The first acquisition module 11 is used to acquire the identification information of the target rosbag data, which is located in a rosbag file stored in the cloud.

[0172] The second acquisition module 12 is used to acquire the offset position of the target rosbag data in the rosbag file based on the identification information of the target rosbag data and the first correspondence relationship stored in advance; the first correspondence relationship includes the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file.

[0173] The third acquisition module 13 is used to acquire target rosbag data from the rosbag file stored in the cloud according to the offset position.

[0174] In an alternative implementation, the apparatus further includes:

[0175] The determination module is used to determine whether the target rosbag data has been cached in the electronic device based on the offset position;

[0176] The third acquisition module is also used to: in the case that the target rosbag data is not cached in the electronic device, acquire the target rosbag data from the rosbag file stored in the cloud according to the offset position.

[0177] In an alternative implementation, the apparatus further includes:

[0178] The fourth acquisition module is used to acquire the target rosbag data cached in the electronic device if the target rosbag data has already been cached in the electronic device.

[0179] In one optional implementation, the determining module includes:

[0180] The lookup unit is used to search for rosbag data corresponding to the offset position in a second correspondence stored locally in the electronic device; the second correspondence includes the correspondence between the offset position of the rosbag data in the rosbag file and the rosbag data itself.

[0181] The first determining unit is used to determine that target rosbag data has been cached in the electronic device when rosbag data corresponding to the offset position is found in the second correspondence.

[0182] Alternatively, the second determining unit is used to determine that the target rosbag data is not cached in the electronic device if no rosbag data corresponding to the offset position is found in the second correspondence.

[0183] In one optional implementation, there are multiple target rosbag data sets, and the offset positions of different target rosbag data sets in the rosbag file are different;

[0184] The third acquisition module includes:

[0185] The first generation unit is used to generate offset position intervals based on the offset positions of multiple target rosbag data in the rosbag file.

[0186] The second generation unit is used to generate a download request, which carries at least an offset position range.

[0187] The sending unit is used to send a download request to the cloud, so that the cloud can obtain the data located in the offset position range from the rosbag file stored in the cloud according to the download request, and return the data located in the offset position range to the electronic device;

[0188] The receiving unit is used to receive data returned from the cloud that is located within the offset position range;

[0189] The acquisition unit is used to acquire each target rosbag data from the data located in the offset position range within the rosbag file, based on the offset position of each target rosbag data.

[0190] In one optional implementation, the rosbag file stored in the cloud includes multiple file blocks; the offset position range of different file blocks in the rosbag file stored in the cloud is different.

[0191] The first generation unit includes:

[0192] The sub-unit is used to obtain the offset position range of any file block in the rosbag file stored in the cloud;

[0193] The selection subunit is used to select an offset position within the offset position range of the file block from the offset positions of multiple target rosbag data in the rosbag file.

[0194] The filtering subunit is used to filter the first and last offset positions within the offset position range of the file block.

[0195] Generate sub-units to generate offset position intervals based on the foremost and last offset positions.

[0196] In one optional implementation, the determining module includes:

[0197] The third determining unit is used to determine the offset position range of the target rosbag data in the rosbag file, which is located in the offset position range of the rosbag file where different file blocks are stored in the cloud.

[0198] The fourth determining unit is used to determine whether the file block containing the target rosbag data has been cached in the electronic device based on the determined offset position range.

[0199] The fifth determining unit is used to determine that the target rosbag data has been cached in the electronic device if the file block containing the target rosbag data has already been cached in the electronic device.

[0200] The sixth determining unit is used to determine, either, that the target rosbag data is not cached in the electronic device if the file block containing the target rosbag data is not cached in the electronic device.

[0201] In one optional implementation, the fifth determining unit includes:

[0202] The lookup subunit is used for local storage in the electronic device; in the third correspondence, the file block corresponding to the determined offset position range is found; the third correspondence includes the correspondence between the offset position range of the file block in the rosbag file and the file block;

[0203] The first determining subunit is used to determine the file block where the cached target rosbag data is located in the electronic device when a file block corresponding to the determined offset position interval is found in the third correspondence.

[0204] Alternatively, the second determining subunit is used to determine the file block in which the target rosbag data is not cached in the electronic device if no file block corresponding to the determined offset position interval is found in the third correspondence.

[0205] In this application, the identification information of the target rosbag data is obtained. The target rosbag data is located in a rosbag file stored in the cloud. Based on the identification information of the target rosbag data and a pre-stored first correspondence, the offset position of the target rosbag data within the rosbag file is obtained. The first correspondence includes the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data within the rosbag file. The target rosbag data is then retrieved from the rosbag file stored in the cloud based on its offset position within the rosbag file.

[0206] By utilizing this application, since a first correspondence between the identifier information of rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file is stored in advance, when the electronic device needs to obtain the offset position of the target rosbag data in the rosbag file based on the identifier information of the target rosbag data, it can directly obtain the offset position of the target rosbag data in the rosbag file by means of the first correspondence, without having to obtain the offset position of the target rosbag data in the rosbag file through data interaction with the cloud. In this way, it is not necessary to trigger a large number of random read and write operations to the cloud, nor is it necessary to establish and then disconnect connections for a large number of acquisition requests, nor is it necessary to perform multiple handshake actions and other connection configuration actions for establishing and disconnecting connections for a large number of acquisition requests. Thus, time can be saved and efficiency can be improved.

[0207] Furthermore, since the offset positions of each rosbag data item within the rosbag file are stored in advance using the identification information of each rosbag data item, the offset position of the target rosbag data within the rosbag file can be directly obtained based on the identification information of the target rosbag data. This achieves the goal of obtaining the offset position of the target rosbag data within the rosbag file without downloading the rosbag file from the cloud to the local device. It is evident that obtaining the offset position of the target rosbag data within the rosbag file does not depend on "the offset position of the rosbag data in the rosbag file that precedes the target rosbag data." In other words, the offset position of the target rosbag data within the rosbag file can be obtained without knowing "the offset position of the rosbag data in the rosbag file that precedes the target rosbag data." This enables streaming acquisition and streaming parsing of the target rosbag data, improving the acquisition and parsing speed, especially when there is a large amount of target rosbag data.

[0208] Reference Figure 10 The diagram shows a structural block diagram of a data processing apparatus according to this application, the apparatus comprising:

[0209] The fifth acquisition module 21 is used to obtain the identification information of each rosbag data and the offset position of each rosbag data in the rosbag file in the scenario where the autonomous driving device uploads the rosbag file to the cloud.

[0210] Storage module 22 is used to store the identification information of each rosbag data and the offset position of each rosbag data in the rosbag file in the first correspondence relationship. The first correspondence relationship includes the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file.

[0211] This application supports the subsequent retrieval of the identification information of the target rosbag data. The target rosbag data is located in a rosbag file stored in the cloud. Based on the identification information of the target rosbag data and a pre-stored first correspondence, the offset position of the target rosbag data within the rosbag file is obtained. The first correspondence includes the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data within the rosbag file. The target rosbag data is then retrieved from the rosbag file stored in the cloud based on its offset position within the rosbag file.

[0212] By utilizing this application, since a first correspondence between the identifier information of rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file is stored in advance, when the electronic device needs to obtain the offset position of the target rosbag data in the rosbag file based on the identifier information of the target rosbag data, it can directly obtain the offset position of the target rosbag data in the rosbag file by means of the first correspondence, without having to obtain the offset position of the target rosbag data in the rosbag file through data interaction with the cloud. In this way, it is not necessary to trigger a large number of random read and write operations to the cloud, nor is it necessary to establish and then disconnect connections for a large number of acquisition requests, nor is it necessary to perform multiple handshake actions and other connection configuration actions for establishing and disconnecting connections for a large number of acquisition requests. Thus, time can be saved and efficiency can be improved.

[0213] Furthermore, since the offset positions of each rosbag data item within the rosbag file are stored in advance using the identification information of each rosbag data item, the offset position of the target rosbag data within the rosbag file can be directly obtained based on the identification information of the target rosbag data. This achieves the goal of obtaining the offset position of the target rosbag data within the rosbag file without downloading the rosbag file from the cloud to the local device. It is evident that obtaining the offset position of the target rosbag data within the rosbag file does not depend on "the offset position of the rosbag data in the rosbag file that precedes the target rosbag data." In other words, the offset position of the target rosbag data within the rosbag file can be obtained without knowing "the offset position of the rosbag data in the rosbag file that precedes the target rosbag data." This enables streaming acquisition and streaming parsing of the target rosbag data, improving the acquisition and parsing speed, especially when there is a large amount of target rosbag data.

[0214] This application also provides a non-volatile readable storage medium storing one or more modules (programs). When these modules are applied to a device, they enable the device to execute the instructions for the method steps in this application.

[0215] This application provides one or more machine-readable media storing instructions that, when executed by one or more processors, cause an electronic device to perform one or more methods as described in the above embodiments. In this application, the electronic device includes a server, a gateway, sub-devices, etc., and the sub-devices are devices such as Internet of Things (IoT) devices.

[0216] Embodiments of this disclosure can be implemented as an apparatus with any suitable hardware, firmware, software, or any combination thereof, configured as desired. This apparatus may include electronic devices such as servers (clusters) and terminal devices such as IoT devices.

[0217] Figure 11 An exemplary apparatus 1300 is schematically shown that can be used to implement the various embodiments of this application.

[0218] In one embodiment, Figure 11An exemplary device 1300 is shown, which includes one or more processors 1302, a control module (chipset) 1304 coupled to at least one of the processors 1302, a memory 1306 coupled to the control module 1304, a non-volatile memory (NVM) / storage device 1308 coupled to the control module 1304, one or more input / output devices 1310 coupled to the control module 1304, and a network interface 1312 coupled to the control module 1304.

[0219] Processor 1302 may include one or more single-core or multi-core processors, and processor 1302 may include any combination of general-purpose processors or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, device 1300 can function as a server device such as a gateway in the embodiments of this application.

[0220] In some embodiments, apparatus 1300 may include one or more computer-readable media (e.g., memory 1306 or NVM / storage device 1308) having instructions 1314 and one or more processors 1302 that are combined with the one or more computer-readable media and configured to execute the instructions 1314 to implement the module and thus perform the actions in this disclosure.

[0221] In one embodiment, the control module 1304 may include any suitable interface controller to provide any suitable interface to at least one of the processors 1302 and / or any suitable device or component communicating with the control module 1304.

[0222] The control module 1304 may include a memory controller module to provide an interface to the memory 1306. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0223] Memory 1306 may be used, for example, to load and store data and / or instructions 1314 for device 1300. In one embodiment, memory 1306 may include any suitable volatile memory, such as suitable DRAM. In some embodiments, memory 1306 may include double data rate quad synchronous dynamic random access memory (DDR4 SDRAM).

[0224] In one embodiment, the control module 1304 may include one or more input / output controllers to provide interfaces to the NVM / storage device 1308 and (one or more) input / output devices 1310.

[0225] For example, NVM / storage device 1308 may be used to store data and / or instructions 1314. NVM / storage device 1308 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (one or more) non-volatile storage devices (e.g., one or more hard disk drives (HDDs), one or more optical disc drives (CDs), and / or one or more digital universal optical disc (DVD) drives).

[0226] NVM / storage device 1308 may include storage resources that are physically part of a device on which device 1300 is mounted, or that can be accessed by the device without needing to be part of the device. For example, NVM / storage device 1308 may be accessed via a network via one or more input / output devices 1310.

[0227] One or more input / output devices 1310 may provide an interface for device 1300 to communicate with any other suitable device. Input / output devices 1310 may include communication components, pinyin components, sensor components, etc. Network interface 1312 may provide an interface for device 1300 to communicate via one or more networks. Device 1300 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, such as accessing wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G, 5G, etc., or combinations thereof.

[0228] In one embodiment, at least one of the processors 1302 may be logically packaged with one or more controllers (e.g., memory controller modules) of the control module 1304. In one embodiment, at least one of the processors 1302 may be logically packaged with one or more controllers of the control module 1304 to form a system-in-package (SiP). In one embodiment, at least one of the processors 1302 may be integrated with the logic of one or more controllers of the control module 1304 on the same die. In one embodiment, at least one of the processors 1302 may be integrated with the logic of one or more controllers of the control module 1304 on the same die to form a system-on-a-chip (SoC).

[0229] In various embodiments, device 1300 may be, but is not limited to, a server, desktop computing device, or mobile computing device (e.g., laptop computing device, handheld computing device, tablet computer, netbook, etc.). In various embodiments, device 1300 may have more or fewer components and / or different architectures. For example, in some embodiments, device 1300 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0230] This application provides an electronic device, including: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, cause the electronic device to perform one or more methods as described in this application.

[0231] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0232] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0233] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable information processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable information processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0234] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable information processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0235] These computer program instructions can also be loaded onto a computer or other programmable information processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0236] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0237] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the element.

[0238] The data processing method and apparatus provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data processing method, characterized in that, Applied to electronic devices, the method includes: Obtain the identification information of the target rosbag data, which is located in a rosbag file stored in the cloud; Based on the identification information of the target rosbag data and the pre-stored first correspondence, obtain the offset position of the target rosbag data in the rosbag file; the first correspondence includes the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file. The target rosbag data is obtained from the rosbag file stored in the cloud based on the offset position; There are multiple target rosbag data, and the offset positions of different target rosbag data in the rosbag file are different; The step of retrieving the target rosbag data from the rosbag file stored in the cloud based on the offset position includes: Generate offset position ranges based on the offset positions of multiple target rosbag data in the rosbag file; Generate a download request, which must include at least an offset position range; Send a download request to the cloud so that the cloud can retrieve the data located in the offset position range from the rosbag file stored in the cloud according to the download request, and return the data located in the offset position range to the electronic device; Receive data returned from the cloud that is located within the offset position range; Based on the offset position of each target rosbag data in the rosbag file, the data of each target rosbag is obtained from the data located in the offset position range.

2. The method according to claim 1, characterized in that, The method further includes: Determine whether the target rosbag data has been cached in the electronic device based on the offset position; If the target rosbag data is not cached in the electronic device, then the step of retrieving the target rosbag data from the rosbag file stored in the cloud based on the offset position is performed again.

3. The method according to claim 2, characterized in that, The method further includes: If the target rosbag data is already cached in the electronic device, retrieve the target rosbag data cached in the electronic device.

4. The method according to claim 2 or 3, characterized in that, The step of determining whether the target rosbag data has been cached in the electronic device based on the offset position includes: In the second correspondence stored locally on the electronic device, search for the rosbag data corresponding to the offset position; the second correspondence includes the correspondence between the offset position of the rosbag data in the rosbag file and the rosbag data itself. If rosbag data corresponding to the offset position is found in the second correspondence, it is determined that the target rosbag data has been cached in the electronic device; Alternatively, if no rosbag data corresponding to the offset position is found in the second correspondence, it is determined that the target rosbag data is not cached in the electronic device.

5. The method according to claim 1, characterized in that, The rosbag file stored in the cloud contains multiple file blocks; the offset position range of different file blocks in the rosbag file stored in the cloud is different. The step of generating offset position intervals based on the offset positions of multiple target rosbag data in the rosbag file includes: For any file block, obtain the offset position range of the file block in the rosbag file stored in the cloud; Among the offset positions of multiple target rosbag data in the rosbag file, select the offset position that is located within the offset position range of the file block; Within the offset position range of the file block, filter the first and last offset positions. An offset position range is generated based on the first and last offset positions.

6. The method according to claim 5, characterized in that, The step of determining whether the target rosbag data has been cached in the electronic device based on the offset position includes: In the offset position range of the rosbag file where different file blocks are stored in the cloud, determine the offset position range of the target rosbag data in the rosbag file. Based on the determined offset range, determine whether the file block containing the target rosbag data has been cached in the electronic device; If the file block containing the target rosbag data is already cached in the electronic device, determine that the target rosbag data is already cached in the electronic device; Alternatively, if the file block containing the target rosbag data is not cached on the electronic device, determine that the target rosbag data is not cached on the electronic device.

7. The method according to claim 6, characterized in that, The step of determining whether the file block containing the target rosbag data has been cached in the electronic device based on the determined offset position range includes: In the third correspondence in the local storage of the electronic device, find the file block corresponding to the determined offset position range; the third correspondence includes the correspondence between the offset position range of the file block in the rosbag file and the file block; If a file block corresponding to the determined offset position range is found in the third correspondence, the file block where the cached target rosbag data is located in the electronic device is determined. Alternatively, if no file block corresponding to the determined offset position interval is found in the third correspondence, the file block containing the uncached target rosbag data in the electronic device is determined.

8. A data processing method, characterized in that, The method includes: In scenarios where autonomous driving devices upload rosbag files to the cloud, the identification information of each rosbag data and the offset position of each rosbag data in the rosbag file are obtained from the rosbag file. The first correspondence stores the identification information of each rosbag data and the offset position of each rosbag data in the rosbag file. The first correspondence includes the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file. An offset range is generated based on the offset positions of multiple target rosbag data in the rosbag file; different target rosbag data have different offset positions in the rosbag file. Generate a download request, which must include at least an offset position range; Send a download request to the cloud so that the cloud can retrieve the data located in the offset position range from the rosbag file stored in the cloud according to the download request, and return the data located in the offset position range to the electronic device; Receive data returned from the cloud that is located within the offset position range; Based on the offset position of each target rosbag data in the rosbag file, the data of each target rosbag is obtained from the data located in the offset position range.

9. A data processing apparatus, characterized in that, Applied to electronic devices, the device includes: The first acquisition module is used to acquire the identification information of the target rosbag data, which is located in a rosbag file stored in the cloud. The second acquisition module is used to acquire the offset position of the target rosbag data in the rosbag file based on the identification information of the target rosbag data and the first correspondence relationship stored in advance; the first correspondence relationship includes the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file. The third acquisition module is used to acquire target rosbag data from the rosbag file stored in the cloud according to the offset position; There are multiple target rosbag data sets, and the offset positions of different target rosbag data sets in the rosbag file are different; the third acquisition module includes: The first generation unit is used to generate offset position intervals based on the offset positions of multiple target rosbag data in the rosbag file. The second generation unit is used to generate a download request, which carries at least an offset position range. The sending unit is used to send a download request to the cloud, so that the cloud can obtain the data located in the offset position range from the rosbag file stored in the cloud according to the download request, and return the data located in the offset position range to the electronic device; The receiving unit is used to receive data returned from the cloud that is located within the offset position range; The acquisition unit is used to acquire each target rosbag data from the data located in the offset position range within the rosbag file, based on the offset position of each target rosbag data.

10. A data processing apparatus, characterized in that, The device includes: The fifth acquisition module is used to obtain the identification information of each rosbag data and the offset position of each rosbag data in the rosbag file in the scenario where the autonomous driving device uploads the rosbag file to the cloud. The storage module is used to store the identification information of each rosbag data and the offset position of each rosbag data in the rosbag file in the first correspondence relationship. The first correspondence relationship includes the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file. An offset range is generated based on the offset positions of multiple target rosbag data in the rosbag file; different target rosbag data have different offset positions in the rosbag file. Generate a download request, which must include at least an offset position range; Send a download request to the cloud so that the cloud can retrieve the data located in the offset position range from the rosbag file stored in the cloud according to the download request, and return the data located in the offset position range to the electronic device; Receive data returned from the cloud that is located within the offset position range; Based on the offset position of each target rosbag data in the rosbag file, the data of each target rosbag is obtained from the data located in the offset position range.

11. An autonomous driving device, characterized in that, The autonomous driving device includes: One or more processors; and memory associated with the one or more processors, the memory storing program instructions that, when read and executed by the one or more processors, perform the following operations: In scenarios where autonomous driving devices upload rosbag files to the cloud, the identification information of each rosbag data and the offset position of each rosbag data in the rosbag file are obtained from the rosbag file. The first correspondence stores the identification information of each rosbag data and the offset position of each rosbag data in the rosbag file. The first correspondence includes the correspondence between the identification information of the rosbag data in the rosbag file and the offset position of the rosbag data in the rosbag file. An offset range is generated based on the offset positions of multiple target rosbag data in the rosbag file; different target rosbag data have different offset positions in the rosbag file. Generate a download request, which must include at least an offset position range; Send a download request to the cloud so that the cloud can retrieve the data located in the offset position range from the rosbag file stored in the cloud according to the download request, and return the data located in the offset position range to the electronic device; Receive data returned from the cloud that is located within the offset position range; Based on the offset position of each target rosbag data in the rosbag file, the data of each target rosbag is obtained from the data located in the offset position range.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the steps of the method as claimed in any one of claims 1 to 8 when executing the program.

13. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as claimed in any one of claims 1 to 8.

Images