Data frame processing method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention provides a data frame processing method and device, electronic equipment and a readable storage medium, and relates to the field of data processing.The data frame processing method comprises the steps that in response to a first page opening operation, a first data address corresponding to each first data frame in a to-be-loaded first data frame set is obtained; determining storage position information corresponding to a first data frame in the first data frame set based on the first data address; the storage position information comprises local storage and remote storage; under the condition that the storage position information corresponding to the first data frame is remote storage, loading the first data frame from a first memory located in a storage server based on the first data address, and storing the first data frame to a local second memory; loading the first data frame from a second memory under the condition that the storage position information corresponding to the first data frame is local storage; and displaying a rendered image generated based on the first data frame.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a data frame processing method, device, electronic device and readable storage medium. Background Art

[0002] Relying on long-term technological accumulation and recent breakthroughs in the field of artificial intelligence, coupled with increasingly mature policies, driverless cars are coming to people. The fact that major Internet companies and automobile giants have invested heavily in the development of driverless technology shows that driverless cars have gradually become the general trend of the information industry and the automobile industry.

[0003] In the process of developing unmanned driving functions and systems, since the autonomous driving AI is not perfect, it is necessary to view, mark, and correct the data generated by the autonomous driving AI when driving on the road. The data can also be used to further train the autonomous driving AI and analyze errors to improve the working effect of the autonomous driving AI. Usually, the staff will store the massive data generated by the vehicle side in the server. When viewing the data on the page, it is necessary to load the vehicle side data required for the page from the server and process the autonomous driving data (for example, marking, correcting, etc.) before the collected autonomous driving data can be used for the training and analysis process of the autonomous driving AI.

[0004] However, since the types and amount of autonomous driving data generated by vehicles are large, in related technologies, the efficiency of technicians loading data frames formed by autonomous driving data from the server to the terminals used by technicians is relatively low. Summary of the invention

[0005] In view of this, the present invention aims to propose a data frame processing method, device, electronic device and readable storage medium to solve the problem of low efficiency in loading data frames formed by autonomous driving data from a server to a terminal in the prior art.

[0006] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] In a first aspect, the present invention provides a data frame processing method, the method comprising:

[0008] In response to the first page opening operation, obtaining first data addresses corresponding to each first data frame in the first data frame set to be loaded;

[0009] Determine storage location information corresponding to a first data frame in the first data frame set based on the first data address; the storage location information includes local storage and remote storage;

[0010] In a case where the storage location information corresponding to the first data frame is the remote storage, based on the first data address, the first data frame is loaded from a first memory located in a storage server, and the first data frame is stored in a second memory located locally;

[0011] When the storage location information corresponding to the first data frame is the local storage, loading the first data frame from the second memory;

[0012] A rendered image generated based on the first data frame is displayed.

[0013] Optionally, loading the first data frame from a first memory located in a storage server based on the first data address includes:

[0014] When the first data frame is the first frame in the first data frame set, obtaining the number of addresses of data to be loaded in the loading queue;

[0015] When the number of addresses is greater than or equal to a preset threshold, cache the addresses of the data to be loaded;

[0016] Adding a first data address corresponding to a first data frame of the first data frame set to the loading queue;

[0017] A first data frame corresponding to the first data address is loaded from a first memory located in a storage server based on the load queue.

[0018] Optionally, the method further comprises:

[0019] In response to the second page opening operation, obtaining a second data address corresponding to each second data frame in the second data frame set to be loaded;

[0020] storing target frame position information of the first data frame currently being loaded in the first data frame set;

[0021] loading the second data frame based on the second data address;

[0022] When all the second data frames in the second data frame set are loaded completely, the first data frames in the first data frame set that have not been loaded completely are continuously loaded starting from the target frame position information.

[0023] Optionally, determining storage location information corresponding to the first data frame in the first data frame set based on the first data address includes:

[0024] Acquire the reading terminal information of the first data frame from the first data address;

[0025] In a case where the reading terminal information matches the local end information, determining that the storage location information corresponding to the first data address is the local storage;

[0026] In a case where the reading terminal information does not match the local end information, it is determined that the storage location information corresponding to the first data address is the remote storage.

[0027] Optionally, the first data frame includes first point cloud data, and loading the first data frame from a first memory located in a storage server based on the first data address includes:

[0028] Sending a point cloud data acquisition instruction including the first data address to the storage server, so that the storage server responds to the point cloud data acquisition instruction, determines the relative distance and relative direction between the point cloud coordinates in the original point cloud data stored at the first data address and the vehicle, and generates the first point cloud data based on the relative distance and the relative direction;

[0029] A first data frame including the first point cloud data is obtained from the storage server.

[0030] Optionally, the method further comprises:

[0031] Acquire data type information contained in the first data frame, and determine a decompression algorithm based on the data type information;

[0032] Decompressing the first data frame by using the decompression algorithm to obtain binary streams corresponding to the data types respectively;

[0033] The binary stream is parsed to obtain data to be rendered corresponding to the data type, and the rendered image is generated based on a plurality of the data to be rendered.

[0034] In a second aspect, the present invention provides a data frame processing device, the device comprising:

[0035] A first response module, configured to obtain, in response to a first page opening operation, a first data address corresponding to each first data frame in a first data frame set to be loaded;

[0036] A storage location module, configured to determine storage location information corresponding to a first data frame in the first data frame set based on the first data address; the storage location information includes local storage and remote storage;

[0037] a storage module, configured to load the first data frame from a first memory located in a storage server based on the first data address and store the first data frame in a second memory located locally when the storage location information corresponding to the first data frame is the remote storage;

[0038] A first data frame module, configured to load the first data frame from the second memory when the storage location information corresponding to the first data frame is the local storage;

[0039] A display module is used to display a rendered image generated based on the first data frame.

[0040] Optionally, the storage module includes:

[0041] An address quantity submodule, used for obtaining the address quantity of data addresses to be loaded in the loading queue when the first data frame is the first frame in the first data frame set;

[0042] A to-be-loaded data address submodule, configured to cache the to-be-loaded data addresses when the number of addresses is greater than or equal to a preset threshold;

[0043] A loading queue submodule, configured to add a first data address corresponding to a first data frame of the first data frame set to the loading queue;

[0044] The first data frame loading submodule is used to load the first data frame corresponding to the first data address from the first memory located in the storage server based on the loading queue.

[0045] Optionally, the device further comprises:

[0046] A second corresponding module, configured to obtain, in response to a second page opening operation, a second data address corresponding to each second data frame in a second data frame set to be loaded;

[0047] A target frame position information module, used to store the target frame position information of the first data frame currently being loaded in the first data frame set;

[0048] A second data frame module, configured to load the second data frame based on the second data address;

[0049] The continuing loading module is used to continue loading the first data frames that have not been loaded in the first data frame set starting from the target frame position information when all the second data frames in the second data frame set have been loaded.

[0050] Optionally, the storage location module includes:

[0051] A terminal information reading submodule, used for obtaining the reading terminal information of the first data frame from the first data address;

[0052] A first storage location information determination submodule, configured to determine, when the reading terminal information matches the local end information, that the storage location information corresponding to the first data address is the local storage;

[0053] The second storage location information determination submodule is used to determine that the storage location information corresponding to the first data address is the remote storage when the reading terminal information does not match the local end information.

[0054] Optionally, the first data frame includes first point cloud data, and the storage module includes:

[0055] an instruction sending submodule, configured to send a point cloud data acquisition instruction including the first data address to the storage server, so that the storage server responds to the point cloud data acquisition instruction, determines the relative distance and relative direction between the point cloud coordinates in the original point cloud data stored at the first data address and the vehicle, and generates the first point cloud data based on the relative distance and the relative direction;

[0056] The point cloud data acquisition submodule is used to acquire a first data frame containing the first point cloud data from the storage server.

[0057] Optionally, the device further comprises:

[0058] A decompression algorithm acquisition module, used to obtain data type information contained in the first data frame, and determine a decompression algorithm based on the data type information;

[0059] A binary stream module, used for decompressing the first data frame by using the decompression algorithm to obtain binary streams corresponding to the data types respectively;

[0060] The rendering image generation module is used to parse the binary stream to obtain the to-be-rendered data corresponding to the data type, and generate the rendering image based on the multiple to-be-rendered data.

[0061] In a third aspect, the present invention provides a readable storage medium, which, when instructions in the readable storage medium are executed by a processor of an electronic device, enables the electronic device to execute the above-mentioned data frame processing method.

[0062] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method of the first aspect are implemented.

[0063] Compared with the prior art, the data frame processing method, device, electronic device and readable storage medium described in the present invention have the following advantages:

[0064] In summary, an embodiment of the present invention provides a data frame processing method, including: in response to a first page opening operation, obtaining a first data address corresponding to each first data frame in a first data frame set to be loaded; determining storage location information corresponding to a first data frame in a first data frame set based on the first data address; the storage location information includes local storage and remote storage; when the storage location information corresponding to the first data frame is remote storage, based on the first data address, loading the first data frame from a first memory located in a storage server, and storing the first data frame in a second memory located locally; when the storage location information corresponding to the first data frame is local storage, loading the first data frame from the second memory; and displaying a rendered image generated based on the first data frame. The method can load the first data frame in different ways according to the storage location of the first data frame, load the first data frame from the server and cache it at the same time when the first data frame is not cached locally, so that the first data frame is subsequently directly loaded from the local, thereby reducing the network bandwidth and server resource consumption required for users to retrieve data frames, and helping to improve the loading speed of the first data frame. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0066] Figure 1 A flow chart of steps of a data frame processing method provided by an embodiment of the present invention;

[0067] Figure 2 A flowchart of another data frame processing method provided by an embodiment of the present invention;

[0068] Figure 3 A loading flow chart provided by an embodiment of the present invention;

[0069] Figure 4 A structural block diagram of a data frame processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0070] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0071] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0072] Reference Figure 1 , shows a step flow chart of a data frame processing method provided by an embodiment of the present invention.

[0073] Step 101 : in response to a first page opening operation, obtaining a first data address corresponding to each first data frame in a first data frame set to be loaded.

[0074] In an embodiment of the present invention, the first page opening operation may include operations that require the retrieval of data frames, such as the operation of viewing the vehicle-side automatic driving data within a certain time period, the operation of playing the vehicle-side automatic driving data within a certain time period. Among them, the vehicle-side automatic driving data refers to the data collected by the automatic driving car during driving, which may specifically include but is not limited to sensor data, vehicle status data, GPS and map data, control command data, system status data, etc. Sensor data may include but is not limited to point cloud data obtained by LiDAR, image data obtained by the vehicle camera, ultrasonic data obtained by the vehicle radar, ultrasonic data obtained by the ultrasonic sensor, etc. Vehicle status data may include but is not limited to the speed, direction, steering angle, brake status, throttle status, etc. of the vehicle. GPS and map data may include but is not limited to the location data of the car, road data, traffic sign data, traffic light data, etc. Control command data may include instructions issued by the automatic driving system to instruct the vehicle how to operate, such as acceleration instructions, deceleration instructions, steering instructions, etc. System status data may include the working status of various sensors and components, battery power, computing power usage, etc.

[0075] The autonomous driving system will continuously receive and process a large amount of vehicle-side autonomous driving data when working. These vehicle-side autonomous driving data are usually organized into "data frames", each of which represents a data snapshot at a specific time point. The content of the data frame may include but is not limited to a time stamp, image data, point cloud data, radar data, ultrasonic data, GPS data, and other combinations of one or more of the above vehicle-side autonomous driving data. In addition, the data frame may also include annotation data generated after the vehicle processes the image data and point cloud data. These annotation data may include but are not limited to target detection results, object recognition results, etc., which are not specifically limited in the embodiments of the present invention.

[0076] In an embodiment of the present invention, the first page opening operation can not only be used to open the first page, but also can be used to indicate the first data frame set that needs to be displayed on the first page. For example, the first page opening operation can include information such as the target vehicle, target time period, and target frame rate. Through this information, it can be determined which data frames need to be loaded for display on the first page.

[0077] After receiving the first page opening operation, the first data address of the first data frame set required for the first page can be obtained, so that the corresponding first data frame can be further obtained based on the first data address in the subsequent process. For example, the user can first select the target vehicle to be played back or viewed, and then the user can select a target time period, and then click the confirmation button to issue the first page opening operation, and at the same time obtain the target vehicle and target time period selected by the user, and query the first data addresses corresponding to each first data frame in the first data frame set formed by the automatic driving data obtained by the target vehicle in the target time period through the target vehicle and the target time period.

[0078] Step 102: Determine storage location information corresponding to the first data frame in the first data frame set based on the first data address; the storage location information includes local storage and remote storage.

[0079] In an embodiment of the present invention, when loading a data frame, this terminal can simultaneously store the data frame and the storage location information corresponding to the data frame in the terminal. Therefore, the storage location information corresponding to the first data frame in the first data frame set can be determined according to the first data address, and the first data address corresponding to the first data frame can be queried in the local storage of this terminal. If the first data address exists in this terminal, it means that this terminal has historically loaded the first data frame corresponding to the first data address, and it can be determined that the storage location information of the first data frame is locally stored; if the first data address does not exist in this terminal, it means that this terminal has not historically loaded the first data frame corresponding to the first data address, and it can be determined that the storage location information of the first data frame is remotely stored.

[0080] In addition, in another case, the data address may include not only the source file address of the data frame in the storage server, but also the copy address of the copy of the data frame, that is, the data address may include multiple storage addresses of the data frame, so that after obtaining the first data address corresponding to the first data frame, if the first data address does not include the copy address, it can be determined that the first data frame has not been read, and the storage location information corresponding to the first data address can be determined to be off-site storage; if the first data address includes a copy address, the copy address can be matched with the address of the terminal, if the match is successful, it can be determined that the storage location information corresponding to the first data address is local storage, if the match is unsuccessful, it can be determined that the storage location information corresponding to the first data address is off-site storage. Among them, the above-mentioned copy address can be composed of a terminal address indicating which terminal is located and a path address indicating a specific file address located in the terminal. In addition, the technician can use other methods to determine the storage location information corresponding to the first data frame in the above-mentioned first data frame set, and the embodiment of the present invention does not make specific limitations.

[0081] Step 103, when the storage location information corresponding to the first data frame is the off-site storage, based on the first data address, load the first data frame from the first memory located in the storage server, and store the first data frame in the second memory located locally.

[0082] In an embodiment of the present invention, the first data frame can be loaded in different loading methods according to the storage location information of the first data frame, thereby improving the efficiency of loading the first data frame. Specifically, if the storage location information corresponding to the first data frame is stored off-site, the first data frame can be loaded from the first memory located on the storage server based on the first data address, and the first data frame can be stored in a second memory located locally. The above-mentioned storage server may include but is not limited to a distributed file system (Distributed File System, DFS), a time series database (such as InfluxDB, OpenTSDB or TimescaleDB), etc., and the embodiment of the present invention is not specifically limited. The first memory may include a physical hard disk, a virtual hard disk, a memory, and other memories of the storage server.

[0083] According to the order of the first data frames in the first data frame set, the corresponding first data frames can be loaded one by one from the storage server based on the storage location information corresponding to the first data frames from the beginning to the end, or multiple first data frames in the first data frame set can be loaded in parallel at one time. The embodiment of the present invention does not specifically limit the specific loading method.

[0084] After loading the first data frame, the first data frame can also be stored in a second memory located locally to complete the localized preloading of the first data frame, so that when the first data frame is needed again in the future, it can be directly loaded from the local, saving the server burden and network bandwidth consumption, and at the same time can greatly improve the loading speed and the response speed of the user's opening of the first page operation.

[0085] Step 104: When the storage location information corresponding to the first data frame is the local storage, load the first data frame from the second memory.

[0086] In an embodiment of the present invention, if the storage location information corresponding to the first data frame is locally stored, the first data frame can be directly loaded from a second storage located locally, thereby avoiding access to a storage server, saving performance resources and bandwidth of the storage server, and improving the speed of obtaining the first data frame. The above-mentioned second storage may include a physical hard disk, a virtual hard disk, a memory, and other storage devices of the local terminal, and may also include an external hard disk mounted on the local terminal, etc., which is not specifically limited in the embodiment of the present invention.

[0087] It should be noted that loading the first data frame may refer to reading the complete data of the first data frame into the running memory of the local end, so as to facilitate subsequent processing of the first data frame. Among them, the local end may be a data viewing platform, a data presentation platform, a data flow platform, etc. for vehicle autonomous driving data, which is not specifically limited in the embodiments of the present invention.

[0088] Step 105: display a rendered image generated based on the first data frame.

[0089] After obtaining the first data frame in the first data frame set, rendering can be performed based on the first data frame to obtain a corresponding rendered image, and then the rendered image can be displayed so that the user can replay and view the vehicle's autonomous driving data. The autonomous driving data generated by the vehicle can also be annotated and edited to make the autonomous driving data more accurate and improve the training effect of subsequent autonomous driving model training based on the autonomous driving data.

[0090] For example, the first data frame may include four visual images collected by four on-board cameras of vehicle A, a point cloud image collected by a lidar of vehicle A, and a target detection result (such as a target detection box, target type information, etc.) generated by a target detection module of vehicle A based on the above visual images and point cloud images. After obtaining the first data frame, the target detection result and each visual image may be superimposed and rendered to obtain a visual rendering image corresponding to each visual image, and the point cloud image and the target detection result may be superimposed and rendered to obtain a point cloud rendering image corresponding to the point cloud image.

[0091] The above-mentioned visual rendering image and point cloud rendering image can be displayed in parallel for user viewing. It should be noted that by continuously reading and rendering the first data frames in the first data frame set and displaying the rendered images obtained by rendering each first data frame, the first data frames in the first data frame set can be dynamically played. In the process of playing the first data frame, the user can also adjust the target detection results superimposed and displayed therein, for example, the position of the target detection frame, the size of the target detection frame, the target type, etc. can be adjusted.

[0092] In summary, an embodiment of the present invention provides a data frame processing method, including: in response to a first page opening operation, obtaining a first data address corresponding to each first data frame in a first data frame set to be loaded; determining storage location information corresponding to a first data frame in a first data frame set based on the first data address; the storage location information includes local storage and remote storage; when the storage location information corresponding to the first data frame is remote storage, based on the first data address, loading the first data frame from a first memory located in a storage server, and storing the first data frame in a second memory located locally; when the storage location information corresponding to the first data frame is local storage, loading the first data frame from the second memory; and displaying a rendered image generated based on the first data frame. The method can load the first data frame in different ways according to the storage location of the first data frame, load the first data frame from the server and cache it at the same time when the first data frame is not cached locally, so that the first data frame is subsequently directly loaded from the local, thereby reducing the network bandwidth and server resource consumption required for users to retrieve data frames, and helping to improve the loading speed of the first data frame.

[0093] Reference Figure 2 , Figure 2 A flowchart of another data frame processing method provided by an embodiment of the present invention is shown.

[0094] Step 201 : in response to a first page opening operation, obtaining a first data address corresponding to each first data frame in a first data frame set to be loaded.

[0095] This step can refer to step 101, and will not be described in detail in this embodiment of the present invention.

[0096] Step 202: Acquire the reading terminal information of the first data frame from the first data address.

[0097] In the embodiment of the present invention, after determining the first data address, the reading terminal information of the first data frame can be obtained from the first data address, wherein the reading terminal information can include all terminal information of reading the first data frame corresponding to the first data address from the first data address.

[0098] For example, the storage server can establish a correspondence between a data frame and a corresponding data address in the address database while storing a data frame. The terminal can obtain the data address of the data frame from the address database, and read the corresponding data frame from the storage server according to the data address. The storage server stores the terminal information of the terminal as the reading terminal information corresponding to the data frame. Before subsequent terminals attempt to read the data frame, they can first obtain the reading terminal information from the storage server and determine whether the data frame has been read by this terminal based on the reading terminal information.

[0099] Step 203: When the reading terminal information matches the local end information, determine that the storage location information corresponding to the first data address is the local storage.

[0100] After obtaining the reading terminal information, the reading terminal information can be matched with the local end information to determine whether the reading terminal information includes the local end. If it does, it means that the local end has read the first data frame stored at the first data address from the storage server, and then it can be determined that the storage location information corresponding to the first data address is local storage.

[0101] Step 204: When the reading terminal information does not match the local terminal information, determine that the storage location information corresponding to the first data address is the remote storage.

[0102] After obtaining the reading terminal information, the reading terminal information can be matched with the local end information to determine whether the reading terminal information includes the local end. If not, it means that the local end has not read the first data frame corresponding to the first data address from the storage server, then it can be determined that the storage location information corresponding to the first data address is stored off-site.

[0103] In addition, in addition to being completed on the local side, the above-mentioned matching process can also be completed by the storage server. For example, the local side can send the first storage address and local side information to the storage server, and the storage server directly matches the reading terminal information corresponding to the first storage address and the above-mentioned local side information to determine whether the reading terminal information contains the local side information. If it does, it means that the local side has read the first data frame stored at the first data address from the storage server, and the storage server can return to the local side a judgment result that the storage location information corresponding to the first data address is locally stored. If it does not contain, it means that the local side has not read the first data frame corresponding to the first data address from the storage server, and the storage server can return to the local side a judgment result that the storage location information corresponding to the first data address is remotely stored.

[0104] In an embodiment of the present invention, the storage location information corresponding to the first data address can be determined through the above steps 201 to 204. For each first data address, its corresponding storage location information can be directly determined by obtaining the recorded reading terminal information, without the need to search the massive data frames stored locally or on the server, thereby improving the efficiency of determining the storage location information.

[0105] Step 205, when the storage location information corresponding to the first data frame is the off-site storage, based on the first data address, load the first data frame from the first memory located in the storage server, and store the first data frame in the second memory located locally.

[0106] This step can refer to the above-mentioned step 103, and will not be described in detail in this embodiment of the present invention.

[0107] Optionally, step 205 may further include:

[0108] Sub-step 2051 , when the first data frame is the first frame in the first data frame set, obtaining the number of addresses of data to be loaded in the loading queue.

[0109] In an embodiment of the present invention, a user may often switch between multiple pages, or change the vehicle, time period, etc. corresponding to the played animation, resulting in that when the user opens the first page, the data frame of the previous page may not have been loaded. Therefore, when loading the first frame in the first data frame set, the number of addresses of the data addresses to be loaded in the loading queue can be obtained, and the timing to start loading the first data frame set can be further determined based on the data address, thereby avoiding the waiting time process for loading the first data frame set and improving the loading response speed.

[0110] Specifically, each time a data frame set is loaded, all data addresses corresponding to the data frame set can be placed in a loading queue to form data addresses to be loaded in the loading queue. When the user opens the first page to determine the corresponding first data frame set and starts loading the first first data frame in the set, the number of data addresses to be loaded in the loading queue can be obtained. It should be noted that the addresses that have been loaded in the loading queue can be removed to ensure that the number of data addresses to be loaded obtained is the addresses that have not yet been loaded.

[0111] Sub-step 2052, when the number of addresses is greater than or equal to a preset threshold, cache the addresses of the data to be loaded.

[0112] In an embodiment of the present invention, in order to ensure the integrity of the task execution of each data set loading, a preset threshold (for example, 10) can be set, and when the number of addresses in the loading queue is greater than or equal to the preset threshold, that is, when it takes a long time to load the remaining addresses in the loading queue, the addresses of the data to be loaded in the loading queue can be cached so as to switch to the loading process of the first data frame, and after the loading of the first data frame is completed, the loading of the data frame corresponding to the data address to be loaded can be resumed.

[0113] Sub-step 2053: adding the first data address corresponding to the first data frame of the first data frame set to the loading queue.

[0114] In an embodiment of the present invention, after the loading queue is cleared, the first data addresses corresponding to the first data frames in the first data frame set can be added to the loading queue in sequence. Furthermore, in order to improve the reconstruction speed of the loading queue, the first data addresses can also be directly overwritten and written into the loading queue. It should be noted that the distribution order of the first data addresses in the loading queue can be made the same as the order of the first data frames corresponding to them in the first data set.

[0115] Sub-step 2054: loading a first data frame corresponding to the first data address from a first memory located in a storage server based on the loading queue.

[0116] In an embodiment of the present invention, since the distribution order of the first data addresses in the loading queue is the same as the order of the corresponding first data frames in the first data set, the first data frames corresponding to the first data addresses can be loaded sequentially from the first memory located in the storage server based on the order of the first data addresses in the loading queue.

[0117] Reference Figure 3 , Figure 3 A loading flow chart provided by an embodiment of the present invention is shown. Figure 3 As shown, after the user opens the first page, the full amount of first data addresses corresponding to the first data frame set required for the first page can be extracted, and then the storage location information of the first frame in the first data frame set can be determined, that is, whether the first frame has been stored in the local second memory. If the first frame is already in the second memory, the first frame is directly obtained from the local and subsequent rendering is performed. If the first frame is not in the second memory, it is further determined whether the first frame is being loaded. If the first frame has not started to be loaded, it means that there are other unfinished tasks in the loading queue. At this time, the processing of the data address to be loaded in the loading queue can be suspended, and the first loading address is rewritten in the loading queue. The first data frame corresponding to the first loading address is preferentially obtained from the storage server. After the first data frame is completely stored in the local second memory, the first data frame is loaded from the second memory. If the above judgment result of whether the first frame is being loaded is that the first frame has started to be loaded, the first data frame can be loaded from the second memory after the first data frame is completely stored in the local second memory, and then subsequent rendering is performed.

[0118] Through the above sub-steps 2051 to 2054, the first data frame can be loaded in time when the first page needs to be displayed, and the integrity of the historical task execution can be taken into account to a certain extent, avoiding frequent suspension of historical tasks and reducing the frequency of rebuilding tasks in the loading queue, thereby helping to improve the overall loading efficiency.

[0119] Optionally, in order to save transmission bandwidth and reduce consumed network resources, the storage server may re-encode the point cloud data recorded at the first data address and send it to the local end. For details, see the following steps A1 to A2:

[0120] Step A1: Send a point cloud data acquisition instruction containing the first data address to the storage server, so that the storage server responds to the point cloud data acquisition instruction, determines the relative distance and relative direction between the point cloud coordinates in the original point cloud data stored at the first data address and the vehicle, and generates the first point cloud data based on the relative distance and the relative direction.

[0121] In an embodiment of the present invention, the first data frame may include first point cloud data. The first point cloud data may be obtained by re-encoding the original point cloud data by the storage server. Specifically, the local end may send a point cloud data acquisition instruction including the first data address to the storage server, so that the storage server responds to the point cloud data acquisition instruction, determines the relative distance and relative direction between the point cloud coordinates in the original point cloud data stored at the first data address and the vehicle, and generates the first point cloud data based on the relative distance and relative direction. Among them, each point in the original point cloud data is represented as a three-dimensional coordinate, and three parameters are required to represent a point cloud point, while each point in the re-encoded first point cloud data is represented by two parameters: the distance and relative direction of the point cloud point compared to the vehicle, which can significantly save the required amount of data, thereby improving the efficiency of transmitting the first data frame.

[0122] Step A2: Acquire a first data frame containing the first point cloud data from the storage server.

[0123] Through steps A1 to A2, the storage server can process the original point cloud data to generate a first data frame with a smaller data volume, and then transmit the first data frame to the local end, which helps to improve the efficiency of transmitting the first data frame and reduce network consumption.

[0124] Step 206: When the storage location information corresponding to the first data frame is the local storage, load the first data frame from the second memory.

[0125] This step can refer to the above-mentioned step 104, and will not be described in detail in this embodiment of the present invention.

[0126] Step 207: display a rendered image generated based on the first data frame.

[0127] This step can be referred to as step 105, and will not be described in detail in this embodiment of the present invention.

[0128] Optionally, the above-mentioned rendering image to be displayed can be generated in the manner shown in the following steps B1 to B3.

[0129] Step B1, obtaining data type information contained in the first data frame, and determining a decompression algorithm based on the data type information.

[0130] In the embodiment of the present invention, since the first data frame may contain multiple types of information, such as timestamp, image data, point cloud data, radar data, ultrasonic data, GPS data, etc., and the number of first data frames may be large, for example, if the user needs to view the autonomous driving data of a vehicle within 10 minutes through the first page, when the frame rate is 30 frames, the corresponding first data frames are 18,000. Therefore, in order to improve the acquisition efficiency and save the transmission bandwidth, the storage server can compress the original data to obtain the first data frame, so that the first data frame obtained by the local end needs to be decompressed before performing subsequent operations.

[0131] Specifically, the data type information contained in the first data frame can be obtained, and compressed data corresponding to multiple data types indicated by the data type information can be extracted from the first data frame based on the data type information. The data type information contained in the first data frame may include image data, point cloud data, and may also include one or more vehicle-side autonomous driving data such as radar data, ultrasonic data, and GPS data.

[0132] In an embodiment of the present invention, the data types in the data frame are different, and the storage server can use different compression methods for compression. For example, the server can select a suitable compression algorithm for independent compression of the original data of each data type, and then collect all the compressed data into a data packet to form the above-mentioned first data frame. Alternatively, the server can treat the original data corresponding to multiple data types as a whole, and select a suitable compression algorithm to compress all data at the same time. It should be noted that for the overall compression method, due to the combined compression effect of the same compression algorithm on different data types, it is possible to determine the compression algorithm with better effect corresponding to each combination of data types through testing, and then set the corresponding relationship between the combination of data types and the compression algorithm. For the original data of the combination of different data types, different types of compression algorithms are used for compression to obtain the first data frame. The above-mentioned compression algorithms may include general compression algorithms, such as lossless data compression algorithms (DEFLATE), LZMA, etc., and may also include self-developed compression algorithms, which are not limited in the embodiments of the present invention.

[0133] Therefore, after obtaining the data type information included in the first data frame, the local end can select a corresponding decompression algorithm according to the correspondence between the above data type combination and the compression algorithm.

[0134] Step B2: decompress the first data frame using the decompression algorithm to obtain binary streams corresponding to the data types.

[0135] In an embodiment of the present invention, the first data frame can be decompressed by the above-determined decompression algorithm to obtain binary streams corresponding to multiple data types. Among them, a binary stream is a data representation format, which is a continuous sequence of binary bits (0 or 1) arranged in a specific order. In an embodiment of the present invention, the storage server can first convert the original data of multiple data types into corresponding binary streams, and then compress the binary streams to obtain a first data frame with a smaller volume, thereby improving the transmission efficiency of the first data frame. Therefore, after receiving the first data frame, the local end can first decompress it to obtain the binary streams corresponding to each data type.

[0136] It should be noted that the first point cloud data involved in the above step A1 can also be further converted into a corresponding binary stream by the storage server and data compressed, so as to further improve the efficiency of the storage server in transmitting the point cloud data to the local end.

[0137] Step B3, parsing the binary stream to obtain the to-be-rendered data corresponding to the data type, and generating the rendered image based on the plurality of to-be-rendered data.

[0138] In an embodiment of the present invention, each binary stream can be parsed based on the data type corresponding to the binary stream to obtain the corresponding original data. For example, for a binary stream obtained by encoding a common image, an original image corresponding to the image data type can be used. The data to be rendered refers to the original data obtained after parsing the binary stream, and the original data refers to the data used by the storage server when converting the binary stream. For example, the storage server can convert the first point cloud data involved in the above step A1 into a binary stream, and the data to be rendered obtained by the local end after parsing the binary stream is the first point cloud data.

[0139] Through the above steps B1 to B3, the storage service end can generate a smaller first data frame through binary stream conversion and data compression, thereby increasing the speed at which the local end obtains the second data frame, reducing the bandwidth requirement between the local end and the server, and reducing the deployment cost of the entire system.

[0140] Optionally, if the user switches to the second page when loading the first data frame corresponding to the first page, the following steps C1 to C4 may be used to switch the loaded data frame.

[0141] Step C1, in response to the second page opening operation, obtaining a second data address corresponding to each second data frame in the second data frame set to be loaded.

[0142] In an embodiment of the present invention, the user may also frequently switch the pages that need to be viewed, that is, switch the vehicles, time periods, etc. that need to be viewed. After the user switches the pages, the required data frames need to be reloaded to ensure that the switched pages are displayed normally. Therefore, in response to the second page opening operation, the second data addresses corresponding to each second data frame in the second data frame set to be loaded can be obtained.

[0143] Step C2, storing the target frame position information of the first data frame currently being loaded in the first data frame set.

[0144] In an embodiment of the present invention, in order to respond to the user's operation of opening the second page as quickly as possible, the second data frame needs to be loaded as quickly as possible. Therefore, the target frame position information of the first data frame currently being loaded in the first data frame set can be stored. Through the target frame position information, the first data frame being loaded can be located from the first data frame set.

[0145] Step C3: loading the second data frame based on the second data address.

[0146] After recording the target frame position information, the second data frame can be loaded based on the second data address. The method of loading the second data frame can be similar to the method of loading the first data frame, and the details can be referred to above steps 202 to 206, which will not be described in detail in the embodiment of the present invention.

[0147] Step C4: when all the second data frames in the second data frame set are loaded completely, continue to load the first data frames in the first data frame set that have not been loaded completely starting from the target frame position information.

[0148] In an embodiment of the present invention, after all the second data frames in the second data frame set are loaded, the first data frames that have not yet been loaded in the first data frame set can continue to be loaded according to the target frame position information recorded above, thereby saving time in loading the first data frame when the user switches back to the first page.

[0149] Through the above steps C1 to C4, the second data frame corresponding to the second page can be responded to and loaded in time after the user switches to the second page, so that the rendering image required for the second page can be quickly generated according to the second data frame, thereby improving the speed and fluency of the user switching pages.

[0150] In summary, an embodiment of the present invention provides another data frame processing method, including: in response to a first page opening operation, obtaining a first data address corresponding to each first data frame in a first data frame set to be loaded; determining storage location information corresponding to a first data frame in a first data frame set based on the first data address; the storage location information includes local storage and remote storage; when the storage location information corresponding to the first data frame is remote storage, based on the first data address, loading the first data frame from a first memory located in a storage server, and storing the first data frame in a second memory located locally; when the storage location information corresponding to the first data frame is local storage, loading the first data frame from the second memory; and displaying a rendered image generated based on the first data frame. The method can load the first data frame in different ways according to the storage location of the first data frame, load the first data frame from the server and cache it at the same time when the first data frame is not cached locally, so that the first data frame is subsequently directly loaded from the local, thereby reducing the network bandwidth and server resource consumption required for users to retrieve data frames, and helping to improve the loading speed of the first data frame.

[0151] Based on the above embodiment, the embodiment of the present invention further provides a data frame processing device.

[0152] refer to Figure 4 , Figure 4 A structural block diagram of a data frame processing device provided by an embodiment of the present invention is shown:

[0153] A first response module 401 is used to obtain, in response to a first page opening operation, a first data address corresponding to each first data frame in a first data frame set to be loaded;

[0154] A storage location module 402, configured to determine storage location information corresponding to a first data frame in the first data frame set based on the first data address; the storage location information includes local storage and remote storage;

[0155] A storage module 403 is configured to load the first data frame from a first memory located in a storage server based on the first data address and store the first data frame in a second memory located locally when the storage location information corresponding to the first data frame is the remote storage;

[0156] A first data frame module 404, configured to load the first data frame from the second memory when the storage location information corresponding to the first data frame is the local storage;

[0157] The display module 405 is used to display the rendered image generated based on the first data frame.

[0158] Optionally, the storage module includes:

[0159] An address quantity submodule, used for obtaining the address quantity of data addresses to be loaded in the loading queue when the first data frame is the first frame in the first data frame set;

[0160] A to-be-loaded data address submodule, configured to cache the to-be-loaded data addresses when the number of addresses is greater than or equal to a preset threshold;

[0161] A loading queue submodule, configured to add a first data address corresponding to a first data frame of the first data frame set to the loading queue;

[0162] The first data frame loading submodule is used to load the first data frame corresponding to the first data address from the first memory located in the storage server based on the loading queue.

[0163] Optionally, the device further comprises:

[0164] A second corresponding module, configured to obtain, in response to a second page opening operation, a second data address corresponding to each second data frame in a second data frame set to be loaded;

[0165] A target frame position information module, used to store the target frame position information of the first data frame currently being loaded in the first data frame set;

[0166] A second data frame module, configured to load the second data frame based on the second data address;

[0167] The continuing loading module is used to continue loading the first data frames that have not been loaded in the first data frame set starting from the target frame position information when all the second data frames in the second data frame set have been loaded.

[0168] Optionally, the storage location module includes:

[0169] A terminal information reading submodule, used for obtaining the reading terminal information of the first data frame from the first data address;

[0170] A first storage location information determination submodule, configured to determine, when the reading terminal information matches the local end information, that the storage location information corresponding to the first data address is the local storage;

[0171] The second storage location information determination submodule is used to determine that the storage location information corresponding to the first data address is the remote storage when the reading terminal information does not match the local end information.

[0172] Optionally, the first data frame includes first point cloud data, and the storage module includes:

[0173] an instruction sending submodule, configured to send a point cloud data acquisition instruction including the first data address to the storage server, so that the storage server responds to the point cloud data acquisition instruction, determines the relative distance and relative direction between the point cloud coordinates in the original point cloud data stored at the first data address and the vehicle, and generates the first point cloud data based on the relative distance and the relative direction;

[0174] The point cloud data acquisition submodule is used to acquire a first data frame containing the first point cloud data from the storage server.

[0175] Optionally, the device further comprises:

[0176] A decompression algorithm acquisition module, used to obtain data type information contained in the first data frame, and determine a decompression algorithm based on the data type information;

[0177] A binary stream module, used for decompressing the first data frame by using the decompression algorithm to obtain binary streams corresponding to the data types respectively;

[0178] The rendering image generation module is used to parse the binary stream to obtain the to-be-rendered data corresponding to the data type, and generate the rendering image based on the multiple to-be-rendered data.

[0179] In summary, an embodiment of the present invention provides a data frame processing device, including: a first response module, for responding to a first page opening operation, obtaining a first data address corresponding to each first data frame in a first data frame set to be loaded; a storage location module, for determining storage location information corresponding to a first data frame in a first data frame set based on the first data address; the storage location information includes local storage and remote storage; a storage module, for loading the first data frame from a first memory located in a storage server based on the first data address when the storage location information corresponding to the first data frame is remote storage, and storing the first data frame in a second memory located locally; a first data frame module, for loading the first data frame from a second memory when the storage location information corresponding to the first data frame is local storage; and a display module, for displaying a rendered image generated based on the first data frame. The first data frame can be loaded in different ways according to the storage location of the first data frame, and the first data frame can be loaded from the server and cached at the same time when the first data frame is not cached locally, so that the first data frame is subsequently directly loaded from the local, thereby reducing the network bandwidth and server resource consumption required for the user to retrieve the data frame, and helping to improve the loading speed of the first data frame.

[0180] An embodiment of the present invention further provides a readable storage medium. When instructions in the readable storage medium are executed by a processor of an electronic device, the electronic device can execute the above-mentioned data frame processing method.

[0181] An embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores a program or instructions running on the processor, and the program or instructions implement the above-mentioned data frame processing method when executed by the processor.

[0182] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned device embodiments and will not be repeated here.

[0183] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0184] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A data frame processing method, characterized in that: The method comprises: In response to the first page opening operation, obtaining first data addresses corresponding to each first data frame in the first data frame set to be loaded; Determine storage location information corresponding to a first data frame in the first data frame set based on the first data address; the storage location information includes local storage and remote storage; In a case where the storage location information corresponding to the first data frame is the remote storage, based on the first data address, the first data frame is loaded from a first memory located in a storage server, and the first data frame is stored in a second memory located locally; When the storage location information corresponding to the first data frame is the local storage, loading the first data frame from the second memory; A rendered image generated based on the first data frame is displayed.

2. The method according to claim 1, characterized in that The step of loading the first data frame from a first memory located at a storage server based on the first data address includes: When the first data frame is the first frame in the first data frame set, obtaining the number of addresses of data to be loaded in the loading queue; When the number of addresses is greater than or equal to a preset threshold, cache the addresses of the data to be loaded; Adding a first data address corresponding to a first data frame of the first data frame set to the loading queue; A first data frame corresponding to the first data address is loaded from a first memory located in a storage server based on the load queue.

3. The method according to claim 1, characterized in that The method further comprises: In response to the second page opening operation, obtaining a second data address corresponding to each second data frame in the second data frame set to be loaded; storing target frame position information of the first data frame currently being loaded in the first data frame set; loading the second data frame based on the second data address; When all the second data frames in the second data frame set are loaded completely, the first data frames in the first data frame set that have not been loaded completely are continuously loaded starting from the target frame position information.

4. The method according to claim 1, characterized in that: The determining, based on the first data address, storage location information corresponding to the first data frame in the first data frame set includes: Acquire the reading terminal information of the first data frame from the first data address; In a case where the reading terminal information matches the local end information, determining that the storage location information corresponding to the first data address is the local storage; In a case where the reading terminal information does not match the local end information, it is determined that the storage location information corresponding to the first data address is the remote storage.

5. The method according to claim 1, characterized in that The first data frame includes first point cloud data, and the step of loading the first data frame from a first memory located on a storage server based on the first data address includes: Sending a point cloud data acquisition instruction including the first data address to the storage server, so that the storage server responds to the point cloud data acquisition instruction, determines the relative distance and relative direction between the point cloud coordinates in the original point cloud data stored at the first data address and the vehicle, and generates the first point cloud data based on the relative distance and the relative direction; A first data frame including the first point cloud data is obtained from the storage server.

6. The method according to claim 1, characterized in that The method further comprises: Acquire data type information contained in the first data frame, and determine a decompression algorithm based on the data type information; Decompressing the first data frame by using the decompression algorithm to obtain binary streams corresponding to the data types respectively; The binary stream is parsed to obtain data to be rendered corresponding to the data type, and the rendered image is generated based on a plurality of the data to be rendered.

7. A data frame processing device, characterized in that: The device comprises: A first response module, configured to obtain, in response to a first page opening operation, a first data address corresponding to each first data frame in a first data frame set to be loaded; A storage location module, configured to determine storage location information corresponding to a first data frame in the first data frame set based on the first data address; the storage location information includes local storage and remote storage; a storage module, configured to load the first data frame from a first memory located in a storage server based on the first data address and store the first data frame in a second memory located locally when the storage location information corresponding to the first data frame is the remote storage; A first data frame module, configured to load the first data frame from the second memory when the storage location information corresponding to the first data frame is the local storage; A display module is used to display a rendered image generated based on the first data frame.

8. The device according to claim 7, characterized in that The storage module comprises: An address quantity submodule, used for obtaining the address quantity of data addresses to be loaded in the loading queue when the first data frame is the first frame in the first data frame set; A to-be-loaded data address submodule, configured to cache the to-be-loaded data addresses when the number of addresses is greater than or equal to a preset threshold; A loading queue submodule, configured to add a first data address corresponding to a first data frame of the first data frame set to the loading queue; The first data frame loading submodule is used to load the first data frame corresponding to the first data address from the first memory located in the storage server based on the loading queue.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores a program or an instruction running on the processor, and when the program or the instruction is executed by the processor, the data frame processing method according to any one of claims 1 to 6 is implemented.

10. A readable storage medium, characterized in that: When the instructions in the readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the data frame processing method as described in any one of claims 1 to 6.

Images