A method, apparatus, device and medium for data processing

By creating bounding boxes and obtaining index values ​​in the cloud, the client loads point cloud data on demand, solving the high cost and low efficiency problems caused by the client directly storing and retrieving all data, and achieving efficient data processing and storage.

CN117725240BActive Publication Date: 2026-11-17IMOTION AUTOMOTIVE TECH (SUZHOU) CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311774855.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-21
Publication Date
2026-11-17
Estimated Expiration
2043-12-21

AI Technical Summary

Technical Problem

The existing client-side approach of directly storing and retrieving all point cloud data results in high data storage costs, low data loading efficiency, and low validity of the loaded data.

Method used

Multiple independent bounding boxes are created in the cloud, and the bounding box index values ​​are obtained. The client determines the target bounding box by obtaining the intersection range between the current field of view and the bounding box based on the bounding box index value, and loads the point cloud data in the target bounding box.

Benefits of technology

It reduces the amount of data stored on the client side, lowers data storage costs, improves data loading efficiency and effectiveness, and makes full use of the data processing capabilities of the cloud.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117725240B_ABST
    Figure CN117725240B_ABST
Patent Text Reader

Abstract

This invention discloses a data processing method, apparatus, device, and medium, relating to the field of data processing technology. In this method, multiple independent bounding boxes are created in the cloud based on point cloud data, and the index values ​​of each bounding box are obtained. The created bounding box index values ​​are sent to the client, so that the client stores only the bounding box index values, thus reducing the amount of data stored on the client and lowering the cost of data storage. To read point cloud data, the client first obtains the bounding boxes corresponding to the bounding box index values, and then takes the intersection of the bounding boxes with the current field of view. This allows loading only the point cloud data corresponding to the bounding boxes within the field of view, achieving on-demand loading of point cloud data without needing to load all point cloud data, improving data loading efficiency and reducing the requirements on computer performance. Simultaneously, the data processing capabilities of the cloud are utilized, further improving data processing efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data processing method, apparatus, device, and medium. Background Technology

[0002] In autonomous driving, 3D LiDAR is being used more and more widely. LiDAR generates 3D point cloud data to represent 3D objects or scenes. For example, 3D point cloud data generated by LiDAR can represent a vehicle in front. Because the amount of data generated by 3D LiDAR is very large, especially in scenarios with continuous frames (such as when information about the front of the vehicle is needed while it is moving), data storage and retrieval have become a major technical challenge.

[0003] In the processing of data generated by the relevant LiDAR, the client directly stores all acquired point cloud information and reads all the data. When storing point cloud data, the client directly stores all acquired point cloud data, resulting in high data storage costs. When reading point cloud data, because all data is read, data loading time is long, high computer performance requirements are placed, and the read data is not necessarily all the necessary data, meaning there are invalid reads.

[0004] Therefore, how to store and process point cloud data to reduce the cost of data storage for clients, improve the efficiency of data loading, and ensure the effectiveness of the loaded data are technical problems that urgently need to be solved by those in the field. Summary of the Invention

[0005] The purpose of this invention is to provide a data processing method, apparatus, device, and medium to solve the technical problems of high client data storage costs, low data loading efficiency, and low validity of loaded data caused by the existing method of client directly storing and reading all point cloud data.

[0006] To address the aforementioned technical problems, this invention provides a data processing method applied in the cloud, the method comprising:

[0007] The system acquires point cloud data sent by the client and creates multiple independent bounding boxes based on the point cloud data; wherein each bounding box contains multiple sets of the point cloud data.

[0008] Obtain and store the bounding box index values ​​of each of the above;

[0009] Upon receiving an instruction to read the point cloud data, the stored bounding box index value is sent to the client so that the client can obtain the current field of view and each bounding box corresponding to the bounding box index value, and determine the target bounding box based on the intersection range of the current field of view and each bounding box, and load the point cloud data in the target bounding box.

[0010] Preferably, the step of acquiring point cloud data sent by the client and creating multiple independent bounding boxes based on the point cloud data includes:

[0011] Obtain the point cloud data in its original format sent by the client;

[0012] The point cloud data in each original format is converted into point cloud data in a three-dimensional format;

[0013] Based on the point cloud data in all three-dimensional formats, obtain the minimum and maximum values ​​of the three axes;

[0014] An initial bounding box is established based on the points formed by the minimum values ​​of the three axes and the points formed by the maximum values ​​of the three axes; wherein, the initial bounding box is an AABB bounding box;

[0015] The initial bounding box is split into multiple independent bounding boxes.

[0016] Preferably, splitting the initial bounding box into multiple independent bounding boxes includes:

[0017] The initial bounding box is divided into a predetermined number of bounding boxes;

[0018] Determine whether the bounding boxes obtained from the splitting meet the preset requirements;

[0019] If not, then the bounding box that does not meet the preset requirements is obtained, and the bounding box that does not meet the preset requirements is split into the preset number of bounding boxes, until the bounding box obtained after splitting the bounding box that does not meet the preset requirements meets the preset requirements, then it is determined that the initial bounding box has been split.

[0020] Preferably, the preset requirement includes at least one of the following: the number of point cloud data in the bounding box is less than a preset value, and the depth value of the bounding box is greater than or equal to a preset depth value; wherein, the depth value of the bounding box is determined by the number of parent bounding boxes of all split layers in the splitting path from the initial bounding box to the point where the bounding box is obtained.

[0021] Preferably, the preset quantity is 8.

[0022] Preferably, the bounding box index value includes the depth value of the bounding box and the position information of the bounding box;

[0023] Obtaining the bounding box index value includes:

[0024] The number of parent bounding boxes of all split layers in the split path from the initial bounding box to the obtained bounding box;

[0025] The depth value of the bounding box is determined based on the number of parent bounding boxes;

[0026] From the initial bounding box to the splitting path where the bounding box is obtained, obtain a three-dimensional binary number representing the position information of the bounding box for all splitting layers; wherein, the value of each bit of the three-dimensional binary number is set according to a preset rule;

[0027] Arrange each of the three-dimensional binary numbers according to the order of the splitting layers in the splitting path to obtain the position information of the bounding box;

[0028] The depth value of the bounding box and the position information are combined to obtain the bounding box index value.

[0029] Preferably, obtaining the current field of view includes:

[0030] The viewing frustum is determined based on the camera's parameters; wherein the camera parameters include the camera's position, lens shooting orientation, up vector, field of view, aspect ratio, near clipping distance, and far clipping distance;

[0031] The visual cone is used as the current field of vision;

[0032] Obtaining the bounding box corresponding to the bounding box index value includes:

[0033] Decompose the bounding box index value to obtain the three-dimensional binary number corresponding to the bounding box of each splitting layer in the splitting path;

[0034] The initial bounding box is updated to obtain the bounding box corresponding to the bounding box index value; wherein, the initial bounding box is updated by updating the initial bounding box according to the three-dimensional binary number corresponding to the bounding box of each split layer in the splitting path;

[0035] Correspondingly, determining the target bounding box based on the intersection range of the current field of view and each of the bounding boxes includes:

[0036] The frustum and each bounding box are respectively used to determine whether the bounding box is within the frustum;

[0037] If so, then the bounding box is determined to be the target bounding box.

[0038] Preferably, updating the initial bounding box based on the three-dimensional binary number corresponding to the bounding box of each splitting layer in the splitting path includes:

[0039] Starting from the first bit of the three-dimensional binary number corresponding to the bounding box of the first split layer in all split layers, and ending at the last bit of the three-dimensional binary number corresponding to the bounding box of the last split layer in all split layers, the previous bounding box is updated sequentially according to the value of each bit and the preset rules to obtain a new bounding box; wherein, except when the first bit of the three-dimensional binary number corresponding to the bounding box of the first split layer is the first bounding box, the previous bounding box is the new bounding box determined according to the value of the previous bit of the current bit and the preset rules when the new bounding box is determined according to the value of the current bit and the preset rules.

[0040] Obtain the final new bounding box, and use the final new bounding box as the bounding box corresponding to the bounding box index value.

[0041] To address the aforementioned technical problems, the present invention also provides a data processing method applied to a client, the method comprising:

[0042] Point cloud data is sent to the cloud so that after receiving the point cloud data sent by the client, the cloud can create multiple independent bounding boxes based on the point cloud data, and obtain and store the index value of each bounding box; wherein each bounding box contains multiple points cloud data.

[0043] Send a command to read point cloud data to the cloud and obtain the bounding box index value sent by the cloud;

[0044] Obtain the current field of view and each bounding box corresponding to the bounding box index value;

[0045] The target bounding box is determined based on the intersection of the current field of view and each of the bounding boxes, and the point cloud data in the target bounding box is loaded.

[0046] To address the aforementioned technical problems, the present invention also provides a data processing apparatus applied in the cloud, the apparatus comprising:

[0047] The first acquisition module is used to acquire point cloud data sent by the client and create multiple independent bounding boxes based on the point cloud data; wherein each bounding box contains multiple points cloud data.

[0048] The second acquisition module is used to acquire and store the bounding box index values ​​of each of the above;

[0049] The sending module is configured to send the stored bounding box index value to the client upon receiving an instruction to read the point cloud data, so that the client can obtain the current field of view and each bounding box corresponding to the bounding box index value, determine the target bounding box based on the intersection range of the current field of view and each bounding box, and load the point cloud data in the target bounding box.

[0050] To address the aforementioned technical problems, the present invention also provides a data processing apparatus, comprising:

[0051] Memory, used to store computer programs;

[0052] A processor is used to implement the steps of the above-described data processing method when executing the computer program.

[0053] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the aforementioned data processing method.

[0054] The data processing method provided by this invention is applied in the cloud. The method includes: acquiring point cloud data sent by a client and creating multiple independent bounding boxes based on the point cloud data; acquiring and storing the index values ​​of each bounding box; upon receiving an instruction to read point cloud data, sending the stored bounding box index values ​​to the client so that the client can obtain the current field of view and the bounding boxes corresponding to the bounding box index values, determine the target bounding box based on the intersection of the current field of view and the bounding boxes, and load the point cloud data in the target bounding box. As can be seen, in this method, the cloud processes the point cloud data sent by the client, specifically by creating multiple independent bounding boxes and acquiring the index values ​​of each bounding box, wherein each bounding box contains multiple points cloud data. When a command to read point cloud data is received, the stored bounding box index values ​​are sent to the client. Compared to the method where the client directly stores point cloud data, the method provided in this invention processes the point cloud data in the cloud and sends the created bounding box index values ​​to the client, so that the client stores only the bounding box index values. Therefore, the amount of data stored on the client is reduced, and the cost of data storage is lowered. In order to read point cloud data, the client first obtains each bounding box corresponding to the bounding box index value, and then takes the intersection of the bounding box with the current field of view. Thus, only the point cloud data corresponding to the bounding box corresponding to the field of view is loaded, which realizes on-demand loading of point cloud data. It does not need to load all point cloud data, which improves the efficiency of data loading and reduces the requirements for computer performance. At the same time, since the point cloud data sent by the client is processed in the cloud, the data processing capabilities of the cloud are fully utilized, which improves the efficiency of data processing.

[0055] In addition, the present invention also provides a method for data processing applied to a client, a data processing apparatus, a data processing device, and a computer-readable storage medium, which have the same or corresponding technical features as the data processing method mentioned above, and have the same effects. Attached Figure Description

[0056] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart of a data processing method provided in an embodiment of the present invention;

[0058] Figure 2 A flowchart illustrating a data storage and loading method provided in an embodiment of the present invention;

[0059] Figure 3A structural diagram of a data processing apparatus provided according to an embodiment of the present invention;

[0060] Figure 4 A structural diagram of a data processing device provided in another embodiment of the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0062] The core of this invention is to provide a data processing method, apparatus, device, and medium to solve the technical problems of high client data storage costs, low data loading efficiency, and low validity of loaded data caused by the existing client-side direct storage and reading of all point cloud data.

[0063] To enable those skilled in the art to better understand the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment provides a data processing method applied to the cloud. Figure 1 A flowchart of a data processing method provided in an embodiment of the present invention is shown below. Figure 1 As shown, the method includes:

[0064] S10: Obtain point cloud data sent by the client and create multiple independent bounding boxes based on the point cloud data; each bounding box contains multiple point cloud data.

[0065] S11: Obtain and store the index values ​​of each bounding box;

[0066] S12: Upon receiving an instruction to read point cloud data, the stored bounding box index values ​​are sent to the client so that the client can obtain the current field of view and each bounding box corresponding to the bounding box index value, and determine the target bounding box based on the intersection of the current field of view and each bounding box, and load the point cloud data in the target bounding box.

[0067] The specific location of the LiDAR installation is not limited; the LiDAR is installed on the vehicle to collect point cloud data around the vehicle. It should be noted that after obtaining the point cloud data, to improve data processing efficiency and capabilities, the point cloud data sent by the client can be sent to the cloud for processing. Previously, all acquired point cloud data was directly stored and read, leading to increased data storage costs, low data loading efficiency, and reduced data loading effectiveness. Therefore, in this embodiment, after obtaining the point cloud data sent by the client, multiple independent bounding boxes (also called nodes) are created based on the point cloud data, and the index value of each bounding box is obtained. Here, an independent bounding box refers to a bounding box with no intersecting regions. The method of creating bounding boxes, the number of bounding boxes created, and the method of setting the index values ​​of each bounding box are not limited and are determined according to the actual situation. After obtaining the index values ​​of each bounding box, the index values ​​of each bounding box are stored. For example, based on 100 point cloud data generated by LiDAR, four bounding boxes are created: bounding box A, bounding box B, bounding box C, and bounding box D. Each bounding box contains multiple point cloud data. The corresponding index values ​​of the four bounding boxes are m, n, o, and p, respectively. When storing, only the index values ​​corresponding to the four bounding boxes need to be stored, instead of storing all the point cloud data, which greatly reduces the amount of data storage and lowers the data storage cost.

[0068] Upon receiving an instruction to read point cloud data, in order to improve the efficiency and validity of data loading, this embodiment obtains the current field of view and uses this field of view to limit the range of data to be loaded. Obtaining the current field of view includes:

[0069] The view frustum is determined based on the camera's parameters, which include the camera's position, lens orientation, up vector, field of view, aspect ratio, near clipping distance, and far clipping distance.

[0070] Use the visual cone as the current field of vision.

[0071] The basic parameters of the camera are as follows:

[0072] ①Position C=(C_x, C_y, C_z);

[0073] ② Oriented towards D;

[0074] ③ The upper vector UP, which is generally (0, 0, 1), represents that the Z-axis is above the camera;

[0075] ④ Field of View (FOV);

[0076] ⑤ Aspect Ratio;

[0077] ⑥ Nearest clipping distance n;

[0078] ⑦ Far clipping distance f.

[0079] A frustum is a pyramid shape defined by the camera's field of view, aspect ratio, near clipping plane, and far clipping plane. It typically consists of six planes: left, right, top, bottom, near, and far.

[0080] The client obtains the current field of view and the bounding boxes corresponding to each bounding box index value. Based on the intersection range between the current field of view and each bounding box, it determines the target bounding box and then loads the point cloud data from the target bounding box. For example, based on the stored index values ​​m, n, o, and p, the corresponding bounding boxes are found to be bounding box A, bounding box B, bounding box C, and bounding box D, respectively. If the current field of view is in Q-space, the client determines that bounding box A is completely within Q-space based on the intersection range between Q-space and bounding boxes A, B, and C, thus identifying bounding box A as the target bounding box, and loading the point cloud data from bounding box A. It should be noted that if the client obtains all bounding box index values ​​sent from the cloud, the bounding boxes determined based on these index values ​​correspond to all bounding boxes; if the client obtains only some index values ​​sent from the cloud, the bounding boxes determined based on these index values ​​correspond to only those specific index values.

[0081] In addition, to reduce redundant loading of point cloud data, after determining the target bounding box, the index value of the target bounding box of the point cloud data to be loaded can be compared with the index value of the bounding box of the point cloud data already loaded. If the index values ​​are the same, the point cloud data corresponding to the target bounding box is not loaded; if the index values ​​are different, the point cloud data corresponding to the target bounding box can be loaded. This loading method reduces the redundant loading of point cloud data.

[0082] The data processing method provided in this invention is applied to the cloud. The method includes: acquiring point cloud data sent by a client and creating multiple independent bounding boxes based on the point cloud data; acquiring and storing the index values ​​of each bounding box; upon receiving an instruction to read point cloud data, sending the stored bounding box index values ​​to the client so that the client can acquire the current field of view and the bounding boxes corresponding to the bounding box index values, determine the target bounding box based on the intersection of the current field of view and the bounding boxes, and load the point cloud data in the target bounding box. As can be seen, in this method, the cloud processes the point cloud data sent by the client, specifically by creating multiple independent bounding boxes and acquiring the index values ​​of each bounding box, wherein each bounding box contains multiple points cloud data. When a command to read point cloud data is received, the stored bounding box index values ​​are sent to the client. Compared to the method where the client directly stores point cloud data, the method provided in this invention processes the point cloud data in the cloud and sends the created bounding box index values ​​to the client, so that the client stores only the bounding box index values. Therefore, the amount of data stored on the client is reduced, and the cost of data storage is lowered. In order to read point cloud data, the client first obtains each bounding box corresponding to the bounding box index value, and then takes the intersection of the bounding box with the current field of view. Thus, only the point cloud data corresponding to the bounding box corresponding to the field of view is loaded, which realizes on-demand loading of point cloud data. It does not need to load all point cloud data, which improves the efficiency of data loading and reduces the requirements for computer performance. At the same time, since the point cloud data sent by the client is processed in the cloud, the data processing capabilities of the cloud are fully utilized, which improves the efficiency of data processing.

[0083] In order to create bounding boxes based on point cloud data, in some embodiments, acquiring point cloud data sent by the client and creating multiple independent bounding boxes based on the point cloud data includes:

[0084] Obtain the point cloud data in its raw format sent by the client;

[0085] Convert point cloud data in various original formats into point cloud data in three-dimensional format;

[0086] Obtain the minimum and maximum values ​​of the three axes from all point cloud data in three-dimensional format;

[0087] An initial bounding box is established based on the points formed by the minimum values ​​of the three axes and the points formed by the maximum values ​​of the three axes; wherein, the initial bounding box is an AABB bounding box;

[0088] The initial bounding box is split into multiple independent bounding boxes.

[0089] After receiving the raw point cloud data sent by the client, the point cloud data can be preprocessed. During preprocessing, statistical outlier analysis can be used to remove noise points and filter the point cloud data. Then, the raw point cloud data is converted into a three-dimensional point cloud data format. The three-dimensional format is not limited; however, in this embodiment of the invention, the raw point cloud data is uniformly converted into Point Cloud Data (PCD) format. The PCD storage format stores a series of spatial positions of points along the X, Y, and Z axes.

[0090] After obtaining the 3D point cloud data, spatial delineation is performed based on the 3D point cloud data, i.e., a bounding box is created. In this embodiment of the invention, the bounding box created is an AABB (Axis-Aligned Bounding Box). An AABB bounding box is a simple cube or cuboid whose sides are parallel to the coordinate axes. It is defined by two main points: the minimum point (usually the lower left rear corner) and the maximum point (usually the upper right front corner).

[0091] When creating the bounding box, iterate through all points and find the minimum (xmin, ymin, zmin) and maximum (xmax, ymax, zmax) values ​​of the XYZ coordinates. Use the found minimum and maximum values ​​to create an initial bounding box (AABB bounding box), and then split the initial bounding box into multiple independent bounding boxes.

[0092] In order to split the initial bounding box into multiple independent bounding boxes, in some embodiments, splitting the initial bounding box into multiple independent bounding boxes includes:

[0093] The initial bounding box is split into a preset number of bounding boxes;

[0094] Determine whether the bounding boxes obtained from the splitting process meet the preset requirements;

[0095] If not, then obtain the bounding boxes that do not meet the preset requirements, and split the bounding boxes that do not meet the preset requirements into a preset number of bounding boxes, until the bounding boxes obtained after splitting the bounding boxes that do not meet the preset requirements meet the preset requirements, then determine that the initial bounding box has been split.

[0096] The preset requirements include at least one of the following: the number of point cloud data in the bounding box is less than a preset value, or the depth value of the bounding box is greater than or equal to a preset depth value; wherein, the depth value of the bounding box is determined by the number of parent bounding boxes of all split layers in the splitting path from the initial bounding box to the obtained bounding box.

[0097] There is no limit to the preset number, but to ensure the same splitting rate across the three axes, a preset number of 8 is preferred. There are no limitations on the preset values ​​of point cloud data within the bounding boxes or the preset depth values ​​set for the bounding boxes. For example, the preset value could be 50 point cloud data points and the preset depth value could be 10 layers. During splitting, the initial bounding box is split into 8 bounding boxes. It is then determined whether the number of point cloud data points in each of these 8 bounding boxes is less than 50 and / or whether the depth value of each bounding box is greater than or equal to 10. If yes, the bounding box splitting is complete; otherwise, each resulting bounding box is further split into 8 bounding boxes, and the same determination is made to ensure that the number of point cloud data points in each bounding box is less than 50 and / or the depth value of each bounding box is greater than or equal to 10.

[0098] To obtain the number of point cloud data points within the bounding box, in practice, the coordinates of the points can be traversed. If a point falls within the range of the minimum and maximum values ​​of its current coordinates, then the point is within the bounding box.

[0099] This section explains how to determine the depth of the current bounding box. The depth of the bounding box is determined by the number of parent bounding boxes in all the splitting layers along the splitting path from the initial bounding box to the obtained bounding box. A parent bounding box is a bounding box that has been split. The initial bounding box has a depth of 0. If the current bounding box is obtained after the initial bounding box is split once (i.e., the splitting path is from the initial bounding box to the current bounding box, and the splitting layers along the path are the layers containing the parent bounding boxes and the current bounding box), and the parent bounding box in all the splitting layers along this path is the initial bounding box (i.e., there is one parent bounding box), then the depth of the current bounding box is 1. If the current bounding box is obtained by splitting the intermediate bounding box obtained after splitting the initial bounding box once, then the splitting path is the initial bounding box, the intermediate bounding box, and the current bounding box, and the splitting layers along the path are the layers containing the parent bounding boxes, the intermediate bounding boxes, and the current bounding box. In this case, the parent bounding boxes in all the splitting layers along the path are the initial bounding box and the intermediate bounding box (two parent bounding boxes in total), then the depth of the current bounding box is 2. The initial bounding box can be considered as the root node, with a depth of 0. The depth increases by 1 for each subsequent layer. For example, if a node is in the third layer, its depth is 2 (counting from 0).

[0100] Previously, point cloud data was stored directly, leading to increased data storage costs. Therefore, in this embodiment of the invention, bounding box index values ​​are stored. In some embodiments, the bounding box index value includes the depth value of the bounding box and the location information of the bounding box.

[0101] Obtaining the bounding box index value includes:

[0102] The number of parent bounding boxes of all split layers in the split path from the initial bounding box to the obtained bounding box;

[0103] The depth value of the bounding box is determined based on the number of parent bounding boxes;

[0104] From the initial bounding box to the splitting path where the bounding box is obtained, obtain the three-dimensional binary numbers of all splitting layers used to represent the position information of the bounding box; wherein, the value of each bit of the three-dimensional binary number is set according to a preset rule;

[0105] Arrange each three-dimensional binary number according to the order of the splitting layers in the splitting path to obtain the position information of the bounding box;

[0106] The depth value and position information of the bounding box are combined to obtain the bounding box index value.

[0107] The method for determining the depth value of the bounding box has been described in detail above and will not be repeated here. This section only describes the positional information within the index value.

[0108] Position information uses a three-bit binary number to describe the position of a child node relative to its parent node. Here, the first bit is used to indicate left (0) or right (1), the second bit is used to indicate up (1) or down (0), and the third bit is used to indicate front (1) or back (0).

[0109] According to this rule, the positions of the eight child nodes can be mapped to binary numbers as follows: Left-Upper-Front: 011; Left-Upper-Back: 010; Right-Upper-Front: 111; Right-Upper-Back: 110; Left-Lower-Front: 001; Left-Lower-Back: 000; Right-Lower-Front: 101; Right-Lower-Back: 100.

[0110] If the current node is in the third layer (depth is 2), and its position relative to its parent node is left-top-back (010), right-bottom-front (101), left-bottom-back (000), then the index value of the node is: D2_P010101000.

[0111] A string consisting of the depth value and position information of the bounding box generates a unique index value for each bounding box, so that the bounding box can be found by using the unique index value.

[0112] After obtaining each bounding box index value, i.e., the depth value and position information, the bounding box index values ​​are stored in the database. In some embodiments, when reading point cloud data within the field of view, obtaining the bounding box corresponding to the bounding box index value includes:

[0113] Decompose the bounding box index values ​​to obtain the three-dimensional binary numbers corresponding to the bounding boxes of each splitting layer in the splitting path;

[0114] The initial bounding box is updated to obtain the bounding box corresponding to the bounding box index value; the method for updating the initial bounding box is as follows: the initial bounding box is updated according to the three-dimensional binary number corresponding to the bounding box of each split layer in the splitting path;

[0115] Correspondingly, determining the target bounding box based on the intersection of the current field of view and each bounding box includes:

[0116] The algorithm for the intersection of the view frustum and each bounding box is used to determine whether the bounding box is inside the view frustum.

[0117] If so, then the bounding box is determined to be the target bounding box.

[0118] Specifically, updating the initial bounding box based on the 3D binary number corresponding to the bounding box of each splitting layer in the splitting path includes:

[0119] Starting from the first bit of the three-dimensional binary number corresponding to the bounding box of the first split layer in all split layers, and ending at the last bit of the three-dimensional binary number corresponding to the bounding box of the last split layer in all split layers, the previous bounding box is updated sequentially according to the value of each bit and the preset rules to obtain a new bounding box. Except when the first bit of the three-dimensional binary number corresponding to the bounding box of the first split layer is the first bounding box, the previous bounding box is the new bounding box determined according to the value of the previous bit of the current bit and the preset rules when a new bounding box is to be determined according to the value of the current bit and the preset rules.

[0120] Obtain the final new bounding box, and use the final new bounding box as the bounding box corresponding to the bounding box index value.

[0121] The specific process of traversing the nodes and calculating the size of the bounding box is as follows:

[0122] 1. Input parameters: Initial bounding box (defined by (x_min, y_min, z_min) and (x_max, y_max, z_max)) and node index (e.g., 'D2_P010101000').

[0123] 2. Initialize the bounding box of the node: Set the initial cur_min and cur_max as the bounding box of the root node: x_cur_min = x_min, y_cur_min = y_min, z_cur_min = z_min and x_cur_max = x_max, y_cur_max = y_max, z_cur_max = z_max.

[0124] 3. Parse the index and traverse each level: Decompose the index to obtain the position code of each level. For example, for 'D2_P010101000', focus on '010101000'.

[0125] 4. Traverse the position code of each layer: For each position code (e.g., '010'), perform the following steps:

[0126] Update bounding box bit by bit:

[0127] 1) X-axis (first position):

[0128] a. If it is 0, then update x_cur_max = (x_cur_min + x_cur_max) / 2.

[0129] b. If it is 1, then update x_cur_min = (x_cur_min + x_cur_max) / 2.

[0130] 2) Y-axis (second position):

[0131] a. If it is 0, then update y_cur_max = (y_cur_min + y_cur_max) / 2.

[0132] b. If it is 1, then update y_cur_min = (y_cur_min + y_cur_max) / 2.

[0133] 3) Z-axis (third position):

[0134] a. If it is 0, then update z_cur_max = (z_cur_min + z_cur_max) / 2.

[0135] b. If it is 1, then update z_cur_min = (z_cur_min + z_cur_max) / 2.

[0136] Repeat until all layers have been traversed: continue processing the position encoding of the next layer until the entire index has been processed.

[0137] 5. Conclusion: The final cur_min and cur_max (x_cur_min, y_cur_min, z_cur_min and x_cur_max, y_cur_max, z_cur_max) represent the bounding boxes corresponding to the index values.

[0138] The following example illustrates the process of determining the bounding box based on the index value, using the initial bounding boxes as (x_min, y_min, z_min) = (0, 0, 0) and (x_max, y_max, z_max) = (100, 100, 100).

[0139] ① First layer: '010':

[0140] aX axis: 0, x_cur_max = (0 + 100) / 2 = 50.

[0141] bY axis: 1, y_cur_min=(0+100) / 2=50.

[0142] cZ axis: 0, z_cur_max=(0+100) / 2=50.

[0143] d. New bounding boxes: (0, 50, 0) and (50, 100, 50).

[0144] ② Second layer: '101':

[0145] aX axis: 1, x_cur_min = (0 + 50) / 2 = 25.

[0146] bY axis: 0, y_cur_max=(50+100) / 2=75.

[0147] cZ axis: 1, z_cur_min=(0+50) / 2=25.

[0148] d. New bounding boxes: (25, 50, 25) and (50, 75, 50).

[0149] ③ Third layer: '000':

[0150] aX axis: 0, x_cur_max = (25 + 50) / 2 = 37.5.

[0151] bY axis: 0, y_cur_max=(50+75) / 2=62.5.

[0152] cZ axis: 0, z_cur_max=(25+50) / 2=37.5.

[0153] d. Final bounding box: (25, 50, 25) and (37.5, 62.5, 37.5).

[0154] The AABB bounding box and view frustum intersection algorithm is used to calculate whether the node is within the current view frustum. If it is, the current node is loaded; otherwise, it is ignored. In implementation, a caching mechanism can be set up to store the indexes of already loaded nodes. For each loaded node index, the index to be loaded is compared with the index to be loaded, and only the necessary index is loaded to reduce duplicate data loading.

[0155] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Figure 2 A flowchart illustrating a data storage and loading method provided in an embodiment of the present invention is shown below. Figure 2 As shown, the method includes:

[0156] S13: Data preprocessing;

[0157] S14: Create an initial bounding box based on the maximum and minimum values ​​of the data in XYZ. This bounding box is the initial node.

[0158] S15: Divide the node into 8 child nodes according to the space;

[0159] S16: Determine if the number of points in the child node is less than the threshold; if yes, proceed to step S17; if no, return to step S15.

[0160] S17: Store node information containing depth and position in the database;

[0161] S18: Read point information from the data based on the location information using a data reading medium;

[0162] S19: Lightweight data loading.

[0163] It should be noted that the axis-aligned bounding box (AABB) created in step S14 is a commonly used geometric structure in computer graphics and game development. This bounding box is a rectangular box parallel to the coordinate axes, typically used to simplify and accelerate various geometric calculations, especially collision detection and spatial queries. The entire data storage and loading process includes data preprocessing (noise filtering, data format conversion), spatial delimitation (creating the initial bounding box), data splitting (splitting the initial bounding box), data storage (storing index values ​​consisting of the depth and position information of the bounding boxes), and on-demand loading and querying of point cloud data (if the bounding box is inside the view frustum, the point cloud data within that bounding box is loaded). This achieves efficient storage and processing of large amounts of point cloud data generated by 3D LiDAR, making it particularly suitable for real-time or near-real-time applications.

[0164] The above provides a data processing method applied to the cloud. This embodiment also provides a data processing method applied to the client, which includes:

[0165] Point cloud data is sent to the cloud so that after receiving the point cloud data sent by the client, the cloud can create multiple independent bounding boxes based on the point cloud data and obtain and store the index value of each bounding box; each bounding box contains multiple point cloud data.

[0166] Send a command to read point cloud data to the cloud and obtain the bounding box index value sent by the cloud;

[0167] Obtain the current field of view and the bounding boxes corresponding to their bounding box index values;

[0168] The target bounding box is determined based on the intersection of the current field of view and each bounding box, and the point cloud data in the target bounding box is loaded.

[0169] The data processing method for the client provided in this embodiment has the same or corresponding technical features as the data processing method for the cloud described above. The data processing method for the cloud has been described in detail above, and the data processing method for the client will not be repeated here. Compared with the method of directly storing point cloud data on the client, the method provided in this embodiment processes the point cloud data in the cloud and sends the created bounding box index values ​​to the client, so that the client stores the bounding box index values. Therefore, the amount of data stored on the client is reduced and the cost of data storage is reduced. In order to read point cloud data, the client first obtains each bounding box corresponding to the bounding box index value, and then takes the intersection of the bounding box with the current field of view. Thus, only the point cloud data corresponding to the bounding box corresponding to the field of view is loaded, which realizes the on-demand loading of point cloud data. It is not necessary to load all point cloud data, which improves the efficiency of data loading and reduces the requirements for computer performance. At the same time, since the point cloud data sent by the client is processed in the cloud, the data processing capabilities of the cloud are fully utilized, which improves the efficiency of data processing.

[0170] In the above embodiments, the data processing method has been described in detail. The present invention also provides embodiments corresponding to a data processing apparatus and a data processing device. It should be noted that the embodiments of the apparatus portion are described from two perspectives: one based on functional modules, and the other based on hardware.

[0171] Figure 3 This is a structural diagram of a data processing apparatus provided according to an embodiment of the present invention. The data processing apparatus is applied in the cloud. This embodiment, based on functional modules, includes:

[0172] The first acquisition module 10 is used to acquire point cloud data sent by the client and create multiple independent bounding boxes based on the point cloud data; wherein each bounding box contains multiple point cloud data.

[0173] The second acquisition module 11 is used to acquire and store the index values ​​of each bounding box;

[0174] The sending module 12 is used to send the stored bounding box index value to the client when it receives the instruction to read point cloud data, so that the client can obtain the current field of view and each bounding box corresponding to the bounding box index value, and determine the target bounding box according to the intersection range of the current field of view and each bounding box, and load the point cloud data in the target bounding box.

[0175] This embodiment also provides an apparatus for data processing on a client side, the apparatus comprising:

[0176] The first sending module is used to send point cloud data to the cloud so that after receiving the point cloud data sent by the client, the cloud can create multiple independent bounding boxes based on the point cloud data and obtain and store the index value of each bounding box; wherein each bounding box contains multiple point cloud data.

[0177] The second sending module is used to send instructions to read point cloud data to the cloud and obtain the bounding box index value sent by the cloud;

[0178] The third acquisition module is used to acquire the current field of view and each bounding box corresponding to the bounding box index value;

[0179] The determination and loading module is used to determine the target bounding box based on the intersection range of the current field of view and each bounding box, and to load the point cloud data in the target bounding box. Since the embodiments of the device section correspond to the embodiments of the method section, the embodiments of the device section are described in the method section and will not be repeated here. Furthermore, it has the same beneficial effects as the data processing method mentioned above.

[0180] Figure 4 This is a structural diagram of a data processing device provided according to another embodiment of the present invention. This embodiment is based on a hardware perspective, such as... Figure 4 As shown, the data processing equipment includes:

[0181] Memory 20 is used to store computer programs;

[0182] The processor 21 is configured to implement the steps of the data processing method as described in the above embodiments when executing a computer program.

[0183] The data processing device provided in this embodiment may include, but is not limited to, smartphones, tablets, laptops, or desktop computers.

[0184] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one of the following hardware forms: Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0185] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 20 is used to store at least the following computer program 201, which, after being loaded and executed by the processor 21, is capable of implementing the relevant steps of the data processing method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, and the storage method may be temporary or permanent storage. The operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include, but is not limited to, the data involved in the data processing method mentioned above.

[0186] In some embodiments, the data processing device may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

[0187] Those skilled in the art will understand that Figure 4 The structure shown does not constitute a limitation on the data processing device and may include more or fewer components than illustrated.

[0188] The data processing device provided in this embodiment of the invention includes a memory and a processor. When the processor executes a program stored in the memory, it can implement the following method: the data processing method has the same effect as above.

[0189] Finally, the present invention also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, performs the steps described in the above method embodiments.

[0190] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0191] The computer-readable storage medium provided by this invention includes the data processing method mentioned above, and has the same effect.

[0192] The present invention has provided a detailed description of a data processing method, apparatus, device, and medium. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0193] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A data processing method, characterized in that, Applied to the cloud, the method includes: The system acquires point cloud data sent by the client and creates multiple independent bounding boxes based on the point cloud data; wherein each bounding box contains multiple sets of the point cloud data. Retrieve and store the index values ​​of each bounding box; Upon receiving an instruction to read the point cloud data, the stored bounding box index value is sent to the client so that the client can obtain the current field of view and each bounding box corresponding to the bounding box index value, and determine the target bounding box based on the intersection of the current field of view and each bounding box, and load the point cloud data in the target bounding box. The step of acquiring point cloud data sent by the client and creating multiple independent bounding boxes based on the point cloud data includes: Obtain the point cloud data in its original format sent by the client; The point cloud data in each original format is converted into point cloud data in a three-dimensional format; Based on the point cloud data in all three-dimensional formats, obtain the minimum and maximum values ​​of the three axes; An initial bounding box is established based on the points formed by the minimum values ​​of the three axes and the points formed by the maximum values ​​of the three axes; wherein, the initial bounding box is an AABB bounding box; The initial bounding box is split into multiple independent bounding boxes; The bounding box index value includes the depth value of the bounding box and the position information of the bounding box; Obtaining the bounding box index value includes: The number of parent bounding boxes of all split layers in the split path from the initial bounding box to the obtained bounding box; The depth value of the bounding box is determined based on the number of parent bounding boxes; From the initial bounding box to the splitting path where the bounding box is obtained, obtain a three-dimensional binary number representing the position information of the bounding box for all splitting layers; wherein, the value of each bit of the three-dimensional binary number is set according to a preset rule; Arrange each of the three-dimensional binary numbers according to the order of the splitting layers in the splitting path to obtain the position information of the bounding box; The depth value of the bounding box and the position information are combined to obtain the bounding box index value.

2. The data processing method according to claim 1, characterized in that, The step of splitting the initial bounding box into multiple independent bounding boxes includes: The initial bounding box is divided into a predetermined number of bounding boxes; Determine whether the bounding boxes obtained from the splitting meet the preset requirements; If not, then the bounding box that does not meet the preset requirements is obtained, and the bounding box that does not meet the preset requirements is split into the preset number of bounding boxes, until the bounding box obtained after splitting the bounding box that does not meet the preset requirements meets the preset requirements, then it is determined that the initial bounding box has been split.

3. The data processing method according to claim 2, characterized in that, The preset requirements include at least one of the following: the number of point cloud data in the bounding box is less than a preset value, and the depth value of the bounding box is greater than or equal to a preset depth value; wherein, the depth value of the bounding box is determined by the number of parent bounding boxes of all split layers in the splitting path from the initial bounding box to the point where the bounding box is obtained.

4. The data processing method according to claim 2 or 3, characterized in that, The preset quantity is 8.

5. The data processing method according to claim 4, characterized in that, Obtaining the current field of view includes: The viewing frustum is determined based on the camera's parameters; wherein the camera parameters include the camera's position, lens shooting orientation, up vector, field of view, aspect ratio, near clipping distance, and far clipping distance; The visual cone is used as the current field of vision; Obtaining the bounding box corresponding to the bounding box index value includes: Decompose the bounding box index value to obtain the three-dimensional binary number corresponding to the bounding box of each splitting layer in the splitting path; The initial bounding box is updated to obtain the bounding box corresponding to the bounding box index value; wherein, the initial bounding box is updated by updating the initial bounding box according to the three-dimensional binary number corresponding to the bounding box of each split layer in the splitting path; Correspondingly, determining the target bounding box based on the intersection range of the current field of view and each of the bounding boxes includes: The frustum and each bounding box are respectively used to determine whether the bounding box is within the frustum; If so, then the bounding box is determined to be the target bounding box.

6. The data processing method according to claim 5, characterized in that, The step of updating the initial bounding box based on the three-dimensional binary number corresponding to the bounding box of each splitting layer in the splitting path includes: Starting from the first bit of the three-dimensional binary number corresponding to the bounding box of the first split layer in all split layers, and ending at the last bit of the three-dimensional binary number corresponding to the bounding box of the last split layer in all split layers, the previous bounding box is updated sequentially according to the value of each bit and the preset rules to obtain a new bounding box; wherein, except when the first bit of the three-dimensional binary number corresponding to the bounding box of the first split layer is the first bounding box, the previous bounding box is the new bounding box determined according to the value of the previous bit of the current bit and the preset rules when the new bounding box is determined according to the value of the current bit and the preset rules. Obtain the final new bounding box, and use the final new bounding box as the bounding box corresponding to the bounding box index value.

7. A data processing method, characterized in that, Applied to a client, the method includes: Point cloud data is sent to the cloud so that after receiving the point cloud data sent by the client, the cloud can create multiple independent bounding boxes based on the point cloud data, and obtain and store the index value of each bounding box; wherein each bounding box contains multiple points cloud data. Send a command to read point cloud data to the cloud and obtain the bounding box index value sent by the cloud; Obtain the current field of view and each bounding box corresponding to the bounding box index value; The target bounding box is determined based on the intersection of the current field of view and each bounding box, and the point cloud data in the target bounding box is loaded. Acquiring point cloud data sent by the client and creating multiple independent bounding boxes based on the point cloud data includes: Obtain the point cloud data in its original format sent by the client; The point cloud data in each original format is converted into point cloud data in a three-dimensional format; Based on the point cloud data in all three-dimensional formats, obtain the minimum and maximum values ​​of the three axes; An initial bounding box is established based on the points formed by the minimum values ​​of the three axes and the points formed by the maximum values ​​of the three axes; wherein, the initial bounding box is an AABB bounding box; The initial bounding box is split into multiple independent bounding boxes; The bounding box index value includes the depth value of the bounding box and the position information of the bounding box; Obtaining the bounding box index value includes: The number of parent bounding boxes of all split layers in the split path from the initial bounding box to the obtained bounding box; The depth value of the bounding box is determined based on the number of parent bounding boxes; From the initial bounding box to the splitting path where the bounding box is obtained, obtain a three-dimensional binary number representing the position information of the bounding box for all splitting layers; wherein, the value of each bit of the three-dimensional binary number is set according to a preset rule; Arrange each of the three-dimensional binary numbers according to the order of the splitting layers in the splitting path to obtain the position information of the bounding box; The depth value of the bounding box and the position information are combined to obtain the bounding box index value.

8. A data processing apparatus, characterized in that, The device, applied in the cloud, includes: The first acquisition module is used to acquire point cloud data sent by the client and create multiple independent bounding boxes based on the point cloud data; wherein each bounding box contains multiple points cloud data. The second acquisition module is used to acquire and store the index values ​​of each bounding box; The sending module is used to send the stored bounding box index value to the client when it receives an instruction to read the point cloud data, so that the client can obtain the current field of view and each bounding box corresponding to the bounding box index value, determine the target bounding box according to the intersection range of the current field of view and each bounding box, and load the point cloud data in the target bounding box. The first acquisition module is specifically used for: Obtain the point cloud data in its original format sent by the client; The point cloud data in each original format is converted into point cloud data in a three-dimensional format; Based on the point cloud data in all three-dimensional formats, obtain the minimum and maximum values ​​of the three axes; An initial bounding box is established based on the points formed by the minimum values ​​of the three axes and the points formed by the maximum values ​​of the three axes; wherein, the initial bounding box is an AABB bounding box; The initial bounding box is split into multiple independent bounding boxes; The bounding box index value includes the depth value of the bounding box and the position information of the bounding box; Obtaining the bounding box index value includes: The number of parent bounding boxes of all split layers in the split path from the initial bounding box to the obtained bounding box; The depth value of the bounding box is determined based on the number of parent bounding boxes; From the initial bounding box to the splitting path where the bounding box is obtained, obtain a three-dimensional binary number representing the position information of the bounding box for all splitting layers; wherein, the value of each bit of the three-dimensional binary number is set according to a preset rule; Arrange each of the three-dimensional binary numbers according to the order of the splitting layers in the splitting path to obtain the position information of the bounding box; The depth value of the bounding box and the position information are combined to obtain the bounding box index value.

9. A data processing device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data processing method as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Browser based mass three-dimensional point cloud data release method

    CN105808672A

  • Lightweight visualization method of building information model based on Web3D

    CN109145366A