Point cloud data labeling method and device, storage medium and electronic equipment

Abstract

Embodiments of the present application provide a point cloud data labeling method and device, a storage medium and an electronic device, wherein the method comprises: determining a target data level matched with a current perspective range of a browser end, wherein scene point cloud data of a target three-dimensional scene is divided into a plurality of point cloud data layers according to a plurality of data levels, and the point cloud density within each point cloud data layer of the plurality of point cloud data layers is positively correlated with the data level corresponding to each point cloud data layer; based on the target data level, obtaining target point cloud data of a group of continuous frames of the target three-dimensional scene from a server end; loading the target point cloud data of the group of continuous frames into the target three-dimensional scene of the browser end for display, so as to perform data labeling on the group of continuous frames. Through the present application, the problem of low labeling efficiency caused by poor point cloud display effect in the related art labeling method of point cloud data is solved.

Description

Technical Field

[0001] This application relates to the field of 3D point cloud annotation technology, and more specifically, to a point cloud data annotation method and apparatus, storage medium and electronic device. Background Technology

[0002] Point cloud continuous frame annotation is a widely used data processing type in autonomous driving scenarios. Annotating continuous frame point cloud data requires high levels of 3D spatial perception and multi-frame processing capabilities. While frame-by-frame annotation can improve efficiency by copying objects, it falls far short of meeting the requirements. 3D (Three-dimensional) point cloud continuous frame annotation is performed based on loaded point cloud data. Traditionally, this involves thinning the continuous frame point cloud data and directly loading the point cloud file on the client side, resulting in poor point cloud display quality, impacting the efficiency and accuracy of manual annotation, and exacerbating the slowness of manual frame-by-frame annotation.

[0003] This shows that the annotation methods for point cloud data in related technologies suffer from low annotation efficiency due to poor point cloud display effects. Summary of the Invention

[0004] This application provides a point cloud data annotation method, apparatus, storage medium, and electronic device to at least solve the problem of low annotation efficiency caused by poor point cloud display effects in related technologies.

[0005] According to one embodiment of this application, a point cloud data annotation method is provided, comprising: determining a target data level that matches the current view range of a browser, wherein the current view range is the view range of a target 3D scene to be displayed on the browser, the scene point cloud data of the target 3D scene is divided into multiple point cloud data layers according to multiple data levels, the point cloud density within each of the multiple point cloud data layers is positively correlated with the data level corresponding to each point cloud data layer, and the view range of the browser and the data level that matches the view range of the browser are defined. Negative correlation, among the multiple point cloud data layers, the point cloud data layer corresponding to the target data level is the target point cloud data layer; based on the target data level, a set of consecutive frames of target point cloud data of the target 3D scene is obtained from the server, wherein the target point cloud data of the set of consecutive frames is the point cloud data in the scene point cloud data of the set of consecutive frames that matches the current view range and is located within the target point cloud data layer; the target point cloud data of the set of consecutive frames is loaded into the target 3D scene on the browser for display, so as to perform data annotation on the set of consecutive frames.

[0006] According to another embodiment of this application, a point cloud data annotation device is provided, comprising: a first determining unit, configured to determine a target data level matching the current view range of a browser, wherein the current view range is the view range of a target 3D scene to be displayed on the browser, the scene point cloud data of the target 3D scene is divided into multiple point cloud data layers according to multiple data levels, the point cloud density within each of the multiple point cloud data layers is positively correlated with the data level corresponding to each point cloud data layer, and the view range of the browser is negatively correlated with the data level matching the view range of the browser. Among the multiple point cloud data layers, the point cloud data layer corresponding to the target data level is the target point cloud data layer; the first acquisition unit is used to acquire a set of target point cloud data of a continuous frame of the target 3D scene from the server based on the target data level, wherein the target point cloud data of the continuous frame is the point cloud data in the scene point cloud data of the continuous frame that matches the current view range and is located within the target point cloud data layer; the loading unit is used to load the target point cloud data of the continuous frame into the target 3D scene on the browser for display, so as to perform data annotation on the continuous frame.

[0007] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the above-described point cloud data annotation method at runtime.

[0008] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the annotation method of the point cloud data through the computer program.

[0009] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in any of the above-described point cloud data annotation method embodiments.

[0010] This application's embodiments employ a layered storage method for point cloud data on the server side. This reduces thinning of continuous frame point cloud data and allows the browser to retrieve point cloud data from the server on demand and load it on the front end (i.e., the browser). The point cloud data file, which is usually loaded by the front end and thinned, is stored using back-end (i.e., server-side) data caching technology. This supports the loading and annotation of large amounts of point cloud data. Point cloud location data information is effectively preserved during the point cloud data loading stage, significantly improving the loading and display effect of large amounts of point cloud data on the client side. This facilitates the speed and accuracy of identifying labeled elements during manual annotation, achieving the technical effect of improving the annotation efficiency of point cloud data. This solves the problem of low annotation efficiency caused by poor point cloud display effects in related point cloud data annotation methods. Attached Figure Description

[0011] Figure 1 This is a hardware structure block diagram of an optional point cloud data annotation method according to an embodiment of this application;

[0012] Figure 2 This is a flowchart illustrating an optional point cloud data annotation method according to an embodiment of this application;

[0013] Figure 3 This is a schematic diagram of an optional continuous frame point cloud storage visualization process according to an embodiment of this application;

[0014] Figure 4 This is a schematic diagram illustrating an optional method for temporarily storing and editing annotation box data according to an embodiment of this application;

[0015] Figure 5 This is a schematic diagram of an optional continuous frame tracking annotation according to an embodiment of this application;

[0016] Figure 6 This is a structural block diagram of an optional point cloud data annotation device according to an embodiment of this application;

[0017] Figure 7 This is a structural block diagram of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] According to one aspect of the embodiments of this application, a method for labeling point cloud data is provided. Optionally, in this embodiment, the above-described method for labeling point cloud data can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102 and server 104. For example... Figure 1 As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0021] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi, Bluetooth. The terminal device 102 may not be limited to devices such as PCs, mobile phones, tablets, etc.

[0022] The point cloud data annotation method of this application embodiment can be executed by the terminal device 102, or by the server 104 and the terminal device 102 jointly. Taking the point cloud data annotation method of this embodiment executed by the terminal device 102 as an example, Figure 2 This is a flowchart illustrating an optional point cloud data annotation method according to an embodiment of this application, as shown below. Figure 2 As shown, the process of this method may include the following steps:

[0023] Step S202: Determine the target data layer that matches the current view range of the browser. The current view range is the view range of the target 3D scene to be displayed on the browser. The scene point cloud data of the target 3D scene is divided into multiple point cloud data layers according to multiple data layers. The point cloud density within each point cloud data layer is positively correlated with the data layer corresponding to each point cloud data layer. The view range of the browser is negatively correlated with the data layer that matches the view range of the browser. Among the multiple point cloud data layers, the point cloud data layer that corresponds to the target data layer is the target point cloud data layer.

[0024] The point cloud data annotation method in this embodiment can be applied to scenarios involving the annotation of 3D point cloud data. 3D point cloud annotation involves labeling point cloud data in 3D space, that is, adding labels or annotations to each point or group of points in the point cloud. These annotations can include information such as the shape, motion trajectory, category, position, pose, and bounding box of the annotated object. Common annotation methods include 3D bounding box annotation, 3D point cloud semantic segmentation, continuous frame annotation of point clouds, and 2D / 3D fusion annotation. Among these, continuous frame annotation of point clouds is a widely used data processing type in autonomous driving scenarios.

[0025] 3D point cloud continuous frame annotation is performed on the basis of loading point cloud data. The traditional method is to thin the continuous frame point cloud data and load the point cloud file directly on the client, resulting in poor point cloud display effect, affecting the efficiency and accuracy of manual annotation, and exacerbating the slowness of manual frame-by-frame annotation.

[0026] To at least partially solve the above problems, in this embodiment, point cloud data is stored in layers on the backend, and the data is obtained by the client and the backend and loaded on the frontend, which can support the loading of large amounts of point cloud data.

[0027] Here, the interaction between the backend and the client can be achieved through WebSocket communication. WebSocket is a communication protocol that establishes a persistent connection between the client and the server, allowing bidirectional communication.

[0028] In this embodiment, to improve the display effect of point cloud loading and facilitate the identification of features by annotators, the backend (i.e., the server) can perform data layering and storage of the acquired point cloud files. Correspondingly, the client (i.e., the browser) can determine the matching target data level based on the current view range, which is the view range of the target 3D scene to be displayed on the browser.

[0029] The point cloud data of the target 3D scene is divided into multiple point cloud data layers according to multiple data levels. The point cloud density within each point cloud data layer is positively correlated with the data level corresponding to each point cloud data layer. That is, the fewer the data levels, the lower the corresponding point cloud density. For example, when the current view range on the browser is large, the fewer the corresponding data levels, the lower the point cloud data density. When the view is zoomed in (the corresponding view range will shrink) to obtain more detailed ground features, the more corresponding data levels, the higher the point cloud data density.

[0030] Here, the target data level is the point cloud data layer with the highest density that matches the current view range of the browser. That is, the 3D point cloud data that matches the current view range of the browser is the target data level and the multiple point cloud data layers smaller than the target data level. Among the multiple point cloud data layers, the point cloud data layer corresponding to the target data level is the target point cloud data layer.

[0031] There is a negative correlation between the browser's field of view and the data hierarchy that matches it; that is, the larger the browser's field of view, the fewer the corresponding data hierarchy; and the smaller the browser's field of view, the more corresponding data hierarchy.

[0032] Step S204: Based on the target data layer, obtain a set of target point cloud data of a continuous frame of the target 3D scene from the server. The target point cloud data of a continuous frame is the point cloud data in the scene point cloud data of a continuous frame that matches the current view range and is located within the target point cloud data layer.

[0033] If the target data level is determined on the browser side and matches the current view range of the browser side, a set of consecutive frames of target point cloud data of the target 3D scene can be obtained from the server based on the target data level. The set of consecutive frames of target point cloud data is the point cloud data in the scene point cloud data of a set of consecutive frames that matches the current view range and is located within the target point cloud data level.

[0034] Here, a set of consecutive frames of target point cloud data can be single-layer or multi-layer point cloud data. When the target point cloud data layer is the lowest density layer, the target point cloud data of a set of consecutive frames is single-layer point cloud data. When the target point cloud data layer is not the lowest density layer, the target point cloud data of a set of consecutive frames is multi-layer point cloud data, including all point cloud data from the lowest density layer to the target point cloud data layer, and including the lowest density layer and the target point cloud data layer itself.

[0035] Optionally, to improve transmission efficiency, if the data size of a set of three-dimensional continuous frame point cloud data is greater than or equal to a preset threshold, the compressed target point cloud data of a set of continuous frames can be obtained from the server; and the decompression tool on the browser side can be called to decompress the compressed target point cloud data of a set of continuous frames.

[0036] Step S206: Load a set of consecutive frame target point cloud data into the target 3D scene on the browser for display, so as to annotate the set of consecutive frames.

[0037] For example, in this embodiment, the backend can first perform data layering and database storage on the obtained point cloud files. This is done through multi-threaded processing and using PostGIS database storage technology to complete the storage of a large number of frame point cloud files. When a frame point cloud needs to be loaded, the client sends a WebSocket request. The backend retrieves the corresponding compressed point cloud data from the Redis cache based on the request parameters (including the range of 3D point cloud coordinates corresponding to four points on the browser screen, the name of the point cloud file, layer parameters, etc.) and sends it to the client.

[0038] The client's data requests are controlled based on the current field of view. When the field of view is large and data accuracy requirements are not high, only the point cloud data of the lowest density layer needs to be requested. When zooming in to obtain more detailed ground features, multiple layers of point cloud data will be requested to enrich the display effect. After receiving the compressed single-layer or multi-layer point cloud data, the client will call the front-end decompression tool to decompress and read the point cloud data, and then perform 3D visualization of the point cloud data. Users can then continue with subsequent point cloud annotation, quality inspection, acceptance, and other operations.

[0039] A flowchart for the visualization of continuous frame point cloud storage can be shown as follows: Figure 3 As shown, specifically, continuous frame point cloud storage visualization may include the following steps:

[0040] Step 1: Input the point cloud files into the pg database (i.e., the postgis database) in layers;

[0041] Step 2: Compress the data and store it in the Redis cache database;

[0042] Step 3: Visualization of the data returned by WebSocket communication;

[0043] Step 4: Update the current visualization content based on the field of view.

[0044] Here, PostGIS is a spatial database extension running on the PostgreSQL relational database, adding support for Geographic Information System (GIS) objects. PostGIS provides the functionality to store and query geospatial data, including geographic objects such as points, lines, and polygons. Its purpose is to enable users to store and process geospatial data within a relational database, thereby facilitating applications such as geospatial analysis and map creation. PostGIS is used to add geospatial data support to relational databases for the storage and analysis of geospatial data. Redis is used as a caching database to improve application performance and scalability.

[0045] Optionally, updating the current visualization content based on the viewing angle range can be done by loading or labeling partitions based on configuration information. The configuration information can be manually set by the user or a fixed partitioning rule. This embodiment does not limit this.

[0046] Through steps S202 to S206, the target data level matching the current view range of the browser is determined. The current view range is the view range of the target 3D scene to be displayed on the browser. The scene point cloud data of the target 3D scene is divided into multiple point cloud data layers according to multiple data levels. The point cloud density within each point cloud data layer is positively correlated with the corresponding data level. The view range of the browser is negatively correlated with the data level matching the view range of the browser. Among the multiple point cloud data layers, the data level corresponding to the target data level... The point cloud data layer is the target point cloud data layer. Based on the target data layer, a set of consecutive frames of target point cloud data of the target 3D scene is obtained from the server. The target point cloud data of a set of consecutive frames is the point cloud data in the scene point cloud data of a set of consecutive frames that matches the current view range and is located within the target point cloud data layer. The target point cloud data of a set of consecutive frames is loaded into the target 3D scene on the browser for display, so as to perform data annotation on a set of consecutive frames. This solves the problem of low annotation efficiency caused by poor point cloud display effect in the annotation methods of related technologies.

[0047] In an exemplary embodiment, the point cloud data in different point cloud data layers of the target 3D scene are stored in different databases.

[0048] Based on the target data hierarchy, a set of consecutive frames of target point cloud data of the target 3D scene is obtained from the server, including:

[0049] S11, send a data acquisition request to the server. The data acquisition request carries the layer identifier of the target data layer and the indication information of a set of continuous frames. The data acquisition request is used to request the acquisition of point cloud data in a set of continuous frames of scene point cloud data that are located within the current view range and whose corresponding data layer is less than or equal to the target data layer.

[0050] S12, receive a set of target point cloud data in a continuous frame returned by the server in response to the data acquisition request. The set of target point cloud data is point cloud data obtained by the server from the database corresponding to the target data level and the database corresponding to the data level lower than the target data level, according to the indication information of the set of continuous frames.

[0051] In the scene point cloud data of the target 3D scene, point cloud data in different point cloud data layers are stored in different databases. The browser can send a data acquisition request to the server. The data acquisition request carries the layer identifier of the target data layer and the indication information of a set of consecutive frames. The data acquisition request is used to request the point cloud data in the scene point cloud data of a set of consecutive frames that are located within the current view range and whose corresponding data layer is less than or equal to the target data layer. That is, the browser obtains the point cloud data that corresponds to the current view range and whose data layer is less than or equal to the target data layer. Here, the target data layer is similar to that in the previous embodiment and will not be described again.

[0052] In response to a data acquisition request, the server returns a set of consecutive frames of target point cloud data to the browser. The browser can receive the set of consecutive frames of target point cloud data returned by the server. Here, the set of consecutive frames of target point cloud data is point cloud data obtained by the server from the database corresponding to the target data level and the database corresponding to the data level lower than the target data level, according to the indication information of the set of consecutive frames.

[0053] Here, the indication information for a set of consecutive frames can be the name of the consecutive frame point cloud file, or other indication information for consecutive frames. This embodiment does not limit this.

[0054] In this embodiment, by obtaining a set of continuous frame point cloud data corresponding to the current viewpoint range and whose data level is less than or equal to the target data level from the server based on the layer identifier of the target data level and the indication information of a set of continuous frames on the browser side, the effect of point cloud loading and display can be improved, avoiding the need to thin out the continuous frame point cloud data and directly load the point cloud file on the client side, which would result in poor point cloud display effect.

[0055] In one exemplary embodiment, after loading a set of consecutive frames of target point cloud data into the target 3D scene on the browser for display, the above method further includes:

[0056] S21, obtain the first annotation data of the target feature in the starting frame of a set of consecutive frames, and obtain the second annotation data of the target feature in the ending frame of a set of consecutive frames.

[0057] S22, based on the first annotation data and the second annotation data, the target features of each intermediate frame in a set of intermediate frames are annotated, wherein a set of intermediate frames is at least one frame located between the start frame and the end frame in a set of consecutive frames.

[0058] In related technologies, the efficiency of annotation can be improved by copying objects in the frame-by-frame annotation mode, but it is far from meeting the annotation requirements. Therefore, an algorithm is introduced to provide tracking annotation for continuous frame point clouds. By using algorithm-assisted annotation, the annotation efficiency can be significantly improved without the need for frame-by-frame annotation.

[0059] After loading a set of consecutive frame target point cloud data into the target 3D scene on the browser for display, the user can annotate the target point cloud data of the set of consecutive frame target points displayed on the browser. The browser can obtain the first annotation data of the target features in the starting frame of the set of consecutive frames, and the second annotation data of the target features in the ending frame of the set of consecutive frames. Based on the first annotation data and the second annotation data, the target features of each intermediate frame in a set of intermediate frames are annotated. The set of intermediate frames is at least one frame located between the starting frame and the ending frame in a set of consecutive frames.

[0060] For example, in this embodiment, for 3D continuous frame point cloud data, the object to be tracked is annotated with a traditional 3D bounding box in the starting frame, the bounding box information of the point cloud in that frame is saved, and then the tracking point cloud is switched to the ending frame. The bounding box of the starting frame can be used as the basic bounding box, and the bounding box of the ending frame is finely adjusted to fit the object. After calculation based on the positions of the bounding boxes in the starting and ending frames, the bounding box of the object in the intermediate frame can be automatically generated according to a specific algorithm.

[0061] This embodiment automatically generates intermediate frame annotation results by tracking the point cloud annotation results of the start and end frames, thereby improving annotation efficiency and accuracy and meeting the needs of automated driving technology for large amounts of training annotation data.

[0062] In an exemplary embodiment, the target feature is labeled by a target label box, the position of the target label box indicated by the first label data is the first label box position, and the position of the target label box indicated by the second label data is the second label box position;

[0063] Based on the first and second annotation data, the target features of each intermediate frame in a set of intermediate frames are annotated, including:

[0064] S31, treat each intermediate frame as the current intermediate frame and perform the following annotation operations to obtain the annotation data of the target features in each intermediate frame:

[0065] When the target feature is a static feature, the position of the target label box in the current intermediate frame is predicted according to the linear relationship based on the positions of the first and second label boxes, thus obtaining the position of the target label box in the current intermediate frame;

[0066] When the target feature is a non-static feature, the position of the target label box in the current intermediate frame is determined based on the position of the first label box, the position of the second label box, the running speed of the target feature, and the frame number of the current intermediate frame; the angle of the target label box in the current intermediate frame is determined based on the angle of the target label box in the starting frame, the angle change rate of the target label box, and the frame number of the current intermediate frame.

[0067] The target elements are labeled using target annotation boxes. The position of the target annotation box indicated by the first annotation data is the first annotation box position, and the position of the target annotation box indicated by the second annotation data is the second annotation box position.

[0068] The size of the annotation boxes in intermediate frames is predicted based on the sizes of the bounding boxes in the start and end frames of the point cloud. Since the same annotation object is used in both frames, the dimensions remain unchanged; the main changes are in the position and orientation of the annotation boxes. When calculating and displaying the annotation boxes in intermediate frames, the front-end and back-end primarily predict the center point position and orientation of the annotation boxes. For the center point position, the displacement value of the intermediate frame annotation box can be calculated based on the center points of the annotation boxes in the start and end frames, thereby predicting the position information of the annotation boxes in each intermediate frame.

[0069] When the target feature is a static feature, the position of the target label box in the current intermediate frame is predicted according to the linear relationship based on the positions of the first and second label boxes, thus obtaining the position of the target label box in the current intermediate frame.

[0070] Optionally, when the target feature is a static feature, the ratio of the position difference between the second and first annotation box positions to the target frame number difference is determined as the position slope, where the target frame number difference is the frame number difference between the end frame and the start frame.

[0071] The position of the target bounding box in the current intermediate frame is determined by multiplying the frame number of the current intermediate frame by 1 and the position slope, and then adding the position of the first bounding box.

[0072] For example, in this embodiment, for static features, a linear relationship can be used to predict the position and angle information of the feature bounding boxes in intermediate frames. Based on the position coordinates of the starting frame, i.e., the position of the first bounding box (x1, y1, z1), and the position information of the bounding box in the ending frame, i.e., the position of the second bounding box (x1, y1, z1),... N y N , z N This allows us to track and calculate the bounding box positions of the corresponding features in the intermediate frame point cloud (Ni) (N-1>i>=1, and are integers):

[0073] x (N-i) =x1+(x N -x1) / (N-1)*(Ni-1),

[0074] y (N-i) =y1+(y N -y1) / (N-1)*(Ni-1),

[0075] z (N-i) =z1+(z N -z1) / (N-1)*(Ni-1).

[0076] Since the orientation and size of static features do not change, we can continue to use the average of the angle, length, width and height of the point clouds from the previous and next frames.

[0077] When the target feature is a non-static feature, the position of the target label box in the current intermediate frame is determined based on the position of the first label box, the position of the second label box, the running speed of the target feature, and the frame number of the current intermediate frame.

[0078] For non-static features, the running speed of the labeled features (target features) needs to be considered. In this embodiment, the position prediction of the labeled box of a non-static object is based on an object moving at a constant speed, such as a vehicle traveling on a lane. The position of the intermediate frame is predicted based on the average speed during the driving process.

[0079] For example, in this embodiment, the position coordinates (x1, y1, z1) of the bounding box (first bounding box) of the starting frame and the position information (x1, y1, z1) of the bounding box (second bounding box) of the ending frame are used. N y N , z N The velocities V in the x, y, and z directions were calculated based on the point cloud time interval t per frame and the average vehicle speed v. x V y V zHere, the calculation speed can be based on the position information of the start frame and the end frame and the time difference. Based on this, the position of the bounding box of the target feature corresponding to the intermediate frame point cloud (Ni) (N-1>i>=1, and is an integer) can be calculated.

[0080] The angle of the target bounding box in the current intermediate frame is determined based on the angle of the target bounding box in the starting frame, the rate of change of the target bounding box angle, and the frame number of the current intermediate frame.

[0081] For example, in this embodiment, the angle calculation for the moving object (target element) to be tracked is based on the angle angle1 of the bounding box (first bounding box) in the starting frame and the angle angle of the bounding box (second bounding box) in the ending frame. N With the average coefficients of angle change dtX, dtY, and dtZ, the angle values ​​of the three dimensions of the intermediate frame (Ni) can be obtained.

[0082] Since the size of the feature to be tracked will not change, we can continue to use the average of the length, width and height of the point cloud in the previous and next frames.

[0083] In an exemplary embodiment, when the target feature is a non-static feature, the position of the target annotation box in the current intermediate frame is determined based on the position of the first annotation box, the position of the second annotation box, the running speed of the target feature, and the frame number of the current intermediate frame; the angle of the target annotation box in the current intermediate frame is determined based on the angle of the target annotation box in the starting frame, the angle change rate of the target annotation box, and the frame number of the current intermediate frame, including:

[0084] S41, when the target element is a non-static element, the ratio of the position difference between the second annotation box position and the first annotation box position to the target frame number difference is determined as the position slope, where the target frame number difference is the frame number difference between the end frame and the start frame.

[0085] S42, the product of the running speed of the target element, the acquisition interval in a set of consecutive frames, and the number of target frames, plus the product of the number of target frames and the position slope, plus the position obtained by the first annotation box position, is determined as the position of the target annotation box in the current intermediate frame, where the number of target frames is the number of frames in the current intermediate frame minus 1.

[0086] S43, the ratio of the angle difference between the angle of the target annotation box in the end frame and the angle of the target annotation box in the start frame to the target time difference is determined as the angle slope, where the target time is the time difference between the acquisition time of the end frame and the acquisition time of the start frame.

[0087] S44. The product of the target frame number and the angle slope is added to the angle of the target bounding box in the starting frame to determine the angle of the target bounding box in the current intermediate frame.

[0088] Similar to the aforementioned embodiments, for non-static elements, such as objects moving at a constant speed, like vehicles traveling on a lane, the position of intermediate frames is predicted based on the average speed during the driving process.

[0089] The position coordinates of the starting frame's bounding box (first bounding box) are (x1, y1, z1), and the position coordinates of the ending frame's bounding box (second bounding box) are (x1, y1, z1). N y N , z N The average speed of the vehicle is v, and the velocities in the x, y, and z directions are V. x V y V z The time interval between each frame of point cloud is t.

[0090] Then the position of the bounding box of the corresponding feature in the intermediate frame point cloud (Ni) (N-1>i>=1, and are integers) is:

[0091] x (N-i) =x1+[(x N -x1) / (N-1)+V x *t]*(Ni-1),

[0092] y (N-i) =y1+[(y N -y1) / (N-1)+V y *t]*(Ni-1),

[0093] z (N-i) =z1+[(z N -z1) / (N-1)+V z *t]*(Ni-1).

[0094] Since the size of the feature to be tracked will not change, we can continue to use the average length, width, and height of the point clouds from the previous and next frames. For the angle calculation of the moving object to be tracked, we use the angle angle1 of the starting frame and the bounding box angle of the ending frame. N Given the average coefficients dtX, dtY, and dtZ of the angle changes, we can derive the angles in the x, y, and z dimensions of the intermediate frame (Ni) as follows:

[0095] angle (N-i) x = angle1x + dtX*(Ni-1),

[0096] angle (N-i) y = angle1y + dtY*(Ni-1),

[0097] angle (N-i)z = angle1z + dtZ*(Ni-1).

[0098] Here, the average coefficient of the angle change is:

[0099] dtX=(angle N x-angle1x) / (N-1),

[0100] dtY=(angle N y-angle1y) / (N-1),

[0101] dtZ=(angle N z-angle1z) / (N-1).

[0102] In one exemplary embodiment, after annotating the target features of each intermediate frame in a set of intermediate frames based on the first annotation data and the second annotation data, the method further includes:

[0103] S51 asynchronously stores the first annotation data, the second annotation data, and the annotation data of each predicted intermediate frame into the offline database by calling the application programming interface of the offline database on the browser side.

[0104] When tracking and annotating point cloud data, unlike single-frame annotation, it is necessary not only to store the geometric information and attribute information of the annotation boxes, but also to mark the association information between the annotation boxes in each frame of the point cloud and other annotation boxes. When there are many consecutive frame point cloud frames or many tracking annotation objects, the traditional localStorage will have a data size limit for temporarily storing these annotation boxes and marking information on the client. It only supports temporary storage of annotation result data within 5M.

[0105] In this embodiment, the offline database can be a LocalForage database. LocalForage is used to temporarily store these point cloud annotation data. Asynchronous storage improves the front-end's support for larger annotation result data, and this storage method is available in all major browsers. When a user uses tracking annotations and edits a tracking annotation box, the data can be temporarily manipulated on the client side, and the precise result data after fine-tuning the annotation box is saved uniformly.

[0106] That is, for client-annotated result data, localforage is used to replace the traditional localStorage to make up for the shortcomings of the client-side temporary storage method and support the temporary storage of larger amounts of annotation result data on the client side.

[0107] Temporary storage and editing diagram of annotation box data, as shown below Figure 4As shown, the annotation data (including coordinate data, dimension data, attribute data, ..., related data) is temporarily stored and edited in localforage, and then stored in the database after editing.

[0108] Here, localStorage is a Web Storage API (Application Programming Interface) used to store data in a browser. It allows web pages to store key-value pairs of data locally for persistence between sessions. This data remains unchanged even after the browser is closed and can be read and written using JavaScript. localStorage is a simple and easy-to-use method for storing small amounts of data on the client side. While localStorage is a browser-provided mechanism for client-side data storage, it has limited performance when handling large amounts of data and complex data structures. localforage is a JavaScript library for offline storage using IndexedDB, WebSQL, or localStorage. localforage provides a simple API for storing and retrieving data.

[0109] This embodiment effectively increases the upper limit of temporary storage for large amounts of annotation results by using localforage, which is supported by most browsers, to temporarily store the client's annotation results data.

[0110] In one exemplary embodiment, before determining the target data level that matches the current view range on the browser side, the method further includes:

[0111] S61, determine the intersection point of each corner point in a set of corner points of the display screen on the browser with the ground plane of the target 3D scene, and obtain a set of intersection points;

[0112] S62, convert the coordinates of each intersection point in a set of intersection points from the screen coordinates of the display screen to the three-dimensional scene coordinates of the target three-dimensional scene, and obtain the three-dimensional scene coordinates of each intersection point;

[0113] S63 determines the current viewpoint range based on the 3D scene coordinates of each intersection point.

[0114] For example, in this embodiment, an infinitely distant ground plane is constructed, and the intersection points of the four corner points of the browser's display screen with the ground plane in the 3D scene are monitored. The screen coordinates are converted to 3D scene coordinates to determine the range of the polygons intersecting with the ground from the current viewpoint, i.e., the current viewpoint range.

[0115] As an optional exemplary embodiment of this application, the annotation method for point cloud data of this application embodiment is explained and described using an example of efficiently tracking and annotating three-dimensional continuous frame point cloud data on the web.

[0116] This solution proposes a method that, based on the efficient loading of continuous frame point cloud data, temporarily stores the 3D continuous frame point cloud annotation results on the client side. During tracking annotation, the server uses appropriate algorithms to calculate the dimensions of the 3D annotation boxes in the start and end frames, ultimately automatically generating the 3D annotation box dimensions of the object in the intermediate frames and displaying the annotation results on the client side. This primarily improves the point cloud loading and display effect, facilitates feature identification by annotators, increases annotation speed, supports temporary storage of large amounts of annotation results on the front end, and provides point cloud tracking annotation functionality by distinguishing between static and non-static features, reducing manual annotation workload and further improving annotation efficiency and accuracy.

[0117] Combination Figure 5 The size of the annotation bounding boxes in the intermediate frame (Ni frame) is predicted based on the bounding box sizes of the starting frame (frame 1) and ending frame (frame N) of the point cloud. Since the same annotation is used in the preceding and following frames, the dimensions remain unchanged; the main changes are to the position and orientation of the bounding boxes. The calculation and display of the intermediate frame bounding boxes primarily involves pre-determining the center point position and orientation (angle) of the bounding boxes. For the center point position, the displacement value of the intermediate frame bounding box can be calculated based on the center points of the bounding boxes in the starting and ending frames, thereby predicting the position of the bounding boxes in each intermediate frame.

[0118] In this embodiment, considering that more complex tracking and annotation algorithms may be involved later, the annotation data of intermediate frames can be predicted on the server side, and the predicted data can be temporarily stored in the localforage database on the browser side. The tracking results can be temporarily stored on the front end, which makes it convenient for the annotators to manually fine-tune the tracking results. That is, after adding, deleting and modifying the annotation data stored in JSON data format on the front end, it can be uniformly saved to the back end database, without having to call the save interface for every very small drag.

[0119] This embodiment effectively preserves point cloud location data during the point cloud data loading stage, significantly improving the client's loading and display of large volumes of point cloud data. This facilitates faster and more accurate identification of labeled features during manual annotation. Using LocalForage, supported by most browsers, to temporarily store the client's annotation results effectively increases the upper limit for temporary storage of large volumes of annotation results. The tracking and annotation scheme, which distinguishes between static and dynamic features, improves the speed of tracking and annotation and the accuracy of the results, alleviating the slowness and low accuracy of manual annotation.

[0120] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0121] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware servers. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM (Read-Only Memory) / RAM (Random Access Memory), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0122] According to another aspect of the embodiments of this application, a point cloud data annotation apparatus for implementing the above-described point cloud data annotation method is also provided. Figure 6 This is a structural block diagram of an optional point cloud data annotation device according to an embodiment of this application, such as... Figure 6 As shown, the device may include:

[0123] The first determining unit 602 is used to determine the target data level that matches the current view range of the browser. The current view range is the view range of the target 3D scene to be displayed on the browser. The scene point cloud data of the target 3D scene is divided into multiple point cloud data layers according to multiple data levels. The point cloud density within each point cloud data layer of the multiple point cloud data layers is positively correlated with the data level corresponding to each point cloud data layer. The view range of the browser is negatively correlated with the data level that matches the view range of the browser. Among the multiple point cloud data layers, the point cloud data layer that corresponds to the target data level is the target point cloud data layer.

[0124] The first acquisition unit 604 acquires a set of target point cloud data of a set of consecutive frames of the target 3D scene from the server based on the target data layer. The set of target point cloud data of a set of consecutive frames is the point cloud data of a set of consecutive frames of scene point cloud data that matches the current view range and is located within the target point cloud data layer.

[0125] The loading unit 606 is used to load a set of consecutive frames of target point cloud data into the target 3D scene on the browser for display, so as to perform data annotation on a set of consecutive frames.

[0126] This application's embodiments determine a target data layer that matches the current view range of the browser. The current view range is the view range of the target 3D scene to be displayed on the browser. The scene point cloud data of the target 3D scene is divided into multiple point cloud data layers according to multiple data levels. The point cloud density within each point cloud data layer is positively correlated with the corresponding data layer, while the view range of the browser is negatively correlated with the data layer matching the view range. Among the multiple point cloud data layers, the point cloud data layer corresponding to the target data layer is the target point cloud data layer. Based on the target data layer, a set of consecutive frames of target point cloud data of the target 3D scene is obtained from the server. The target point cloud data of a set of consecutive frames is the point cloud data within the scene point cloud data of a set of consecutive frames that matches the current view range and is located within the target point cloud data layer. The target point cloud data of a set of consecutive frames is loaded into the target 3D scene on the browser for display, thus annotating the set of consecutive frames. This solves the problem of low annotation efficiency caused by poor point cloud display effects in related technologies, improving annotation efficiency.

[0127] In an exemplary embodiment, the point cloud data in different point cloud data layers of the target 3D scene are stored in different databases.

[0128] The first acquisition unit includes:

[0129] The sending module is used to send a data acquisition request to the server. The data acquisition request carries the layer identifier of the target data layer and the indication information of a set of consecutive frames. The data acquisition request is used to request the acquisition of point cloud data in a set of consecutive frames of scene point cloud data that are located within the current view range and whose corresponding data layer is less than or equal to the target data layer.

[0130] The receiving module is used to receive a set of consecutive frames of target point cloud data returned by the server in response to the data acquisition request. The set of consecutive frames of target point cloud data is point cloud data obtained by the server from the database corresponding to the target data level and the database corresponding to the data level lower than the target data level, according to the indication information of the set of consecutive frames.

[0131] In one exemplary embodiment, the above-described apparatus further includes:

[0132] The second acquisition unit is used to acquire the first annotation data of the target elements in the starting frame of a set of consecutive frames and the second annotation data of the target elements in the ending frame of a set of consecutive frames after loading the target point cloud data of a set of consecutive frames into the target 3D scene on the browser side for display.

[0133] The annotation unit is used to annotate the target features of each intermediate frame in a set of intermediate frames based on the first annotation data and the second annotation data, wherein the set of intermediate frames is at least one frame located between the start frame and the end frame in a set of consecutive frames.

[0134] In an exemplary embodiment, the target feature is labeled by a target label box, the position of the target label box indicated by the first label data is the first label box position, and the position of the target label box indicated by the second label data is the second label box position;

[0135] The annotation unit includes:

[0136] The execution module is used to perform the following annotation operations on each intermediate frame as the current intermediate frame, to obtain the annotation data of the target features in each intermediate frame:

[0137] When the target feature is a static feature, the position of the target label box in the current intermediate frame is predicted according to the linear relationship based on the positions of the first and second label boxes, thus obtaining the position of the target label box in the current intermediate frame;

[0138] When the target feature is a non-static feature, the position of the target label box in the current intermediate frame is determined based on the position of the first label box, the position of the second label box, the running speed of the target feature, and the frame number of the current intermediate frame; the angle of the target label box in the current intermediate frame is determined based on the angle of the target label box in the starting frame, the angle change rate of the target label box, and the frame number of the current intermediate frame.

[0139] In one exemplary embodiment, the execution module includes:

[0140] The first determining submodule is used to determine the ratio of the position difference between the second annotation box position and the first annotation box position to the target frame number difference as the position slope when the target feature is a non-static feature, wherein the target frame number difference is the frame number difference between the end frame and the start frame.

[0141] The second determining submodule is used to determine the position of the target annotation box in the current intermediate frame by multiplying the running speed of the target element, the acquisition interval in a set of consecutive frames, and the number of target frames, plus the product of the number of target frames and the position slope, plus the position obtained by the first annotation box position, where the number of target frames is the number of frames in the current intermediate frame minus 1.

[0142] The third determining submodule is used to determine the angle slope as the ratio of the angle difference between the angle of the target annotation box in the end frame and the angle of the target annotation box in the start frame to the target time difference, where the target time is the time difference between the acquisition time of the end frame and the acquisition time of the start frame.

[0143] The fourth determination submodule is used to add the product of the target frame number and the angle slope to the angle of the target bounding box in the starting frame, and then determine the angle of the target bounding box in the current intermediate frame.

[0144] In one exemplary embodiment, the above-described apparatus further includes:

[0145] The storage unit is used to asynchronously store the first annotation data, the second annotation data, and the predicted annotation data of each intermediate frame into the offline database by calling the application programming interface of the offline database on the browser side after the target features of each intermediate frame in a set of intermediate frames are annotated based on the first annotation data and the second annotation data.

[0146] In one exemplary embodiment, the above-described apparatus further includes:

[0147] The second determining unit is used to determine the intersection point of each corner point in a set of corner points of the display screen on the browser with the ground plane of the target 3D scene before determining the target data level that matches the current view range of the browser, so as to obtain a set of intersection points.

[0148] The conversion unit is used to convert the coordinates of each intersection point in a set of intersection points from the screen coordinates of the display screen to the 3D scene coordinates of the target 3D scene, so as to obtain the 3D scene coordinates of each intersection point.

[0149] The third determining unit is used to determine the current view range based on the three-dimensional scene coordinates of each intersection point.

[0150] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of a device, can operate in environments such as... Figure 1 The hardware environment shown can be implemented through software or hardware, and the hardware environment includes the network environment.

[0151] According to another aspect of the embodiments of this application, a storage medium is also provided. Optionally, in this embodiment, the storage medium can be used to execute program code for the annotation method of any of the point cloud data described in the embodiments of this application.

[0152] Optionally, in this embodiment, the storage medium may be located on at least one of the network devices in the network shown in the above embodiment.

[0153] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

[0154] S1, determine the target data level that matches the current view range of the browser, where the current view range is the view range of the target 3D scene to be displayed on the browser, the scene point cloud data of the target 3D scene is divided into multiple point cloud data layers according to multiple data levels, the point cloud density within each point cloud data layer of the multiple point cloud data layers is positively correlated with the data level corresponding to each point cloud data layer, the view range of the browser is negatively correlated with the data level that matches the view range of the browser, and among the multiple point cloud data layers, the point cloud data layer that corresponds to the target data level is the target point cloud data layer;

[0155] S2, based on the target data layer, obtain a set of consecutive frames of target point cloud data of the target 3D scene from the server. The target point cloud data of a set of consecutive frames is the point cloud data that matches the current view range and is located within the target point cloud data layer in the scene point cloud data of a set of consecutive frames.

[0156] S3 loads a set of consecutive frame target point cloud data into the target 3D scene on the browser for display, in order to annotate the data of a set of consecutive frames.

[0157] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated in this embodiment.

[0158] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0159] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described point cloud data annotation method is also provided. The electronic device may be a smart device, a server, a terminal, or a combination thereof.

[0160] Figure 7 This is a structural block diagram of an optional electronic device according to an embodiment of this application, such as... Figure 7 As shown, it includes a processor 702, a communication interface 704, a memory 706, and a communication bus 708. The processor 702, communication interface 704, and memory 706 communicate with each other via the communication bus 708.

[0161] Memory 706 is used to store computer programs;

[0162] When processor 702 executes a computer program stored in memory 706, it performs the following steps:

[0163] S1, determine the target data level that matches the current view range of the browser, where the current view range is the view range of the target 3D scene to be displayed on the browser, the scene point cloud data of the target 3D scene is divided into multiple point cloud data layers according to multiple data levels, the point cloud density within each point cloud data layer of the multiple point cloud data layers is positively correlated with the data level corresponding to each point cloud data layer, the view range of the browser is negatively correlated with the data level that matches the view range of the browser, and among the multiple point cloud data layers, the point cloud data layer that corresponds to the target data level is the target point cloud data layer;

[0164] S2, based on the target data layer, obtain a set of consecutive frames of target point cloud data of the target 3D scene from the server. The target point cloud data of a set of consecutive frames is the point cloud data that matches the current view range and is located within the target point cloud data layer in the scene point cloud data of a set of consecutive frames.

[0165] S3 loads a set of consecutive frame target point cloud data into the target 3D scene on the browser for display, in order to annotate the data of a set of consecutive frames.

[0166] Optionally, the communication bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The symbol is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0167] The memory may include RAM, or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0168] As an example, the memory 706 described above may include, but is not limited to, the first determining unit 602, the first acquiring unit 604, and the loading unit 606 from the point cloud data annotation device. Furthermore, it may include, but is not limited to, other module units from the point cloud data annotation device, which will not be elaborated upon in this example.

[0169] The processors mentioned above can be general-purpose processors, including but not limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; they can also be DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0170] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0171] Those skilled in the art will understand that Figure 7 The structure shown is for illustrative purposes only. The device that implements the above point cloud data annotation method can be a terminal device, such as a smartphone (e.g., an Android phone, an iOS phone), a tablet computer, a PDA, a mobile internet device (MID), a PAD, or other terminal devices. Figure 7This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 7 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 7 The different configurations shown.

[0172] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0173] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0174] The embodiments described herein also provide a computer program that includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in any of the above method embodiments.

[0175] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, ROM, RAM, disk or optical disk, etc.

[0176] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0177] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0178] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0179] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0180] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the solution provided in this embodiment, depending on actual needs.

[0181] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or at least two units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0182] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for labeling point cloud data, characterized in that, include: A target data layer matching the current view range of the browser is determined, wherein the current view range is the view range of the target 3D scene to be displayed on the browser, the scene point cloud data of the target 3D scene is divided into multiple point cloud data layers according to multiple data layers, the point cloud density within each of the multiple point cloud data layers is positively correlated with the data layer corresponding to each point cloud data layer, the view range of the browser is negatively correlated with the data layer matching the view range of the browser, and among the multiple point cloud data layers, the point cloud data layer corresponding to the target data layer is the target point cloud data layer; Based on the target data layer, a set of consecutive frames of target point cloud data of the target 3D scene is obtained from the server. The target point cloud data of the set of consecutive frames are the point cloud data in the scene point cloud data of the set of consecutive frames that match the current view range and are located within the target point cloud data layer. The target point cloud data of the set of consecutive frames is loaded into the target 3D scene on the browser for display, so as to perform data annotation on the set of consecutive frames.

2. The method according to claim 1, characterized in that, In the scene point cloud data of the target 3D scene, the point cloud data in different point cloud data layers are stored in different databases; The step of obtaining a set of consecutive frames of target point cloud data of the target 3D scene from the server based on the target data hierarchy includes: Send a data acquisition request to the server, wherein the data acquisition request carries the level identifier of the target data level and the indication information of the set of consecutive frames. The data acquisition request is used to request the acquisition of point cloud data in the scene point cloud data of the set of consecutive frames that are located within the current view range and whose corresponding data level is less than or equal to the target data level. The server receives the target point cloud data of a set of consecutive frames returned in response to the data acquisition request, wherein the target point cloud data of the set of consecutive frames is point cloud data obtained by the server from the database corresponding to the target data level and the database corresponding to the data level lower than the target data level, according to the indication information of the set of consecutive frames.

3. The method according to claim 1, characterized in that, After loading the target point cloud data of the set of consecutive frames into the target 3D scene on the browser for display, the method further includes: Obtain first annotation data of the target feature in the starting frame of the set of consecutive frames, and obtain second annotation data of the target feature in the ending frame of the set of consecutive frames; Based on the first annotation data and the second annotation data, the target features of each intermediate frame in a set of intermediate frames are annotated, wherein the set of intermediate frames is at least one frame located between the start frame and the end frame in the set of consecutive frames.

4. The method according to claim 3, characterized in that, The target elements are labeled using target annotation boxes. The position of the target annotation box indicated by the first annotation data is the first annotation box position, and the position of the target annotation box indicated by the second annotation data is the second annotation box position. The step of annotating the target features in each intermediate frame of a set of intermediate frames based on the first annotation data and the second annotation data includes: Perform the following annotation operation on each intermediate frame as the current intermediate frame to obtain the annotation data of the target feature in each intermediate frame: When the target element is a static element, the position of the target annotation box in the current intermediate frame is predicted according to a linear relationship based on the position of the first annotation box and the position of the second annotation box, so as to obtain the position of the target annotation box in the current intermediate frame; When the target element is a non-static element, the position of the target annotation box in the current intermediate frame is determined based on the position of the first annotation box, the position of the second annotation box, the running speed of the target element, and the frame number of the current intermediate frame; the angle of the target annotation box in the current intermediate frame is determined based on the angle of the target annotation box in the starting frame, the angle change rate of the target annotation box, and the frame number of the current intermediate frame.

5. The method according to claim 4, characterized in that, When the target element is a non-static element, the position of the target annotation box in the current intermediate frame is determined based on the position of the first annotation box, the position of the second annotation box, the running speed of the target element, and the frame number of the current intermediate frame; the angle of the target annotation box in the current intermediate frame is determined based on the angle of the target annotation box in the starting frame, the angle change rate of the target annotation box, and the frame number of the current intermediate frame, including: When the target element is a non-static element, the ratio of the position difference between the second annotation box position and the first annotation box position to the target frame number difference is determined as the position slope, wherein the target frame number difference is the frame number difference between the end frame and the start frame; The position of the target annotation box in the current intermediate frame is determined by multiplying the running speed of the target element, the acquisition interval in the set of consecutive frames, and the number of target frames, plus the product of the number of target frames and the position slope, plus the position obtained by the first annotation box position, wherein the number of target frames is the number of frames in the current intermediate frame minus 1. The angle slope is determined by the ratio of the angle difference between the target bounding box angle in the end frame and the target bounding box angle in the start frame to the target time difference, wherein the target time is the time difference between the acquisition time of the end frame and the acquisition time of the start frame. The angle of the target frame in the current intermediate frame is determined by adding the product of the target frame number and the angle slope to the angle of the target bounding box in the starting frame.

6. The method according to any one of claims 3 to 5, characterized in that, After annotating the target features of each intermediate frame in a set of intermediate frames based on the first annotation data and the second annotation data, the method further includes: By calling the application programming interface of the offline database on the browser side, the first annotation data, the second annotation data, and the predicted annotation data of each intermediate frame are asynchronously stored in the offline database.

7. The method according to claim 1, characterized in that, Before determining the target data level that matches the current view range on the browser side, the method further includes: The intersection point of each corner point in a set of corner points of the display screen on the browser end with the ground plane of the target 3D scene is determined to obtain a set of intersection points; The coordinates of each intersection point in the set of intersection points are converted from the screen coordinates of the display screen to the three-dimensional scene coordinates of the target three-dimensional scene to obtain the three-dimensional scene coordinates of each intersection point; The current viewpoint range is determined based on the three-dimensional scene coordinates of each intersection point.

8. A point cloud data annotation device, characterized in that, include: The first determining unit is used to determine the target data level that matches the current view range of the browser, wherein the current view range is the view range of the target 3D scene to be displayed on the browser, the scene point cloud data of the target 3D scene is divided into multiple point cloud data layers according to multiple data levels, the point cloud density within each of the multiple point cloud data layers is positively correlated with the data level corresponding to each point cloud data layer, the view range of the browser is negatively correlated with the data level that matches the view range of the browser, and among the multiple point cloud data layers, the point cloud data layer corresponding to the target data level is the target point cloud data layer; The first acquisition unit is used to acquire a set of target point cloud data of a set of consecutive frames of the target three-dimensional scene from the server based on the target data layer, wherein the set of consecutive frames of target point cloud data are the point cloud data in the scene point cloud data of the set of consecutive frames that match the current view range and are located within the target point cloud data layer. The loading unit is used to load the target point cloud data of the set of consecutive frames into the target 3D scene on the browser side for display, so as to perform data annotation on the set of consecutive frames.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

Images