Real-time marking method and system for point cloud data based on Web browser and medium

Abstract

The invention discloses a point cloud data real-time labeling method and system based on a Web browser and a medium. The method comprises the following steps of: loading a preview file at a Web front end by using a Potree library; wherein the preview file is a point cloud file to be marked in a Potree format; performing front-end labeling on the point cloud data in the preview file through a prefabricated Potree labeling tool to obtain a point cloud range; wherein the prefabricated Potree labeling tool is used for cutting the point cloud data for multiple times, and determining a cut label; and sending the point cloud range to a back-end service, so that the back-end service adds a labeling field for the point cloud data through the point cloud range to obtain a labeled point cloud file. According to the embodiment of the invention, the point cloud data can be labeled at the Web front end by prefabricating the Potree labeling tool, the labeling complexity can be reduced, and the loading efficiency can be improved. The method can be widely applied to the technical field of computers.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, system and medium for real-time annotation of point cloud data based on a Web browser. Background Art

[0002] In the related technology, point cloud data is usually displayed and annotated in real time on the web page through the following methods: Based on Three.js, point cloud data in PCD format is loaded locally or on the Internet. When processing files with a large number of points in the point cloud, this method takes a long time to load and the front-end page will freeze. Based on commercial front-end frameworks such as Cesium 3D GIS Web front-end framework or ArcGIS Javascript SDK, point cloud data that has been published as an online service is loaded; this method cannot annotate point cloud data on the Web front-end. When you need to preview the annotated point cloud data, you need to manually publish the point cloud data as a point cloud service again; there is a problem of complex annotation process. Summary of the invention

[0003] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0004] To this end, the purpose of the present invention is to provide an efficient method, system and medium for real-time annotation of point cloud data based on a Web browser.

[0005] In order to achieve the above technical objectives, one aspect of an embodiment of the present invention provides a method for real-time annotation of point cloud data based on a Web browser, comprising the following steps: using the Potree library to load a preview file on the Web front end; wherein the preview file is a point cloud file to be annotated in the Potree format; performing front-end annotation of the point cloud data in the preview file using a prefabricated Potree annotation tool to obtain a point cloud range; wherein the prefabricated Potree annotation tool is used to perform multiple cropping of the point cloud data and determine the cropped labels; the point cloud range is sent to the back-end service so that the back-end service adds annotation fields to the point cloud data through the point cloud range to obtain an annotated point cloud file. The embodiment of the present application implements annotation of point cloud data on the Web front end through the prefabricated Potree annotation tool, which is conducive to reducing the complexity of annotation and improving loading efficiency.

[0006] In some embodiments, in the method for real-time annotation of point cloud data based on a Web browser of an embodiment of the present invention, the prefabricated Potree annotation tool is determined by the following steps:

[0007] Increase the maximum number of nodes for drawing polygons in Potree;

[0008] After selecting the circle points and classification categories, the clipping parameters are passed to the vertex shader of WebGL for rendering; the clipping parameters include: label index, polygon index, clipping type, polygon node coordinate sequence, and color field value.

[0009] In some embodiments, in one embodiment of the present invention, the vertex shader performs rendering by the following steps:

[0010] Obtain the current point cloud point; determine whether the point cloud point needs to be drawn according to the clipping parameters;

[0011] If the point cloud point needs to be drawn, determining whether the point cloud point is displayed in the viewport according to the spatial relationship between the point cloud point and the polygon node coordinate sequence;

[0012] If the point cloud point needs to be displayed in the viewport, determine the color value of the point cloud point marked.

[0013] In some embodiments, in one embodiment of the present invention, the point cloud data in the preview file is annotated by a prefabricated Potree annotation tool to obtain a point cloud range, including:

[0014] Display the point cloud preview interface;

[0015] Based on the box selection point cloud operation and the closing operation received on the point cloud preview interface, determine the point cloud range and display the box selection interface;

[0016] Based on the selection operation received in the box selection interface, determining the point cloud to be retained, and displaying the point cloud to be retained in the point cloud preview interface;

[0017] Return to the step of determining the point cloud range based on the box selection point cloud operation and the closing operation received in the point cloud preview interface, and displaying the box selection interface, and update the point cloud range.

[0018] In some embodiments, in one embodiment of the present invention, the backend service performs point cloud annotation through the following steps:

[0019] According to the spatial relationship between the points in the point cloud and the clipping polygon, the point cloud to be annotated is determined;

[0020] Add annotation fields to the point cloud to be annotated to obtain an annotated point cloud file.

[0021] In some embodiments, in one embodiment of the present invention, the method further comprises:

[0022] The backend service uses Potree Converter to convert the point cloud file into a preview file.

[0023] In some embodiments, in one embodiment of the present invention, the method further comprises:

[0024] The web front end obtains page refresh information through HTTP polling and loads the preview file;

[0025] Alternatively, the Web front end obtains page refresh information through WebSocket and loads the preview file.

[0026] On the other hand, an embodiment of the present invention proposes a point cloud data real-time annotation system based on a Web browser, comprising:

[0027] The first module is used to load the preview file using the Potree library on the Web front end; wherein the preview file is a point cloud file to be annotated in the Potree format;

[0028] The second module is used to perform front-end annotation on the point cloud data in the preview file by using a prefabricated Potree annotation tool to obtain a point cloud range; wherein the prefabricated Potree annotation tool is used to perform multiple cropping of the point cloud data and determine the cropped labels;

[0029] The third module is used to send the point cloud range to the backend service, so that the backend service adds annotation fields to the point cloud data through the point cloud range to obtain the annotated point cloud file.

[0030] On the other hand, an embodiment of the present invention provides a point cloud data real-time annotation device based on a Web browser, comprising:

[0031] at least one processor;

[0032] at least one memory for storing at least one program;

[0033] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned point cloud data real-time annotation method based on a Web browser.

[0034] On the other hand, an embodiment of the present invention provides a storage medium, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to implement the above-mentioned real-time annotation method of point cloud data based on a Web browser.

[0035] The embodiments of the present application include at least the following beneficial effects: The method provided by the embodiments of the present invention includes: using the Potree library to load a preview file on the Web front end; wherein the preview file is a point cloud file to be annotated in the Potree format; performing front-end annotation of the point cloud data in the preview file by a prefabricated Potree annotation tool to obtain a point cloud range; wherein the prefabricated Potree annotation tool is used to perform multiple cropping of the point cloud data and determine the cropped labels; sending the point cloud range to the back-end service so that the back-end service adds annotation fields to the point cloud data through the point cloud range to obtain an annotated point cloud file. The embodiments of the present application implement annotation of point cloud data on the Web front end through the prefabricated Potree annotation tool, which is conducive to reducing the complexity of annotation and improving loading efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0037] Figure 1 A schematic diagram of a flow chart of an embodiment of a method for real-time annotation of point cloud data based on a Web browser provided by the present invention;

[0038] Figure 2 A schematic flow chart of another embodiment of the method for real-time annotation of point cloud data based on a Web browser provided by the present invention;

[0039] Figure 3 This is an interface display diagram of an embodiment of the point cloud preview interface provided by this application;

[0040] Figure 4 An interface display diagram of an embodiment of the box selection interface provided by this application;

[0041] Figure 5 An interface display diagram of an embodiment of the point cloud range provided by this application;

[0042] Figure 6 Interface display diagram of another embodiment of the point cloud range provided by this application

[0043] Figure 7 An interface display diagram of an embodiment determined by the classification category provided by this application;

[0044] Figure 8 A schematic diagram of a scenario of an embodiment of saving a preview file provided by the present application;

[0045] Fig. 9 An interface display diagram of an embodiment of loading the front end of the preview file provided by this application;

[0046] Fig.10 An interface display diagram of an embodiment of front-end loading of annotated preview files provided by the present application;

[0047] Fig.11 A schematic diagram of the structure of an embodiment of a point cloud data real-time annotation system based on a Web browser provided by the present invention;

[0048] Fig.12 A schematic structural diagram of an embodiment of a point cloud data real-time annotation device based on a Web browser provided by the present invention. DETAILED DESCRIPTION

[0049] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0050] Currently, there are several ways to display and annotate point cloud data in real time on a web page:

[0051] 1. Load point cloud data in PCD format locally or on the Internet based on Three.js.

[0052] The point cloud files that need to be previewed in this way do not need to be pre-published as services or generate preview files. They can be directly uploaded in a web browser or previewed using an Internet network path. The disadvantage is that only point cloud files with small file sizes and a small number of points in the point cloud can be loaded. When the number of points in the point cloud file is greater than 100,000 points, it takes a long time to load the data when the user previews the point cloud, and the front-end page will be obviously stuck or unable to respond to the user's mouse operation.

[0053] When annotating point clouds in real time, you can use WebGL shaders to annotate the front-end point clouds and modify the categories corresponding to the point cloud point objects. At this time, you can generate a point cloud file with annotated category scalar fields on the server backend corresponding to the Web page.

[0054] 2. Use commercial front-end frameworks such as Cesium 3D GIS Web front-end framework or ArcGIS Javascript SDK to load point cloud data that has been published as an online service.

[0055] When publishing point cloud services in this way, the publishing service will pre-process the point cloud data according to the LOD display level and generate slice data of different LOD display levels. When browsing the point cloud, the Web browser will calculate the point cloud slices that need to be loaded according to the viewport and field of view parameters of the 3D camera in the current scene. This method loads point clouds quickly, has less delay when browsing point cloud data, and has better operation fluency. However, this method cannot annotate point cloud data on the Web front end. When you need to preview the annotated point cloud data, you need to manually publish the point cloud data as a point cloud service again, or use ArcObject or ArcGIS SOE and other methods to update and publish the point cloud data through the Web API. These operations require the purchase of ArcGIS and other software licenses.

[0056] 3. Use Potree Converter to create a point cloud preview file in Potree format, and use the Potree front-end Javascript library to load the point cloud data.

[0057] In this method, Potree is a dedicated Javascript library for point cloud Web front-end display built on Three.js. It contains three files: metadata.json, hierarchy.bin, and octree.bin. These three files can be generated by converting the point cloud file in las format through Potree Converter. During the conversion, the point cloud will be downsampled according to the octree hierarchy according to the spatial octree division method. The octree hierarchy information and the corresponding point cloud data offset value in octree.bin are recorded in the hierarchy.bin file, and octree.bin records the point cloud data of each level of the octree. The metadata.json file records the basic information of the point cloud file, such as bounding box coordinates, field information, etc. When browsing using Potree, the data loading method is the same as the second method. Only the point cloud data that needs to be displayed is loaded, and the data that does not need to be loaded does not need to be loaded. However, the point cloud data loaded by Potree will query the point cloud binary stream that needs to be displayed from octree.bin instead of the point cloud slice data. Potree contains annotations for point cloud data using an external json5 file format. The json5 file only records a spatial point and its annotation title and description information. Potree does not have the function of re-annotating part of the point cloud data as a certain category.

[0058] After the point cloud data is updated, you need to use Potree Converter again to create a preview file in Potree format.

[0059] The following describes in detail a method and system for real-time annotation of point cloud data based on a Web browser according to an embodiment of the present invention with reference to the accompanying drawings. First, the method for real-time annotation of point cloud data based on a Web browser according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0060] Reference Figure 1 In an embodiment of the present invention, a method for real-time annotation of point cloud data based on a Web browser is provided. The method for real-time annotation of point cloud data based on a Web browser in an embodiment of the present invention can be applied to a terminal, a server, or software running in a terminal or a server. The terminal can be a tablet computer, a laptop computer, a desktop computer, etc., but is not limited to this. The server can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The method for real-time annotation of point cloud data based on a Web browser in an embodiment of the present invention mainly includes the following steps:

[0061] S100: Using the Potree library to load a preview file on the Web front end; wherein the preview file is a point cloud file to be annotated in the Potree format;

[0062] S200: front-end annotation is performed on the point cloud data in the preview file by using a prefabricated Potree annotation tool to obtain a point cloud range; wherein the prefabricated Potree annotation tool is used to perform multiple cropping of the point cloud data and determine the cropped labels;

[0063] S300: Sending the point cloud range to a backend service, so that the backend service adds a labeling field to the point cloud data through the point cloud range to obtain a labeled point cloud file.

[0064] In some possible implementations, reference Figure 2The present invention realizes a method of real-time annotation of point cloud data based on a Web browser, uses Potree Converter and a Potree framework with modified source code to perform front-end annotation of point cloud, and submits the annotated point cloud range to the back-end service. The back-end service annotates the original point cloud file according to the annotated point cloud range, and calls Potree Converter to convert the point cloud file with annotation information into Potree format. After the conversion is completed, the Web browser automatically refreshes and loads. This method has the characteristics of fast loading speed, smooth browsing, real-time annotation of point cloud in a Web browser, and real-time preview of annotation results. Back-end development does not need to rely on commercial authorizations such as ArcGIS or Cesium three-dimensional GIS frameworks. It can be understood that the present application uses a prefabricated Potree annotation tool to perform front-end annotation of the point cloud data in the preview file, and crops the point cloud data to obtain the cropped point cloud range. The back-end service adds annotation fields to the point cloud data through the point cloud range, and after obtaining the annotated point cloud file, the preview file is loaded through the Web front end to display the annotated point cloud file.

[0065] Optionally, in one embodiment of the present invention, the prefabricated Potree annotation tool is determined by the following steps:

[0066] Increase the maximum number of nodes for drawing polygons in Potree;

[0067] After selecting the circle points and classification categories, the clipping parameters are passed to the vertex shader of WebGL for rendering; the clipping parameters include: label index, polygon index, clipping type, polygon node coordinate sequence, and color field value.

[0068] Optionally, in one embodiment of the present invention, the vertex shader performs rendering through the following steps:

[0069] Obtain the current point cloud point; determine whether the point cloud point needs to be drawn according to the clipping parameters;

[0070] If the point cloud point needs to be drawn, determining whether the point cloud point is displayed in the viewport according to the spatial relationship between the point cloud point and the polygon node coordinate sequence;

[0071] If the point cloud point needs to be displayed in the viewport, determine the color value of the point cloud point marked.

[0072] In the present invention, the polygon clipping tool of Potreee is modified to realize the real-time annotation and rendering of point cloud data on the Web front end. The polygon clipping tool can be used to perform multiple clippings, gradually reduce the number of point cloud points of the point cloud object to be annotated, and select the label value of the point cloud set. The modification method is as follows:

[0073] 1. Modify the maximum number of nodes in Potree and increase the maximum number of nodes to 18 to meet the needs of clipping complex objects.

[0074] 2. After circling the points and selecting the classification category, pass the relevant clipping parameters to the WebGL VertexShader graphics rendering method. The parameters passed are:

[0075] (1) labelIndex, which shows the number of labels, such as the i-th label. A label can contain 1 to N spatial clipping polygons;

[0076] (2) polygonIndex, which shows the operation step of the nth annotation for this clipping, such as the jth clipping step of the i-th annotation. Each operation step will generate a spatial clipping polygon.

[0077] (3) clipType, which shows the way to retain points in this clipping, such as whether to retain points inside the clipping polygon or outside the clipping polygon. All spatial clipping polygons contain this field, which is used to gradually narrow the range of points that need to be marked.

[0078] (4) The node coordinate XYZ value sequence of the spatial polygon;

[0079] (5) red, displays the value of the red color field of the nth annotation;

[0080] (6) green, displays the value of the green color field of the nth annotation;

[0081] (7)blue, displays the value of the blue color field of the nth annotation.

[0082] When VertexShader renders each point cloud point, it first determines whether the point needs to be drawn. When drawing, it uses the array of 7 types of parameters passed previously to determine the spatial relationship between each point and the clipping polygon array to determine whether each point should be displayed in the 3D viewport. If it should be displayed, the point is given a marked color value.

[0083] Optionally, in one embodiment of the present invention, the point cloud data in the preview file is annotated by a prefabricated Potree annotation tool to obtain a point cloud range, including:

[0084] Display the point cloud preview interface;

[0085] Based on the box selection point cloud operation and the closing operation received on the point cloud preview interface, determine the point cloud range and display the box selection interface;

[0086] Based on the selection operation received in the box selection interface, determining the point cloud to be retained, and displaying the point cloud to be retained in the point cloud preview interface;

[0087] Return to the step of determining the point cloud range based on the box selection point cloud operation and the closing operation received in the point cloud preview interface, and displaying the box selection interface, and update the point cloud range.

[0088] In some possible implementations, the main steps for performing real-time point cloud annotation in a web browser are as follows:

[0089] (1)Reference Figure 3 As shown, in the point cloud preview interface, use the custom polygon annotation tool to draw a spatial polygon by clicking the left mouse button to select the point cloud (i.e., the point cloud selection operation), and right click the mouse to close the spatial polygon (i.e., the closing operation). Of course, it can be understood that the point cloud selection operation and the closing operation can be implemented through other operation forms, and this application does not specifically limit them.

[0090] (2) In the pop-up dialog box (i.e., the box selection interface), select "Keep points inside the polygon (Inside)" or "Discard points inside the polygon (Outside)" to decide which points to keep. Figure 4 As shown, define the type value of "keep points within the polygon" as 0, and define "discard points within the polygon" as 1 to retain and display the point cloud. This step is one step. Figure 4 The embodiment is the result of selecting the point cloud within the polygon after the first step of polygon selection. Figure 5 As shown, after adjusting the viewing direction with the mouse, you can continue to select the points to be retained or discarded by cropping.

[0091] If you choose to keep the points within the polygon, the clipping parameters are as follows, and the polygon field is a list of node coordinates of the spatial polygon.

[0092]

[0093] (3) Repeat the first step until the point cloud points that need to be marked are retained. Figure 5 The protruding parts on the left and right ends of the middle pipe are cut and formed Figure 6 The cropped result is shown.

[0094] (4) Select the classification type of the last retained point cloud, such as Figure 7 Select its label value classtype as PipelineCombine and its value as 25.

[0095] Optionally, in one embodiment of the present invention, the backend service performs point cloud annotation through the following steps:

[0096] According to the spatial relationship between the points in the point cloud and the clipping polygon, the point cloud to be annotated is determined;

[0097] Add annotation fields to the point cloud to be annotated to obtain an annotated point cloud file.

[0098] In some possible implementations, after the backend service obtains the annotated point cloud file, it uses PotreeConverter to convert the point cloud file into a preview file; then, the Web frontend uses the Potree library to load the preview file.

[0099] Optionally, in one embodiment of the present invention, the method further comprises:

[0100] The backend service uses Potree Converter to convert the point cloud file into a preview file.

[0101] Optionally, in one embodiment of the present invention, the method further comprises:

[0102] The web front end obtains page refresh information through HTTP polling and loads the preview file;

[0103] Alternatively, the Web front end obtains page refresh information through WebSocket and loads the preview file.

[0104] The method provided by the present application is described in detail below with a specific embodiment, which specifically includes the steps of:

[0105] First, determine whether the point cloud file uploaded to the Web server has been generated as a point cloud file. If not, execute step A; if it has been generated, execute step B.

[0106] A. Reference Figure 8 As shown, use Potree Converter to convert the point cloud file into a preview file in Potree format, and save the preview file on the HTTP server.

[0107] B. The Web front end uses the Potree library to load the Potree format preview file, refer to Fig. 9 shown.

[0108] C. The user uses the customized Potree annotation tool on the Web front end to annotate. Potree's original polygon clipping can only be clipped once, not multiple times, and its clipping function cannot be used for point cloud annotation. Its annotation function only marks a point in three-dimensional space and its text description on the point cloud. Its original built-in clipping tool can only clip once and display a clipping result. The clipping result can only be viewed on the web page and cannot be used for other purposes. When using polygon clipping, the polygon drawn by the mouse can only use up to 8 nodes. At the 8th node, the polygon will automatically close.

[0109] D. After the marking is completed, the Web front end sends the marking information to the back end service.

[0110] The final submitted data is shown below, where there are two spatial polygon annotation step information in steps. The first step is to retain the points within the spatial polygon, and the second step is to discard the points within the spatial polygon.

[0111]

[0112]

[0113]

[0114] E. The backend service adds / modifies annotation field values ​​to the point cloud data according to the annotation information, and then saves it as a new point cloud file.

[0115] For example, you can use the self-written LidarManualClassify Linux command (manual classification of point clouds) to annotate point clouds. LidarManualClassify is a point cloud annotation command compiled by adding functional codes based on the Point Cloud Library C++ library. It mainly generates new annotated point cloud files from the data sent by the Web front end. It filters out the annotated points by judging the spatial relationship between the points in the point cloud and the spatial clipping polygons, and adds or modifies the annotation classification field values ​​for these points. It has the following parameters:

[0116] “-i” specifies the origin point cloud file that needs to be annotated.

[0117] “-o” specifies the new point cloud file after annotation.

[0118] "--param" specifies the annotation information sent by the Web front end. LidarManualClassify also adds the specified classification value to the specified point cloud in the same way as filtering the required point cloud points by spatial polygons. Point cloud points that are not assigned classification values ​​will be assigned classification values ​​of 0 (if the point cloud file has not been annotated before) or the original classification value (if the point cloud file has been assigned classification value 0 or the classification value of the previous annotation).

[0119] The command call example is as follows:

[0120] / usr / local / bin / LidarManualClassify

[0121] -i / app / source.las

[0122] --param="{\"manual\":[{\"classtype\":25,\"steps\":[{\"type\":0,\"polygon\":[{\"x\":201146.77792743265,\"y\": 2488376.0667338474,\"z\":257.6004469558181},{\"x\":201216.01054112884,\"y\":2488355.2928754976,\"z\":255.536 97639059766},{\"x\":201168.94736163938,\"y\":2488209.8427423327,\"z\":201.96928051747543},{\"x\":201085.2156 851653,\"y\":2488254.3182609966,\"z\":211.1310898270541},{\"x\":201147.069811476,\"y\":2488376.244004501,\"z\ ":257.68298577842694}]},{\"type\":1,\"polygon\":[{\"x\":201113.48550045915,\"y\":2488484.2823817646,\"z\":19 0.53654743629278},{\"x\":201091.4067861979,\"y\":2488403.8610067,\"z\":201.70192495821556},{\"x\":201034.5272 8157534,\"y\":2488411.9093756536,\"z\":151.7076972481136},{\"x\":201065.37059937385,\"y\":2488502.5793731166 ,\"z\":144.04191566589796},{\"x\":201113.75449784528,\"y\":2488484.435034568,\"z\":190.70319486199315}]}]}]}"

[0123] -o / app / dest.las

[0124] After the LidarManualClassify command generates a new annotated point cloud file, it will automatically start executing step A to generate the Potree preview and file of the point cloud.

[0125] Finally, the Web front end obtains the page refresh information through HTTP polling or WebSocket, prompting the user that the annotation is complete, and refreshes the page to preview the annotated point cloud file. Fig.10 Refreshes the page content with annotated point cloud preview file for the portion shown.

[0126] The method for real-time annotation of point cloud data provided by the present invention can annotate point clouds in real time on the Web front end to classify point clouds, generate annotation results in real time, and quickly preview the annotation results. It has the advantages of fast point cloud loading and smooth annotation process, and can quickly obtain the annotation result point cloud file and preview it.

[0127] The present invention provides a point cloud real-time annotation process based on a Web browser and a point cloud annotation process based on a Potree polygon clipping method, which has the following advantages:

[0128] 1. No need to install desktop software, point cloud annotation can be performed in a web browser.

[0129] 2. Point cloud data loads quickly, and the full picture of the point cloud can be seen within seconds.

[0130] 3. The labeling process is concise, which facilitates the classification and labeling of point clouds.

[0131] 4. The annotation results can be obtained quickly, and the annotation result point cloud file can be obtained quickly and previewed.

[0132] In summary, the method provided by the present application includes: using the Potree library to load a preview file on the Web front end; wherein the preview file is a point cloud file to be annotated in the Potree format; performing front-end annotation of the point cloud data in the preview file through a prefabricated Potree annotation tool to obtain a point cloud range; wherein the prefabricated Potree annotation tool is used to perform multiple cropping of the point cloud data and determine the cropped labels; sending the point cloud range to the back-end service so that the back-end service adds annotation fields to the point cloud data through the point cloud range to obtain an annotated point cloud file. The embodiment of the present application realizes the annotation of point cloud data on the Web front end through the prefabricated Potree annotation tool, which is conducive to reducing the complexity of annotation and improving loading efficiency.

[0133] Secondly, refer to the attached Fig.11 A point cloud data real-time annotation system based on a Web browser is described according to an embodiment of the present invention.

[0134] Fig.11 1 is a schematic diagram of a real-time annotation system for point cloud data based on a Web browser according to an embodiment of the present invention. The system specifically includes:

[0135] The first module 110 is used to load a preview file using the Potree library on the Web front end; wherein the preview file is a point cloud file to be annotated in the Potree format;

[0136] The second module 120 is used to perform front-end annotation on the point cloud data in the preview file by using a prefabricated Potree annotation tool to obtain a point cloud range; wherein the prefabricated Potree annotation tool is used to perform multiple cropping on the point cloud data and determine the cropped labels;

[0137] The third module 130 is used to send the point cloud range to the backend service, so that the backend service adds a labeling field to the point cloud data through the point cloud range to obtain a labeled point cloud file.

[0138] It can be seen that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0139] Reference Fig.12 The embodiment of the present invention provides a point cloud data real-time annotation device based on a Web browser, comprising:

[0140] at least one processor 210;

[0141] At least one memory 220, used to store at least one program;

[0142] When the at least one program is executed by the at least one processor 210, the at least one processor 210 implements the point cloud data real-time annotation method based on a Web browser.

[0143] Similarly, the contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0144] An embodiment of the present invention further provides a computer-readable storage medium, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to execute the above-mentioned real-time annotation method of point cloud data based on a Web browser.

[0145] Similarly, the contents of the above method embodiments are all applicable to the present storage medium embodiments. The functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0146] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0147] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0148] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several programs to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0149] The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable programs for implementing the logical functions, and may be embodied in any computer-readable medium for use by a program execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch and execute a program from a program execution system, device or apparatus), or in conjunction with such program execution systems, devices or apparatuses. For purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by a program execution system, device or apparatus, or in conjunction with such program execution systems, devices or apparatuses.

[0150] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0151] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable program execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0152] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0153] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0154] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A method for real-time annotation of point cloud data based on a Web browser, characterized in that: The following steps are involved: Use the Potree library to load the preview file on the Web front end; wherein the preview file is a point cloud file to be annotated in the Potree format; Performing front-end annotation on the point cloud data in the preview file by using a prefabricated Potree annotation tool to obtain a point cloud range; wherein the prefabricated Potree annotation tool is used to perform multiple cropping on the point cloud data and determine the cropped labels; The point cloud range is sent to a backend service, so that the backend service adds a labeling field to the point cloud data through the point cloud range to obtain a labeled point cloud file.

2. The method for real-time annotation of point cloud data based on a Web browser according to claim 1, characterized in that: The prefabricated Potree annotation tool is determined by the following steps: Increase the maximum number of nodes for drawing polygons in Potree; After selecting the circle points and classification categories, the clipping parameters are passed to the vertex shader of WebGL for rendering; the clipping parameters include: label index, polygon index, clipping type, polygon node coordinate sequence, and color field value.

3. The method for real-time annotation of point cloud data based on a Web browser according to claim 2, characterized in that: The vertex shader performs rendering through the following steps: Obtain the current point cloud point; determine whether the point cloud point needs to be drawn according to the clipping parameters; If the point cloud point needs to be drawn, determining whether the point cloud point is displayed in the viewport according to the spatial relationship between the point cloud point and the polygon node coordinate sequence; If the point cloud point needs to be displayed in the viewport, determine the color value of the point cloud point marked.

4. The method for real-time annotation of point cloud data based on a Web browser according to claim 1, characterized in that: The point cloud data in the preview file is annotated by the prefabricated Potree annotation tool to obtain the point cloud range, including: Display the point cloud preview interface; Based on the box selection point cloud operation and the closing operation received on the point cloud preview interface, determine the point cloud range and display the box selection interface; Based on the selection operation received in the box selection interface, determining the point cloud to be retained, and displaying the point cloud to be retained in the point cloud preview interface; Return to the step of determining the point cloud range based on the box selection point cloud operation and the closing operation received in the point cloud preview interface, and displaying the box selection interface, and update the point cloud range.

5. The method for real-time annotation of point cloud data based on a Web browser according to claim 1, characterized in that: The backend service performs point cloud annotation through the following steps: According to the spatial relationship between the points in the point cloud and the clipping polygon, the point cloud to be annotated is determined; Add annotation fields to the point cloud to be annotated to obtain an annotated point cloud file.

6. The method for real-time annotation of point cloud data based on a Web browser according to claim 1, characterized in that: The method further comprises: The web front end obtains page refresh information through HTTP polling and loads the preview file; Alternatively, the Web front end obtains page refresh information through WebSocket and loads the preview file.

7. The method for real-time annotation of point cloud data based on a Web browser according to claim 1, characterized in that: The method further comprises: The backend service uses Potree Converter to convert the point cloud file into a preview file.

8. A point cloud data real-time annotation system based on a Web browser, characterized in that: include: The first module is used to load the preview file using the Potree library on the Web front end; wherein the preview file is a point cloud file to be annotated in the Potree format; The second module is used to perform front-end annotation on the point cloud data in the preview file by using a prefabricated Potree annotation tool to obtain a point cloud range; wherein the prefabricated Potree annotation tool is used to perform multiple cropping of the point cloud data and determine the cropped labels; The third module is used to send the point cloud range to the backend service, so that the backend service adds annotation fields to the point cloud data through the point cloud range to obtain the annotated point cloud file.

9. A point cloud data real-time annotation device based on a Web browser, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the point cloud data real-time annotation method based on a Web browser as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the real-time annotation method of point cloud data based on a Web browser as described in any one of claims 1 to 7 when executed by the processor.

Images