A three-dimensional fitting label frame generation method and device

Abstract

The application relates to a three-dimensional fitting label frame generation method and device, which comprises the following steps: obtaining a first point set of points framed by a three-dimensional label frame of a user; filtering the first point set through the distribution of height values to obtain a second point set; calculating the covariance matrix of the second point set, a plurality of eigenvalues and a plurality of eigenvectors of the covariance matrix, and determining the orientation of the three-dimensional label frame according to the covariance matrix; calculating the projection of all points in the second point set in the orientation of the three-dimensional label frame and the direction of the normal vector of the three-dimensional label frame; and calculating the position of the three-dimensional label frame and the size of three-dimensional directions and generating the three-dimensional label frame based on the projection in the orientation of the three-dimensional label frame and the direction of the normal vector of the three-dimensional label frame and the height distribution of the second point set. The three-dimensional label frame labeling can be completed through a small amount of adjustment, the problem that the adjustment step is time-consuming in the laser point cloud three-dimensional label frame labeling process is solved, and the labeling efficiency of the laser point cloud three-dimensional label frame is improved.

Description

Technical Field

[0001] This invention belongs to the field of laser point cloud data processing technology, specifically relating to a method and apparatus for generating three-dimensional fitting annotation boxes. Background Technology

[0002] In the field of autonomous driving, AI is often used to identify objects such as vehicles and pedestrians from data acquired by sensors such as cameras and radar. Training an AI that can identify objects accurately and quickly typically requires multiple rounds of training using a large amount of ground-value data, most of which is generated through data annotation. Therefore, efficient data annotation can significantly shorten the training time for AI.

[0003] For laser point cloud data generated by LiDAR, the common ground truth data is 3D bounding boxes. Annotating these 3D bounding boxes requires a specialized annotation platform and typically involves three steps: drawing the box, adjusting it, and filling in attributes. The adjustment step has the greatest impact on annotation efficiency and accuracy. Annotators first need to observe and judge the object to be annotated. Once the position, size, and orientation of the object are determined, the 3D bounding box can be translated, scaled, and rotated to align with the object's position, size, and orientation. This series of operations usually takes 20 to 30 seconds, occupying the majority of the annotation time. Therefore, optimizing this process can significantly improve annotation efficiency. Summary of the Invention

[0004] To improve the annotation efficiency of optical point cloud data, a method for generating a three-dimensional bounding box is provided in a first aspect of the present invention, comprising: obtaining a first set of points selected by a user's three-dimensional bounding box; filtering the first set of points by the distribution of height values ​​to obtain a second set of points; calculating the covariance matrix of the second set of points, as well as multiple eigenvalues ​​and multiple eigenvectors of the covariance matrix, and determining the orientation of the three-dimensional bounding box based on the multiple eigenvalues ​​and multiple eigenvectors; calculating the projection of all points in the second set onto the orientation of the three-dimensional bounding box and its normal vector direction; calculating the position and the magnitude of the three-dimensional bounding box in three dimensions based on the projection of all points in the second set onto the orientation of the three-dimensional bounding box and its normal vector direction, and the height distribution of the second set of points; and generating a three-dimensional bounding box based on the position and the magnitude of the three-dimensional bounding box in three dimensions.

[0005] In some embodiments of the present invention, calculating the covariance matrix of the second set of points, and the plurality of eigenvalues ​​and eigenvectors of the covariance matrix, includes:

[0006] Set the height of all points in the second set of points to 0, and calculate the covariance matrix of all points in the second set of points, as well as the three eigenvalues ​​of the covariance matrix and the eigenvector corresponding to each eigenvalue.

[0007] Furthermore, determining the orientation of the 3D bounding box based on the plurality of feature values ​​and the plurality of feature vectors includes: taking the feature vector corresponding to the largest feature value among the three feature values ​​as the orientation of the 3D bounding box.

[0008] In some embodiments of the present invention, calculating the projection of all points in the second set of points onto the orientation and normal vector direction of the three-dimensional annotation box includes: calculating the projection value of the orientation of all points in the second set of points onto the three-dimensional annotation box; rotating the orientation by 90° in the plane containing the length and width directions to obtain the normal vector of the orientation; and calculating the projection value of all points in the second set onto the normal vector of the orientation.

[0009] Furthermore, the step of calculating the position and size of the three-dimensional annotation box in three dimensions based on the projection of all points in the second set onto the orientation and normal vector direction of the three-dimensional annotation box, and the height distribution of the second set of points, includes: calculating the projection value of the orientation of the second set of points and the maximum and minimum values ​​of the orientation normal vector; and calculating the position and size of the three-dimensional annotation box in three dimensions based on the projection value, the maximum and minimum values ​​of the orientation normal vector, and the maximum and minimum values ​​of the height of the second set of points.

[0010] In the above embodiments, the step of filtering the first set of points by the distribution of height values ​​to obtain the second set of points includes: calculating the maximum and minimum height values ​​of the points in the first set of points; dividing the first set of points into multiple intervals based on the maximum and minimum height values; counting the number of points contained in each interval, and taking the median height of the interval containing the most points as the new minimum height value; and selecting points from the first set whose height falls within the interval between the new minimum height value and the original maximum height value to obtain the second set of points.

[0011] A second aspect of the present invention provides a three-dimensional fitting annotation box generation device, comprising: an acquisition module, configured to acquire a first set of points selected by a user's three-dimensional annotation box; filter the first set of points by the distribution of height values ​​to obtain a second set of points; a determination module, configured to calculate the covariance matrix of the second set of points, and a plurality of eigenvalues ​​and a plurality of eigenvectors of the covariance matrix, and determine the orientation of the three-dimensional annotation box based on the plurality of eigenvalues ​​and the plurality of eigenvectors; a first calculation module, configured to calculate the projection of all points in the second set onto the orientation of the three-dimensional annotation box and its normal vector direction; a second calculation module, configured to calculate the position and the magnitude of the three-dimensional annotation box in three dimensions based on the projection of all points in the second set onto the orientation of the three-dimensional annotation box and its normal vector direction, and the height distribution of the second set of points; and a generation module, configured to generate a three-dimensional annotation box based on the position and the magnitude of the three-dimensional annotation box in three dimensions.

[0012] A third aspect of the present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the three-dimensional fitting annotation box generation method provided in the first aspect of the present invention.

[0013] In a fourth aspect, the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the three-dimensional fitting annotation box generation method provided in the first aspect of the present invention.

[0014] The beneficial effects of this invention are:

[0015] Compared to conventional 3D annotation without automatic alignment and 3D annotation without orientation, the 3D alignment annotation generation method proposed in this invention allows users to directly generate 3D annotation boxes with accurate orientation, position, and size simply by dragging the box during the 3D annotation process. This requires little or no adjustment and attribute filling to complete the 3D annotation, reducing the time users spend adjusting the 3D annotation boxes. It solves the problem of time-consuming adjustment steps in the 3D annotation process for laser point clouds, improves the annotation efficiency of 3D annotation boxes for laser point clouds, and facilitates the rapid generation of ground truth data for AI training, thus shortening AI training time. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the basic process of the three-dimensional fitting annotation box generation method in some embodiments of the present invention;

[0017] Figure 2This is a schematic diagram illustrating the specific process of the three-dimensional fitting annotation box generation method in some embodiments of the present invention;

[0018] Figure 3 This is a schematic diagram of the structure of a three-dimensional fitting annotation box generation device in some embodiments of the present invention;

[0019] Figure 4 This is a schematic diagram of the structure of an electronic device in some embodiments of the present invention. Detailed Implementation

[0020] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0021] refer to Figure 1 and Figure 2 In a first aspect of the present invention, a method for generating a three-dimensional fitting annotation box is provided, comprising: S100. obtaining a first set of points selected by a user's three-dimensional annotation box; filtering the first set of points by the distribution of height values ​​to obtain a second set of points; S200. calculating the covariance matrix of the second set of points, and multiple eigenvalues ​​and multiple eigenvectors of the covariance matrix, and determining the orientation of the three-dimensional annotation box based on the multiple eigenvalues ​​and multiple eigenvectors; S300. calculating the projection of all points in the second set onto the orientation of the three-dimensional annotation box and its normal vector direction; S400. calculating the position and the size of the three-dimensional annotation box in three dimensions based on the projection of all points in the second set onto the orientation of the three-dimensional annotation box and its normal vector direction, and the height distribution of the second set of points; S500. generating the three-dimensional annotation box based on the position and the size of the three-dimensional annotation box in three dimensions.

[0022] It should be understood that the above steps can also be equivalent to the following steps: After the user completes the drawing of the 3D annotation box on the annotation platform, obtain the set of points selected by the user, obtain the 3D coordinates of all points in the set, filter the points in the set according to height, and obtain the filtered set of points. Calculate the extreme values ​​of the z-values ​​(height direction or elevation direction) of the filtered set of points, and the absolute value of the difference between the extreme values ​​of z-values ​​is taken as the height of the 3D annotation box. Calculate the covariance matrix of the x and y dimensions of the filtered set of points, and calculate the eigenvalues ​​and eigenvectors of the covariance matrix. Take the eigenvector corresponding to the largest eigenvalue as the orientation of the 3D annotation box, and calculate the vector perpendicular to this vector in the x-y plane as the direction perpendicular to the orientation. Project all points in the filtered set of points into the orientation and the direction perpendicular to the orientation, respectively, and obtain the extreme values ​​of the projection results in the orientation and the direction perpendicular to the orientation, respectively. The absolute value of the difference between the extreme values ​​of the projection results in the orientation direction is taken as the length of the 3D annotation box, and the absolute value of the difference between the extreme values ​​of the projection results in the direction perpendicular to the orientation is taken as the width of the 3D annotation box. Take the average of the extreme values ​​in the x, y, and z directions as the position of the 3D annotation box. The 3D annotation frame can be drawn based on its position, orientation, and dimensions (length, width, and height). This 3D annotation frame is one with accurate orientation, precise position, and matching dimensions.

[0023] In step S100 of this embodiment, filtering the first set of points by the distribution of height values ​​to obtain the second set of points includes: S101. Calculating the maximum and minimum heights of the points in the first set of points; S102. Dividing the first set of points into multiple intervals based on the maximum and minimum heights; S103. Counting the number of points contained in each interval, and taking the median height of the interval containing the most points as the new minimum height; S104. Selecting points from the first set whose heights fall within the interval between the new minimum height and the original maximum height to obtain the second set of points.

[0024] Specifically, the extreme values ​​of the z-coordinates of the user's selected points set are calculated, and the maximum value of the z-coordinate of the selected points set is denoted as z. max The minimum value of the z-coordinate is denoted as z. min Then z max -z min This refers to the z-coordinate interval of the selected point set. Dividing this interval into 100 equal parts, the size of each part is z. size =(z max -z min ) / 100, therefore interval 1 is [z min ,z min +z size ), interval 2 is [z min +z size ,z min +2z sizeThis process continues. The points in the selected set are divided into corresponding intervals based on their z-value ranges, and the number of points in each interval is calculated. Since ground points are almost all on the same plane, the interval with the largest number of points can be considered the ground interval. The midpoint of this interval is taken as the ground height, and z is updated using this value. min , with the new z min Filter the selected point set; points whose z-coordinate is less than the new z-coordinate. min Points below ground level are considered to be below ground and should not be considered in the annotation calculation; therefore, they are filtered out. The remaining set of filtered points represents the z-coordinates in the new z-coordinate system. min To z max Points within the range.

[0025] In step S200 of some embodiments of the present invention, the calculation of the covariance matrix of the second set of points, and the plurality of eigenvalues ​​and eigenvectors of the covariance matrix includes: setting the height of all points in the second set of points to 0, and calculating the covariance matrix of all points in the second set of points, and the three eigenvalues ​​of the covariance matrix and the eigenvector corresponding to each eigenvalue.

[0026] Specifically, the z-coordinates of all points in the filtered point set are set to 0 to eliminate the influence of the z-coordinates on the x and y coordinates during the calculation of the covariance matrix and eigenvalues / eigenvectors. The covariance matrix C of the filtered point set is calculated using the methods shown in Definitions 1, 2, and 3. After obtaining C, all eigenvalues ​​and their corresponding eigenvectors of C are calculated according to Definition 4. Since the number of combinations of eigenvalues ​​and eigenvectors will not exceed the order of the matrix, according to Definition 1, matrix C is a third-order square matrix, therefore C has at most three pairs of eigenvalue and eigenvector combinations. The three obtained eigenvalues ​​are denoted as λ1, λ2, and λ3, and the corresponding eigenvectors are denoted as x1, x2, and x3. According to Definition 4, x1, x2, and x3 are all three-dimensional non-zero column vectors. The maximum value of eigenvalues ​​λ1, λ2, and λ3 is calculated; the eigenvector corresponding to this maximum eigenvalue is the orientation of the three-dimensional annotation box corresponding to the selected point, denoted as V. dir Since the z-coordinates of all points in the filtered point set were set to 0 before the calculation, V dir The z-value is 0, that is, V dir It lies in the x-y plane.

[0027] in: Definition 1;

[0028] cov(X,Y)=E[(XE(Y))(YE(X))]=E(XY)-E(X)E(Y), definition 2;

[0029] Definition 3;

[0030] Let A be an n-order matrix. If a number λ and an n-dimensional non-zero column vector x satisfy Ax = λx, then the number λ is called an eigenvalue of A, and x is called an eigenvector of A corresponding to the eigenvalue λ. (Definition 4)

[0031] d=P·V / V.length, formula 1;

[0032] P·V=P x V x +P y V y +P z V z , formula 2;

[0033]

[0034] Furthermore, determining the orientation of the 3D bounding box based on the plurality of feature values ​​and the plurality of feature vectors includes: taking the feature vector corresponding to the largest feature value among the three feature values ​​as the orientation of the 3D bounding box.

[0035] In step S300 of some embodiments of the present invention, calculating the projection of all points in the second set of points onto the orientation and normal vector direction of the three-dimensional annotation box includes: calculating the projection value of the orientation of all points in the second set of points onto the three-dimensional annotation box; rotating the orientation by 90° in the plane containing the length and width directions to obtain the normal vector of the orientation; and calculating the projection value of all points in the second set onto the normal vector of the orientation.

[0036] Specifically, V dir Rotating the normal vector around the x-y plane, i.e., the z-axis of the point cloud coordinate system, 90° clockwise or counterclockwise, yields a vector perpendicular to V in the x-y plane. dir The vector, denoted as V vert . All points in the filtered point set are respectively directed to V dir and V vert The projection is performed, and the calculation method for the projection is shown in Formulas 1, 2, and 3. In Formula 1, d is the projection result, which is a constant, P is a point in the filtered point set, and V is V dir or V vert .

[0037] Furthermore, in step S400, calculating the position and size of the three-dimensional annotation box in three dimensions based on the projection of all points in the second point set onto the orientation and normal vector direction of the three-dimensional annotation box, and the height distribution of the second point set, includes: calculating the projection value of the orientation of the second point set and the maximum and minimum values ​​of the orientation normal vector; and calculating the position and size of the three-dimensional annotation box in three dimensions based on the projection value, the maximum and minimum values ​​of the orientation normal vector, and the maximum and minimum values ​​of the height of the second point set.

[0038] Specifically, to obtain V dir maximum value of the upper projection dir max and minimum value dir min , and in V vert The maximum value of the upper projection vert max and minimum value vert min The length of the 3D annotation box is length = dir. max -dir min The width is width=vert max -vert min The height is z max -z min The position of the 3D annotation box is

[0039] position=((dir max +dir min ) / 2,(vert max +vert min ) / 2,(z max +z min ) / 2). According to the orientation V dir By specifying the position, length, width, and height, a 3D annotation frame with accurate orientation, position, and size can be drawn.

[0040] refer to Figure 2 In one embodiment of the present invention, the method for generating a three-dimensional fitting annotation box includes the following steps:

[0041] 1. Obtain the set P1 of points selected by the user;

[0042] 2. Calculate the maximum z-coordinate zmax and the minimum z-coordinate zmin in P1;

[0043] 3. Divide zmax and zmin into 100 intervals, calculate the value range corresponding to each interval, and divide all points in P1 into 100 intervals according to the value range of their z coordinates;

[0044] 4. Count the interval with the most points among 100 intervals, and take the median value of that interval as the new zmin;

[0045] 5. Filter the points in P1 to select points whose z-coordinates are within the range of zmax to the new zmin, and generate point set P2;

[0046] 6. Set the z-coordinate of all points in P2 to 0, and calculate the covariance matrix C of all points in P2 and the corresponding eigenvalues ​​a1, a2, a3 and eigenvectors x1, x2, x3;

[0047] 7. Calculate the maximum value amax of a1, a2, and a3, and its corresponding feature vector is dir;

[0048] 8. Rotate dir by 90° in the x-y plane to obtain its normal vector vert;

[0049] 9. Calculate the projection values ​​of all points in P2 onto dir, and calculate the maximum and minimum projection values ​​dirmax and dirmin.

[0050] 10. Calculate the projection values ​​of all points in P2 onto vert, and calculate the maximum and minimum projection values, vertmax and vertmin.

[0051] 11. Calculate the position pos, length l, width w, and height h of the 3D annotation box based on zmax, zmin, dirmax, dirmin, vertmax, and vertmin.

[0052] Draw a 3D annotation frame based on the orientation dir, position pos, length l, width w, and height h.

[0053] Example 2

[0054] refer to Figure 3In a second aspect, the present invention provides a three-dimensional fitting annotation box generation device 1, comprising: an acquisition module 11, configured to acquire a first set of points selected by a user's three-dimensional annotation box; filter the first set of points by the distribution of height values ​​to obtain a second set of points; a determination module 12, configured to calculate the covariance matrix of the second set of points, as well as multiple eigenvalues ​​and multiple eigenvectors of the covariance matrix, and determine the orientation of the three-dimensional annotation box based on the multiple eigenvalues ​​and multiple eigenvectors; a first calculation module 13, configured to calculate the projection of all points in the second set onto the orientation of the three-dimensional annotation box and its normal vector direction; a second calculation module 14, configured to calculate the position and the size of the three-dimensional annotation box in three dimensions based on the projection of all points in the second set onto the orientation of the three-dimensional annotation box and its normal vector direction, and the height distribution of the second set of points; and a generation module 15, configured to generate a three-dimensional annotation box based on the position and the size of the three-dimensional annotation box in three dimensions.

[0055] Furthermore, the determining module 12 includes: a first calculation unit for calculating the projection values ​​of the orientation of all points in the second set of points onto the three-dimensional annotation box; a rotation unit for rotating the orientation by 90° in the plane containing the length and width directions to obtain the normal vector of the orientation; and a second calculation unit for calculating the projection values ​​of all points in the second set of points onto the normal vector of the orientation.

[0056] Example 3

[0057] refer to Figure 4 In a third aspect, the present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the three-dimensional fitting annotation box generation method of the first aspect of the present invention.

[0058] Electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0059] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, hard disks; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 4 Each box shown can represent a device or multiple devices as needed.

[0060] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by a processing device 501, it performs the functions defined in the methods of embodiments of this disclosure. It should be noted that the computer-readable medium described in embodiments of this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0061] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more computer programs, which, when executed by the electronic device, cause the electronic device to:

[0062] Computer program code for performing the operations of embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, C++, and Python—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0063] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0064] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for generating a three-dimensional fitting annotation box, characterized in that, include: Get the first set of points selected by the user's 3D annotation box; By filtering the first set of points based on the distribution of height values, a second set of points is obtained: The maximum and minimum height values ​​of the points in the first set are calculated; based on the maximum and minimum height values, the first set of points is divided into multiple intervals; the number of points contained in each interval is counted, and the median height of the interval containing the most points is taken as the new minimum height; points in the first set whose height falls within the interval between the new minimum height and the original maximum height are selected to obtain the second set of points. Calculate the covariance matrix of the second set of points, as well as multiple eigenvalues ​​and multiple eigenvectors of the covariance matrix, and determine the orientation of the 3D annotation box based on the multiple eigenvalues ​​and multiple eigenvectors; The calculation of the covariance matrix of the second set of points, and the multiple eigenvalues ​​and multiple eigenvectors of the covariance matrix, includes: setting the height of all points in the second set of points to 0, and calculating the covariance matrix of all points in the second set of points, as well as the three eigenvalues ​​of the covariance matrix and the eigenvector corresponding to each eigenvalue; Calculate the projection of all points in the second set onto the orientation of the 3D annotation box and the direction of its normal vector; Based on the orientation of all points in the second set of points in the 3D annotation box and their projections in the direction of their normal vectors, as well as the height distribution of the second set of points, calculate the position and the size of the three-dimensional annotation box in three dimensions. A three-dimensional annotation box is generated based on the position and size of the three-dimensional annotation box in the three dimensions. 2.The method of claim 1, wherein, The process of determining the orientation of the 3D bounding box based on the plurality of feature values ​​and the plurality of feature vectors includes: The eigenvector corresponding to the largest eigenvalue among the three eigenvalues ​​is used as the orientation of the 3D bounding box. 3.The method of claim 1, wherein, The calculation of the projection of all points in the second set onto the orientation and normal direction of the 3D annotation box includes: Calculate the projection values ​​of the orientation of all points in the second set of points onto the 3D annotation box; Rotate the orientation by 90° in the plane containing the length and width directions to obtain the normal vector of the orientation; Calculate the projection values ​​of all points in the second set onto the orientation normal vector.

4. The three-dimensional fitting bounding box generation method of claim 3, wherein, The step of calculating the position and magnitude of the three-dimensional annotation box in three dimensions based on the projection of all points in the second set onto the orientation of the 3D annotation box and its normal vector direction, as well as the height distribution of the second set of points, includes: Calculate the maximum and minimum values ​​of the projection values ​​of the second set of points and the normal vector of the orientation. Calculate the position of the 3D annotation box and the size of its three-dimensional directions based on the maximum and minimum values ​​of the projection value and the orientation normal vector, as well as the maximum and minimum values ​​of the height of the second set of points. 5.A three-dimensional fitting bounding box generation device, characterized in that, include: The acquisition module is used to acquire the first set of points selected by the user's 3D annotation box; By filtering the first set of points based on the distribution of height values, a second set of points is obtained; The determination module is used to calculate the covariance matrix of the second set of points, as well as multiple eigenvalues ​​and multiple eigenvectors of the covariance matrix, and determine the orientation of the three-dimensional annotation box based on the multiple eigenvalues ​​and multiple eigenvectors. The first calculation module is used to calculate the projection of all points in the second point set onto the orientation of the 3D annotation box and the direction of its normal vector. The second calculation module is used to calculate the position and the size of the three-dimensional annotation box in three dimensions based on the projection of all points in the second point set onto the orientation of the three-dimensional annotation box and its normal vector direction, as well as the height distribution of the second point set. The generation module is used to generate a three-dimensional annotation box based on the position and size of the three-dimensional annotation box in three dimensions. 6.The three-dimensional bounding box generation device of claim 5, wherein, The first computing module includes: The first calculation unit is used to calculate the projection values ​​of the orientation of all points in the second set of points onto the 3D annotation box; A rotation unit is used to rotate the orientation by 90° in the plane containing the length and width directions to obtain the normal vector of the orientation; The second calculation unit is used to calculate the projection values ​​of all points in the second set onto the orientation normal vector.

7. An electronic device, comprising: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the three-dimensional fitting annotation box generation method as described in any one of claims 1 to 4.

8. A computer readable medium having stored thereon a computer program, wherein, When the computer program is executed by the processor, it implements the three-dimensional fitting annotation box generation method as described in any one of claims 1 to 4.

Images