Target bounding box adjustment method and device, electronic equipment and storage medium
By parsing and registering point cloud frames, the target bounding box is automatically adjusted, solving the problems of low efficiency and difficulty in guaranteeing quality in manual adjustment, and achieving efficient and accurate bounding box adjustment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING BAIDU NETCOM SCI & TECH CO LTD
- Filing Date
- 2023-12-20
- Publication Date
- 2026-08-04
AI Technical Summary
In existing technologies, manually adjusting the target bounding box during point cloud annotation is inefficient and the quality is difficult to guarantee, affecting the efficiency and quality of point cloud annotation.
By parsing multiple consecutive point cloud frames, the initial bounding box of the target in each point cloud frame is obtained, and the Iterative Closest Point (ICP) algorithm is used for registration to obtain the registration contour and registration matrix, and the target bounding box is automatically adjusted.
It reduces the cost of adjusting the target bounding box, improves the efficiency and quality of adjustment, and realizes automated bounding box adjustment.
Smart Images
Figure CN117745775B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and in particular to the field of artificial intelligence technology such as autonomous driving and environmental perception, specifically to a method, apparatus, electronic device and storage medium for adjusting a target bounding box. Background Technology
[0002] Currently, point cloud annotation requires a target bounding box containing relevant point cloud points to represent these points as obstacles. The same obstacle at different times will form an obstacle trajectory. For a complete obstacle trajectory, each target bounding box requires manual modification of its size and pose in the point cloud annotation tool to ensure that the target bounding box fully encompasses and fits the relevant point cloud points. However, this manual adjustment method affects the efficiency of point cloud annotation and cannot guarantee the quality of the annotation results. Summary of the Invention
[0003] This disclosure aims to at least partially address one of the technical problems in the related art.
[0004] The first aspect of this disclosure provides a method for adjusting a target bounding box, comprising:
[0005] Analyze multiple consecutive point cloud frames to obtain the first point cloud set and initial bounding box of the target in each point cloud frame;
[0006] Register each of the first point sets to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point set;
[0007] Based on the second point set corresponding to the registration contour, the first initial bounding box is adjusted to obtain the adjusted first bounding box;
[0008] Based on the adjusted first bounding box and each of the registration matrices, the other initial bounding boxes are adjusted respectively to obtain the second bounding box.
[0009] A second aspect of this disclosure provides a target bounding box adjustment device, comprising:
[0010] The acquisition module is used to parse multiple consecutive point cloud frames and obtain the first point cloud set and initial bounding box of the target in each point cloud frame;
[0011] The registration module is used to register each of the first point sets to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point set.
[0012] The first adjustment module is used to adjust the first initial bounding box based on the second point set corresponding to the registration contour, so as to obtain the adjusted first bounding box.
[0013] The second adjustment module is used to adjust other initial bounding boxes based on the adjusted first bounding box and each of the registration matrices to obtain a second bounding box.
[0014] A third aspect of this disclosure provides a computer device including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements a method for adjusting a target bounding box as proposed in a first aspect of this disclosure.
[0015] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a method for adjusting a target bounding box as proposed in a first aspect of this disclosure.
[0016] A fifth aspect of this disclosure provides a computer program product including a computer program that, when executed by a processor, implements a method for adjusting a target bounding box as described in a first aspect of this disclosure.
[0017] The target bounding box adjustment method, apparatus, electronic device, and storage medium provided in this disclosure have the following beneficial effects:
[0018] In this embodiment, multiple consecutive point cloud frames are first parsed to obtain the first point cloud set and initial bounding box corresponding to the target in each point cloud frame. Then, each first point cloud set is registered to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point cloud set. Next, based on the second point cloud set corresponding to the registration contour, the first initial bounding box is adjusted to obtain the adjusted first bounding box. Finally, based on the adjusted first bounding box and each registration matrix, the other initial bounding boxes are adjusted respectively to obtain the second bounding boxes. This reduces the cost of adjusting the target bounding box and improves the efficiency and quality of target bounding box adjustment.
[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0020] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:
[0021] Figure 1This is a flowchart illustrating a method for adjusting a target bounding box according to an embodiment of the present disclosure.
[0022] Figure 2 This is a flowchart illustrating a method for adjusting a target bounding box according to an embodiment of the present disclosure.
[0023] Figure 3 This is a flowchart illustrating a method for adjusting a target bounding box according to an embodiment of the present disclosure.
[0024] Figure 4 This is a flowchart illustrating a method for adjusting a target bounding box according to an embodiment of the present disclosure.
[0025] Figure 5 A schematic diagram of the structure of the target bounding box adjustment device provided in the embodiments of this disclosure;
[0026] Figure 6 A block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation
[0027] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.
[0028] This disclosure relates to the fields of artificial intelligence technology, such as autonomous driving and environmental perception.
[0029] Artificial Intelligence (AI) is a new technological science that studies and develops theories, methods, technologies, and application systems to simulate, extend, and expand human intelligence.
[0030] Autonomous driving, also known as driverless driving, computer-controlled driving, or wheeled mobile robots, is a cutting-edge technology that relies on computer and artificial intelligence technologies to achieve complete, safe, and efficient driving without human intervention.
[0031] Environmental perception is the core technology of autonomous driving. It is responsible for detecting various moving and stationary obstacles (such as vehicles, pedestrians, buildings, etc.) and collecting various information on the road (such as drivable areas, lane lines, traffic signs, traffic lights, etc.). It requires the use of various sensors (such as cameras, LiDAR, millimeter-wave radar, etc.).
[0032] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0033] The following description, with reference to the accompanying drawings, describes a method, apparatus, electronic device, and storage medium for adjusting the target bounding box according to embodiments of the present disclosure.
[0034] Figure 1 This is a flowchart illustrating a method for adjusting a target bounding box provided in an embodiment of the present disclosure.
[0035] like Figure 1 As shown, the method for adjusting the target bounding box may include the following steps:
[0036] Step 101: parse multiple consecutive point cloud frames to obtain the first point cloud set and initial bounding box of the target in each point cloud frame.
[0037] It should be noted that multiple consecutive point cloud frames refer to multiple point cloud frames acquired within a continuous time period. The target can be any moving or stationary object, such as vehicles, pedestrians, landmarks, etc., and this disclosure does not impose any limitations on it.
[0038] The first point cloud can be a point cloud set composed of point clouds corresponding to the target.
[0039] In some possible implementations, target recognition can be performed on each point cloud frame based on camera data corresponding to multiple point cloud frames to obtain the center position of the target in each point cloud frame. Then, based on the center position and a preset size, the initial bounding box of the target in each point cloud frame is determined. Finally, all the point clouds contained in the initial bounding box are determined as the first point cloud set corresponding to the target in each point cloud frame, thereby providing conditions for reducing the cost of adjusting the target bounding box.
[0040] The preset size refers to the size of the initial bounding box, which can be set to any size. For example, the preset size can be 1 meter in length, width, and height, etc., and this disclosure does not limit it.
[0041] Step 102: Register each first point cloud to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point cloud.
[0042] It should be noted that the registration of each first point set can be based on the Iterative Closest Point (ICP) algorithm, and this disclosure does not limit this.
[0043] Among them, the registration profile is the 3D point cloud profile of the target after registration based on each point cloud set.
[0044] In some possible implementations, a reference point cloud can be determined first based on the number of points contained in each first point cloud. Then, based on the reference point cloud, the other first point clouds are registered to obtain the registration contour of the target in the first point cloud frame where the reference point cloud is located, as well as the registration matrix corresponding to each other first point cloud, thereby providing conditions for improving the adjustment quality of the target bounding box.
[0045] The reference point set is the point cloud set used as the basis for configuring the point cloud set, and it can be any first point cloud set. For example, the reference point set can be the point cloud set containing the largest number of point clouds among all first point cloud sets, etc., and this disclosure does not limit it in this way.
[0046] The registration matrix is the transformation matrix used to register other first point cloud sets to the first point cloud frame where the reference point cloud set is located.
[0047] In this disclosure, after obtaining the first point cloud set and initial bounding box corresponding to the target in each point cloud frame, in order to obtain a clearer and more complete target point cloud contour, the first point cloud set can be registered based on the reference point cloud set using the ICP algorithm to obtain the registered contour corresponding to the target.
[0048] Step 103: Based on the second point set corresponding to the registration contour, adjust the first initial bounding box to obtain the adjusted first bounding box.
[0049] The second point set is the point set of the target registration contour that makes up the point cloud frame where the reference point set is located.
[0050] The first initial bounding box is the initial bounding box that needs to be adjusted. It can be the initial bounding box corresponding to any point cloud frame, and this disclosure does not limit it.
[0051] In this disclosure, after determining the registration profile of the target in the first point cloud frame where the reference point set is located, in order to make the first initial bounding box corresponding to any point cloud frame contain all point clouds in the second point cloud set, the first initial bounding box can be adjusted based on the second point cloud set corresponding to the registration profile to obtain the first bounding box.
[0052] Step 104: Based on the adjusted first bounding box and each registration matrix, adjust the other initial bounding boxes respectively to obtain the second bounding box.
[0053] In this disclosure, after adjusting the first initial bounding box corresponding to any point cloud frame, other initial bounding boxes can be adjusted based on the obtained adjusted first bounding box and the registration matrix associated with the point cloud frame where each first point cloud set is located, to obtain the second bounding box.
[0054] In this embodiment, multiple consecutive point cloud frames are first parsed to obtain the first point cloud set and initial bounding box corresponding to the target in each point cloud frame. Then, each first point cloud set is registered to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point cloud set. Next, based on the second point cloud set corresponding to the registration contour, the first initial bounding box is adjusted to obtain the adjusted first bounding box. Finally, based on the adjusted first bounding box and each registration matrix, the other initial bounding boxes are adjusted respectively to obtain the second bounding boxes. Thus, by first registering multiple first point cloud sets corresponding to the target, and then adjusting each initial bounding box based on the registration matrix and registration contour, automatic adjustment of the bounding box containing the target is achieved, reducing the cost of adjusting the target bounding box and improving the adjustment efficiency and quality of the target bounding box.
[0055] Figure 2 This is a flowchart illustrating a method for adjusting a target bounding box according to an embodiment of the present disclosure.
[0056] like Figure 2 As shown, the method for adjusting the target bounding box may include the following steps:
[0057] Step 201: parse multiple consecutive point cloud frames to obtain the first point cloud set and initial bounding box of the target in each point cloud frame.
[0058] Step 202: Register each first point cloud to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point cloud.
[0059] The specific implementation of steps 201 to 202 can be found in the detailed descriptions of other embodiments in this disclosure, and will not be repeated here.
[0060] Step 203: Select any one of the multiple initial bounding boxes as the first initial bounding box.
[0061] In this disclosure, after obtaining the registration matrix corresponding to each first point set, any one of the initial bounding boxes in the frame containing each first point set can be determined as the first initial bounding box, thereby providing conditions for improving the adjustment efficiency of the target bounding box. For example, the initial bounding box corresponding to the reference point set can be determined as the first initial bounding box, or any other initial bounding box corresponding to any first point set other than the reference point set can be determined as the first initial bounding box, etc. This disclosure does not limit this.
[0062] Step 204: If the first initial bounding box is the initial bounding box corresponding to the reference point cloud set, adjust the size and / or pose of any initial bounding box to obtain the adjusted first bounding box, wherein the reference point cloud set is the point cloud set on which the point cloud set is based when configuring the point cloud set, and all point clouds in the second point cloud set are located within the adjusted first bounding box.
[0063] In this disclosure, after determining the initial bounding box corresponding to the reference point cloud as the first initial bounding box, the size and / or pose of the first initial bounding box can be adjusted based on the second point cloud corresponding to the registration profile to obtain the adjusted first bounding box, so that all point clouds in the second point cloud are located in the adjusted first bounding box.
[0064] Step 205: Based on the adjusted first bounding box and each registration matrix, adjust the other initial bounding boxes respectively to obtain the second bounding box.
[0065] In this disclosure, after adjusting the initial bounding box corresponding to the reference point set, the initial bounding box corresponding to each other first point set can be adjusted based on the adjusted bounding box and the inverse matrix of the registration matrix corresponding to each other first point set.
[0066] The specific implementation of step 205 can be found in the detailed descriptions of other embodiments in this disclosure, and will not be repeated here.
[0067] In this embodiment, multiple consecutive point cloud frames are first parsed to obtain the first point cloud set and initial bounding box corresponding to the target in each point cloud frame. Then, each first point cloud set is registered to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point cloud set. Next, any one of the multiple initial bounding boxes is determined as the first initial bounding box. With the first initial bounding box being the initial bounding box corresponding to the reference point cloud set, the size and / or pose of any initial bounding box is adjusted to obtain the adjusted first bounding box. Finally, based on the adjusted first bounding box and each registration matrix, the other initial bounding boxes are adjusted respectively to obtain the second bounding box. Therefore, by adjusting the size and / or pose of the initial bounding box corresponding to the reference point cloud set based on the point cloud set corresponding to the registration contour, and then adjusting the other initial bounding boxes based on the registration matrix corresponding to each first point cloud set and the adjusted bounding box of the reference point cloud set, the efficiency and accuracy of target bounding box adjustment are improved.
[0068] Figure 3 This is a flowchart illustrating a method for adjusting a target bounding box according to an embodiment of the present disclosure.
[0069] like Figure 3As shown, the method for adjusting the target bounding box may include the following steps:
[0070] Step 301: parse multiple consecutive point cloud frames to obtain the first point cloud set and initial bounding box of the target in each point cloud frame.
[0071] Step 302: Register each first point cloud to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point cloud.
[0072] Step 303: Select any one of the multiple initial bounding boxes as the first initial bounding box.
[0073] The specific implementation of steps 301 to 303 can be found in the detailed descriptions of other embodiments in this disclosure, and will not be repeated here.
[0074] Step 304: If the first initial bounding box is not the initial bounding box corresponding to the reference point cloud set, the second point cloud set is mapped to the first point cloud frame where the first initial bounding box is located based on the registration matrix corresponding to the first initial bounding box to obtain the third point cloud set, wherein the reference point cloud set is the point cloud set on which the point cloud set is based when configuring the point cloud set.
[0075] The third point cloud is the point cloud that forms the target contour in the first point cloud frame where the first initial bounding box is located.
[0076] In this disclosure, when the first initial bounding box is not the initial bounding box corresponding to the reference point cloud set, in order to make the target contour in the first point cloud frame where the first initial bounding box is located clearer and more complete, the second point cloud set can be mapped to the first point cloud frame where the first initial bounding box is located based on the inverse matrix of the registration matrix corresponding to the point cloud frame where the first initial bounding box is located, so as to obtain the third point cloud set.
[0077] Step 305: Based on the third point cloud set, adjust the size and / or pose of any initial bounding box to obtain the adjusted first bounding box, wherein all point clouds in the third point cloud set are located within the adjusted first bounding box.
[0078] In this disclosure, after obtaining the third point cloud, the size and / or pose of the initial bounding box in the first point cloud frame containing the third point cloud can be adjusted based on the third point cloud, so that the adjusted bounding box can contain all the point clouds in the third point cloud.
[0079] Step 306: Based on the adjusted first bounding box and each registration matrix, adjust the other initial bounding boxes respectively to obtain the second bounding box.
[0080] The specific implementation of step 306 can be found in the detailed descriptions of other embodiments in this disclosure, and will not be repeated here.
[0081] In this embodiment, multiple consecutive point cloud frames are first parsed to obtain the first point cloud set and initial bounding box corresponding to the target in each point cloud frame. Each first point cloud set is then registered to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point cloud set. Then, any one of the multiple initial bounding boxes is determined as the first initial bounding box. If the first initial bounding box is not the initial bounding box corresponding to the reference point cloud set, based on the registration matrix corresponding to the first initial bounding box, the second point cloud set is mapped to the first point cloud frame containing the first initial bounding box to obtain the third point cloud set. Then, based on the third point cloud set, the size and / or pose of any initial bounding box is adjusted to obtain the adjusted first bounding box. Finally, based on the adjusted first bounding box and each registration matrix, the other initial bounding boxes are adjusted respectively to obtain the second bounding box. Therefore, by adjusting the size and / or pose of the initial bounding box corresponding to any point cloud frame, and adjusting the other initial bounding boxes based on the adjusted bounding boxes and each registration matrix, the cost of adjusting the target bounding box is reduced, and the quality of the target bounding box is improved.
[0082] Figure 4 This is a flowchart illustrating a method for adjusting a target bounding box according to an embodiment of the present disclosure.
[0083] like Figure 4 As shown, the method for adjusting the target bounding box may include the following steps:
[0084] Step 401: parse multiple consecutive point cloud frames to obtain the first point cloud set and initial bounding box of the target in each point cloud frame.
[0085] Step 402: Register each first point cloud to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point cloud.
[0086] Step 403: Based on the second point set corresponding to the registration contour, adjust the first initial bounding box to obtain the adjusted first bounding box.
[0087] The specific implementation of steps 401 to 403 can be found in the detailed descriptions of other embodiments in this disclosure, and will not be repeated here.
[0088] Step 404: Based on the adjusted size and / or pose of the first bounding box and the first registration matrix associated with the adjusted first bounding box, adjust the second initial bounding box in the first point cloud frame where the reference point set is located to obtain a second bounding box.
[0089] The second initial bounding box is the initial bounding box in the first point cloud frame where the reference point cloud is located.
[0090] In this disclosure, after obtaining the adjusted first bounding box, the second initial bounding box in the first point cloud frame where the reference point set is located can be adjusted based on the size and / or pose of the adjusted first bounding box and the first registration matrix associated with the adjusted first bounding box, so as to obtain the second bounding box corresponding to the reference point set.
[0091] Step 405: Based on the second registration matrix associated with a second bounding box and other initial bounding boxes, adjust the other initial bounding boxes respectively to obtain other second bounding boxes.
[0092] The second registration matrix is the inverse of the registration matrices associated with the other initial bounding boxes.
[0093] In this disclosure, after adjusting the second initial bounding box in the first point cloud frame where the reference point set is located, other initial bounding boxes can be adjusted based on the adjusted second bounding box and the second registration matrix associated with the initial bounding boxes in the point cloud frames where other point sets are located, so as to obtain other second bounding boxes, thereby improving the adjustment quality and efficiency of the target bounding box.
[0094] Step 406: Determine the spatial density of the point cloud contained in each adjusted bounding box.
[0095] In this disclosure, after obtaining the adjusted bounding box in the point cloud frame where each point cloud set is located, in order to make the adjusted bounding box fit the point cloud set better, the spatial density of the point cloud contained in each adjusted bounding box can be determined first.
[0096] Step 407: If the spatial density of the point cloud adjacent to the first boundary of any adjusted bounding box is greater than a density threshold, the first boundary is adjusted based on the position of each point cloud adjacent to the first boundary to obtain a third bounding box, wherein the distance between the first boundary of the third bounding box and the point cloud within the third bounding box is less than a distance threshold.
[0097] The first boundary can be any boundary in the adjusted bounding box, and this disclosure does not limit it.
[0098] The density threshold is a critical value for the spatial density of the point cloud used to determine whether to adjust the first boundary of the bounding box. It can be preset, and this disclosure does not limit it.
[0099] The distance threshold is the critical distance between the point cloud points within the first boundary and the third bounding box. It can be preset, and this disclosure does not limit it.
[0100] In this disclosure, if the spatial density of the point cloud adjacent to the first boundary of any adjusted bounding box is greater than the density threshold, it can be considered that the spatial density of the point cloud adjacent to the first boundary is large, that is, the number of point clouds adjacent to the first boundary is large. In this case, in order to make the first boundary fit the adjacent point clouds better, the first boundary can be adjusted based on the position of each point cloud adjacent to the first boundary to obtain the third bounding box, thereby improving the accuracy of the target bounding box.
[0101] In this embodiment, multiple consecutive point cloud frames are first parsed to obtain the first point cloud set and initial bounding box corresponding to the target in each point cloud frame. Then, each first point cloud set is registered to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point cloud set. Based on the second point cloud set corresponding to the registration contour, the first initial bounding box is adjusted to obtain the adjusted first bounding box. Based on the size and / or pose of the adjusted first bounding box and the first registration matrix associated with the adjusted first bounding box, the second initial bounding box in the first point cloud frame where the reference point cloud set is located is adjusted to obtain a second bounding box. Then, based on the second registration matrix associated with the second bounding box and other initial bounding boxes, other initial bounding boxes are adjusted to obtain other second bounding boxes. Finally, the spatial density of the point cloud contained in each adjusted bounding box is determined. If the spatial density of the point cloud adjacent to the first boundary of any adjusted bounding box is greater than a density threshold, the first boundary is adjusted based on the position of each point cloud adjacent to the first boundary to obtain a third bounding box. Therefore, by adjusting the first bounding box and its associated first registration matrix, the bounding box in the first point cloud frame where the reference point set is located is adjusted. Then, based on the adjusted bounding box of the reference point set and the second registration matrix associated with other initial bounding boxes, other initial bounding boxes are adjusted. Furthermore, based on the position of the boundary in the bounding box and the adjacent point cloud and the spatial density of the point cloud, the corresponding boundary is adjusted, thereby making the target bounding box more accurate and improving the quality of the adjusted target bounding box.
[0102] To achieve the above embodiments, this disclosure also proposes a target bounding box adjustment device.
[0103] Figure 5 This is a schematic diagram of the structure of the target bounding box adjustment device provided in an embodiment of the present disclosure.
[0104] like Figure 5 As shown, the target bounding box adjustment device 500 includes: an acquisition module 501, a registration module 502, a first adjustment module 503, and a second adjustment module 504.
[0105] The acquisition module 501 is used to parse multiple consecutive point cloud frames and obtain the first point cloud set and initial bounding box of the target in each point cloud frame.
[0106] Registration module 502 is used to register each first point cloud to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point cloud.
[0107] The first adjustment module 503 is used to adjust the first initial bounding box based on the second point set corresponding to the registration contour, so as to obtain the adjusted first bounding box.
[0108] The second adjustment module 504 is used to adjust other initial bounding boxes based on the adjusted first bounding box and each registration matrix to obtain the second bounding box.
[0109] In one possible implementation of this disclosure, the aforementioned acquisition module 501 is specifically used for:
[0110] Based on camera data corresponding to multiple point cloud frames, target recognition is performed on each point cloud frame to obtain the center position of the target in each point cloud frame.
[0111] Based on the center position and preset size, determine the initial bounding box of the target in each point cloud frame;
[0112] All point clouds contained in the initial bounding box are determined as the first point cloud set corresponding to the target in each point cloud frame.
[0113] In one possible implementation of this disclosure, the registration module 502 is specifically used for:
[0114] The baseline point set is determined based on the number of points contained in each first point set;
[0115] Using the reference point cloud as a reference, register the other first point clouds to obtain the registration contour of the target in the first point cloud frame where the reference point cloud is located, and the registration matrix corresponding to each other first point cloud.
[0116] In one possible implementation of this disclosure, the first adjustment module 503 is further configured to:
[0117] Choose any one of the multiple initial bounding boxes as the first initial bounding box.
[0118] In one possible implementation of this disclosure, the first adjustment module 503 is further configured to:
[0119] When the first initial bounding box is the initial bounding box corresponding to the reference point cloud set, the size and / or pose of any initial bounding box are adjusted to obtain the adjusted first bounding box, wherein the reference point cloud set is the point cloud set on which the point cloud set is based when configuring the point cloud set, and all point clouds in the second point cloud set are located within the adjusted first bounding box.
[0120] In one possible implementation of this disclosure, the first adjustment module 503 is further configured to:
[0121] If the first initial bounding box is not the initial bounding box corresponding to the reference point cloud set, the second point cloud set is mapped to the first point cloud frame where the first initial bounding box is located based on the registration matrix corresponding to the first initial bounding box to obtain the third point cloud set, wherein the reference point cloud set is the point cloud set on which the point cloud set is based when configuring the point cloud set.
[0122] Based on the third point cloud, the size and / or pose of any initial bounding box are adjusted to obtain an adjusted first bounding box, wherein all point clouds in the third point cloud are located within the adjusted first bounding box.
[0123] In one possible implementation of this disclosure, the second adjustment module 504 is specifically used for:
[0124] Based on the adjusted size and / or pose of the first bounding box, and the first registration matrix associated with the adjusted first bounding box, the second initial bounding box in the first point cloud frame where the reference point set is located is adjusted to obtain a second bounding box;
[0125] Based on a second registration matrix that associates a second bounding box with other initial bounding boxes, the other initial bounding boxes are adjusted to obtain other second bounding boxes.
[0126] In one possible implementation of this disclosure, the second adjustment module 504 is further configured to:
[0127] Determine the spatial density of the point cloud contained in each adjusted bounding box;
[0128] If the spatial density of the point cloud adjacent to the first boundary of any adjusted bounding box is greater than a density threshold, the first boundary is adjusted based on the position of each point cloud adjacent to the first boundary to obtain a third bounding box, wherein the distance between the first boundary of the third bounding box and the point cloud within the third bounding box is less than a distance threshold.
[0129] The functions and specific implementation principles of the modules described in this embodiment can be found in the above method embodiments, and will not be repeated here.
[0130] In this embodiment, multiple consecutive point cloud frames are first parsed to obtain the first point cloud set and initial bounding box corresponding to the target in each point cloud frame. Then, each first point cloud set is registered to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point cloud set. Next, based on the second point cloud set corresponding to the registration contour, the first initial bounding box is adjusted to obtain the adjusted first bounding box. Finally, based on the adjusted first bounding box and each registration matrix, the other initial bounding boxes are adjusted respectively to obtain the second bounding boxes. Thus, by first registering multiple first point cloud sets corresponding to the target, and then adjusting each initial bounding box based on the registration matrix and registration contour, automatic adjustment of the bounding box containing the target is achieved, reducing the cost of adjusting the target bounding box and improving the adjustment efficiency and quality of the target bounding box.
[0131] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0132] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0133] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.
[0134] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0135] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the target bounding box adjustment method. For example, in some embodiments, the target bounding box adjustment method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the target bounding box adjustment method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the target bounding box adjustment method by any other suitable means (e.g., by means of firmware).
[0136] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0137] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0138] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0139] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0140] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.
[0141] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.
[0142] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.
[0143] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified. In the description of this disclosure, the words "if" and "suppose" as used may be interpreted as "when," "when," "in response to determination," or "in the circumstances."
[0144] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.
Claims
1. A method for adjusting a target bounding box, comprising: Analyze multiple consecutive point cloud frames to obtain the first point cloud set and initial bounding box of the target in each point cloud frame; Each of the first point cloud sets is registered to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point cloud set. The reference point cloud set is used as a reference to register the other first point cloud sets to obtain the registration contour of the target in the first point cloud frame where the reference point cloud set is located, and the registration matrix corresponding to each other first point cloud set. The registration contour is the three-dimensional point cloud contour of the target after registration based on each point cloud set. Any one of the plurality of initial bounding boxes is determined as the first initial bounding box; Based on the second point cloud corresponding to the registration profile, the size and / or pose of the first initial bounding box are adjusted to obtain the adjusted first bounding box. The second point cloud is the point cloud that makes up the registration profile in the point cloud frame where the reference point cloud is located, and all point clouds in the second point cloud are located within the adjusted first bounding box. Based on the adjusted first bounding box and each of the registration matrices, the other initial bounding boxes are adjusted respectively to obtain the second bounding box.
2. The method as described in claim 1, wherein, The step of parsing multiple consecutive point cloud frames to obtain the first point cloud set and initial bounding box corresponding to the target in each point cloud frame includes: Based on the camera data corresponding to the multiple point cloud frames, target recognition is performed on each point cloud frame to obtain the center position of the target in each point cloud frame; Based on the center position and the preset size, the initial bounding box of the target in each point cloud frame is determined; All point clouds contained in the initial bounding box are determined as the first point cloud set corresponding to the target in each point cloud frame.
3. The method as described in claim 1, wherein, The method further includes: A reference point set is determined based on the number of point clouds contained in each of the first point sets.
4. The method of claim 1, wherein, The step of adjusting the size and / or pose of the first initial bounding box based on the second point set corresponding to the registration contour to obtain the adjusted first bounding box includes: When the first initial bounding box is the initial bounding box corresponding to the reference point cloud set, the size and / or pose of the first initial bounding box are adjusted to obtain the adjusted first bounding box, wherein the reference point cloud set is the point cloud set on which the point cloud set is based when configuring the point cloud set, and all point clouds in the second point cloud set are located within the adjusted first bounding box.
5. The method of claim 1, wherein, The step of adjusting the size and / or pose of the first initial bounding box based on the second point set corresponding to the registration contour to obtain the adjusted first bounding box includes: If the first initial bounding box is not the initial bounding box corresponding to the reference point cloud set, the second point cloud set is mapped to the first point cloud frame where the first initial bounding box is located based on the registration matrix corresponding to the first initial bounding box to obtain the third point cloud set, wherein the reference point cloud set is the point cloud set on which the point cloud set is based when configuring the point cloud set. Based on the third point cloud, the size and / or pose of the first initial bounding box are adjusted to obtain the adjusted first bounding box, wherein all point clouds in the third point cloud are located within the adjusted first bounding box.
6. The method as described in any one of claims 1-5, wherein, The step of adjusting other initial bounding boxes based on the adjusted first bounding box and each of the registration matrices to obtain a second bounding box includes: Based on the size and / or pose of the adjusted first bounding box, and the first registration matrix associated with the adjusted first bounding box, the second initial bounding box in the first point cloud frame where the reference point set is located is adjusted to obtain a second bounding box. Based on the second registration matrix associated with the first second bounding box and other initial bounding boxes, the other initial bounding boxes are adjusted respectively to obtain other second bounding boxes.
7. The method of claim 6, wherein, After obtaining the second bounding box, the process further includes: Determine the spatial density of the point cloud contained in each adjusted bounding box; If the spatial density of the point cloud adjacent to the first boundary of any adjusted bounding box is greater than a density threshold, the first boundary is adjusted based on the position of each point cloud adjacent to the first boundary to obtain a third bounding box, wherein the distance between the first boundary of the third bounding box and the point cloud within the third bounding box is less than a distance threshold.
8. A target bounding box adjustment device, comprising: The acquisition module is used to parse multiple consecutive point cloud frames and obtain the first point cloud set and initial bounding box of the target in each point cloud frame; The registration module is used to register each of the first point clouds to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point cloud. The reference point cloud is used as a reference to register the other first point clouds to obtain the registration contour of the target in the first point cloud frame where the reference point cloud is located, and the registration matrix corresponding to each other first point cloud. The registration contour is the three-dimensional point cloud contour of the target after registration based on each point cloud. The first adjustment module is used to determine any one of the multiple initial bounding boxes as the first initial bounding box; and to adjust the size and / or pose of the first initial bounding box based on the second point cloud corresponding to the registration profile to obtain the adjusted first bounding box, wherein the second point cloud is the point cloud that makes up the registration profile in the point cloud frame where the reference point cloud is located, and all point clouds in the second point cloud are located within the adjusted first bounding box. The second adjustment module is used to adjust other initial bounding boxes based on the adjusted first bounding box and each of the registration matrices to obtain a second bounding box.
9. The apparatus of claim 8, wherein, The acquisition module is specifically used for: Based on the camera data corresponding to the multiple point cloud frames, target recognition is performed on each point cloud frame to obtain the center position of the target in each point cloud frame; Based on the center position and the preset size, the initial bounding box of the target in each point cloud frame is determined; All point clouds contained in the initial bounding box are determined as the first point cloud set corresponding to the target in each point cloud frame.
10. The apparatus of claim 8, wherein, The registration module is also used for: A reference point set is determined based on the number of point clouds contained in each of the first point sets.
11. The apparatus of claim 8, wherein, The first adjustment module is further configured to: When the first initial bounding box is the initial bounding box corresponding to the reference point cloud set, the size and / or pose of the first initial bounding box are adjusted to obtain the adjusted first bounding box, wherein the reference point cloud set is the point cloud set on which the point cloud set is based when configuring the point cloud set, and all point clouds in the second point cloud set are located within the adjusted first bounding box.
12. The apparatus of claim 8, wherein, The first adjustment module is further configured to: If the first initial bounding box is not the initial bounding box corresponding to the reference point cloud set, the second point cloud set is mapped to the first point cloud frame where the first initial bounding box is located based on the registration matrix corresponding to the first initial bounding box to obtain the third point cloud set, wherein the reference point cloud set is the point cloud set on which the point cloud set is based when configuring the point cloud set. Based on the third point cloud, the size and / or pose of the first initial bounding box are adjusted to obtain the adjusted first bounding box, wherein all point clouds in the third point cloud are located within the adjusted first bounding box.
13. The apparatus according to any one of claims 8-12, wherein, The second adjustment module is specifically used for: Based on the size and / or pose of the adjusted first bounding box, and the first registration matrix associated with the adjusted first bounding box, the second initial bounding box in the first point cloud frame where the reference point set is located is adjusted to obtain a second bounding box. Based on the second registration matrix associated with the first second bounding box and other initial bounding boxes, the other initial bounding boxes are adjusted respectively to obtain other second bounding boxes.
14. The apparatus of claim 13, wherein, The second adjustment module is also used for: Determine the spatial density of the point cloud contained in each adjusted bounding box; If the spatial density of the point cloud adjacent to the first boundary of any adjusted bounding box is greater than a density threshold, the first boundary is adjusted based on the position of each point cloud adjacent to the first boundary to obtain a third bounding box, wherein the distance between the first boundary of the third bounding box and the point cloud within the third bounding box is less than a distance threshold.
15. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.
17. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.