3D target detection methods, electronic devices and storage media

By using an improved SSD network and a 3D object model library, and combining camera image information to calculate the 3D position of objects, the problem of slow speed and high cost of existing 3D object detection algorithms is solved, achieving fast and low-cost object detection.

CN117218365BActive Publication Date: 2025-10-28HON HAI PRECISION INDUSTRY CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202210616195.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-31
Publication Date
2025-10-28
Estimated Expiration
2042-05-31

AI Technical Summary

Technical Problem

Existing 3D target detection algorithms are slow and costly when detecting object categories and 3D positions, and their use of regression calculations and light or radar results in low efficiency.

Method used

An improved Single Shot MultiBox Detector (SSD) network is used to construct a target detection model. Combined with a 3D object model library, the 3D position of the object is calculated using camera image information, avoiding regression calculations and the use of light or radar.

Benefits of technology

It enables fast and low-cost object category and 3D position detection, improving the driving safety of autonomous driving systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117218365B_ABST
    Figure CN117218365B_ABST
Patent Text Reader

Abstract

This application provides a three-dimensional object detection method, electronic device, and storage medium. The method includes: acquiring a detection image; inputting the detection image into a trained object detection model, using the object detection model to determine the object category, the object's two-dimensional bounding box, and the object's rotation angle in the detection image; based on the object category, searching a three-dimensional object model library to determine the object model corresponding to the object and the three-dimensional bounding box corresponding to the object model; determining the distance from the camera to the object model based on the size of the object's two-dimensional bounding box, image information of the detection image, and the camera's focal length; determining the position of the object model in three-dimensional space based on the object's rotation angle, the distance from the camera to the object model, and the three-dimensional bounding box, and using the position of the object model in three-dimensional space as the position of the object in three-dimensional space. This application can quickly obtain the position of an object in three-dimensional space.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to computer vision and deep learning technologies, and more particularly to a three-dimensional object detection method, electronic device, and storage medium. Background Technology

[0002] In the field of autonomous driving, different sensors detect objects in front of or near the vehicle, enabling autonomous driving systems to make corresponding decisions. Therefore, autonomous driving systems need to quickly and accurately detect the type and location of objects to ensure driving safety. Currently, most 3D object detection algorithms rely on regression calculations to obtain the 3D position of objects; however, using regression calculations results in long prediction processing times for these algorithms. Furthermore, when detecting the distance between the vehicle and objects in front, current 3D object detection algorithms use LiDAR or radar to obtain depth information, but using LiDAR or radar is costly and has a relatively small field of view. Summary of the Invention

[0003] In view of the above, it is necessary to provide a three-dimensional target detection method, electronic device and storage medium to solve the problem of slow detection speed of object category and three-dimensional position, and reduce detection cost.

[0004] This application provides a three-dimensional object detection method, which includes: acquiring a detection image captured by a camera; inputting the detection image into a trained object detection model, using the object detection model to determine the object category, the two-dimensional bounding box of the object, and the rotation angle of the object in the detection image; searching a three-dimensional object model library according to the object category to determine the object model corresponding to the object and the three-dimensional bounding box corresponding to the object model; determining the distance from the camera to the object model according to the size of the two-dimensional bounding box of the object, the image information of the detection image, and the focal length of the camera; determining the position of the object model in three-dimensional space according to the rotation angle of the object, the distance from the camera to the object model, and the three-dimensional bounding box, and using the value of the object model in three-dimensional space as the position of the object in three-dimensional space.

[0005] In an optional implementation, the method further includes: constructing the target detection model and improving the target detection model based on a SingleShot MultiBox Detector (SSD) network, wherein the SSD network includes a backbone network and a first head network; the improvement based on the SSD network includes: adding a second head network after the backbone network in the SSD network; after improvement, the target detection model includes the backbone network, the first head network, and the second head network.

[0006] In an optional implementation, the method further includes: acquiring training sample images; extracting features from the training sample images using the backbone network in the object detection model to obtain multiple training feature maps of different scales; generating multiple first default boxes on the multiple training feature maps of different scales and inputting each training feature map of different scales into the first head network for convolution, outputting the object category score of the object within the multiple first default boxes and the position of the multiple first default boxes; performing non-maximum suppression on the multiple first default boxes to output two-dimensional bounding boxes of objects in the training sample images, wherein the two-dimensional bounding boxes of objects in the training sample images include the object category within the two-dimensional bounding boxes and the position of the two-dimensional bounding boxes; inputting each training feature map of different scales into the second head network for convolution, outputting the rotation angle of the object in the training sample images; minimizing the loss values ​​of the first head network and the second head network to obtain the trained object detection model.

[0007] In an optional implementation, the method further includes: performing data augmentation processing on the training sample images to enhance the training sample images, wherein the data augmentation processing includes flipping, rotating, scaling, and shifting the training sample images.

[0008] In one optional implementation, the step of searching a 3D object model library to determine the object model corresponding to the object and the 3D bounding box corresponding to the object model according to the object category includes: establishing a 3D object model library, wherein the 3D object model library includes multiple object models corresponding to different object categories and 3D bounding boxes corresponding to each object model, and the 3D bounding box includes the length, width and height corresponding to the object category.

[0009] In one optional implementation, determining the distance from the camera to the object model based on the size of the object's two-dimensional bounding box, the image information of the detected image, and the camera's focal length includes: calculating the distance from the camera to the object model based on the width and / or length of the object's two-dimensional bounding box, the camera's focal length, the resolution of the detected image, and the pixel width.

[0010] In one optional implementation, determining the position of the object model in three-dimensional space based on the rotation angle of the object, the distance from the camera to the object model, and the three-dimensional bounding box includes: using the rotation angle of the object as the rotation angle of the object model; determining the orientation of the object model in three-dimensional space based on the rotation angle of the object model; and determining the position of the object model in three-dimensional space based on the orientation of the object model in three-dimensional space, the distance from the camera to the object model, and the three-dimensional bounding box.

[0011] In an optional implementation, the method further includes: outputting the object category and the object's position in three-dimensional space, and displaying the object category and the object's position in three-dimensional space on a display screen.

[0012] This application also provides an electronic device, which includes a processor and a memory. The processor is used to execute a computer program stored in the memory to implement the three-dimensional target detection method.

[0013] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the three-dimensional target detection method.

[0014] By utilizing the technical solutions provided in the various embodiments of this application, complex calculations are not required, reducing labor costs and enabling the rapid acquisition of the three-dimensional position of an object. Attached Figure Description

[0015] Figure 1 This is a flowchart of a three-dimensional target detection method provided in an embodiment of this application.

[0016] Figure 2 This is a flowchart of the nonmaximum suppression method provided in the embodiments of this application.

[0017] Figure 3 A schematic diagram of the three-dimensional object model library provided in the embodiments of this application.

[0018] Figure 4 This is a schematic diagram illustrating a method for calculating the distance from a camera to an object model provided in an embodiment of this application.

[0019] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] To better understand the above-mentioned objectives, features, and advantages of this application, the application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0021] Numerous specific details are set forth in the following description to provide a thorough understanding of this application. The described embodiments are merely some, not all, of the embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0023] In the field of autonomous driving, different sensors detect objects in front of or near the vehicle, enabling the autonomous driving system to make corresponding decisions. Therefore, autonomous driving systems need to quickly and accurately detect the type and location of objects to ensure driving safety. Currently, most 3D object detection algorithms use regression calculations to obtain the object's 3D position after detecting its type and location information. However, using regression calculations results in a long prediction processing time for 3D object detection algorithms. Furthermore, when detecting the distance between the vehicle and objects in front, current 3D object detection algorithms use LiDAR or radar to obtain depth information, but using LiDAR or radar is costly and has a relatively small field of view.

[0024] To address the aforementioned issues, this application provides a three-dimensional target detection method that eliminates the need for regression calculations, enabling rapid determination of the object's category and three-dimensional position. Furthermore, when acquiring the distance between the detection vehicle and objects ahead, it eliminates the need for using light-emitting diodes (LEDs) or radar to obtain depth information, thus reducing costs. The three-dimensional target detection method will be described in detail below.

[0025] See Figure 1 The diagram shown is a flowchart of a three-dimensional target detection method provided in an embodiment of this application. The method is applied to electronic devices (e.g., Figure 5 In the electronic device 5 shown, the electronic device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (Internet Protocol Television, IPTV), smart wearable device, etc.

[0026] The electronic device is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to: microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0027] The electronic device may also include network devices and / or user devices. The network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0028] The networks in which the electronic devices are located include, but are not limited to, the Internet, wide area networks, metropolitan area networks, local area networks, and virtual private networks (VPNs).

[0029] The method specifically includes the following.

[0030] 101. Obtain training sample images and detection images captured by the camera.

[0031] In at least one embodiment of this application, the training sample images include, but are not limited to, various scene images on urban and rural roads at different time periods. The detection images captured by the camera include, but are not limited to, scene images on urban and rural roads captured by a monocular camera at different time periods. It should be noted that although both the training sample images and the images captured by the monocular camera are scene images of urban and rural areas at different time periods, the training sample images and the images captured by the camera are not the same.

[0032] In at least one embodiment of this application, obtaining training sample images further includes: performing data augmentation operations on the training sample images to obtain more diverse training sample images. The data augmentation operations include, but are not limited to, flipping, rotating, scaling, and cropping images. These data augmentation operations can effectively expand the sample data, allowing the target detection model to be trained and optimized using training sample images from more diverse scenarios, thus making the target detection model more robust.

[0033] 102. Construct an object detection model and train the object detection model using training sample images to obtain a trained object detection model.

[0034] In at least one embodiment of this application, constructing the target detection model includes:

[0035] The target detection model is improved based on a Single Shot MultiBox Detector (SSD) network, wherein the SSD network includes a backbone network and a first head network. In this embodiment, the improvement of the target detection model based on the SSD network includes adding a second head network after the backbone network in the SSD network. After the improvement, the target detection model includes the backbone network, the first head network, and the second head network.

[0036] In at least one embodiment of this application, training the target detection model using training sample images to obtain the trained target detection model includes:

[0037] The backbone network in the target detection model is used to extract features from the training sample images to obtain multiple training feature maps of different scales for the training sample images;

[0038] Multiple first default boxes are generated on the multiple training feature maps of different scales, and each training feature map of different scales is input into the first head network for convolution, and the object category score of the objects in the multiple default boxes and the position of the multiple first default boxes are output.

[0039] The multiple first default boxes are subjected to non-maximum suppression operation to output the two-dimensional bounding boxes of objects in the training sample image. The two-dimensional bounding boxes of objects in the training sample image include the object category within the two-dimensional bounding box and the position of the two-dimensional bounding box.

[0040] Each training feature map at a different scale is input into the second head network for convolution, and the rotation angle of the object in the training sample image is output.

[0041] Minimize the loss values ​​of the first head network and the second head network to obtain the trained object detection model.

[0042] In at least one embodiment of this application, the first head network includes multiple convolutional layers.

[0043] In at least one embodiment of this application, the second head network comprises multiple convolutional layers and fully connected layers. After training feature maps of different scales are input into the convolutional layers of the second head network for convolution, the fully connected layers output the rotation angle of the objects in the training sample images.

[0044] In at least one embodiment of this application, the loss function of the SSD network is used as the loss function of the first head network. The loss function of the SSD network is prior art and will not be described in detail here.

[0045] In at least one embodiment of this application, the loss function of the second head network is:

[0046]

[0047] Where LOSS is the loss function of the second head network, Y i f(x) represents the rotation angle output value of the object in the second head network. i () represents the actual value of the object's rotation angle.

[0048] In at least one embodiment of this application, the target detection model is trained using the loss functions of the first head network and the second head network. The training is iterated until the target detection model converges, thereby minimizing the loss value and obtaining the trained target detection model.

[0049] In at least one embodiment of this application, the step of performing non-maximum suppression operation on the plurality of first default boxes to output the two-dimensional bounding boxes of objects in the training sample image includes:

[0050] As described above, the scores of the object categories containing objects within the plurality of first default boxes and the positions of the plurality of default boxes are considered. The Non-Maximum Suppression (NMS) operation is referenced... Figure 2 The flowchart specifically includes:

[0051] 21. Sort the multiple first default boxes according to their scores and select the first default box with the highest score.

[0052] 22. Traverse the other first default boxes, calculate the Intersection Over Union (IOU) ratio between the other first default boxes and the selected first default box, and delete the first default boxes with an IOU ratio greater than a preset threshold. In this embodiment, the IOU ratio is the degree of overlap between the selected first default box (i.e., the one with the highest score) and other first default boxes.

[0053] 23. Determine whether there are other first default boxes besides the selected first default box. If there are other first default boxes, return to 21. If there are no other first default boxes, execute 24, and output the selected first default box as the 2D bounding box of the object in the training image.

[0054] 103. Input the detected image into the trained target detection model, and use the target detection model to determine the object category, the two-dimensional bounding box of the object, and the rotation angle of the object in the detected image.

[0055] In at least one embodiment of this application, the detected image is input into a trained target detection model, and features of the detected image are extracted by a backbone network to obtain multiple feature maps of different scales. Multiple second default boxes are generated on the multiple feature maps of different scales, and each feature map of different scales is input into a first head network for convolution. The scores of the object categories of the objects within the multiple second default boxes and the positions of the multiple second default boxes are output. Non-maximum suppression is performed on the multiple second default boxes to output two-dimensional bounding boxes of the objects in the detected image. The two-dimensional bounding boxes of the objects in the detected image include the object category within the two-dimensional bounding boxes and the position of the two-dimensional bounding boxes. Each feature map of different scales is input into a second head network for convolution, and the rotation angle of the objects in the detected image is output. Thus, the detected image is input into the trained target detection model, and the object category, the two-dimensional bounding box of the object, and the rotation angle of the object in the detected image are output.

[0056] 104. Based on the object category, search the 3D object model library to determine the object model corresponding to the object and the 3D bounding box corresponding to the object model.

[0057] In at least one embodiment of this application, the method further includes: establishing a three-dimensional object model library, wherein the three-dimensional object model library includes multiple object models corresponding to different object categories and a three-dimensional bounding box corresponding to each object model, and the three-dimensional bounding box includes the length, width and height corresponding to the object category.

[0058] In this embodiment, the object model is determined based on the object category and the 3D object model library, and the 3D bounding box of the object model is determined based on the object model. For example, as... Figure 3The diagram illustrates the determination of a 3D bounding box according to an embodiment of this application. When the object category is a car, the object model of the car is searched based on the 3D object model library, and the 3D bounding box of the car is then searched based on the object model of the car. When the object category is a truck, the object model of the truck is searched based on the 3D object model library, and the 3D bounding box of the truck is then searched based on the object model of the truck. When the object category is an electric vehicle, the object model of the electric vehicle is searched based on the 3D object model library, and the 3D bounding box of the electric vehicle is then searched based on the object model of the electric vehicle. When the object category is a bus, the object model of the bus is searched based on the 3D object model library, and the 3D bounding box of the bus is then searched based on the object model of the bus. In this embodiment, the object model includes, but is not limited to, a 3D model.

[0059] 105. The distance from the camera to the object model is determined based on the size of the two-dimensional outline of the object, the image information of the detected image, and the focal length of the camera.

[0060] In at least one embodiment of this application, the size of the two-dimensional outline of the object includes the width and / or length of the two-dimensional outline of the object, and the image information of the detected image includes the resolution of the detected image and the pixel width of the detected image.

[0061] In at least one embodiment of this application, the distance from the camera to the object model is calculated based on the width and / or length of the object's two-dimensional outline, the camera's focal length, the resolution of the detected image, and the pixel width of the detected image. For example, refer to... Figure 4 The diagram shown is a schematic representation of a method for calculating the distance from a camera to an object model provided in an embodiment of this application. Where X represents the pixel width of the detected image, f represents the focal length of the camera, G represents the optical center of the camera, d represents the distance from the camera to the object model, and Y represents the width of the two-dimensional bounding box. Based on the principle of triangle similarity, formula ① can be obtained as follows:

[0062]

[0063] Formula ② is:

[0064]

[0065] Where Y represents the width of the two-dimensional bounding box, X represents the pixel width of the detected image, and P represents the resolution of the detected image.

[0066] The distance from the camera to the object model can be obtained from the above formulas ① and ②.

[0067] 106. Determine the position of the object model in three-dimensional space based on the rotation angle of the object, the distance from the camera to the object model, and the three-dimensional boundary frame, and use the position of the object model in three-dimensional space as the position of the object in three-dimensional space.

[0068] In at least one embodiment of this application, the rotation angle of the object is used as the rotation angle of the object model.

[0069] In at least one embodiment of this application, determining the position of the object model in three-dimensional space based on the rotation angle of the object, the distance from the camera to the object model, and the three-dimensional bounding box includes:

[0070] The orientation of the object model in the three-dimensional space is determined based on the rotation angle of the object model. The position of the object model in the three-dimensional space is determined based on the orientation of the object model in the three-dimensional space, the distance from the camera to the object model, and the three-dimensional bounding box.

[0071] In at least one embodiment of this application, the position of the object model in three-dimensional space is taken as the position of the object in three-dimensional space, the object category and the position of the object in three-dimensional space are output, and the object category and the position of the object in three-dimensional space are displayed on the display screen.

[0072] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. For those skilled in the art, improvements can be made without departing from the inventive concept of this application, but these all fall within the scope of protection of this application.

[0073] like Figure 5 As shown, Figure 5 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. The electronic device 5 includes a memory 501, at least one processor 502, a computer program 503 stored in the memory 501 and executable on the at least one processor 502, and at least one communication bus 504.

[0074] Those skilled in the art will understand that Figure 5 The schematic diagram shown is merely an example of the electronic device 5 and does not constitute a limitation on the electronic device 5. It may include more or fewer components than shown, or combine certain components, or different components. For example, the electronic device 5 may also include input / output devices, network access devices, etc.

[0075] The at least one processor 502 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The at least one processor 502 may be a microprocessor or any conventional processor, etc. The at least one processor 502 is the control center of the electronic device 5, connecting various parts of the electronic device 5 via various interfaces and lines.

[0076] The memory 501 can be used to store the computer program 503. The at least one processor 502 implements various functions of the electronic device 5 by running or executing the computer program 503 stored in the memory 501 and calling data stored in the memory 501. The memory 501 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 5 (such as audio data), etc. In addition, the memory 501 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0077] If the modules / units integrated in the electronic device 5 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0078] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A three-dimensional target detection method, characterized in that, The three-dimensional target detection method includes: Acquire training sample images and detection images captured by the camera; A target detection model is constructed, and the target detection model is improved based on the Single Shot MultiBox Detector (SSD) network, wherein the SSD network includes a backbone network and a first head network; the improvement based on the SSD network includes adding a second head network after the backbone network in the SSD network; the improved target detection model includes the backbone network, the first head network, and the second head network. The improved target detection model is trained using the training sample images to obtain the trained target detection model; The detection image is input into a trained target detection model, and the target detection model is used to determine the object category, the two-dimensional bounding box of the object, and the rotation angle of the object in the detection image. This includes: inputting the detection image into the trained target detection model, extracting features of the detection image through the backbone network to obtain multiple feature maps of different scales; inputting each feature map of different scales into the second head network for convolution, and outputting the rotation angle of the object in the detection image. Based on the object category, search the 3D object model library to determine the object model corresponding to the object and the 3D bounding box corresponding to the object model; Determining the distance from the camera to the object model based on the size of the object's two-dimensional bounding box, the image information of the detected image, and the camera's focal length includes: calculating the distance from the camera to the object model based on the width and / or length of the object's two-dimensional bounding box, the camera's focal length, the resolution of the detected image, and the pixel width of the detected image. The distance from the camera to the object model can be calculated using formulas ① and ②. Formula ① is: ; Formula ② is: ; Where X represents the pixel width of the detected image, f represents the focal length of the camera, d represents the distance from the camera to the object model, Y represents the width of the two-dimensional bounding box, and P represents the resolution of the detected image. The position of the object model in three-dimensional space is determined based on the rotation angle of the object, the distance from the camera to the object model, and the three-dimensional bounding box. The position of the object model in three-dimensional space is then used as the position of the object in three-dimensional space.

2. The three-dimensional target detection method according to claim 1, characterized in that, The method further includes: The backbone network in the target detection model is used to extract features from the training sample images to obtain multiple training feature maps of different scales for the training sample images; Multiple first default boxes are generated on the multiple training feature maps of different scales, and each training feature map of different scales is input into the first head network for convolution, and the object category score of the objects in the multiple first default boxes and the position of the multiple first default boxes are output. Non-maximum suppression is performed on the plurality of first default boxes to output two-dimensional bounding boxes of objects in the training sample image. The two-dimensional bounding boxes of objects in the training sample image include the object category within the two-dimensional bounding box and the position of the two-dimensional bounding box. Each training feature map at a different scale is input into the second head network for convolution, and the rotation angle of the object in the training sample image is output. Minimize the loss values ​​of the first head network and the second head network to obtain the trained object detection model.

3. The three-dimensional target detection method according to claim 2, characterized in that, The method further includes: The training sample images are augmented with data to enhance them, wherein the data augmentation includes flipping, rotating, scaling, and shifting the training sample images.

4. The three-dimensional target detection method according to claim 1, characterized in that, The step of searching a 3D object model library to determine the object model corresponding to the object and the 3D bounding box corresponding to the object model based on the object category includes: A three-dimensional object model library is established, wherein the three-dimensional object model library includes multiple object models corresponding to different object categories and a three-dimensional bounding box corresponding to each object model, and the three-dimensional bounding box includes the length, width and height corresponding to the object category.

5. The three-dimensional target detection method according to any one of claims 1 to 4, characterized in that, The step of determining the distance from the camera to the object model based on the size of the object's two-dimensional bounding box, the image information of the detected image, and the camera's focal length includes: The distance from the camera to the object model is calculated based on the width and / or length of the object's two-dimensional outline, the camera's focal length, and the resolution and pixel width of the detected image.

6. The three-dimensional target detection method according to any one of claims 1 to 4, characterized in that, Determining the position of the object model in three-dimensional space based on the object's rotation angle, the distance from the camera to the object model, and the three-dimensional bounding box includes: The rotation angle of the object is taken as the rotation angle of the object model; The orientation of the object model in the three-dimensional space is determined based on the rotation angle of the object model; The position of the object model in the three-dimensional space is determined based on the orientation of the object model in the three-dimensional space, the distance from the camera to the object model, and the three-dimensional bounding box.

7. The three-dimensional target detection method according to claim 6, characterized in that, The method further includes: Output the object category and the object's position in three-dimensional space, and display the object category and the object's position in three-dimensional space on the display screen.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory, the processor being configured to execute a computer program stored in the memory to implement the three-dimensional target detection method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, which, when executed by a processor, implements the three-dimensional target detection method as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Learning geometric differentials for matching 3D models to objects in a 2d image

    US20190026917A1

  • Method and system for processing an image and determining viewpoints of objects

    US20210374988A1

  • Monocular 3D object detection from image semantics network

    US20220026917A1