A method, device and apparatus for detecting a three-dimensional object frame based on a single RGB image
Through the 3D object box detection method based on RGB single image, convolutional neural network is used to process RGB two-dimensional monocular images, which solves the application limitations caused by hardware dependence in existing technologies and realizes 3D object box detection without specific hardware conditions. It is suitable for a wide range of applications and real-time scenarios on mobile terminals.
Patent Information
- Application Number
- CN202210492947.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-07
- Publication Date
- 2025-10-03
- Estimated Expiration
- 2042-05-07
AI Technical Summary
Existing 3D object detection solutions rely on specific hardware or multiple cameras, which results in significant limitations in application scenarios.
Through the 3D object frame detection method based on RGB single image, a convolutional neural network is used to build a model to obtain RGB 2D monocular images without depth information. The heat map and displacement field of the target object are obtained through fitting regression processing, and post-processing is performed to obtain the 3D object frame.
The depth or point cloud of the target can be obtained without relying on specific hardware devices. It has a wider range of uses and is suitable for real-time scenarios on mobile devices.
Smart Images

Figure CN115019298B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to a method, device and equipment for detecting a three-dimensional object frame based on an RGB single image. Background Art
[0002] Object detection has always been a fundamental research area in computer vision. In the field of two-dimensional object detection, relatively mature technologies are already available. Furthermore, in three-dimensional space, many depth- or point-cloud-based object detection technologies are also widely used in fields such as autonomous driving. However, three-dimensional object detection often requires specialized hardware or sensors, or multiple cameras, to acquire the depth or point cloud of the target. While these solutions can achieve higher accuracy, they also significantly limit the application scenarios of these algorithms. Summary of the Invention
[0003] In view of this, the purpose of the present invention is to propose a method, device and equipment for detecting three-dimensional object frames based on a single RGB image, aiming to solve the problem that existing target detection solutions require reliance on specific hardware, resulting in significant limitations in their application scenarios.
[0004] To achieve the above objectives, the present invention provides a method for detecting a three-dimensional object frame based on a single RGB image, the method comprising:
[0005] Acquire an image to be processed, wherein the image to be processed is an RGB two-dimensional monocular image without depth information;
[0006] Inputting the image to be processed into a pre-trained detection model for fitting and regression processing to obtain a heat map and displacement field of the target object;
[0007] The heat map and the displacement field are post-processed to obtain a three-dimensional object frame corresponding to the target object.
[0008] Preferably, the detection model is constructed based on a convolutional neural network, and the detection model includes an encoding and decoding network and a head structure, wherein:
[0009] The encoding and decoding network encodes the input two-dimensional image to obtain a feature map;
[0010] The head structure detects the feature map to obtain a heat map and a displacement field corresponding to the two-dimensional image.
[0011] Preferably, the detection model further includes an upsampling module connected to the end of the codec network, and the upsampling module supervises the regression of the feature map and heat map of the two-dimensional image based on the high-dimensional feature space.
[0012] Preferably, the head structure detects the feature map to obtain a heat map and a displacement field corresponding to the two-dimensional image, including:
[0013] according to Calculate the heat map of the two-dimensional image, where {p} represents all pixels of the two-dimensional image, O represents the set of all objects in the two-dimensional image, and μ i is the center position of object i, σ i It is the experience value;
[0014] according to Calculate the displacement field of the two-dimensional image, where x i Represents the projection coordinates of the 3D object box on the 2D image plane.
[0015] Preferably, the post-processing of the heat map and the displacement field to obtain a three-dimensional object frame corresponding to the target object includes:
[0016] Extracting the peak coordinates of the heat map, and calculating the projection coordinates of eight corner points of the three-dimensional object frame on the two-dimensional image plane based on the peak coordinates and the displacement field;
[0017] The projection coordinates are combined with the camera intrinsic parameters of the image to be processed, and the three-dimensional object frame is calculated using the EPnP algorithm.
[0018] Preferably, calculating the projection coordinates of eight corner points of the three-dimensional object frame on the two-dimensional image plane according to the peak coordinates and the displacement field includes:
[0019] according to Calculate the projection coordinates, where p represents the peak coordinates, represents the displacement field.
[0020] Preferably, the combining of the projection coordinates with the camera intrinsic parameters of the image to be processed and calculating the three-dimensional object frame by an EPnP algorithm includes:
[0021] according to Calculate the coordinates of the eight corner points of the three-dimensional object frame, where {C j} represents the N selected control points, α ij are the coefficients of the basis under the rigid transformation.
[0022] To achieve the above object, the present invention further provides a device for detecting a three-dimensional object frame based on a single RGB image, the device comprising:
[0023] An acquisition unit, configured to acquire an image to be processed, wherein the image to be processed is an RGB two-dimensional monocular image without depth information;
[0024] A first processing unit is configured to input the image to be processed into a pre-trained detection model for fitting and regression processing to obtain a heat map and a displacement field of the target object;
[0025] The second processing unit is configured to post-process the heat map and the displacement field to obtain a three-dimensional object frame corresponding to the target object.
[0026] In order to achieve the above-mentioned objectives, the present invention also proposes a device comprising a processor, a memory, and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of a method for detecting a three-dimensional object frame based on a single RGB image as described in the above-mentioned embodiment.
[0027] In order to achieve the above-mentioned objectives, the present invention also proposes a computer-readable storage medium, on which a computer program is stored. The computer program is executed by a processor to implement the steps of a method for detecting a three-dimensional object frame based on a single RGB image as described in the above embodiment.
[0028] Beneficial effects:
[0029] The above solution fits and regresses a single RGB two-dimensional image without depth information through the detection model. The obtained heat map and displacement field of the target object are post-processed to obtain the three-dimensional object box corresponding to the target object. There is no need to rely on specific hardware equipment to obtain the depth or point cloud of the target, and its application range is wider. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0031] Figure 1 A flowchart of a method for detecting a three-dimensional object frame based on a single RGB image is provided in accordance with an embodiment of the present invention.
[0032] Figure 2 An embodiment of the present invention provides
[0033] Figure 3 A schematic diagram of the network structure of a detection model provided in one embodiment of the present invention.
[0034] Figure 4 A schematic structural diagram of a device for detecting a three-dimensional object frame based on a single RGB image provided by one embodiment of the present invention.
[0035] The realization of the objectives of the invention, the functional features and advantages will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0036] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention for which protection is sought, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0037] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include one or more of the features.
[0038] The present invention is described in detail below with reference to the embodiments.
[0039] Reference Figure 1 FIG2 is a flow chart of a method for detecting a three-dimensional object frame based on a single RGB image provided by an embodiment of the present invention.
[0040] In this embodiment, the method includes:
[0041] S11, obtaining an image to be processed, wherein the image to be processed is an RGB two-dimensional monocular image without depth information.
[0042] S12: Input the image to be processed into a pre-trained detection model for fitting and regression processing to obtain a heat map and displacement field of the target object.
[0043] The detection model is constructed based on a convolutional neural network, and includes a codec network and a head structure.
[0044] The encoding and decoding network encodes the input two-dimensional image to obtain a feature map;
[0045] The head structure detects the feature map to obtain a heat map and a displacement field corresponding to the two-dimensional image.
[0046] like Figure 2 As shown. In this embodiment, the detection model is a convolutional neural network, including an encoder-decoder network and a head structure, wherein the head structure includes a detection head and a shape regression head. Specifically, the encoding network (Encoder) of the encoder-decoder network uses the classic MobileNetV2 as the basic backbone network, and the input is a single RGB three-channel two-dimensional image; the decoding network (Decoder) uses several deconvolution structures, and the bottleneck module (bottleneck) of the encoding network is connected (concat) with the feature map (feature map) of the corresponding size output by the decoding network. In the head structure, anchor-free is used for detection, and the outputs are a heat map (Heatmap) of the probability distribution of the center position of the encoded target and a displacement field of the three-dimensional object frame position offset of the encoded target.
[0047] The head structure detects the feature map to obtain a heat map and a displacement field corresponding to the two-dimensional image, including:
[0048] according to Calculate the heat map of the two-dimensional image, where {p} represents all pixels of the two-dimensional image, O represents the set of all objects in the two-dimensional image, and μ i is the center position of object i, σ i It is the experience value;
[0049] according to Calculate the displacement field of the two-dimensional image, where x i Represents the projection coordinates of the 3D object box on the 2D image plane.
[0050] In this embodiment, the 2D image is an RGB 2D monocular image without depth information. One of the outputs of the codec network is a single-channel heat map with a size half that of the input image. Specifically, for all pixels {p} in the 2D image, the heat map is obtained using the following formula for the bivariate normal distribution:
[0051]
[0052] in, is the set of all targets in the two-dimensional image, μ i is the center position of target i, σ i Is the kernel size suitable for the object size (empirical value). During training, this part uses L2 Loss (mean square error) for training supervision and back propagation.
[0053] The displacement field is used to represent the offset field of each pixel position compared to the target outer box vector. In particular, for a box vector X i , x i Represents the projection coordinates of the three-dimensional object frame on the two-dimensional image plane. The displacement field can be calculated using the following formula:
[0054]
[0055] During the training phase, L1 Loss (mean absolute error) is used for training supervision and back propagation. The calculation formula is as follows:
[0056]
[0057] Among them, ‖·‖1 represents a norm and ∈ is an empirical threshold.
[0058] Furthermore, the detection model also includes an upsampling module connected to the end of the codec network, and the upsampling module supervises the regression of the feature map and heat map of the two-dimensional image based on the high-dimensional feature space.
[0059] Reference Figure 3 As shown. In this embodiment, the end of the basic backbone network is further extended, the upsampling module is connected, and a full convolution subnetwork is constructed. The full convolution subnetwork uses the two-dimensional segmentation annotation of the object for supervision, and uses pixel-level L2 Loss (mean square error) for training supervision and back propagation. The full convolution subnetwork only participates in training during the network training phase and does not participate in the forward direction of the network. The full convolution subnetwork shares the basic backbone network with the heat map module and the displacement field regression module, and the three modules form multi-task training. The module used to supervise the two-dimensional instance segmentation introduces supervisory information of the high-dimensional feature space to the basic backbone network. Experiments have shown that it has a significant positive impact on the performance of the heat map and displacement field regression.
[0060] S13: Post-process the heat map and the displacement field to obtain a three-dimensional object frame corresponding to the target object.
[0061] The post-processing of the heat map and the displacement field to obtain a three-dimensional object frame corresponding to the target object includes:
[0062] S13-1, extracting the peak coordinates of the heat map, and calculating the projection coordinates of eight corner points of the three-dimensional object frame on the two-dimensional image plane based on the peak coordinates and the displacement field;
[0063] Step S13-1 further includes:
[0064] according to Calculate the projection coordinates, where p represents the peak coordinates, represents the displacement field.
[0065] S13-2, combining the projection coordinates with the camera intrinsic parameters of the image to be processed, and calculating the three-dimensional object frame using an EPnP algorithm.
[0066] Step S13-2 further includes:
[0067] according to Calculate the coordinates of the eight corner points of the three-dimensional object frame, where {C j} represents the N selected control points, α ij are the coefficients of the basis under the rigid transformation.
[0068] In this embodiment, after the detection model is trained according to the above network structure and loss function, the target's heat map and displacement field are obtained from the output of the detection model. To obtain the three-dimensional object frame, corresponding post-processing is required, including peak extraction and EPnP. Specifically,
[0069] (1) The purpose of peak extraction is to obtain the projection of the three-dimensional object frame on the two-dimensional image plane through the heat map and displacement field. By extracting the peak coordinate p of the heat map output by the detection module, the projection of the eight corner points of the three-dimensional object frame on the two-dimensional image plane {x i} can be calculated by the following formula:
[0070]
[0071] in, is the displacement field output by the displacement field module.
[0072] (2) After obtaining the projection of the three-dimensional object frame on the two-dimensional image plane through the above steps, the camera parameters of the image are required. The EPnP algorithm is used to calculate the corresponding three-dimensional object frame coordinates. The camera intrinsic parameters need to be input for calculation through the EPnP algorithm. The specific process is: the rotation vector rvec and the translation vector tvec are obtained using the coordinates of the object coordinate system, the coordinates of the two-dimensional image coordinate system, and the camera intrinsic parameters. rvec and tvec can transform the coordinates of the model coordinate system into the camera coordinate system, thereby obtaining the 3D coordinates in the camera coordinate system. By selecting 4 control points {C j} (located in the object coordinate system), three of which are located on the coordinate axis, then the eight corner points of the three-dimensional object box can be expressed by four control points using the following formula:
[0073]
[0074] Among them, α ij are the coefficients of the basis under the rigid transformation.
[0075] In summary, this embodiment eliminates the need for specialized hardware to acquire object depth or point clouds. Instead, it only requires a three-channel monocular image, combined with camera intrinsics, to obtain the target object's 3D bounding box. This approach offers a wider range of applications. Furthermore, its lightweight network design makes it possible to deploy this solution in real-time scenarios on mobile devices.
[0076] Reference Figure 4 FIG2 is a schematic structural diagram of a device for detecting a three-dimensional object frame based on a single RGB image according to an embodiment of the present invention.
[0077] In this embodiment, the device 40 includes:
[0078] An acquisition unit 41 is configured to acquire an image to be processed, wherein the image to be processed is an RGB two-dimensional monocular image without depth information;
[0079] The first processing unit 42 is configured to input the image to be processed into a pre-trained detection model for fitting and regression processing to obtain a heat map and a displacement field of the target object;
[0080] The second processing unit 43 is configured to perform post-processing on the heat map and the displacement field to obtain a three-dimensional object frame corresponding to the target object.
[0081] Furthermore, the detection model is constructed based on a convolutional neural network, and the detection model includes an encoding and decoding network and a head structure, wherein:
[0082] The encoding and decoding network encodes the input two-dimensional image to obtain a feature map;
[0083] The head structure detects the feature map to obtain a heat map and a displacement field corresponding to the two-dimensional image.
[0084] Furthermore, the detection model also includes an upsampling module connected to the end of the codec network, and the upsampling module supervises the regression of the feature map and heat map of the two-dimensional image based on the high-dimensional feature space.
[0085] Furthermore, the head structure detects the feature map to obtain a heat map and a displacement field corresponding to the two-dimensional image, including:
[0086] according to Calculate the heat map of the two-dimensional image, where {p} represents all pixels of the two-dimensional image, O represents the set of all objects in the two-dimensional image, and μ i is the center position of object i, σ i It is the experience value;
[0087] according to Calculate the displacement field of the two-dimensional image, where x i Represents the projection coordinates of the 3D object box on the 2D image plane.
[0088] Furthermore, the second processing unit 43 further includes:
[0089] A first calculation unit is configured to extract peak coordinates of the heat map, and calculate projection coordinates of eight corner points of the three-dimensional object frame on a two-dimensional image plane based on the peak coordinates and the displacement field;
[0090] The second calculation unit is used to combine the projection coordinates with the camera intrinsic parameters of the image to be processed, and calculate the three-dimensional object frame through the EPnP algorithm.
[0091] Furthermore, the first computing unit is further configured to:
[0092] according to Calculate the projection coordinates, where p represents the peak coordinates, represents the displacement field.
[0093] Furthermore, the second computing unit is further configured to:
[0094] according to Calculate the coordinates of the eight corner points of the three-dimensional object frame, where {C j} represents the N selected control points, α ij are the coefficients of the basis under the rigid transformation.
[0095] Each unit module of the device 40 can respectively execute the corresponding steps in the above method embodiment, so each unit module will not be described in detail here. Please refer to the description of the corresponding steps above for details.
[0096] The embodiment of the present invention further provides a device, which includes the above-mentioned 3D object frame detection device based on RGB single image, wherein the 3D object frame detection device based on RGB single image can adopt Figure 4 The structure of the embodiment can be executed accordingly. Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects. For details, please refer to the relevant records in the above embodiments and will not be repeated here.
[0097] The device includes: a mobile phone, digital camera, tablet computer, or other device with a camera function, or a device with an image processing function, or a device with an image display function. The device may include components such as a memory, a processor, an input unit, a display unit, and a power supply.
[0098] Among them, the memory can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as an image playback function, etc.), etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor and the input unit with access to the memory.
[0099] The input unit can be used to receive input digital, character, or image information, and generate keyboard, mouse, joystick, optical, or trackball signal input related to user settings and function control. Specifically, the input unit of this embodiment includes not only a camera, but also a touch-sensitive surface (such as a touch display) and other input devices.
[0100] The display unit can be used to display information input by the user or information provided to the user and various graphical user interfaces of the device, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit may include a display panel. Optionally, the display panel can be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), etc. Furthermore, the touch-sensitive surface can cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor to determine the type of touch event. The processor then provides a corresponding visual output on the display panel based on the type of touch event.
[0101] The embodiment of the present invention further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the memory in the above embodiment; or a computer-readable storage medium that exists independently and is not assembled into a device. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement Figure 1 The computer readable storage medium may be a read-only memory, a magnetic disk or an optical disk.
[0102] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. For similar or identical parts between the various embodiments, reference can be made to each other. For the apparatus embodiments, device embodiments, and storage medium embodiments, since they are generally similar to the method embodiments, their descriptions are relatively simple. For relevant parts, reference can be made to the descriptions of the method embodiments.
[0103] Furthermore, in this document, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.
[0104] While the foregoing description shows and describes preferred embodiments of the present invention, it should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. Rather, the present invention can be used in various other combinations, modifications, and environments, and can be modified within the scope of the present invention by the teachings herein or by techniques or knowledge in the relevant art. Modifications and variations made by those skilled in the art without departing from the spirit and scope of the present invention are intended to be within the scope of the appended claims.
Claims
1. A method for detecting a three-dimensional object frame based on a single RGB image, characterized in that: The method comprises: Acquire an image to be processed, wherein the image to be processed is an RGB two-dimensional monocular image without depth information; Inputting the image to be processed into a pre-trained detection model for fitting and regression processing to obtain a heat map and displacement field of the target object; Furthermore, the detection model is constructed based on a convolutional neural network, and the detection model includes an encoding and decoding network and a head structure, wherein: The encoding and decoding network encodes the input two-dimensional image to obtain a feature map; The head structure detects the feature map to obtain a heat map and a displacement field corresponding to the two-dimensional image; The encoding network of the encoding and decoding network uses MobileNetV2 as the basic backbone network, and the decoding network uses several deconvolution structures. The bottleneck module of the encoding network is connected to the feature map of the corresponding size output by the decoding network; Furthermore, the detection model further includes connecting an upsampling module at the end of the codec network, and supervising the regression of the feature map and heat map of the two-dimensional image based on the high-dimensional feature space through the upsampling module; By connecting to the upsampling module, a fully convolutional subnetwork is constructed. This fully convolutional subnetwork uses the two-dimensional segmentation annotation of the object for supervision and uses the pixel-level mean squared error for training supervision and backpropagation. The heat map and the displacement field are post-processed to obtain a three-dimensional object frame corresponding to the target object.
2. The method for detecting a three-dimensional object frame based on a single RGB image according to claim 1, wherein: The head structure detects the feature map to obtain a heat map and a displacement field corresponding to the two-dimensional image, including: according to Calculate the heat map of the two-dimensional image, where {p} represents all pixels of the two-dimensional image, O represents the set of all objects in the two-dimensional image, and μ i is the center position of object i, σ i It is the experience value; according to Calculate the displacement field of the two-dimensional image, where x i Represents the projection coordinates of the 3D object box on the 2D image plane.
3. The method for detecting a three-dimensional object frame based on a single RGB image according to claim 1, wherein: The post-processing of the heat map and the displacement field to obtain a three-dimensional object frame corresponding to the target object includes: Extracting the peak coordinates of the heat map, and calculating the projection coordinates of eight corner points of the three-dimensional object frame on the two-dimensional image plane based on the peak coordinates and the displacement field; The projection coordinates are combined with the camera intrinsic parameters of the image to be processed, and the three-dimensional object frame is calculated using the EPnP algorithm.
4. The method for detecting a three-dimensional object frame based on a single RGB image according to claim 3, wherein: Calculating the projection coordinates of eight corner points of the three-dimensional object frame on the two-dimensional image plane according to the peak coordinates and the displacement field includes: according to Calculate the projection coordinates, where p represents the peak coordinates, represents the displacement field.
5. The method for detecting a three-dimensional object frame based on a single RGB image according to claim 4, wherein: The step of combining the projection coordinates with the camera intrinsic parameters of the image to be processed and calculating the three-dimensional object frame using an EPnP algorithm includes: according to Calculate the coordinates of the eight corner points of the three-dimensional object frame, where {C j } represents the N selected control points, α ij are the coefficients of the basis under the rigid transformation.
6. A 3D object frame detection device based on a single RGB image, characterized in that: The device comprises: An acquisition unit, configured to acquire an image to be processed, wherein the image to be processed is an RGB two-dimensional monocular image without depth information; A first processing unit is configured to input the image to be processed into a pre-trained detection model for fitting and regression processing to obtain a heat map and a displacement field of the target object; Furthermore, the detection model is constructed based on a convolutional neural network, and the detection model includes an encoding and decoding network and a head structure, wherein: The encoding and decoding network encodes the input two-dimensional image to obtain a feature map; The head structure detects the feature map to obtain a heat map and a displacement field corresponding to the two-dimensional image; Furthermore, the encoding network of the encoding and decoding network uses MobileNetV2 as the basic backbone network, the decoding network uses several deconvolution structures, and the bottleneck module of the encoding network is connected to the feature map of the corresponding size output by the decoding network; Furthermore, the detection model further includes connecting an upsampling module at the end of the codec network, and supervising the regression of the feature map and heat map of the two-dimensional image based on the high-dimensional feature space through the upsampling module; Furthermore, by connecting the upsampling module, a fully convolutional subnetwork is constructed. This fully convolutional subnetwork uses the two-dimensional segmentation annotation of the object for supervision and uses the pixel-level mean squared error for training supervision and backpropagation. The second processing unit is configured to post-process the heat map and the displacement field to obtain a three-dimensional object frame corresponding to the target object.
7. A device, characterized in that The method comprises a processor, a memory, and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of a method for detecting a three-dimensional object frame based on a single RGB image as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is executed by a processor to implement the steps of a method for detecting a three-dimensional object frame based on a single RGB image as described in any one of claims 1 to 5.
Citation Information
Patent Citations
Mobile 3D river crab real-time detection method for 2D images
CN112215116A
Monocular 3D target detection method based on dynamic convolution
CN114266900A