A method and device for estimating plane normal vectors, and a storage medium

Abstract

An embodiment of the present application discloses a planar normal estimation method, device, and storage medium, including: obtaining an RGB image; inputting the RGB image into a planar normal estimation model to obtain normal information corresponding to the RGB image; the planar normal estimation model is a model trained according to a first RGB sample image with a first sample normal label and a second RGB sample image without a sample normal label.

Description

Technical Field

[0001] The present application relates to the technical field of plane normal estimation, and in particular, to a method and device for plane normal estimation and a storage medium. Background Art

[0002] Plane normals can be used in multiple Augmented Reality (AR) tasks, such as plane advertisement embedding, automatic floor replacement, etc. Therefore, estimating plane normals is of great significance.

[0003] In the prior art, a large number of RGBD images are collected by a depth camera, and a large number of normal labels corresponding to the large number of RGBD images are determined according to the image depth information in the large number of RGBD images, so as to estimate the plane normal of the image to be processed based on the large number of RGBD images and the large number of normal labels. When the number of RGBD images is small, that is, the number of normal labels is small, the accuracy of the predicted plane normal of the image to be processed is very low. Summary of the Invention

[0004] To solve the above technical problems, embodiments of the present application are expected to provide a method and device for plane normal estimation and a storage medium, which can improve the accuracy of estimating the plane normal corresponding to the image to be processed.

[0005] The technical solution of the present application is implemented as follows:

[0006] Embodiments of the present application provide a method for plane normal estimation, and the method for plane normal estimation includes:

[0007] Obtain an RGB image;

[0008] Input the RGB image into a plane normal estimation model to obtain the normal information corresponding to the RGB image; the plane normal estimation model is a model trained according to a first RGB sample image with a first sample normal label and a second RGB sample image without a sample normal label.

[0009] Embodiments of the present application provide a device for plane normal estimation, and the device includes:

[0010] An obtaining unit, configured to obtain an RGB image;

[0011] An input unit, configured to input the RGB image into a plane normal estimation model to obtain the normal information corresponding to the RGB image; the plane normal estimation model is a model trained according to a first RGB sample image with a first sample normal label and a second RGB sample image without a sample normal label.

[0012] Embodiments of the present application provide a device for plane normal estimation, and the device includes:

[0013] A memory, a processor, and a communication bus. The memory communicates with the processor through the communication bus. The memory stores a program for plane normal estimation executable by the processor. When the program for plane normal estimation is executed, the above-mentioned plane normal estimation method is executed by the processor.

[0014] An embodiment of the present application provides a storage medium, on which a computer program is stored and applied to a plane normal estimation device. It is characterized in that when the computer program is executed by a processor, the above-mentioned plane normal estimation method is implemented.

[0015] An embodiment of the present application provides a plane normal estimation method, device, and storage medium. The plane normal estimation method includes: acquiring an RGB image; inputting the RGB image into a plane normal estimation model to obtain normal information corresponding to the RGB image; the plane normal estimation model is a model trained according to a first RGB sample image with a first sample normal label and a second RGB sample image without a sample normal label. By adopting the above method implementation, the plane normal estimation device trains a plane normal estimation model by using a first RGB sample image with a first sample normal label and a second RGB image without a sample normal label. In the case where the number of first RGB sample images with a first sample normal label is small, the plane normal estimation device can jointly train a plane normal estimation model by using a small number of first RGB sample images, a small number of first sample normal labels, and second RGB sample images, improving the generalization ability of the plane normal estimation model and the accuracy of estimating the plane normal corresponding to the image to be processed. Description of the Drawings

[0016] Figure 1 It is a flowchart of a plane normal estimation method provided by an embodiment of the present application;

[0017] Figure 2 It is a schematic diagram of obtaining normal information corresponding to an RGB image provided by an embodiment of the present application;

[0018] Figure 3 It is a schematic flowchart of training a plane normal model by using semi-supervised learning provided by an embodiment of the present application;

[0019] Figure 4 It is a schematic diagram of an image of an image with added noise and an image with added patches provided by an embodiment of the present application;

[0020] Figure 5 It is a schematic composition structure of a plane normal estimation device provided by an embodiment of the present application Figure 1 ;

[0021] Figure 6 Schematic diagram of the composition structure of a plane normal estimation device provided by an embodiment of the present application Figure 2 。 Specific implementation manners

[0022] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0023] Embodiment 1

[0024] An embodiment of the present application provides a plane normal estimation method. A plane normal estimation method is applied to a plane normal estimation device. Figure 1 For a flowchart of a plane normal estimation method provided by an embodiment of the present application, as Figure 1 shown, the plane normal estimation method may include:

[0025] S101. Obtain an RGB image.

[0026] The plane normal estimation method provided by an embodiment of the present application is applicable to a scenario where the normal information of the obtained RGB image is estimated using a plane normal estimation model.

[0027] In an embodiment of the present application, the plane normal estimation device may be implemented in various forms. For example, the plane normal estimation device described in the present application may include devices such as mobile phones, cameras, tablet computers, laptop computers, palmtop computers, personal digital assistants (Personal Digital Assistant, PDA), portable media players (Portable Media Player, PMP), navigation devices, wearable devices, smart bracelets, pedometers, etc., and devices such as digital TVs, desktop computers, servers, etc.

[0028] In an embodiment of the present application, the RGB image may be an image stored in the plane normal estimation device; the RGB image may also be an image transmitted by other devices received by the plane normal estimation device; the RGB image may also be an image obtained by the plane normal estimation device from the client; the specific acquisition method of the RGB image may be determined according to the actual situation.

[0029] In an embodiment of the present application, the RGB image may be a human image; the RGB image may also be an animal image; the RGB image may also be a scene image; the RGB image may also be an indoor image; the specific content of the RGB image may be determined according to the actual situation, and the embodiments of the present application do not limit this.

[0030] In the embodiments of the present application, if the RGB image is an image transmitted by other devices received by the planar normal estimation device, the RGB image can be an image transmitted by other devices received by the planar normal estimation device through wired transmission; the RGB image can also be an image transmitted by other devices received by the planar normal estimation device through wireless transmission; specifically, the manner in which other devices transmit the RGB image to the planar normal estimation device can be determined according to the actual situation, and the embodiments of the present application do not limit this.

[0031] Exemplarily, if the RGB image is an image transmitted by other devices received by the planar normal estimation device through wired transmission, the corresponding wired transmission methods include Asymmetric Digital Subscriber Line (ADSL), Local Area Network (LAN), Fibre (Fiber) To The Home (FTTH), etc. Specifically, the wired transmission method can be determined according to the actual situation, and the embodiments of the present application do not limit this.

[0032] Exemplarily, if the RGB image is an image transmitted by other devices received by the planar normal estimation device through wireless transmission, the corresponding wireless transmission methods include General packet radio service (GPRS), Narrow Band Internet of Things (NB-IoT), Long Range Radio (LoRA), etc. Specifically, the wireless transmission method can be determined according to the actual situation, and the embodiments of the present application do not limit this.

[0033] In the embodiments of the present application, the number of RGB images can be one; the number of RGB images can also be two; the number of RGB images can also be multiple; specifically, the number of RGB images can be determined according to the actual situation, and the embodiments of the present application do not limit this.

[0034] S102. Input the RGB image into the planar normal estimation model to obtain the normal information corresponding to the RGB image; the planar normal estimation model is a model trained according to the first RGB sample image with the first sample normal label and the second RGB sample image without the sample normal label.

[0035] In the embodiments of the present application, after the planar normal estimation device obtains the RGB image, the planar normal estimation device can input the RGB image into the planar normal estimation model to obtain the normal information corresponding to the RGB image.

[0036] It should be noted that the plane normal estimation model is a model trained based on a first RGB sample image with a first sample normal label and a second RGB sample image without a sample normal label.

[0037] In the embodiment of the present application, the process of the plane normal estimation device inputting an RGB image into the plane normal estimation model to obtain the normal information corresponding to the RGB image includes: the plane normal estimation device inputs the RGB image into the plane normal estimation model to obtain a plane mask corresponding to the RGB image and a plane normal corresponding to the RGB image; the plane normal estimation device uses the plane mask and the plane normal as the normal information.

[0038] It should be noted that the plane normal is the normal corresponding to the plane mask.

[0039] In the embodiment of the present application, the plane normal estimation model can be a semantic segmentation network model; the plane normal estimation model can also be an instance segmentation network model; the plane normal estimation model can also be other network models; specifically, it can be determined according to the actual situation, and the embodiment of the present application does not limit this.

[0040] Exemplarily, as Figure 2 shown, when the plane normal estimation device obtains an RGB image in the H*W*3 format, the plane normal estimation device inputs the RGB image into the plane normal estimation model, and uses the first Convolutional Neural Networks (CNN) module in the plane normal estimation model to obtain a plane mask (M1,..., Mn) in the N*H*W form, and uses the second CNN module in the plane normal estimation model to obtain the plane normal ([ad, bd, cd]1,..., [ad, bd, cd]n) corresponding to the plane mask.

[0041] In the embodiment of the present application, before the plane normal estimation device inputs the RGB image into the plane normal estimation model to obtain the normal information corresponding to the RGB image, the plane normal estimation device will also obtain a first RGB sample image and a first sample normal label; the plane normal estimation device obtains a second RGB sample image, and performs image enhancement processing on the second RGB sample image to obtain a processed second RGB sample image; the plane normal estimation device uses the first RGB sample image, the first sample normal label, the second RGB sample image, and the processed second RGB sample image to train an initial plane normal estimation model to obtain a plane normal estimation model.

[0042] In the embodiments of the present application, the first RGB sample image may be an image stored in the planar normal estimation device; the first RGB sample image may also be an image transmitted by other devices received by the planar normal estimation device; the first RGB sample image may also be an image obtained by the planar normal estimation device from the client; specifically, the acquisition method of the first RGB sample image may be determined according to the actual situation.

[0043] In the embodiments of the present application, the first RGB sample image may be a human image; the first RGB sample image may also be an animal image; the first RGB sample image may also be a scenery image; the first RGB sample image may also be an indoor image; specifically, the content of the first RGB sample image may be determined according to the actual situation, and the embodiments of the present application do not limit this.

[0044] In the embodiments of the present application, the first sample normal label includes a first sample planar normal label and a first sample planar mask label.

[0045] In the embodiments of the present application, the process of performing image enhancement processing on the second RGB sample image to obtain the processed second RGB sample image may be that the planar normal estimation device adds noise processing to the second RGB sample image to obtain the processed second RGB sample image; or the planar normal estimation device adds patch processing to the second RGB sample image to obtain the processed second RGB sample image; it may also be that the planar normal estimation device performs other methods of enhancement processing on the second RGB sample image to obtain the processed second RGB sample image; specifically, the method of the planar normal estimation device performing enhancement processing on the second RGB sample image may be determined according to the actual situation, and the embodiments of the present application do not limit this.

[0046] In the embodiments of the present application, the second RGB sample image may be an image stored in the planar normal estimation device; the second RGB sample image may also be an image transmitted by other devices received by the planar normal estimation device; the second RGB sample image may also be an image obtained by the planar normal estimation device from the client; specifically, the acquisition method of the second RGB sample image may be determined according to the actual situation.

[0047] In the embodiments of the present application, the second RGB sample image may be a human image; the second RGB sample image may also be an animal image; the second RGB sample image may also be a scenery image; the second RGB sample image may also be an indoor image; specifically, the content of the second RGB sample image may be determined according to the actual situation, and the embodiments of the present application do not limit this.

[0048] In an embodiment of the present application, the process of the planar normal estimation device training an initial planar normal estimation model by using a first RGB sample image, a first sample normal label, a second RGB sample image, and a processed second RGB sample image to obtain a planar normal estimation model includes: the planar normal estimation device inputs the first RGB sample image into the initial planar normal estimation model to obtain a first output normal label; the planar normal estimation device inputs the second RGB sample image into the initial planar normal estimation model to obtain a second output normal label; the planar normal estimation device inputs the processed second RGB sample image into the initial planar normal estimation model to obtain a third output normal label; the planar normal estimation device trains the initial planar normal estimation model according to the first output normal label, the second output normal label, the third output normal label, and the first sample normal label to obtain a planar normal estimation model.

[0049] In an embodiment of the present application, the first output normal label includes a first output planar normal label and a first output planar mask label. The second output normal label includes a second output planar normal label and a second output planar mask label. The third output normal label includes a third output planar normal label and a third output planar mask label.

[0050] In an embodiment of the present application, the process of the planar normal estimation device training the initial planar normal estimation model according to the first output normal label, the second output normal label, the third output normal label, and the first sample normal label to obtain a planar normal estimation model includes: the planar normal estimation device determines a first loss between the first output normal label and the first sample normal label; the planar normal estimation device determines a second loss between the second output normal label and the third output normal label; the planar normal estimation device determines a total loss between the first loss and the second loss; the planar normal estimation device trains the initial planar normal estimation model according to the total loss to obtain a planar normal estimation model.

[0051] In an embodiment of the present application, the first output normal label includes a first output planar normal label and a first output planar mask label; the first sample normal label includes a first sample planar normal label and a first sample planar mask label; the process of the planar normal estimation device determining a first loss between the first output normal label and the first sample normal label includes: the planar normal estimation device determines a first planar normal loss between the first output planar normal label and the first sample planar normal label; the planar normal estimation device determines a first planar mask loss between the first output planar mask label and the first sample planar mask label; the planar normal estimation device uses the first planar normal loss and the first planar mask loss as the first loss.

[0052] In an embodiment of the present application, the second output normal label includes a second output plane normal label and a second output plane mask label; the third output normal label includes a third output plane normal label and a third output plane mask label; the process by which the plane normal estimation device determines a second loss between the second output normal label and the third output normal label includes: the plane normal estimation device determines a second plane normal loss between the second output plane normal label and the third output plane normal label; the plane normal estimation device determines a second plane mask loss between the second output plane mask label and the third output plane mask label; the plane normal estimation device uses the second plane normal loss and the second plane mask loss as the second loss.

[0053] Exemplarily, the process of training a plane normal model using semi-supervised learning is as Figure 3 shown: First, labeled data and unlabeled data are collected; the first RGB sample image (I_labeled) is input into the initial plane normal estimation model (Model) to obtain a first output plane normal label (normal1) and a first output plane mask label (mask1). The plane normal estimation device determines a first plane normal loss (L_normal1) and a first plane mask loss (L_mask1) using the first sample plane normal label (normal_gt) and the first sample plane mask label (mask_gt) corresponding to the first RGB sample image respectively; the second RGB sample image (I_unlabeled) is data-augmented to obtain a processed second RGB sample image (I_enhance). The second RGB sample image is input into the initial plane normal estimation model to obtain a second output plane normal label (normal2) and a second output plane mask label (mask2). The processed second RGB sample image is input into the initial plane normal estimation model to obtain a third output plane normal label (normal3) and a third output plane mask label (mask3). A second plane normal loss (L_normal2) is determined according to the second output plane normal label and the third output plane normal label; a second plane mask loss (L_mask2) is determined according to the second output plane mask label and the third output plane mask label; a total loss (L) is determined according to the first plane normal loss, the first plane mask loss, the second output plane mask label, and the third output plane mask label to train the initial plane normal estimation model using the total loss, thereby obtaining a plane normal estimation model.

[0054] In an embodiment of the present application, the process of the plane normal estimation device performing image enhancement processing on the second RGB sample image to obtain the processed second RGB sample image includes: the plane normal estimation device adding noise to the second RGB sample image to obtain the processed second RGB sample image; or, the plane normal estimation device adding patches to the second RGB sample image to obtain the processed second RGB sample image.

[0055] Exemplarily, for the image with added noise, its plane mask and plane normal remain unchanged compared to before adding the noise; for the image with added patches, its plane mask needs to set the position of the patches as the background, as Figure 4 shown: Among them, Figure 4 Figure (a) in is the unlabeled image; Figure (b) is the mask image of a certain plane in the image; Figure (c) is the image with added patches; Figure (d) is the mask image of a certain plane after adding patches, and the plane normal of the image with added patches remains unchanged.

[0056] It can be understood that the plane normal estimation device trains a plane normal estimation model by using the first RGB sample image with the first sample normal label and the second RGB image without the sample normal label. In the case where the number of the first RGB sample images with the first sample normal label is small, the plane normal estimation device can jointly train a plane normal estimation model by using a small number of the first RGB sample images, a small number of the first sample normal labels, and the second RGB sample images, improving the generalization ability of the plane normal estimation model and the accuracy of estimating the plane normal corresponding to the image to be processed.

[0057] Embodiment 2

[0058] Based on the same inventive concept as Embodiment 1, an embodiment of the present application provides a plane normal estimation device 1, corresponding to a plane normal estimation method; Figure 5 It is a schematic composition structure of a plane normal estimation device provided by an embodiment of the present application Figure 1 , and the plane normal estimation device 1 may include:

[0059] An acquisition unit 11, configured to acquire an RGB image;

[0060] An input unit 12, configured to input the RGB image into the plane normal estimation model to obtain the normal information corresponding to the RGB image; the plane normal estimation model is a model trained according to the first RGB sample image with the first sample normal label and the second RGB sample image without the sample normal label.

[0061] In some embodiments of the present application, the device further includes an enhancement unit and a training unit;

[0062] The obtaining unit 11 is configured to obtain the first RGB sample image and the first sample normal label; and obtain the second RGB sample image;

[0063] The enhancement unit is configured to perform image enhancement processing on the second RGB sample image to obtain the processed second RGB sample image;

[0064] The training unit is configured to train an initial plane normal estimation model by using the first RGB sample image, the first sample normal label, the second RGB sample image, and the processed second RGB sample image to obtain the plane normal estimation model.

[0065] In some embodiments of the present application, the input unit 12 is configured to input the first RGB sample image into the initial plane normal estimation model to obtain a first output normal label; input the second RGB sample image into the initial plane normal estimation model to obtain a second output normal label; and input the processed second RGB sample image into the initial plane normal estimation model to obtain a third output normal label;

[0066] The training unit is configured to train the initial plane normal estimation model according to the first output normal label, the second output normal label, the third output normal label, and the first sample normal label to obtain the plane normal estimation model.

[0067] In some embodiments of the present application, the apparatus further includes a determination unit;

[0068] The determination unit is configured to determine a first loss between the first output normal label and the first sample normal label; determine a second loss between the second output normal label and the third output normal label; and determine a total loss between the first loss and the second loss;

[0069] The training unit is configured to train the initial plane normal estimation model according to the total loss to obtain the plane normal estimation model.

[0070] In some embodiments of the present application, the first output normal label includes a first output plane normal label and a first output plane mask label; the first sample normal label includes a first sample plane normal label and a first sample plane mask label;

[0071] The determining unit is configured to determine a first plane normal loss between the first output plane normal label and the first sample plane normal label; determine a first plane mask loss between the first output plane mask label and the first sample plane mask label; and use the first plane normal loss and the first plane mask loss as the first loss.

[0072] In some embodiments of the present application, the second output normal label includes a second output plane normal label and a second output plane mask label; the third output normal label includes a third output plane normal label and a third output plane mask label;

[0073] The determining unit is configured to determine a second plane normal loss between the second output plane normal label and the third output plane normal label; determine a second plane mask loss between the second output plane mask label and the third output plane mask label; and use the second plane normal loss and the second plane mask loss as the second loss.

[0074] In some embodiments of the present application, the apparatus further includes an adding unit;

[0075] The adding unit is configured to add noise to the second RGB sample image to obtain the processed second RGB sample image; or add a patch to the second RGB sample image to obtain the processed second RGB sample image.

[0076] In some embodiments of the present application, the input unit 12 is configured to input the RGB image into a plane normal estimation model to obtain a plane mask corresponding to the RGB image and a plane normal corresponding to the RGB image; the plane normal is the normal corresponding to the plane mask; and use the plane mask and the plane normal as the normal information.

[0077] It should be noted that in practical applications, the above-mentioned obtaining unit 11 and input unit 12 can be implemented by a processor 13 on the plane normal estimation device 1, specifically implemented by a CPU (Central Processing Unit), an MPU (Microprocessor Unit), a DSP (Digital Signal Processing), or a field programmable gate array (FPGA), etc.; the above-mentioned data storage can be implemented by a memory 14 on the plane normal estimation device 1.

[0078] Embodiments of the present application further provide a plane normal estimation device 1, as Figure 6As shown, the plane normal estimation device 1 includes: a processor 13, a memory 14, and a communication bus 15. The memory 14 communicates with the processor 13 through the communication bus 15. The memory 14 stores programs executable by the processor 13. When the programs are executed, the plane normal estimation method as described above is executed by the processor 13.

[0079] In practical applications, the above-mentioned memory 14 can be a volatile memory, such as a Random-Access Memory (RAM); or a non-volatile memory, such as a Read-Only Memory (ROM), a flash memory, a Hard Disk Drive (HDD), or a Solid-State Drive (SSD); or a combination of the above types of memories, and provides instructions and data to the processor 13.

[0080] An embodiment of the present application provides a computer-readable storage medium with a computer program thereon. When the program is executed by the processor 13, the plane normal estimation method as described above is implemented.

[0081] It can be understood that the plane normal estimation device trains a plane normal estimation model by using a first RGB sample image with a first sample normal label and a second RGB image without a sample normal label. In the case where the number of first RGB sample images with a first sample normal label is small, the plane normal estimation device can use a small number of first RGB sample images, a small number of first sample normal labels, and second RGB sample images to jointly train a plane normal estimation model, improving the generalization ability of the plane normal estimation model and the accuracy of estimating the plane normal corresponding to the image to be processed.

[0082] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories and optical memories, etc.) containing computer-usable program codes.

[0083] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices produce means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in one block or multiple blocks.

[0084] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in one block or multiple blocks.

[0085] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in one block or multiple blocks.

[0086] As described above, it is only a preferred embodiment of the present application and is not used to limit the protection scope of the present application.

Claims

1. A method for estimating a plane normal, characterized in that, The method includes: Obtain an RGB image; Input the RGB image into a plane normal estimation model to obtain the normal information corresponding to the RGB image; the plane normal estimation model is obtained according to the following steps: Obtain a first RGB sample image and a first sample normal label; Obtain a second RGB sample image, and perform image enhancement processing on the second RGB sample image to obtain the processed second RGB sample image; Input the first RGB sample image into an initial plane normal estimation model to obtain a first output normal label; Input the second RGB sample image into the initial plane normal estimation model to obtain a second output normal label; Input the processed second RGB sample image into the initial plane normal estimation model to obtain a third output normal label; Train the initial plane normal estimation model according to the first output normal label, the second output normal label, the third output normal label, and the first sample normal label to obtain the plane normal estimation model.

2. The method according to claim 1, wherein The training of the initial plane normal estimation model according to the first output normal label, the second output normal label, the third output normal label, and the first sample normal label to obtain the plane normal estimation model includes: Determine a first loss between the first output normal label and the first sample normal label; Determine a second loss between the second output normal label and the third output normal label; Determine a total loss between the first loss and the second loss; Train the initial plane normal estimation model according to the total loss to obtain the plane normal estimation model.

3. The method according to claim 2, wherein The first output normal label includes a first output plane normal label and a first output plane mask label; the first sample normal label includes a first sample plane normal label and a first sample plane mask label; the determination of the first loss between the first output normal label and the first sample normal label includes: Determine a first plane normal loss between the first output plane normal label and the first sample plane normal label; Determine a first plane mask loss between the first output plane mask label and the first sample plane mask label; Take the first plane normal loss and the first plane mask loss as the first loss.

4. The method according to claim 2, wherein The second output normal label includes a second output plane normal label and a second output plane mask label; the third output normal label includes a third output plane normal label and a third output plane mask label; the determination of the second loss between the second output normal label and the third output normal label includes: Determine a second plane normal loss between the second output plane normal label and the third output plane normal label; Determine a second plane mask loss between the second output plane mask label and the third output plane mask label; Take the second plane normal loss and the second plane mask loss as the second loss.

5. The method according to claim 1, wherein The performing of image enhancement processing on the second RGB sample image to obtain the processed second RGB sample image includes: Add noise to the second RGB sample image to obtain the processed second RGB sample image; Alternatively, add patches to the second RGB sample image to obtain the processed second RGB sample image.

6. The method according to claim 1, wherein The step of inputting the RGB image into a plane normal estimation model to obtain the normal information corresponding to the RGB image includes: Input the RGB image into a plane normal estimation model to obtain a plane mask corresponding to the RGB image and a plane normal corresponding to the RGB image; the plane normal is the normal corresponding to the plane mask; Use the plane mask and the plane normal as the normal information.

7. A planar normal estimation device, characterized in that, The device includes: An acquisition unit for acquiring an RGB image; An input unit for inputting the RGB image into a plane normal estimation model to obtain the normal information corresponding to the RGB image; the plane normal estimation model is obtained according to the following steps: Acquire a first RGB sample image and a first sample normal label; Acquire a second RGB sample image and perform image enhancement processing on the second RGB sample image to obtain a processed second RGB sample image; Input the first RGB sample image into an initial plane normal estimation model to obtain a first output normal label; Input the second RGB sample image into the initial plane normal estimation model to obtain a second output normal label; Input the processed second RGB sample image into the initial plane normal estimation model to obtain a third output normal label; Train the initial plane normal estimation model according to the first output normal label, the second output normal label, the third output normal label, and the first sample normal label to obtain the plane normal estimation model.

8. A planar normal estimation device, characterized in that, The device includes: A memory, a processor, and a communication bus. The memory communicates with the processor through the communication bus. The memory stores a program for plane normal estimation executable by the processor. When the program for plane normal estimation is executed, the method according to any one of claims 1 to 6 is executed by the processor.

9. A storage medium having a computer program stored thereon, which is applied to a planar normal estimation device, characterized in that, When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images