Self-supervised occupancy network training method, system and device based on difficulty

Abstract

The invention relates to the field of network training, in particular to a difficulty-based self-supervised occupancy network training method, system and device, and the method comprises the steps: carrying out the sampling of a current frame camera image, and obtaining the features of voxels; predicting the rough occupancy probability of the voxel, defining the difficulty of the voxel, and obtaining the final occupancy probability of the voxel based on the difficulty of the voxel; rendering based on a volume rendering method to obtain a current camera depth map; calculating a depth difficulty map, and performing iterative up-sampling from coarse to fine based on the depth difficulty map to obtain a target resolution depth map; and pixel-by-pixel weighting is carried out on the luminosity reprojection loss function by using the difficulty degree of the target resolution depth map. According to the method, more accurate occupancy probability prediction can be obtained, so that a difficult area gets enough attention; the calculation amount can be reduced, so that memory and calculation resources are saved; therefore, the model can pay more attention to occupation prediction and depth rendering of the difficult area during training, so that a better training effect is achieved.

Images