Medicine box detection method based on three-dimensional point cloud and image data fusion and detection system thereof

Abstract

The invention discloses a medicine box detection method based on three-dimensional point cloud and image data fusion, and the method comprises the steps: inputting a medicine box image collected by a camera into an optimized U-shaped full convolutional neural network, and extracting a medicine box feature image; in the feature extraction part, a grouping residual error convolution module is used for extracting a preliminary feature image, a cavity space convolution pooling pyramid module is used for extracting feature image information of different scales of the preliminary feature image, and a mixed attention module is used for fusing the feature image information of different scales to obtain a two-dimensional fusion feature image; a segmented medicine box image is obtained through up-sampling; and whether the segmented medicine box image meets the detection requirement is judged, if not, three-dimensional information of the medicine box is extracted, a target in the image is positioned through a two-dimensional target detection network, and a cone point cloud corresponding to the two-dimensional detection frame is obtained according to a camera geometric imaging model. A Point Net point cloud network and a feature fusion network layer are adopted to carry out instance segmentation on the cone point cloud to obtain all target points. A target centroid is estimated by using a T-Net network, a target point cloud is moved to a centroid coordinate system, then estimation of parameters of a three-dimensional bounding box is obtained through a parameter estimation network and a feature fusion network layer, finally, the size and orientation of a medicine box are obtained, the type of the medicine box is judged, and the character 0 of the medicine box is recognized in combination with image information. The problems of long time consumption, high false detection rate and the like in existing medicine box detection are solved.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and in particular relates to a medicine box detection method based on fusion of three-dimensional point cloud and image data. Background technique [0002] With the rapid development of convolutional neural network, it has demonstrated its powerful ability in feature learning, and has made very significant breakthroughs and progress in many computer vision tasks. A large number of computer vision researches mainly focus on two-dimensional images. However, our real world is a three-dimensional world, and there is an inevitable loss of information when the camera projects a three-dimensional scene to a two-dimensional image. 3D data is relatively insensitive to factors such as illumination changes and texture changes. The image processing of medicine boxes is prone to algorithm performance degradation under strong and weak light conditions and insufficient texture information. However, 3D d...

AI Technical Summary

Problems solved by technology

[0003] Aiming at the deficiencies in the prior art, the present invention provides a medicine box detection method based on the fusion of 3D point cloud and image data, which overcomes the problems of long time consumption and high false detection rate in the existing medicine box detection

Images