Cargo classification method based on monocular vision

Images are collected by a monocular camera and combined with the depth prediction network to obtain depth information. The YOLOv5 network plus ReCBS module is used for classification, which solves the problem of insufficient information and poor robustness of two-dimensional image recognition in express parcel classification, achieving higher Classification accuracy and lower hardware requirements.

CN119091430APending Publication Date: 2024-12-06蒋薇
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202311554936.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-11-20
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In the prior art, two-dimensional image recognition is used for express parcel classification with limited information, low accuracy, poor robustness, high requirements for image quality and shooting angle, and high demand for hardware equipment.

Method used

A monocular camera is used to acquire images and combine the depth prediction network to obtain depth information, input the item classification network for classification, and use the YOLOv5 network plus the ReCBS module to improve classification accuracy and reduce hardware requirements.

Benefits of technology

It improves the accuracy of classification results and the robustness of the system, reduces the requirements for image quality and shooting angle, and reduces hardware costs and manual re-inspect costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091430A_ABST
    Figure CN119091430A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of calculators, and discloses a monocular vision-based cargo classification method, which comprises the following steps of: carrying out image acquisition on a target object by adopting a monocular camera; the collected images are preprocessed; inputting the preprocessed image into a depth prediction network to obtain depth information of the target object; and inputting the image acquired by the monocular camera and the depth information of the target object into an article classification network to obtain a classification result. Compared with a traditional 2D image processing method, the method has the advantages that more information is collected, the requirement for subsequent detection work is reduced, and the final classification result is higher in precision; the cost of acquisition equipment is reduced, and image acquisition can be completed by only one monocular camera; compared with a traditional method, the detection precision is improved, and the manual reinspection cost is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of computers, and in particular relates to a cargo classification method based on monocular vision. Background Art

[0002] In order to achieve the classification of express parcels, most of the existing technical solutions currently use two-dimensional image recognition methods, and the classification effect is poor. In the process of realizing the present invention, the inventor found that there are at least the following problems in the existing technology: the amount of information in the two-dimensional image is limited and cannot be accurately classified; the recognition accuracy for different categories of express parcels is low and the robustness is poor; the image quality and shooting angle are high; the existing image processing solutions have high requirements for running hardware equipment. Summary of the invention

[0003] The present invention aims to solve the above technical problems at least to a certain extent. To this end, the present invention aims to provide a cargo classification method based on monocular vision.

[0004] The technical solution adopted by the present invention is:

[0005] A cargo classification method based on monocular vision comprises the following steps:

[0006] A monocular camera is used to capture images of the target object;

[0007] Preprocess the collected images;

[0008] Input the preprocessed image into the depth prediction network to obtain the depth information of the target object;

[0009] The image captured by the monocular camera and the depth information of the target object are input into the object classification network to obtain the classification result.

[0010] Preferably, a monocular camera is calibrated before capturing an image of the target object to obtain the intrinsic and extrinsic parameters of the monocular camera. The shooting angle of the monocular camera is perpendicular to the surface of the target object. The image preprocessing includes color space enhancement and scale transformation enhancement.

[0011] Preferably, the method for acquiring the training data set of the depth prediction network includes: using a monocular camera to simulate the actual operating environment to collect training images, and using a 3D camera to collect corresponding depth images as ground truth during training.

[0012] Preferably, the object classification network is based on the YOLOv5 network, and a ReCBS module is added. The ReCBS module is used to add the input to the original input after passing through a 3*3 convolution layer and a 1*1 convolution layer, and then perform BN layer and SiLu layer processing.

[0013] The beneficial effects of the present invention are:

[0014] The present invention provides a cargo classification method based on monocular vision, which uses a monocular camera as the acquisition end, and then inputs the image into a trained depth prediction network and a depth prediction network to classify the items; compared with the traditional 2D image processing method, more information is collected, the requirements for subsequent detection work are reduced, and the final classification result is more accurate; the cost of the acquisition equipment is reduced, and only one monocular camera is required to complete the image acquisition; compared with the traditional method, the detection accuracy is improved and the cost of manual re-inspection is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of the cargo classification method based on monocular vision of the present invention.

[0016] Figure 2 is a schematic diagram of the ReCBS module of the present invention. DETAILED DESCRIPTION

[0017] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] It should be understood that it should also be noted that in the embodiment, the functions / actions that appear may be different from the order in which the drawings appear. For example, depending on the functions / actions involved, they can actually be executed substantially concurrently, or two figures shown in succession can sometimes be executed in the reverse order.

[0019] like Figure 1 As shown, a cargo classification method based on monocular vision in this embodiment includes the following steps:

[0020] S1. Arrange the monocular camera so that the shooting angle of the monocular camera is perpendicular or nearly perpendicular to the surface of the target object to ensure the accuracy of the depth information obtained later and avoid the projection error that may be introduced due to the large angle. At the same time, ensure that the field of view of the monocular camera is large enough to capture the entire target object; then calibrate the monocular camera, obtain the intrinsic and extrinsic parameters of the monocular camera, and use the monocular camera to collect images of the target object.

[0021] S2. Preprocess the collected images; the preprocessing of the images includes color space enhancement, scale transformation enhancement, etc. The purpose of the preprocessing operation is to perform a series of processing on the image before inputting the image into the deep prediction network, improve the image quality and information, make it more suitable for subsequent processing, and improve the performance of the deep prediction network and the understanding of the image.

[0022] Preprocessing operations include:

[0023] Denoising and enhancement: Some images may contain noise or unclear areas of detail. Denoising and enhancement operations help improve image quality to ensure that the deep prediction network can identify important features;

[0024] Scale transformation: changing the size or scale of the image to fit the network's input requirements. This is very important to ensure that the input images are of uniform size, as depth prediction networks usually require input images to have the same dimensions;

[0025] Reduce redundant information: Sometimes images contain unimportant redundant information, such as borders or backgrounds; this information can be reduced by cropping or using edge detection to reduce the computational burden of the depth prediction network;

[0026] Contrast enhancement: Increasing the contrast of an image helps to better highlight features in the image; this is usually achieved through methods such as histogram equalization;

[0027] Reducing lighting effects: Lighting variations can have a negative impact on image processing and feature extraction; by applying lighting correction techniques, this effect can be mitigated.

[0028] S3. Input the preprocessed image into the depth prediction network to obtain the depth information of the target object. The depth prediction network is a conventional depth prediction network and belongs to an absolute depth prediction network, but it needs to use a self-made data set as a training data set to make up for the lack of generalization of using a public data set. The method for obtaining the self-made data set includes: using a monocular camera to simulate the actual operating environment to collect training images, and using a 3D camera to collect corresponding depth images as ground truth during training.

[0029] The preprocessed image is input into the depth prediction network. The task of the depth prediction network is to estimate the depth information of objects in the image. Finally, the depth prediction network generates a depth map in which each pixel corresponds to the distance from each point in the image to the camera.

[0030] S4. The original image (which can be an RGB image) captured by the monocular camera and the depth information of the target object are input into the object classification network. The object classification network is trained to analyze this information to determine the category of the object in the image. The output of the object classification network is a label indicating the category to which the object in the image belongs.

[0031] The key to achieving high-precision classification in object classification networks is to use depth information, not just traditional 2D image information. Depth information allows the network to better understand the three-dimensional spatial properties of objects, which can bring more accurate classification results. In addition, depth information also helps to overcome some problems in 2D images, such as perspective deformation or occlusion.

[0032] The object classification network is based on the YOLOv5 network and adds the ReCBS module. Figure 2 As shown in the figure, the ReCBS module is used to add the input to the original input after passing it through a 3*3 convolution layer and a 1*1 convolution layer, and then perform BN layer and SiLu layer processing. Adding the ReCBS module can enhance the classification accuracy of the classification network model by improving feature representation, reducing overfitting risks, or improving the model's ability to distinguish different categories of items, thereby improving network efficiency with the same number of parameters.

[0033] The ReCBS module adopts a parameter sharing strategy. Since the number of parameters is usually proportional to the number of layers and width of the network, the introduction of parameter sharing allows some parameters to be shared by multiple parts, which can significantly reduce the number of parameters and reduce the computational burden during training and inference, thus greatly shortening the running time. At the same time, parameter sharing in the ReCBS module can also reduce the complexity of the object classification network and improve efficiency.

[0034] In this embodiment, a cargo classification method based on monocular vision not only obtains the surface appearance (2D information) of the image through monocular 3D vision, but also obtains the depth information of the target object, that is, its position in three-dimensional space. This depth information provides the spatial position and shape of the object, which is much more detailed than using only 2D information;

[0035] The acquisition of depth information makes the system more robust and reduces the high requirements on image quality and shooting angle, because even under different lighting conditions or shooting angles, the depth information can still be used for accurate positioning and classification of objects, which reduces the work requirements and makes the method more reliable in different environments;

[0036] Since depth information provides more spatial position data of objects, the classification results of this method are more accurate. It can determine the position of objects more accurately without being disturbed by occlusion or other factors.

[0037] The present invention is not limited to the above-mentioned optional implementation modes. Anyone can derive other various forms of products under the inspiration of the present invention. However, no matter what changes are made in the shape or structure, all technical solutions that fall within the scope defined by the claims of the present invention fall within the protection scope of the present invention.

Claims

1. A cargo classification method based on monocular vision, characterized in that: The following steps are involved: A monocular camera is used to capture images of the target object; Preprocess the collected images; Input the preprocessed image into the depth prediction network to obtain the depth information of the target object; The image captured by the monocular camera and the depth information of the target object are input into the object classification network to obtain the classification result.

2. The cargo classification method according to claim 1, characterized in that: Before capturing images of the target object, the monocular camera is calibrated to obtain the intrinsic and extrinsic parameters of the monocular camera.

3. The cargo classification method according to claim 1, characterized in that: The shooting angle of the monocular camera is perpendicular to the surface of the target object.

4. The cargo classification method according to claim 1, characterized in that: Image preprocessing includes color space enhancement and scale transformation enhancement.

5. The cargo classification method according to claim 1, characterized in that: The method for obtaining the training data set of the depth prediction network includes: using a monocular camera to simulate the actual operating environment to collect training images, and using a 3D camera to collect corresponding depth images as ground truth during training.

6. The cargo classification method according to any one of claims 1 to 5, characterized in that: The object classification network is based on the YOLOv5 network and adds the ReCBS module.

7. The cargo classification method according to claim 6, characterized in that: The ReCBS module is used to add the input to the original input after passing it through a 3*3 convolution layer and a 1*1 convolution layer, and then perform processing on the BN layer and SiLu layer.

Citation Information

Patent Citations

  • Convolutional neural network object identification method based on depth information pre-segmentation

    CN107563388A

  • Article classification method based on depth recovery information

    CN108520535A

  • Image-based object recognition method, commodity self-service cashier system and electronic device

    CN109886112A

  • Robust single-camera depth map estimation method

    CN110766732A

  • Data processing method and device

    CN112241978A