Automatic recognition method of power cable reels based on depth images

Abstract

The present invention relates to the field of computer image processing, and in particular to the application of computer image processing in large-scale warehousing, and more specifically to an automatic recognition method for power cable reels based on depth images, which uses depth images to fuse feature points in different fields of view to complete feature point matching between images, and obtains the center coordinates of each cable reel according to machine vision system calculations. Furthermore, by further converting the cable reel depth image into point cloud data, the spatial three-dimensional coordinates (camera coordinate system) of the center coordinates of each cable reel are obtained, and the reference point is used to convert the cable reel center point into the loading and unloading site coordinates in millimeters, thereby realizing the automatic intelligent loading and unloading operation of the overhead crane.

Description

Technical Field

[0001] The present invention relates to the field of computer image processing, in particular to the application field of computer image processing in large-scale warehouses, and more specifically to an automatic recognition method for power cable reels based on depth images. Background Art

[0002] Machine vision is a technology that uses computers to simulate human visual functions, analyze and collect image data, extract key information, and perform structured processing and analysis. The machine vision recognition system covers light sources, image acquisition systems, image processing and intelligent analysis systems, and mechanical control systems. It obtains image signals through image acquisition devices such as cameras and converts them into digital signals and transmits them to dedicated image processing systems. It analyzes the characteristics of the target image based on information such as pixel distribution, brightness and color, and then conducts extended analysis and application.

[0003] With the improvement of the automation level of power material warehouses, cable reels, as the basic storage units of power cables in warehouses, are affected by the differences in specifications and models of the cable reels themselves and the warehouse infrastructure equipment, making it difficult to achieve fully automatic unmanned loading and unloading operations.

[0004] One of the most important technical difficulties is the inability to accurately identify the position of the cable reel on the vehicle panel during the warehousing process.

[0005] Therefore, applying visual recognition technology to intelligently identify the center coordinates of cable reels of different specifications and models is a prerequisite for realizing the automation of loading and unloading operations, and is also the focus of research in this field. Summary of the invention

[0006] The purpose of the invention is to provide a method for automatic identification of power cable reels with high accuracy and strong applicability to realize the automation of cable reel loading and unloading operations in power warehouses, in order to address the problem that the current warehousing process cannot accurately and automatically identify the coordinate position of cable reels in the warehousing process, thereby limiting the development of warehousing automation and unmanned operations.

[0007] In order to achieve the above-mentioned object of the invention, the present invention discloses a method for automatic identification of power cable reels based on depth images, comprising the following steps:

[0008] Step 1: Depth cameras are set up at different locations in the warehouse unloading area. The depth cameras at different locations take pictures of the cable reels respectively to form a set of cable reel depth image scene graphs based on different angles;

[0009] Step 2: Label different depth image scene graphs respectively, and mark the cable reels to be loaded and unloaded, the outline edges of the cable reels to be loaded and unloaded, and the background of the cable reels to be loaded and unloaded with different colors, so that the area of ​​each cable reel in the image is clear and independent;

[0010] Step 3: Select several feature points and mark them in the depth image scene graph at each angle, and then complete the data association of the depth images at different angles based on the feature points;

[0011] Step 4: Use the computer vision system to calculate the center point position of the cable drum; the specific method is: extract the independent operation area information of the cable drum and match it with all the image feature points obtained by the depth camera, mark the image feature points that successfully match the edge of the independent area as the outer edge contour coordinates (multiple coordinates of the outer circle edge correspond to the same center of the circle), compare the remaining image feature point information with the image library, identify the grasping point coordinates (multiple inner circle edge coordinates and one center coordinate), all of which are marked as grasping point coordinates according to the serial number, calculate the spatial position and distance between the grasping point coordinates and the edge contour coordinates, form a set of position parameters, compare the position parameter information with the coordinate data of each type of cable in the data, and determine the final reasonable grasping point.

[0012] Step 5: Convert the depth image scene graph into point cloud data, and generate the coordinate value of the center point position in the camera coordinate system (i.e., the three-dimensional spatial coordinate system);

[0013] Step 6: Using the depth camera as the calibration point, obtain the conversion relationship between the camera coordinate system and the actual coordinate system, and then convert the spatial three-dimensional coordinate value of the cable drum center point obtained in step 5 into the actual site coordinate value.

[0014] As a preferred technical solution, step 7 is also included: according to the communication protocol, the actual site coordinate value of the center position of the cable reel obtained in step 6 is transmitted to the intelligent traveling crane, and the intelligent traveling crane automatically completes the lifting of the cable reel according to the actual site coordinate value.

[0015] In a preferred technical solution, the present invention further discloses that the method for obtaining the conversion relationship in step 6 is to select a point in the depth image as a reference point for relationship calculation, and calculate the spatial three-dimensional coordinate value (camera coordinate system) of the reference point in the point cloud image and the coordinate value in the visual recognition system coordinate system through the relationship, so as to obtain the conversion relationship between the spatial three-dimensional coordinate system and the visual recognition system coordinate system; then use the depth camera as a calibration point to further obtain the conversion relationship between the spatial three-dimensional coordinate value and the actual site coordinate value. Specifically, the imaging principle of the depth camera is used to obtain the conversion relationship between the spatial three-dimensional coordinate value of the reference point and the actual site coordinate value.

[0016] The present invention uses depth images to fuse feature points in different fields of view to match feature points between images, and obtains the center coordinates of each cable drum according to the machine vision system calculation. The cable drum depth image is further converted into point cloud data to obtain the spatial three-dimensional coordinates (camera coordinate system) of the center coordinates of each cable drum, and the reference point is used to convert the center point of the cable drum into the loading and unloading site coordinates in millimeters, thereby realizing the automatic intelligent loading and unloading operation of the overhead crane. DETAILED DESCRIPTION

[0017] In order to better understand the present invention, the present invention is further described below in conjunction with specific embodiments.

[0018] In this embodiment, a method for automatically identifying a power cable reel based on a depth image is disclosed, comprising the following steps:

[0019] Step 1: Depth cameras are set up at different locations in the warehouse unloading area. The depth cameras at different locations take pictures of the cable reels respectively to form a set of cable reel depth image scene graphs based on different angles;

[0020] Step 2: Label different depth image scene graphs respectively, and mark the cable reels to be loaded and unloaded, the outline edges of the cable reels to be loaded and unloaded, and the background of the cable reels to be loaded and unloaded with different colors, so that the area of ​​each cable reel in the image is clear and independent;

[0021] Step 3: Select several feature points and mark them in the depth image scene graph at each angle, and then complete the data association of the depth images at different angles based on the feature points;

[0022] Step 4: Use the computer vision system to calculate the center point of the cable drum; select the first cable drum, extract its independent operation area information, match it with the image feature points of the outer edge of the first cable drum obtained by the depth camera, and mark the image feature points that successfully match the outer edge of the independent area as edge contour coordinates a1, a2, a3..a n , and the corresponding circle center coordinates O1, compare the remaining image feature point information with the image library, and identify the grab point coordinates (inner edge position coordinates) c1, c2, c3..c n , and the corresponding circle center coordinate O2 is marked as the grasping point coordinate. First match the coordinate positions of O1 and O2, and calculate the coordinates of each grasping point c1, c2, c n The spatial position and distance before the edge contour coordinates form three sets of position parameters d1, d2, d n , compare the position parameter information of each grabbing point coordinate with the data of each type of cable reel in the data to determine the final reasonable grabbing point Ct.

[0023] Step 5: Convert the depth image scene graph into point cloud data, and generate the coordinate value of the center point position in the camera coordinate system (i.e., the three-dimensional spatial coordinate system);

[0024] Step 6: Using the depth camera as the calibration point, obtain the conversion relationship between the camera coordinate system and the actual coordinate system, and then convert the spatial three-dimensional coordinate value of the cable drum center point obtained in step 5 into the actual site coordinate value.

[0025] Specifically, a certain point is selected in the depth image as a reference point for relationship calculation, and the spatial three-dimensional coordinate value (camera coordinate system) of the reference point in the point cloud image and the coordinate value in the visual recognition system coordinate system are calculated through the relationship, so as to obtain the conversion relationship between the spatial three-dimensional coordinate system and the visual recognition system coordinate system; then the depth camera is used as a calibration point to further obtain the conversion relationship between the spatial three-dimensional coordinate value and the actual site coordinate value. Specifically, the imaging principle of the depth camera is used to obtain the conversion relationship between the spatial three-dimensional coordinate value of the reference point and the actual site coordinate value. Preferably, the present embodiment also includes step 7: according to the communication protocol, the actual site coordinate value at the center of the cable drum obtained in step 6 is transmitted to the intelligent traveling crane, and the intelligent traveling crane automatically completes the lifting of the cable drum according to the actual site coordinate value.

[0026] In a specific application scenario, after the carrier (usually a flatbed truck) loaded with cables arrives at the designated loading and unloading area, depth cameras at different positions respectively photograph the carrier plate and cable reel, thereby outputting multi-frame depth image scene graph information.

[0027] This image is a complete image of the vehicle loaded with cables. That is to say, if there are multiple cable drums on the vehicle board, the scene image containing multiple cable drums is captured.

[0028] Then, the cable reels in the above overall scene are marked, specifically, the cable reels to be loaded and unloaded in the image, the outline edges of the cable reels to be loaded and unloaded, and the background of the cable reels to be loaded and unloaded are marked with different colors. Through this marking operation, the independent area of ​​each cable reel on the vehicle board can be distinguished.

[0029] Then, feature points are selected, and feature points in different fields of view are fused to complete feature point matching between images. The center coordinates of each cable reel are obtained according to the calculation of the machine vision system.

[0030] Finally, the cable reel depth image is converted into point cloud data, and the spatial three-dimensional coordinates (camera coordinate system) of the center coordinates of each cable reel are obtained. The conversion matrix calculated using the reference point (the position of the depth camera) is used to convert the cable reel center point into loading and unloading site coordinates in millimeters. This coordinate information is transmitted to the intelligent overhead crane, and special lifting equipment is further mobilized to complete the automated loading and unloading operations of the cable reel.

[0031] The above is a specific implementation of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also considered to be within the protection scope of the present invention.

Claims

1. A method for automatic identification of power cable reels based on depth images, characterized in that: The following steps are involved: Step 1: Depth cameras are set up at different locations in the warehouse unloading area. The depth cameras at different locations take pictures of the cable reels respectively to form a set of cable reel depth image scene graphs based on different angles; Step 2: Label different depth image scene graphs respectively, and mark the cable reels to be loaded and unloaded, the outline edges of the cable reels to be loaded and unloaded, and the background of the cable reels to be loaded and unloaded with different colors, so that the area of ​​each cable reel in the image is clear and independent; Step 3: Select several feature points and mark them in the depth image scene graph at each angle, and then complete the data association of the depth images at different angles based on the feature points; Step 4: Use the computer vision system to calculate the center point position of the cable drum; First, extract the independent operation area information of the cable drum and match it with all the image feature points obtained by the depth camera, and mark the image feature points that successfully match the edge of the independent area as the outer edge contour coordinates and the corresponding circle center coordinates O1; then compare the remaining image feature point information with the image library, identify the grasping point coordinates and the corresponding circle center coordinates O2, and mark them as grasping point coordinates according to the serial number, match the O1 and O2 coordinate positions, calculate the spatial position and distance between the grasping point coordinates and the edge contour coordinates, and form a set of position parameters. Finally, compare the position parameter information with the coordinate data of each type of cable in the data to determine the final reasonable grasping point; Step 5: Convert the depth image scene graph into point cloud data, and generate the coordinate value of the center point position in the camera coordinate system (i.e., the three-dimensional spatial coordinate system); Step 6: Using the depth camera as the calibration point, obtain the conversion relationship between the camera coordinate system and the actual coordinate system, and then convert the spatial three-dimensional coordinate value of the cable drum center point obtained in step 5 into the actual site coordinate value.

2. The method for automatic identification of power cable reels based on depth images according to claim 1 is characterized in that: The method also includes step 7: according to the communication protocol, the actual site coordinate value of the center position of the cable reel obtained in step 6 is transmitted to the intelligent traveling crane, and the intelligent traveling crane automatically completes the lifting of the cable reel according to the actual site coordinate value.

3. The method for automatic identification of power cable reels based on depth images according to claim 1, characterized in that: The method for obtaining the conversion relationship in step 6 is to select a point in the depth image as a reference point for relationship calculation, and calculate the spatial three-dimensional coordinate value (camera coordinate system) of the reference point in the point cloud image and the coordinate value in the visual recognition system coordinate system through the relationship, so as to obtain the conversion relationship between the spatial three-dimensional coordinate system and the visual recognition system coordinate system; then use the depth camera as the calibration point to further obtain the conversion relationship between the spatial three-dimensional coordinate value and the actual site coordinate value. Specifically, the imaging principle of the depth camera is used to obtain the conversion relationship between the spatial three-dimensional coordinate value of the reference point and the actual site coordinate value.