Stack position coordinate calibration device and method based on image shape detection and image fusion

Abstract

The present invention provides a stack position coordinate calibration device and method based on image shape detection and image fusion. By adopting the method of simultaneously shooting with multiple cameras and image fusion technology, the problem that the viewing angle range of a single camera is small in a large-scene range and cannot completely cover all stack positions within the entire warehouse is solved. The accurate stitching of the panoramic picture of the entire warehouse is realized, and a complete stack position picture can be obtained immediately. By adopting the contour extraction technology of markers in the image, the accurate image coordinates of the center points of the stack position markers are obtained, and the coordinate conversion algorithm between image coordinates and actual coordinates is used to obtain the coordinates of all markers in the real scene to represent the actual coordinates of the stack positions. The device provided by the present invention has a high degree of automation. Workers only need to calibrate the positions of several stack positions, and the system can automatically calculate the positions of other stack positions.

Description

Technical Field

[0001] The present invention relates to the field of stack position coordinate calibration, and specifically, to a stack position coordinate calibration device and method based on image shape detection and image fusion technology. Background Art

[0002] With the development of target detection technology and ranging technology, the application of automatic control bridge crane technology in steel product warehouses has become more and more extensive. In the daily operation of the automatic control bridge crane in the storage area, the main task is to lift and transport steel products. Specifically, the requirements for lifting and transporting steel products can be divided into two cases: picking up steel products from the stack position and clamping and placing steel products on the stack position. For example, it is required to pick up steel products from the stack position in the first column of the second row and lift and transport them to the empty stack position in the third column of the fifth row.

[0003] Since the actual physical coordinate system of the stack position is different from the coordinate system of the bridge crane, the prerequisite for completing the above automatic operation is to calibrate the accurate position of the stack position first, so that the physical coordinates of each stack position can be reflected in the coordinate system of the bridge crane.

[0004] The existing calibration method - plumb bob measurement method:

[0005] The calibration tools are a plumb bob and a tape measure. This calibration method generally has two steps.

[0006] (1) Obtain the center point of the stack position to be calibrated

[0007] Use a tape measure to draw the diagonal of the stack position to obtain the center point position of the stack position. After obtaining the center point, make a mark with a prominent sign.

[0008] (2) Obtain the difference between the center point of the stack position to be calibrated and the position coordinates of the bridge crane

[0009] The ground personnel command the bridge crane clamp to move as close as possible to the center of the stack position (the mark made in the first step). Read the coordinates (x, y) at this time on the vehicle-mounted terminal of the bridge crane. Then fix the plumb bob at the center position of the clamp. After the plumb bob stabilizes and does not shake, measure the coordinate difference (Δx, Δy) between the position of the plumb bob and the center of the stack position. If the deviation is in the positive direction, the coordinates of the center point of the stack position in the coordinate system of the bridge crane are (x + Δx, y + Δy); if the deviation is in the negative direction, the coordinates of the center point of the stack position in the coordinate system of the bridge crane are (x - Δx, y - Δy).

[0010] The obtained stack position coordinates are stored in the database, and then the bridge crane control system will perform operations according to the coordinates in the future.

[0011] Disadvantages: Measuring each stack position one by one in this way involves a large amount of work. Moreover, since it is necessary to coordinate with a bridge crane for cooperation, it takes a long time. In addition, because ground personnel need to be near the clamp all the time, there is a certain degree of danger relatively speaking.

[0012] Regarding the calibration of the entire row of stack positions.

[0013] The above steps illustrate the calibration method for a single stack position. For the calibration of the entire row of stack positions, if the position accuracy requirements for the stack positions are not very high, the following method is generally used for calibration.

[0014] 1. Obtain the coordinates of the head and tail stack positions through the single-stack-position calibration method: (x 1 , y 1 ), (x 8 , y 8 )

[0015] 2. Since the stack positions on site are not in a straight line, calculate the average value of the Y-axis error through (x 1 , y 1 ), (x 8 , y 8 ):

[0016] 3. Calculate the Y coordinates of other stack positions:

[0017] 4. Calculate the distance between each stack position on the X-axis through (x 1 , x 8 ):

[0018] 5. Calculate the X coordinates of other stack positions:

[0019] Disadvantages: This method reduces the calibration workload by sacrificing accuracy. If it is necessary to improve the calibration accuracy of the stack position, the number of stack positions to be calibrated must be increased, which will greatly increase the workload.

[0020] Therefore, the current traditional method of using a plumb bob to calibrate and measure the stack position coordinates has a huge labor cost and low automation, seriously restricting the efficiency of the system debugging work.

[0021] The patent document "Intelligent Steel Coil Storage Area Lifting Steel Coils and Anti-Collision Detection System for Stored Steel Coils" with the publication number "CN108155915A" discloses an intelligent steel coil storage area lifting steel coils and anti-collision detection system for stored steel coils, which is different from the technical field of the present invention.

[0022] The patent document "An Automatic Identification and Positioning Device for On-vehicle Steel Coils and Vehicle Stacking Positions" with the publication number "CN106327044A" discloses "An Automatic Identification and Positioning Device for On-vehicle Steel Coils and Vehicle Stacking Positions", which also has the above problems. Summary of the Invention

[0023] Aiming at the defects in the prior art, the purpose of the present invention is to provide a stack position coordinate calibration device and method based on image shape detection and image fusion.

[0024] According to one aspect of the present invention, there is provided a stack position coordinate calibration device based on image shape detection and image fusion, including:

[0025] Module M1: Measure the center of each stack position and assemble a prominent marker at each center;

[0026] Module M2: Collect the top images of the stack positions through multiple cameras on the overhead crane;

[0027] Module M3: Use image recognition technology to extract the contour information of the marker from the top image of the stack position to obtain the shape information of the marker;

[0028] Module M4: Perform image preprocessing on the color picture of the marker position at the stack position center to obtain the coordinates of the stack position center point in the image. And identify the position of the marker and the relative positions between the markers;

[0029] Module M5: Measure the world coordinates of a certain stack position marker, assign them to the stack position coordinate calibration system, and obtain the world coordinates of the stack position through the matrix conversion between the coordinates of the stack position center point in the image and the overhead crane coordinate system;

[0030] Module M6: For the top images of the stack positions taken by multiple cameras respectively, use image feature point extraction and registration technology to extract the feature points of the images in the video stream of multiple frames of stack position images obtained, and then perform matching, registration, fusion and boundary processing splicing of the feature points to obtain the panoramic image of the warehouse;

[0031] Module M7: Taking the origin of the overhead crane coordinate as the reference point, use image recognition and extraction technology to convert the world coordinates of all stack positions in the panoramic image into the complete stack position coordinates of the entire warehouse;

[0032] For the case where steel products have been placed on the stack position, the marker can be directly placed at the center position on the top of the steel product, and the system will handle it in the same way as the marker placed on the stack position. Or after the steel product is lifted off, supplementary calibration can be carried out in combination with the nearby calibrated stack positions.

[0033] Preferably, the prominent markers in the module M1 include: regularly shaped objects that can be distinguished by color; if the stack position itself has a certain regular shape or color, the stack position can be directly used as a marker.

[0034] Preferably, after the shooting ranges of the multiple cameras in the module M2 are stitched together, they can cover all the stack position placement areas where coordinates need to be obtained.

[0035] Preferably, in the module M7, the image recognition includes converting the color picture of the position of the central marker of the stack position taken into a grayscale picture, then performing Gaussian filtering on the input grayscale picture, making a grayscale histogram, and extracting the threshold.

[0036] Preferably, in the module M7, the conversion includes converting the image coordinates of the center point of the marker into world coordinates in the bridge crane coordinate system through a matrix.

[0037] According to another aspect of the present invention, there is provided a stack position coordinate calibration method based on image shape detection and image fusion, including:

[0038] Step 1: Measure the center of each stack position and install prominent markers at each center;

[0039] Step 2: Collect the top images of the stack positions through multiple cameras on the bridge crane;

[0040] Step 3: Use image recognition technology to extract the contour information of the marker from the top image of the stack position to obtain the shape information of the marker;

[0041] Step 4: Perform image preprocessing on the color picture of the position of the central marker of the stack position to obtain the coordinates of the center point of the stack position in the image. And identify the position of the marker and the relative positions between the markers;

[0042] Step 5: Measure the world coordinates of a marker of a stack position, assign them into the stack position coordinate calibration system, and obtain the world coordinates of the stack position through the matrix conversion between the coordinates of the center point of the stack position in the image and the bridge crane coordinate system;

[0043] Step 6: For the top images of the stack positions taken by multiple cameras respectively, use image feature point extraction and registration technology to extract the feature points of the images in the video stream from multiple frames of stack position images obtained, and then perform matching, registration, fusion and boundary processing and stitching of the feature points to obtain a panoramic image of the warehouse;

[0044] Step 7: Taking the origin of the bridge crane coordinate as a reference point, use image recognition and extraction technology to convert the world coordinates of all stack positions in the panoramic image into the complete stack position coordinates of the entire warehouse;

[0045] Step 8: For the case where steel products have already been placed on the stack, the marker can be directly placed at the center of the top of the steel products, and the system will handle it in the same way as the marker placed on the stack. Or, after the steel products are lifted off, supplementary calibration can be carried out in combination with the calibrated stacks in the vicinity.

[0046] Preferably, the eye-catching marker in step 1 includes: a regularly shaped object that can be distinguished by color. If the stack itself has a certain regular shape or color, the stack can be directly used as the marker.

[0047] Preferably, after the shooting ranges of the multi-channel cameras in step 2 are stitched together, they can cover all the stack placement areas where coordinates need to be obtained.

[0048] Preferably, in step 7, the image recognition includes converting the color picture of the position of the center marker of the stack into a grayscale picture, then performing Gaussian filtering on the input grayscale picture, making a grayscale histogram, and extracting the threshold.

[0049] Preferably, in step 7, the conversion includes converting the image coordinates of the center point of the marker into the world coordinates in the coordinate system of the overhead crane through a matrix.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] (1) The manual efficiency is improved, and there is no need to coordinate the downtime of the overhead crane, so the calibration efficiency is greatly improved;

[0052] (2) Since it does not require a large amount of occupation of the operation time of the overhead crane, the influence on the operation of the overhead crane is very small;

[0053] (3) The cost is low, and only ordinary cameras are used to complete the calibration of the stacks. Moreover, each device in this device can be removed for use in other storage areas after calibration;

[0054] (4) The degree of automation is high. Workers only need to calibrate the positions of several stacks, and the system can automatically calculate the positions of other stacks. Description of the Drawings

[0055] By reading the detailed description of the non-restrictive embodiments with reference to the following drawings, other features, purposes and advantages of the present invention will become more obvious:

[0056] Figure 1 It is a schematic diagram of the steps of the present invention;

[0057] Figure 2 It is a schematic diagram of the connection method of the multi-channel cameras and the system host;

[0058] Figure 3Recognition and extraction effect diagrams of various regular shapes in the present invention;

[0059] Figure 4 Images of identification objects taken by multiple cameras;

[0060] Figure 5 Diagrams of the image fusion process - feature point extraction and matching, image stitching and fusion results. Specific implementation manners

[0061] The present invention will be described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several changes and improvements can still be made. These all belong to the protection scope of the present invention.

[0062] In this embodiment, a stack position coordinate calibration device based on image shape detection and image fusion provided by the present invention is adopted, as Figure 1 shown, including:

[0063] Module M1: Measure the center of each stack position and assemble a prominent marker at each center;

[0064] Module M2: Collect the top images of the stack positions through multiple cameras on the overhead crane. The connection method between the multiple cameras and the system host is as Figure 2 shown;

[0065] Module M3: Adopt image recognition technology to extract the contour information of the marker from the top image of the stack position to obtain the shape information of the marker;

[0066] Module M4: Perform image preprocessing on the color picture of the marker position at the stack position center to obtain the coordinates of the stack position center point in the image. And identify the position of the marker and the relative positions between the markers. The recognition and extraction effects are as Figure 3 shown;

[0067] Module M5: Measure the world coordinates of a certain stack position marker, assign them into the stack position coordinate calibration system, and obtain the world coordinates of the stack position through the matrix conversion between the coordinates of the stack position center point in the image and the overhead crane coordinate system;

[0068] Module M6: For the top images of the stack positions taken by multiple cameras respectively, use image feature point extraction and registration technology to extract the feature points of the images in the video stream from multiple frames of stack position images in the acquired video stream, and then perform feature point matching, registration, fusion and boundary processing stitching to obtain the panoramic image of the warehouse;

[0069] Module M7: Taking the origin of the coordinate system of the bridge crane as the reference point, using image recognition and extraction technology to convert the world coordinates of all stack positions in the panoramic image into the complete stack position coordinates of the entire warehouse;

[0070] Module M8: For the case where steel products have been placed on the stack position, the marker can be directly placed at the center of the top of the steel product, and the system will handle it in the same way as the marker placed on the stack position. Or after the steel product is lifted off, supplementary calibration can be carried out in combination with the nearby stack positions that have been calibrated.

[0071] The prominent markers in Module M1 include: regularly shaped objects that can be distinguished by color; if the stack position itself has a certain regular shape or color, the stack position can be directly used as a marker.

[0072] After the shooting ranges of the multiple cameras in Module M2 are stitched together, they can cover the entire stack position placement area where coordinates need to be obtained.

[0073] In Module M7, the image recognition includes converting the color picture of the position of the marker at the center of the stack position taken into a grayscale image, then performing Gaussian filtering on the input grayscale image, making a grayscale histogram, and extracting the threshold.

[0074] In Module M7, the conversion includes converting the image coordinates of the center point of the marker into the world coordinates in the coordinate system of the bridge crane through a matrix.

[0075] Specifically, first, the staff installs 4 network cameras on the bridge crane at the top of the stack position to ensure that the shooting areas of the 4 cameras can cover the entire stack position placement area in the warehouse and correctly collect the top images of the stack positions. Assemble prominent circular markers at the center of each stack position. The staff operates the bridge crane to drive from one end of the warehouse to the other end to shoot the stack position video and input it into the stack position coordinate calibration system to convert the color picture of the position of the marker at the center of the stack position taken in the video into a grayscale image; then perform Gaussian filtering on the input grayscale image, make a grayscale histogram, extract the threshold, and perform binarization processing; the effects of the marker images taken by multiple cameras are as Figure 4 shown; extract the picture contour according to the edge detection algorithm, as Figure 5 shown.

[0076] Extract the image coordinates of the center of the circle according to the set circular edge segmentation rule, artificially specify the actual origin of the coordinate system of the bridge crane in the warehouse stack position area, and manually measure the actual bridge crane coordinates of the center point of the first marker.

[0077] For the top images of the stack positions taken by the 4 cameras, detect the feature points in all the input images for image registration. The corner detection algorithm is used to provide a better feature matching effect. Its features are rotation invariance and scale variability, and in the intensity change under displacement:

[0078] E(u, v) = x,y w(x, y)[I(x + u, y + u)I(x, y)] 2

[0079] x,y w(x, y) is a window function, I(x + u, y + u) is the intensity after movement, and I(x, y) is the intensity at a single pixel position.

[0080] The corner detection algorithm first calculates the autocorrelation matrix M for each pixel I(x, y) point in the image, where

[0081]

[0082] I x , I y are the partial derivatives of I(x, y). Gaussian filtering is performed on each pixel point in the image to obtain a new matrix M. The discrete two-dimensional zero-mean Gaussian function is

[0083] Gauss = exp(-u 2 + v 2 ) / 2δ 2

[0084] Calculate the corner metric for each pixel point (x, y) to obtain

[0085] R = Det(M) - k * trace(M)

[0086] k is a coefficient, and its value range can be designed according to the actual situation, such as 0.04 < k < 0.06.

[0087] Select the local maximum points. The algorithm assumes that the feature points correspond to the pixel values of the local maximum interest points.

[0088] Set a threshold T to detect corners. If the maximum value of R is higher than the threshold T, then this point is a corner.

[0089] After detecting the image feature points, match the images by the sum of squared differences method. Cross-correlation works by analyzing the pixel windows around each point in the first image and correlating them with the pixel Windows around each point in the second image. The points with the maximum bidirectional correlation are taken as corresponding pairs. Based on the similarity between the image intensity values calculated in each of the two image displacements (shifts) to the "windows"

[0090] Calculate the homography matrix and remove the unnecessary angles that do not belong to the overlapping region.

[0091] A homograph is a mapping between two spaces, usually used to represent the correspondence between two images of the same scene. It can match most relevant feature points and can achieve image projection, so that one image can be projected to largely overlap with another image.

[0092] The last step of image stitching is to transform all input images and fuse them into a consistent output image. All input images can be simply transformed into a plane called the composite panoramic plane. Pixel colors are blended in the overlapping areas to avoid seams. The simplest form of use is feathering, which uses weighted average color values to blend overlapping pixels. An alpha factor is usually used, commonly known as the alpha channel, which has a value of 1 at the central pixel and becomes 0 when linearly decreasing to the boundary pixels. When there are at least two overlapping images in the output mosaic image, the following alpha value is used to calculate the color of a pixel: Assume that two images overlap in the output image; each pixel in the image, where (R, G, B) are the color values of the pixel. The pixel value at (x, y) in the stitched output image will be calculated.

[0093] Stitch all stack images to obtain a panoramic image of the stacks in the warehouse.

[0094] Input the coordinates of the marker points in the first image into the stack calibration system as the initial values. The true coordinates of all markers in the stack positions of the panoramic image are obtained through the matrix transformation between the image coordinates of the marker center in the captured image and the coordinate system of the bridge crane.

[0095] Those skilled in the art know that in addition to implementing the system and its various devices, modules, and units provided by the present invention in the form of pure computer-readable program code, the method steps can be logically programmed to enable the system and its various devices, modules, and units provided by the present invention to be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers, etc. to achieve the same functions. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a kind of hardware component, and the devices, modules, and units included therein for implementing various functions can also be regarded as the structure within the hardware component; the devices, modules, and units for implementing various functions can also be regarded as either software modules for implementing the method or the structure within the hardware component.

[0096] In the description of the present application, it should be understood that the orientation or positional relationship indicated by the terms "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present application.

[0097] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A stack position coordinate calibration device based on image shape detection and image fusion, characterized in that, it includes: Module M1: Measure the center of each stack position and assemble eye-catching markers at each center; Module M2: Collect the top images of the stack positions through multiple cameras on the overhead crane; Module M3: Use image recognition technology to extract the contour information of the markers from the top images of the stack positions to obtain the shape information of the markers; Module M4: Preprocess the color picture of the position of the marker at the center of the stack position to obtain the coordinates of the center point of the stack position in the image, and identify the position of the marker and the relative positions between the markers; Module M5: Measure the world coordinates of a marker at a stack position, assign them to the stack position coordinate calibration system, and obtain the world coordinates of the stack position through the matrix transformation between the coordinates of the center point of the stack position in the image and the overhead crane coordinate system; Module M6: For the top images of the stack positions taken by multiple cameras respectively, use image feature point extraction and registration technology to extract the feature points of the images in the video stream of multiple frames of stack position images in the obtained video stream, and then perform matching, registration, fusion and boundary processing and stitching of the feature points to obtain a panoramic image of the warehouse; Module M7: Taking the origin of the overhead crane coordinate as a reference point, use image recognition and extraction technology to convert the world coordinates of all stack positions in the panoramic image into the complete stack position coordinates of the entire warehouse; Module M8: For the case where steel products have been placed on the stack position, place the marker at the center position on the top of the steel products, and the system will handle it in the same way as the marker placed on the stack position, or after the steel products are lifted off, supplement the calibration in combination with the nearby calibrated stack positions; The said Module M6 includes: Install multiple network cameras on the overhead crane at the top of the stack position to ensure that the shooting areas of the multiple cameras cover the placement areas of the stack positions in the entire warehouse, correctly collect the top images of the stack positions, assemble markers at the center of each stack position, operate the overhead crane to drive from one end of the warehouse to the other end to shoot the stack position video, and input it into the stack position coordinate calibration device, and convert the color picture of the marker position taken in the stack position video into a grayscale picture; then perform Gaussian filtering on the input grayscale picture, make a grayscale histogram, extract the threshold, perform binarization processing, and extract the picture contour according to the edge detection algorithm; Extract the center image coordinates of the circle according to the set circular edge segmentation rule, specify the actual origin of the overhead crane coordinate system in the warehouse stack position area, and measure the actual overhead crane coordinates of the center point of the first marker; For the top images of the stack positions taken by multiple cameras, detect the feature points in all input images for image registration, and use the corner detection algorithm. In the intensity change under displacement: E(u,v) = x,y w(x,y)[I(x + u,y + u)I(x,y)] 2 x,y w(x,y) is a window function, I(x+u,y+u) is the intensity after movement, and I(x,y) is the intensity at a single pixel position; The corner detection algorithm first calculates the autocorrelation matrix M for each I(x,y) in the image, where I x , I y is the partial derivative of I(x, y). Gaussian filtering is performed on each pixel point in the image to obtain a new matrix M. The discrete two-dimensional zero-mean Gaussian function is Gauss = exp(-u 2 + v 2 ) / 2δ 2 Calculate the corner metric of each pixel point (x,y) to obtain R = Det(M) - k * trace(M) k is a coefficient, and its value range can be designed according to the actual situation; Select the local maximum points, and the pixel values of the feature points correspond to the local maximum interest points; Set the threshold T to detect corners. If the maximum value of R is higher than the threshold T, then this point is a corner. After detecting the image feature points, the images are matched by the sum of squared differences method. Cross-correlation works by analyzing the pixel windows around each point in the first image and correlating them with the pixel Windows around each point in the second image, and taking the points with the maximum bidirectional correlation as corresponding pairs, based on the similarity between the image intensity values calculated in each of the two image displacements to the "windows". Calculate the homography matrix and remove the unnecessary angles that do not belong to the overlapping area. Convert all input images and fuse them into a consistent output image. Convert all input images to a plane called the composite panoramic plane, and blend the pixel colors in the overlapping area to avoid seams. Stitch all the stack images to obtain a panoramic image of the stacks in the warehouse.

2. The stack coordinate calibration device based on image shape detection and image fusion according to claim 1, characterized in that the prominent markers in the module M1 include: regularly shaped objects that can be distinguished by color; if the stack itself has a certain regular shape or color, the stack can be directly used as a marker.

3. The stack coordinate calibration device based on image shape detection and image fusion according to claim 1, characterized in that the shooting ranges of the multiple cameras in the module M2 can cover all the stack placement areas where coordinates need to be obtained after stitching.

4. The stack coordinate calibration device based on image shape detection and image fusion according to claim 1, characterized in that in the module M7, the image recognition includes converting the color picture of the position of the center marker of the stack taken into a grayscale picture, then performing Gaussian filtering on the input grayscale picture, making a grayscale histogram, and extracting the threshold.

5. The stack coordinate calibration device based on image shape detection and image fusion according to claim 1, characterized in that in the module M7, the conversion includes converting the image coordinates of the center point of the marker into the world coordinates in the bridge crane coordinate system through a matrix.

6. A stack coordinate calibration method based on image shape detection and image fusion, characterized in that it includes: Step 1: Measure the center of each stack and install prominent markers at each center; Step 2: Collect the top images of the stacks through multiple cameras on the bridge crane; Step 3: Adopt image recognition technology to extract the contour information of the markers from the top images of the stacks to obtain the shape information of the markers; Step 4: Perform image preprocessing on the color picture of the position of the center marker of the stack to obtain the coordinates of the center point of the stack in the image, and identify the position of the markers and the relative positions between the markers; Step 5: Measure the world coordinates of a certain stack marker, assign them into the stack coordinate calibration system, and obtain the world coordinates of the stack through the matrix conversion between the coordinates of the center point of the stack in the image and the bridge crane coordinate system; Step 6: For the top images of the stacks taken by multiple cameras respectively, use image feature point extraction and registration technology to extract the feature points of the images in the video stream from multiple frames of stack images in the acquired video stream, and then perform matching, registration, fusion and boundary processing and stitching of the feature points to obtain a panoramic image of the warehouse; Step 7: Taking the origin of the coordinate system of the bridge crane as the reference point, use image recognition and extraction technology to convert the world coordinates of all stack positions in the panoramic image into the complete stack position coordinates of the entire warehouse; Step 8: For the case where steel products have been placed on the stack position, directly place the marker at the center of the top of the steel product, and the system will handle it in the same way as the marker placed on the stack position; Or after the steel product is lifted off, supplement the calibration in combination with the nearby calibrated stack positions; The said step 6 includes: Install multi-channel network cameras on the bridge crane at the top of the stack position, ensure that the shooting areas of the multi-channel cameras cover the placement area of the stack positions in the entire warehouse, correctly collect the top images of the stack positions, assemble markers at the center of each stack position, operate the bridge crane to drive from one end of the warehouse to the other end to shoot stack position videos, and input them into the stack position coordinate calibration device. Convert the color pictures of the marker positions taken in the stack position videos into grayscale pictures; then perform Gaussian filtering on the input grayscale pictures, make a grayscale histogram, extract the threshold, perform binarization processing, and extract the picture contours according to the edge detection algorithm; Extract the center image coordinates of the circle according to the set circular edge segmentation rule, specify the actual origin of the coordinate system of the bridge crane in the warehouse stack position area, and measure the actual bridge crane coordinates of the center point of the first marker; For the top images of the stack positions taken by the multi-channel cameras, detect the feature points in all the input images for image registration. Using the corner detection algorithm, in the intensity change under displacement: E(u,v) = x,y w(x,y)[I(x + u,y + u)I(x,y)] 2 x,y w(x, y) is the window function, I(x + u, y + u) is the intensity after movement, and I(x, y) is the intensity at a single pixel position; The corner detection algorithm first calculates the autocorrelation matrix M for each I(x,y) in the image, where I x , I y is the partial derivative of I(x, y). Gaussian filtering is performed on each pixel point in the image to obtain a new matrix M. The discrete two-dimensional zero-mean Gaussian function is Gauss = exp(-u 2 + v 2 ) / 2δ 2 Calculate the corner metric of each pixel point (x,y) to obtain R = Det(M) - k * trace(M) k is a coefficient, and its value range can be designed according to the actual situation; Select the local maximum points, and the pixel values of the feature points correspond to the local maximum interest points; Set the threshold T to detect corners. If the maximum value of R is higher than the threshold T, then this point is a corner; After detecting the image feature points, match the images by the sum of squared differences method. Cross-correlation works by analyzing the pixel windows around each point in the first image and associating them with the pixel Windows around each point in the second image. Take the point with the maximum bidirectional correlation as the corresponding pair, based on the similarity between the image intensity values calculated in each of the two image displacements to the "window"; Calculate the homography matrix and delete the unnecessary angles that do not belong to the overlapping area; Convert all the input images and fuse them into a consistent output image. Convert all the input images into a plane called the composite panoramic plane, and mix the pixel colors in the overlapping area to avoid seams; Stitch all the stack position images to obtain the panoramic image of the stack positions in the warehouse.

7. The stack position coordinate calibration method based on image shape detection and image fusion according to claim 6, Characterized in that, The eye-catching markers in the said step 1 include: regular-shaped objects that can be distinguished by color; if the stack position itself has a certain regular shape or color, the stack position can be directly used as a marker.

8. The stack position coordinate calibration method based on image shape detection and image fusion according to claim 6, Characterized in that, After the shooting ranges of the multiple cameras in step 2 are stitched, they can cover all the stacking areas where coordinates need to be obtained.

9. The stacking position coordinate calibration method based on image shape detection and image fusion according to claim 6, characterized in that in step 7, the image recognition includes converting the color picture of the position of the center marker of the stacking position taken into a grayscale picture, then performing Gaussian filtering on the input grayscale picture, making a grayscale histogram, and extracting the threshold.

10. The stacking position coordinate calibration method based on image shape detection and image fusion according to claim 6, characterized in that in step 7, the conversion includes converting the image coordinates of the center point of the marker into world coordinates in the bridge crane coordinate system through a matrix.

Images