Crane transferring method based on binocular vision

Abstract

The invention relates to a crane transshipment method based on binocular vision, which can effectively solve the problem of identification of the accurate position of a product transport vehicle in the prior art, and adopts the technical scheme that a crane visual navigation positioning system is utilized, and then the crane transshipment method comprises the following steps: 1) a crane drives a visual measurement camera to move to a specified position; (2) the visual positioning system controls the camera to photograph; 3) processing and analyzing the image acquired by the camera, and judging the orientation of the vehicle according to the number of the coordinates of the target point; 4) obtaining vehicle position coordinates; 5, the vehicle position coordinates are fed back to a controller of a domestic Linux system, and measurement is ended. According to the transferring method, the accurate position of the transport vehicle can be accurately measured, recognized position information is sent to transferring equipment, product transferring work is rapidly completed, manual intervention is reduced, and the working efficiency is improved. And meanwhile, the product quality safety is improved, the production detection period is shortened, the working efficiency is improved, and the method is an innovation of a transshipment method in industrial production.

Description

Technical Field

[0001] The invention relates to a crane visual navigation and positioning system, in particular to a crane transfer method based on binocular vision. Background Art

[0002] With the increasing requirements for automated measurement in industrial production, visual measurement technology has been widely introduced into the industrial sector. It has achieved rapid and in-depth development in both basic research and applied research, forming a relatively complete directional system of theoretical methods, technical processes, system development and product applications. It shows a development trend of theoretical systematization, multidimensional methods, precision and speed, and has become an indispensable and preferred technology for intelligent manufacturing process control, product quality inspection and assurance, equipment service testing, and product automation docking.

[0003] At present, visual measurement technology has been widely used in many industrial fields such as object recognition, product guided docking, 3D reconstruction, skeleton tracking, digital twins, automatic transfer, deformation tracking, etc. In industrial production and manufacturing, industrial photogrammetry technology is needed for guided docking, measurement and positioning, etc., which can ensure the automatic docking measurement of products, quickly complete assembly work, reduce manual intervention, and at the same time help improve product quality, shorten production inspection cycle, improve work efficiency, etc. Therefore, it is imperative to invent a crane transfer method based on binocular vision. Summary of the invention

[0004] In view of the above situation, in order to solve the defects of the prior art, the purpose of the present invention is to provide a crane transfer method based on binocular vision, which can effectively solve the problem of identifying the accurate position of product transport vehicles in the prior art.

[0005] The technical solution of the present invention is to use the crane visual navigation and positioning system and then proceed according to the following steps: 1) Start transferring, and the crane drives the visual measurement camera to the specified position; 2) After car A or car B arrives at the designated location, the crane sends a start measurement and positioning command to the visual positioning system. After receiving the start measurement and positioning command, the visual positioning system controls the camera to take pictures; 3) The image processing and visual algorithm processing module in the crane visual navigation and positioning system processes and analyzes the image acquired by the camera. If the image solution fails, the measurement is performed again. If the second measurement still fails, the failure information is fed back to the controller based on the domestic Linux system to end the measurement and positioning. If the image solution is successful, the coordinates of the target point installed on the vehicle can be obtained, and the vehicle direction can be determined according to the number of the target point coordinates; 4) The image processing and visual algorithm processing module calculates the relative position relationship between the target point coordinates on the vehicle and the vehicle to obtain the vehicle position coordinates; 5) Feedback the vehicle position coordinates to the controller based on the domestic Linux system to end the measurement.

[0006] The crane visual navigation and positioning system includes 4 navigation and positioning cameras (explosion-proof treatment), 1 controller based on the domestic Linux system (installed in the explosion-proof cabinet of the crane), 1 set of visual processing software, target points, 1 reference ruler and several cables. The navigation and positioning cameras use high-precision measurement cameras for explosion-proof treatment and are installed under the two beams of the crane. Two cameras are installed on each beam, with the same direction as the crane beam and extending vertically outward by 1m along the beam to ensure that after the crane is in place, the camera is near the top of the target point on the vehicle.

[0007] The power input of the crane visual navigation and positioning system is 24V DC, the hardware interface of the communication is an aviation connector, and the navigation and positioning system communicates with the crane controller via Ethernet.

[0008] The transfer method of the present invention can accurately measure the exact position of the transport vehicle and send the identified position information to the transfer equipment controller, which can ensure the automatic lifting and dropping of the product, quickly complete the product transfer work, reduce manual intervention, and at the same time be beneficial to improving product quality and safety, shortening the production inspection cycle, and improving work efficiency. It is an innovation in the transfer method in industrial production. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a working flow chart of the crane visual navigation and positioning system of the present invention.

[0010] Figure 2 This is the electrical schematic diagram of the visual measurement system of the present invention.

[0011] Figure 3 This is a schematic diagram of the installation position of the visual camera of the present invention.

[0012] Figure 4 This is a workflow diagram of the navigation and positioning camera of the present invention.

[0013] Figure 5 This is the definition of the crane coordinate system of the present invention.

[0014] Figure 6 It is a schematic diagram of orientation and establishment of coordinate system of the present invention. DETAILED DESCRIPTION

[0015] The specific implementation modes of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0016] Depend on Figure 1-6 Given, the present invention utilizes the crane visual navigation and positioning system, and then proceeds according to the following steps: 1) Start transferring, and the crane drives the visual measurement camera to the specified position; 2) After car A or car B arrives at the designated location, the crane sends a start measurement and positioning command to the visual positioning system. After receiving the start measurement and positioning command, the visual positioning system controls the camera to take pictures; 3) The image processing and visual algorithm processing module in the crane visual navigation and positioning system processes and analyzes the image acquired by the camera. If the image solution fails, the measurement is performed again. If the second measurement still fails, the failure information is fed back to the controller based on the domestic Linux system to end the measurement and positioning. If the image solution is successful, the coordinates of the target point installed on the vehicle can be obtained, and the vehicle direction can be determined according to the number of the target point coordinates; 4) The image processing and visual algorithm processing module calculates the relative position relationship between the target point coordinates on the vehicle and the vehicle to obtain the vehicle position coordinates; 5) Feedback the vehicle position coordinates to the controller based on the domestic Linux system to end the measurement.

[0017] In order to ensure the use effect, the crane visual navigation and positioning system includes 4 navigation and positioning cameras (explosion-proof treatment), 1 controller based on the domestic Linux system (installed in the explosion-proof cabinet of the crane), 1 set of visual processing software, target points, 1 reference ruler and several cables. The navigation and positioning cameras use high-precision measurement cameras for explosion-proof treatment and are installed under the two girders of the crane. Two cameras are installed on each girder, with the same direction as the crane girder and extending vertically outward by 1m along the girder to ensure that after the crane is in place, the camera is near the top of the target point on the vehicle.

[0018] The power input of the crane visual navigation and positioning system is 24V DC, the hardware interface of the communication is an aviation connector, and the navigation and positioning system communicates with the crane controller via Ethernet.

[0019] The visual processing software is written in C / C++ language, the hardware system is Linux system, and the visual measurement software is installed on a controller based on the domestic Linux system. It is the core of the entire visual measurement and includes the following functional modules: 1. Visual camera driver module: It is a Gigabit network communication driver developed based on the GenCam protocol, which is built on the basis of the TCP network protocol.

[0020] 2. RS485 communication module: mainly used to set communication parameters, command verification and data sending and receiving command analysis for RS485.

[0021] 3. Camera control and acquisition module: mainly controls the camera's acquisition parameters (including time, gain, transmission parameters, etc.) and performs image data acquisition and storage.

[0022] 4. Network data communication module: mainly communicates with the host computer, obtains the measurement instructions sent by the host computer and feeds back the measurement results to the host computer.

[0023] 5. Image processing and visual algorithm processing module: It mainly processes the collected images to obtain the image coordinates of the photographic mark, and orients the camera and measures the mark accordingly. The measurement points are all in the visual measurement system coordinate system.

[0024] 6. Data analysis and calculation module: mainly performs analysis and calculation based on the measured coordinates to finally obtain the positioning results.

[0025] The crane visual navigation and positioning system of the present invention adopts binocular photogrammetry for positioning, and uses 4 navigation and positioning cameras to cooperate with the target points originally provided on the vehicle for positioning measurement. The coded points installed on the vehicle can be guaranteed to be seen in a certain specific area, and have been reproduced and used in this model product.

[0026] The navigation and positioning cameras are explosion-proof high-precision measuring cameras that are installed under the two girders of the crane. Two cameras are installed on each girder in the same direction as the crane girder and extend vertically outward about 1m along the girder to ensure that after the crane is in place, the camera is near the top of the coding point on the vehicle.

[0027] The visual positioning system controller provides power for the camera, communicates with the four cameras, and controls the four cameras for measurement. At the same time, the visual positioning system software is installed in the visual navigation and positioning system controller to perform image recognition on the images obtained by the camera, obtain the coordinates of the coding points installed on the vehicle, and obtain the orientation of the vehicle, and calculate the current coordinates of the vehicle based on the target point coordinates.

[0028] Communication and power supply cables provide power to the camera and communicate at the same time, and can be explosion-proof.

[0029] System calibration The visual positioning system is unified with the crane moving direction coordinate system, and the crane moving coordinate system is used as the system reference coordinate system, such as Figure 5 shown.

[0030] The orientation of the visual positioning system refers to the process of calibrating the relative position and posture of a group of visual cameras. The orientation of the present invention adopts relative orientation combined with one-dimensional reference ruler constraint. The establishment of the system coordinate system is as follows: Figure 6 shown.

[0031] The hoist provides multiple image points with the same name for relative orientation by moving to multiple positions. The distance between the two photographic coding points is used as the distance constraint to establish a relative orientation model, and the relative position and posture of the two cameras can be calculated.

[0032] After the visual camera completes the orientation, its coordinate system origin and direction need to be converted to the coordinate system of the spreader movement in the image coordinate system of one of the visual cameras. This process is to establish the coordinate system of the visual orientation system. Similar to the visual camera orientation process, move the spreader about 500mm (not a fixed value) in the X direction of the crane movement, and the visual positioning system records its starting and ending positions to calibrate the X direction; move the spreader 500mm in the Y direction of the crane movement, and the visual positioning system records its starting and ending positions to calibrate the Y direction. In this way, the coordinate system of the visual positioning system is unified with the coordinate system of the crane movement direction.

[0033] The origin of the coordinate system of the visual positioning system is unified with the origin of the crane coordinate system and is measured and unified when the load is first lifted.

[0034] The sources of error in the measurement process of the crane visual navigation positioning measurement system of the present invention mainly include the following aspects: the measurement point error of the navigation positioning camera, the calibration error of the special directional ruler, the calibration error of the tooling, and the error of the conversion between the tooling and the measurement point. After repeated practice, the accuracy is as follows: 1) The present invention uses a navigation and positioning camera, and one system contains two cameras. The measurement point accuracy of the navigation and positioning camera system is 0.5mm+0.3mm / m. The farthest distance between the camera and the point is about 6m. The measurement point accuracy of the camera position to the farthest point can reach 2mm; 2) Since the present invention uses a reference ruler for orientation, the length of the orientation ruler is 400 mm, and two coding marks are distributed on each surface. The reference ruler orients the camera on site, and the orientation result can reach 0.5 mm; 3) The present invention requires the use of a positioning target originally installed on the vehicle, and the installation accuracy of the positioning target is about 0.5 mm.

[0035] To sum up, according to the law of error propagation, the accuracy of the digital measurement system of the transfer method of the present invention is estimated to be about 3mm, which is less than the technical requirement of 5mm. It can accurately measure the exact position of the transport vehicle and send the identified position information to the transfer equipment controller, which can ensure the automatic lifting and dropping of the product, quickly complete the product transfer work, reduce manual intervention, and at the same time is conducive to improving product quality and safety, shortening the production inspection cycle, and improving work efficiency. It is an innovation in the transfer method in industrial production and has good economic and social benefits.

[0036] It should be pointed out that the above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any technician familiar with the profession can make changes or modify the technical contents disclosed above into equivalent embodiments with equivalent changes without departing from the scope of the technical solution of the present invention, which all fall within the protection scope of the present invention.

Claims

1. A crane transfer method based on binocular vision, characterized in that: Use the crane vision navigation positioning system and then follow these steps: 1) Start transferring, and the crane drives the visual measurement camera to the specified position; 2) After car A or car B arrives at the designated location, the crane sends a start measurement and positioning command to the visual positioning system. After receiving the start measurement and positioning command, the visual positioning system controls the camera to take pictures; 3) The image processing and visual algorithm processing modules in the crane visual navigation and positioning system process and analyze the images acquired by the camera. If the image solution fails, the measurement is performed again. If the second measurement still fails, the failure information is fed back to the controller of the domestic Linux system to end the measurement and positioning. If the image solution is successful, the coordinates of the target point installed on the vehicle can be obtained, and the vehicle direction can be determined according to the number of the target point coordinates. 4) The image processing and visual algorithm processing module calculates the relative position relationship between the target point coordinates on the vehicle and the vehicle to obtain the vehicle position coordinates; 5) Feedback the vehicle position coordinates to the controller of the domestic Linux system to end the measurement.

2. The binocular vision-based crane transfer method according to claim 1, characterized in that: The crane visual navigation and positioning system includes 4 navigation and positioning cameras, 1 controller based on the domestic Linux system installed in the explosion-proof cabinet of the crane, 1 set of visual processing software, target points, 1 reference ruler and several cables. The navigation and positioning cameras use high-precision measurement cameras for explosion-proof treatment and are installed under the two beams of the crane. Two cameras are installed on each beam, with the same direction as the crane beam and extending vertically outward by 1m along the beam to ensure that after the crane is in place, the camera is near the top of the target point on the vehicle.

3. The binocular vision-based crane transfer method according to claim 2, characterized in that: The power input of the crane visual navigation and positioning system is 24V DC, the hardware interface of the communication is an aviation connector, and the navigation and positioning system communicates with the crane controller via Ethernet.

4. The binocular vision-based crane transfer method according to claim 2, characterized in that: The visual processing software is written in C / C++ language, the hardware system is a domestic Linux system, and the visual measurement software is installed on the domestic Linux system, including the following functional modules:

1. Visual camera driver module: It is a Gigabit network communication driver developed based on the GenCam protocol, which is based on the TCP network protocol; 2. RS485 communication module: set communication parameters for RS485, command verification and data sending and receiving command analysis; 3. Camera control and acquisition module: controls the camera acquisition parameters, including time, gain, and transmission parameters, and performs image data acquisition and storage; 4. Network data communication module: communicate with the host computer, obtain the measurement instructions sent by the host computer and feed back the measurement results to the host computer; 5. Image processing and visual algorithm processing module: The image coordinates of the photographic mark are obtained by processing the collected images, and the camera is oriented and the mark is measured accordingly. The measurement points are all in the visual measurement system coordinate system; 6. Data analysis and calculation module: perform analysis and calculation based on the measured coordinates to finally obtain the positioning results.

Images