Field transfer method based on binocular stereoscopic vision

Abstract

The invention relates to a field transfer method based on binocular stereoscopic vision, which can effectively solve the problem of automatic field transfer of a specific product by a transfer vehicle, and adopts the technical scheme that the field transfer method based on binocular stereoscopic vision utilizes a vision measurement system mounted on the transfer vehicle, and then comprises the following steps: 1, automatic grabbing; 2, automatic falling; the transferring method has the advantages of high precision, real-time dynamic measurement, non-contact, real-time output of measurement results, signal synchronization, module customization function, capability of guiding a mechanical arm or a mechanism to carry out posture adjustment in real time and completion of butt joint or transferring tasks, capability of ensuring automatic hoisting and falling of products, capability of rapidly completing product transferring work and capability of improving the product transferring efficiency. And meanwhile, improvement of product quality safety, shortening of a production detection period, improvement of working efficiency and the like are facilitated, and the method is an innovation of a transshipment method in industrial production.

Description

1. Technical Field

[0001] The invention relates to the field of automated measurement, in particular to a field transfer method based on binocular stereo vision. 2. Background Technology

[0002] With the increasing requirements for automated measurement in industrial production, visual measurement technology has been widely introduced into the industrial sector. It has achieved rapid and in-depth development in both basic research and applied research, forming a relatively complete directional system of theoretical methods, technical processes, system development and product applications. It shows a development trend of theoretical systematization, multidimensional methods, precision and speed, and has become an indispensable and preferred technology for intelligent manufacturing process control, product quality inspection and assurance, equipment service testing, and product automation docking.

[0003] At present, visual measurement technology has been widely used in many industrial fields such as object recognition, product guided docking, 3D reconstruction, skeleton tracking, digital twins, automatic transfer, deformation tracking, etc. In industrial production and manufacturing, industrial photogrammetry technology is needed for guided docking, measurement and positioning, etc., which can ensure the automatic docking measurement of products, quickly complete assembly work, reduce manual intervention, and at the same time help improve product quality, shorten production inspection cycle, and improve work efficiency. At present, there is much manual intervention in the automatic transportation of specific products by field transfer vehicles, and the work efficiency is low. Therefore, it is imperative to invent a new transfer method based on binocular stereo vision. III. Summary of the invention

[0004] In view of the above situation, in order to solve the defects of the prior art, the purpose of the present invention is to provide a field transfer method based on binocular stereo vision, which can effectively solve the problem of automatic field transfer of specific products by transfer vehicles.

[0005] The technical solution provided by the present invention is that the field transfer method based on binocular stereo vision utilizes a visual measurement system installed on a transfer vehicle, and then proceeds according to the following steps:

[0006] 1. Automatic crawling

[0007] 1) Start the visual measurement system

[0008] The camera online controller is in the self-starting state after the transfer vehicle is powered on;

[0009] 2) Product hoisting starts

[0010] After the spreader reaches a certain position, the transfer vehicle sends a start measurement guidance instruction to the visual software measurement module;

[0011] 3) Photograph the product’s location

[0012] First, the robotic arm will move to the top of the drop rack, take pictures of the drop rack's coordinates and record its position;

[0013] 4) Shoot the target at the product hoisting corner piece position

[0014] After the drop rack position is photographed, the robot arm will move over the product, photograph the product position and start grabbing;

[0015] 5) Calculate the relative position and posture relationship between the hanger and the product hanging

[0016] After the visual software measurement module obtains the three-dimensional spatial coordinates of the measuring tooling mark points through calculation, it obtains the position and posture relationship between the hanger and the product corner parts through coordinate conversion according to the tooling coordinate system file established in the early stage, and automatically transmits the data adjustment amount to the transfer vehicle for feedback;

[0017] 6) Adjust the position of the transfer vehicle hoist

[0018] The transfer vehicle is adjusted according to the posture adjustment amount fed back by the visual software measurement module;

[0019] 7) The transfer vehicle is adjusted to the correct position

[0020] When the adjustment amount obtained by the visual software measurement module reaches the preset tolerance range, the visual software measurement module stops measuring, and the transfer vehicle automatically performs hoisting until the product is accurately hoisted and the measurement guidance ends;

[0021] 2. Automatic placement

[0022] 1) Product placement begins

[0023] After the product is hoisted to a certain position, the transfer vehicle sends a start measurement guidance instruction to the visual software measurement module;

[0024] 2) Photograph the product landing position coding tooling

[0025] The robotic arm hoist will move to the top of the drop rack and take real-time photos of the coded point coordinates on the side of the drop rack;

[0026] 3) Calculate the relative position and posture relationship between the hanger and the product hanging

[0027] After the visual software measurement module obtains the three-dimensional spatial coordinates of the measuring tooling mark point through calculation, it obtains the position and posture relationship between the hanger and the product through coordinate conversion according to the tooling coordinate system file established in the early stage, and automatically transmits the data adjustment amount to the transfer vehicle for feedback;

[0028] 4) The transfer vehicle adjusts its position

[0029] The transfer vehicle is adjusted according to the posture adjustment amount fed back by the visual software measurement module;

[0030] 5) The transfer vehicle is adjusted to the correct position

[0031] When the adjustment amount obtained by the visual software measurement module reaches the predetermined tolerance range, the product is guided to accurately fall into position, the visual software measurement module stops measuring, and the product is automatically placed into position.

[0032] The visual measurement system includes four measuring cameras, a visual measurement controller, targets, cables and a visual software measurement module. The four measuring cameras are respectively installed on both sides of the sling and connected to the visual measurement controller via cables. When the sling reaches under the product, the measuring cameras measure the targets placed at the lifting point of the product and the target at the landing point, and can calculate the relative position X, Y, Z and rotation angles Rx, Ry, Rz of the sling relative to the product in the horizontal plane, and send the results to the visual measurement controller via the CAN bus.

[0033] The four measuring cameras are DHEx-M20IR infrared sensitive cameras.

[0034] A directional ruler is arranged in front of the four measuring cameras.

[0035] The visual measurement controller is a controller based on a domestic Linux system.

[0036] The target is made of directional reflective material with a high return light reflectivity. The target is placed at a corresponding position, and the three-dimensional coordinates of the position can be obtained by identifying the mark, and the corresponding position of the product can be fed back.

[0037] The visual software measurement module comprises a visual camera driving module, a communication module, a camera control and acquisition module, a network data communication module, an image processing and visual algorithm processing module, a data analysis and calculation module, and a log storage module.

[0038] The visual camera driver module is a Gigabit network communication driver developed based on the GenCam protocol, which is built on the basis of the TCP network protocol.

[0039] The communication module is mainly used to set communication parameters for the serial port, command verification and data transmission and reception command analysis, etc.

[0040] The camera control and acquisition module is mainly used to control the acquisition parameters of the camera (including time, gain, transmission parameters, etc.) and perform image data acquisition and storage.

[0041] The network data communication module is mainly used to communicate with the host computer, obtain the measurement instructions sent by the host computer and feed back the measurement results to the host computer. Communication failures and system operation failures can return error codes.

[0042] The image processing and visual algorithm processing module is mainly used to process the collected images to obtain the image coordinates of the photographic mark, and then orient the camera and measure the mark, and the measurement points are all in the visual measurement system coordinate system.

[0043] The data analysis and calculation module mainly performs analysis and calculation based on the measured coordinates to finally obtain the positioning result.

[0044] The log storage module is used to store system operation records, mainly including the functions of receiving and sending instruction records, system operation records, and measurement data storage.

[0045] The transfer method of the present invention has the advantages of high precision, real-time dynamic measurement, non-contact, real-time output of measurement results, signal synchronization, and module customization function. It can guide the robot arm or mechanism to adjust the posture in real time to complete the docking or transfer task. It can ensure the automatic lifting and dropping of products, quickly complete the product transfer work, reduce manual intervention, and at the same time is beneficial to improving product quality and safety, shortening the production inspection cycle, and improving work efficiency. It is an innovation in the transfer method in industrial production. IV. Description of the drawings

[0046] Figure 1 It is a schematic diagram of the automatic grabbing process of the present invention.

[0047] Figure 2 It is a schematic diagram of the automatic placement process of the present invention.

[0048] Figure 3 This is the architecture diagram of the visual software measurement module of the present invention. V. Specific implementation methods

[0049] The specific implementation modes of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0050] Depend on Figure 1-3 Given, the present invention utilizes a visual measurement system installed on a transfer vehicle, and then proceeds according to the following steps:

[0051] 1. Automatic crawling

[0052] 1) Start the visual measurement system

[0053] The camera online controller is in the self-starting state after the transfer vehicle is powered on;

[0054] 2) Product hoisting starts

[0055] After the spreader reaches a certain position, the transfer vehicle sends a start measurement guidance instruction to the visual software measurement module;

[0056] 3) Photograph the product’s location

[0057] First, the robotic arm will move to the top of the drop rack, take pictures of the drop rack's coordinates and record its position;

[0058] 4) Shoot the target at the product hoisting corner piece position

[0059] After the drop rack position is photographed, the robot arm will move over the product, photograph the product position and start grabbing;

[0060] 5) Calculate the relative position and posture relationship between the hanger and the product hanging

[0061] After the visual software measurement module obtains the three-dimensional spatial coordinates of the measuring tooling mark points through calculation, it obtains the position and posture relationship between the hanger and the product corner parts through coordinate conversion according to the tooling coordinate system file established in the early stage, and automatically transmits the data adjustment amount to the transfer vehicle for feedback;

[0062] 6) Adjust the position of the transfer vehicle hoist

[0063] The transfer vehicle is adjusted according to the posture adjustment amount fed back by the visual software measurement module;

[0064] 7) The transfer vehicle is adjusted to the correct position

[0065] When the adjustment amount obtained by the visual software measurement module reaches the preset tolerance range, the visual software measurement module stops measuring, and the transfer vehicle automatically performs hoisting until the product is accurately hoisted and the measurement guidance ends;

[0066] 2. Automatic placement

[0067] 1) Product placement begins

[0068] After the product is hoisted to a certain position, the transfer vehicle sends a start measurement guidance instruction to the visual software measurement module;

[0069] 2) Photograph the product landing position coding tooling

[0070] The robotic arm hoist will move to the top of the drop rack and take real-time photos of the coded point coordinates on the side of the drop rack;

[0071] 3) Calculate the relative position and posture relationship between the hanger and the product hanging

[0072] After the visual software measurement module obtains the three-dimensional spatial coordinates of the measuring tooling mark point through calculation, it obtains the position and posture relationship between the hanger and the product through coordinate conversion according to the tooling coordinate system file established in the early stage, and automatically transmits the data adjustment amount to the transfer vehicle for feedback;

[0073] 4) The transfer vehicle adjusts its position

[0074] The transfer vehicle is adjusted according to the posture adjustment amount fed back by the visual software measurement module;

[0075] 5) The transfer vehicle is adjusted to the correct position

[0076] When the adjustment amount obtained by the visual software measurement module reaches the predetermined tolerance range, the product is guided to accurately fall into position, the visual software measurement module stops measuring, and the product is automatically placed into position.

[0077] In order to ensure the use effect, the visual measurement system includes four measuring cameras, a visual measurement controller, targets, cables and a visual software measurement module. The four measuring cameras are respectively installed on both sides of the hanger and connected to the visual measurement controller via cables. When the hanger reaches under the product, the measuring camera measures the target placed at the hanging point of the product and the target at the landing point, and can calculate the relative position X, Y, Z and rotation angles Rx, Ry, Rz of the hanger relative to the product in the horizontal plane, and send the results to the visual measurement controller through the CAN bus.

[0078] The four measuring cameras are DHEx-M20IR infrared sensitive cameras.

[0079] A directional ruler is arranged in front of the four measuring cameras.

[0080] The visual measurement controller is a controller based on a domestic Linux system.

[0081] The target is made of directional reflective material with a high return light reflectivity. The target is placed at a corresponding position, and the three-dimensional coordinates of the position can be obtained by identifying the mark, and the corresponding position of the product can be fed back.

[0082] The visual software measurement module includes a visual camera driving module, a communication module, a camera control and data acquisition module, a network data communication module, an image processing and visual algorithm module, a data analysis and calculation module, and a log storage module.

[0083] The visual camera driver module is a Gigabit network communication driver developed based on the GenCam protocol, which is built on the basis of the TCP network protocol.

[0084] The communication module is mainly used to set communication parameters for the serial port, command verification and data transmission and reception command analysis, etc.

[0085] The camera control and data acquisition module is mainly used to control the acquisition parameters of the camera (including time, gain, transmission parameters, etc.) and perform image data acquisition and storage.

[0086] The network data communication module is mainly used to communicate with the host computer, obtain the measurement instructions sent by the host computer and feed back the measurement results to the host computer. Communication failures and system operation failures can return error codes.

[0087] The image processing and visual algorithm module is mainly used to process the collected images to obtain the image coordinates of the photographic mark, and then orient the camera and measure the mark, and the measurement points are all in the visual measurement system coordinate system.

[0088] The data analysis and calculation module mainly performs analysis and calculation based on the measured coordinates to finally obtain the positioning result.

[0089] The log storage module is used to store system operation records, mainly including the functions of receiving and sending instruction records, system operation records, and measurement data storage.

[0090] The transfer method of the present invention has the advantages of high precision, real-time dynamic measurement, non-contact, real-time output of measurement results, signal synchronization, and module customization function. It can guide the mechanical arm or mechanism to adjust the posture in real time to complete the docking or transfer task. It can ensure the automatic lifting and placement of the product, quickly complete the product transfer work, reduce manual intervention, and at the same time be beneficial to improve product quality and safety, shorten the production inspection cycle, and improve work efficiency. The system can work all-weather in an outdoor open-air environment, with a protection level of IP67, and 100% of the components are domestically produced. It can be applied to field automatic transfer, automatic docking of barrel sections, intelligent product loading, automatic transfer of aerial vehicles, automatic warehousing of products, product entry and exit, etc., among which:

[0091] The measuring camera is a DHEx-M20IR infrared sensitive camera, which uses an infrared light source and is mainly responsible for image acquisition.

[0092]

[0093]

[0094] The visual measurement controller is used to synchronize the camera flash, receive camera images, process images, and output measurement results. The master control system does not adopt a protective design. As a part of the visual positioning controller, it is used to power the camera module and communicate with the camera at the same time, control the camera to take pictures and set related parameters. The controller and the control combination communicate through the CAN line. The controller uses the vehicle-mounted 28V power supply with a power supply voltage range of 18 to 36V. The controller adopts a waterproof design with a waterproof grade of IP67.

[0095] Serial number project Technical indicators 1 Supply voltage 18~36VDC 2 power 95W 3 CPU Loongson 3A5000 4 Frequency 2.0GHz 5 Hard disk capacity Default 64G, expandable to 512G 6 Memory 8GB 7 Other interfaces Two CAN interfaces, 1 debugging network port, 1 debugging USB, 1 HDMI

[0096] The orienteering ruler is used for position calibration between visual cameras. The positional relationship between two or more visual cameras can be calculated by using the orienteering ruler at different positions in front of the visual camera. Thus, the two or more cameras can measure the coordinates of the target through the principle of front intersection, thereby obtaining the position and posture of the platform.

[0097] The target is made of a directional reflective material with a high return reflectivity. The target is placed at the corresponding position. The three-dimensional coordinates of the position can be obtained by identifying the mark, and the corresponding position of the product can be fed back.

[0098] The visual measurement system uses four visual cameras, which are installed on both sides of the hanger. When the hanger reaches under the product, the visual cameras measure the targets placed at the hanger point and the landing point of the product, and calculate the relative position X, Y, Z and rotation angles Rx, Ry, Rz of the hanger relative to the product in the horizontal plane, and send the results to the visual measurement controller through the CAN bus.

[0099] The visual measurement module is used to measure the relative position relationship between the spreader and the product on the transfer vehicle. When the spreader moves to the top of the transfer vehicle and enters the measurement range of the visual device, the visual device can measure the spreader's relative position X, Y, Z and posture Rx, Ry, Rz (the posture includes the spreader's yaw, pitch, and roll angles relative to the transfer vehicle) in the horizontal plane, and send it to the host computer control system through Can for automatic grasping and automatic placement. The specific process is as follows: Figure 1 , Figure 2 shown.

[0100] The visual software measurement module is installed on the industrial computer. It is the core of the entire visual measurement and is built on the domestically produced Galaxy Kirin system. It mainly uses its safety, stability and high efficiency. Its main architecture is as follows: Figure 3 As shown, it includes visual camera driver module, communication module, camera control and acquisition module, network data communication module, image processing and visual algorithm processing module, data analysis and calculation module, log storage function, etc., as follows:

[0101] The visual camera driver module is a Gigabit network communication driver developed based on the GenCam protocol, which is built on the basis of the TCP network protocol.

[0102] The communication module is mainly used to set communication parameters for the serial port, command verification, and data transmission and reception command analysis.

[0103] The camera control and acquisition module mainly controls the camera's acquisition parameters (including time, gain, transmission parameters, etc.) and performs image data acquisition and storage.

[0104] The network data communication module mainly communicates with the host computer, obtains the measurement instructions sent by the host computer and feeds back the measurement results to the host computer. Communication failures and system operation failures may return error codes.

[0105] The image processing and visual algorithm processing module mainly processes the collected images to obtain the image coordinates of the photographic mark, and then orients the camera and measures the mark. The measurement points are all in the visual measurement system coordinate system.

[0106] The data analysis and calculation module mainly performs analysis and calculation based on the measured coordinates to finally obtain the positioning results.

[0107] The log storage module is used to save system operation records, mainly including records of receiving and sending instructions, system operation records, measurement data storage and other functions.

[0108] The visual measurement method of the present invention mainly solves the problem of automatic field transfer of specific products by a certain type of transfer vehicle, which consists of two vehicles A and B. Vehicle A is a product transfer vehicle, and vehicle B is a transport vehicle product. The transfer mechanical arm carried on vehicle A can complete the lifting and transfer of products in vehicle A and workshop B. The main implementation method is to install four visual cameras above the transfer vehicle mechanical arm, and the cameras are installed on both sides of the mechanical arm sling. When the sling reaches under the product, the visual camera measures the target placed at the lifting point of the product and the target at the landing point, and can calculate the relative position X, Y, Z and rotation angle Rx, Ry, Rz of the sling relative to the product in the horizontal plane, and send the results to the controller system through the CAN bus to guide the mechanical arm on vehicle A to complete the transfer and placement of the product.

[0109] The visual measurement method of the present invention can ensure that the product can be automatically docked and measured, and has high-precision, real-time dynamic measurement, non-contact, real-time output of measurement results, and signal synchronization functions. It can guide the robot arm or mechanism to adjust the posture in real time to complete the docking or transfer task. It can ensure that the product can be automatically hoisted and dropped, and the product transfer work can be completed quickly, reducing manual intervention. At the same time, it is beneficial to improve product quality and safety, shorten the production inspection cycle, and improve work efficiency. The system can work all-weather in outdoor open-air environments, with a protection level of IP67, and 100% of the components are domestically produced. It can be applied to field automatic transfer, automatic docking of barrel sections, intelligent loading of products, automatic transfer of aerial vehicles, automatic warehousing of products, and product entry and exit. It is an innovation in transfer methods in industrial production and has good economic and social benefits.

[0110] It should be pointed out that the above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any technician familiar with the profession can make changes or modify the technical contents disclosed above into equivalent embodiments with equivalent changes without departing from the scope of the technical solution of the present invention, which all fall within the protection scope of the present invention.

Claims

1. A field reprinting method based on binocular stereo vision, characterized in that: The field transfer method based on binocular stereo vision utilizes a visual measurement system installed on a transfer vehicle, and then proceeds according to the following steps:

1. Automatic crawling 1) Start the visual measurement system The camera online controller is in the self-starting state after the transfer vehicle is powered on; 2) Product hoisting starts After the spreader reaches a certain position, the transfer vehicle sends a start measurement guidance instruction to the visual software measurement module; 3) Photograph the product’s location First, the robotic arm will move to the top of the drop rack, take pictures of the drop rack's coordinates and record its position; 4) Photograph the target at the product hoisting corner fittings After the drop rack position is photographed, the robot arm will move over the product, photograph the product position and start grabbing; 5) Calculate the relative position and posture between the hanger and the product After the visual software measurement module obtains the three-dimensional spatial coordinates of the measuring tooling mark points through calculation, it obtains the position and posture relationship between the hanger and the product corner parts through coordinate conversion according to the tooling coordinate system file established in the early stage, and automatically transmits the data adjustment amount to the transfer vehicle for feedback; 6) Adjust the position of the transfer vehicle hoist The transfer vehicle is adjusted according to the posture adjustment amount fed back by the visual software measurement module; 7) The transfer vehicle is adjusted to the correct position When the adjustment amount obtained by the visual software measurement module reaches the preset tolerance range, the visual software measurement module stops measuring, and the transfer vehicle automatically performs hoisting until the product is accurately hoisted and the measurement guidance ends; 2. Automatic placement 1) Product placement begins After the product is hoisted to a certain position, the transfer vehicle sends a start measurement guidance instruction to the visual software measurement module; 2) Photograph the product landing position coding tooling The robotic arm hoist will move to the top of the drop rack and take real-time photos of the coded point coordinates on the side of the drop rack; 3) Calculate the relative position and posture relationship between the hanger and the product hanging After the visual software measurement module obtains the three-dimensional spatial coordinates of the measuring tooling mark point through calculation, it obtains the position and posture relationship between the hanger and the product through coordinate conversion according to the tooling coordinate system file established in the early stage, and automatically transmits the data adjustment amount to the transfer vehicle for feedback; 4) The transfer vehicle adjusts its position The transfer vehicle is adjusted according to the posture adjustment amount fed back by the visual software measurement module; 5) The transfer vehicle is adjusted to the correct position When the adjustment amount obtained by the visual software measurement module reaches the predetermined tolerance range, the product is guided to accurately fall into position, the visual software measurement module stops measuring, and the product is automatically placed into position.

2. The field transfer method based on binocular stereo vision according to claim 1, characterized in that: The visual measurement system includes four measuring cameras, a visual measurement controller, targets, cables and a visual software measurement module. The four measuring cameras are respectively installed on both sides of the sling and connected to the visual measurement controller via cables. When the sling reaches under the product, the measuring cameras measure the targets placed at the lifting points and the landing points of the product, and can calculate the relative positions X, Y, Z and rotation angles Rx, Ry, Rz of the sling relative to the product in the horizontal plane, and send the results to the visual measurement controller via the CAN bus.

3. The field reprinting method based on binocular stereo vision according to claim 2 is characterized in that: The four measuring cameras are DHEx-M20IR infrared sensitive cameras.

4. The method for field reprinting based on binocular stereo vision according to claim 2, characterized in that: A directional ruler is arranged in front of the four measuring cameras.

5. The method for field reprinting based on binocular stereo vision according to claim 2, characterized in that: The visual measurement controller is a controller based on a domestic Linux system.

6. The method for outdoor reprinting based on binocular stereo vision according to claim 2, characterized in that: The target is made of directional reflective material with a high return light reflectivity. The target is placed at a corresponding position, and the three-dimensional coordinates of the position can be obtained by identifying the mark, and the corresponding position of the product can be fed back.

7. The method for field reprinting based on binocular stereo vision according to claim 2, characterized in that: The visual software measurement module includes a visual camera driving module, a communication module, a camera control and acquisition module, a network data communication module, an image processing and visual algorithm processing module, a data analysis and calculation module, and a log storage module; The visual camera driver module is a Gigabit network communication driver developed based on the GenCam protocol, which is based on the TCP network protocol; The communication module is mainly used to set communication parameters for the serial port, verify instructions and analyze data transmission and reception commands; The camera control and acquisition module is mainly used to control the acquisition parameters of the camera, including time, gain, and transmission parameters, and to acquire and store image data; The network data communication module is mainly used to communicate with the host computer, obtain the measurement instructions sent by the host computer and feed back the measurement results to the host computer. Communication failure and system operation failure can return error codes; The image processing and visual algorithm processing module is mainly used to process the collected images to obtain the image coordinates of the photographic mark, and then to orient the camera and measure the mark, and the measurement points are all in the visual measurement system coordinate system; The data analysis and calculation module mainly performs analysis and calculation based on the measured coordinates to finally obtain the positioning result; The log storage module is used to store system operation records, including records of receiving and sending instructions, system operation records, and measurement data storage functions.

Images